Skip to contentSkip to conversation

drm/etnaviv: don't trigger OOM killer when page allocation fails

PATCH In review Lucas Stach 4 participants 8 messages Jun 6, 2017 – Jun 26, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/3 drm/etnaviv: don't trigger OOM killer when page allocation fails

GPU buffers can be quite large, so userspace is expected to deal with allocation failure. Don't trigger the OOM killer when page allocation for the GEM objects fails, as this opens an easy possiblity for unprivileged applications to DOS the system,a s the shmem pages are not fully accounted to the allocating process.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +2 -1
index f0efc5d..4c53508 100644
@@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev,
 		 * going to pin these pages.
 		 */
 		mapping = obj->filp->f_mapping;
-		mapping_set_gfp_mask(mapping, GFP_HIGHUSER);
+		mapping_set_gfp_mask(mapping, GFP_HIGHUSER |
+				     __GFP_NORETRY | __GFP_NOWARN);
 	}
 
 	if (ret)
Signature · 6 lines
2.9.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#1 lore raw

PATCH 2/3 drm/etnaviv: reduce allocation failure message severity

The GPU userspace is expected to deal with failure to allocate memory for the GPU buffers, there is no need to spam the log on failure.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +1 -1
index 4c53508..98243ba 100644
@@ -68,7 +68,7 @@ static int etnaviv_gem_shmem_get_pages(struct etnaviv_gem_object *etnaviv_obj)
 	struct page **p = drm_gem_get_pages(&etnaviv_obj->base);
 
 	if (IS_ERR(p)) {
-		dev_err(dev->dev, "could not get pages: %ld\n", PTR_ERR(p));
+		dev_dbg(dev->dev, "could not get pages: %ld\n", PTR_ERR(p));
 		return PTR_ERR(p);
 	}
 
Signature · 6 lines
2.9.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#2 lore raw

PATCH 3/3 drm/etnaviv: populate GEM objects on cpu_prep

In reply to Lucas Stach

CPU prep is the point where we can reasonably return an error to userspace when something goes wrong while populating the object. If we leave the object unpopulated at this point, the allocation will happen in the fault handler when userspace accesses the object through the mmap space, where we don't have any other option than to OOM the system.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +10 -10
index 98243ba..f6ac3f7 100644
@@ -413,6 +413,16 @@ int etnaviv_gem_cpu_prep(struct drm_gem_object *obj, u32 op,
 	bool write = !!(op & ETNA_PREP_WRITE);
 	int ret;
 
+	if (!etnaviv_obj->sgt) {
+		void *ret;
+
+		mutex_lock(&etnaviv_obj->lock);
+		ret = etnaviv_gem_get_pages(etnaviv_obj);
+		mutex_unlock(&etnaviv_obj->lock);
+		if (IS_ERR(ret))
+			return PTR_ERR(ret);
+	}
+
 	if (op & ETNA_PREP_NOSYNC) {
 		if (!reservation_object_test_signaled_rcu(etnaviv_obj->resv,
 							  write))
@@ -427,16 +437,6 @@ int etnaviv_gem_cpu_prep(struct drm_gem_object *obj, u32 op,
 	}
 
 	if (etnaviv_obj->flags & ETNA_BO_CACHED) {
-		if (!etnaviv_obj->sgt) {
-			void *ret;
-
-			mutex_lock(&etnaviv_obj->lock);
-			ret = etnaviv_gem_get_pages(etnaviv_obj);
-			mutex_unlock(&etnaviv_obj->lock);
-			if (IS_ERR(ret))
-				return PTR_ERR(ret);
-		}
-
 		dma_sync_sg_for_cpu(dev->dev, etnaviv_obj->sgt->sgl,
 				    etnaviv_obj->sgt->nents,
 				    etnaviv_op_to_dma_dir(op));
Signature · 6 lines
2.9.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#3 lore raw
In reply to Lucas Stach
Quoted 16 lines · Lucas Stach

GPU buffers can be quite large, so userspace is expected to deal with allocation failure. Don't trigger the OOM killer when page allocation for the GEM objects fails, as this opens an easy possiblity for unprivileged applications to DOS the system,a s the shmem pages are not fully accounted to the allocating process.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +0 -0
index f0efc5d..4c53508 100644

drivers/gpu/drm/etnaviv/etnaviv_gem.c static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, · line 666

Quoted 6 lines
drivers/gpu/drm/etnaviv/etnaviv_gem.c +2 -1
@@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev,
 		 * going to pin these pages.
 		 */
 		mapping = obj->filp->f_mapping;
-		mapping_set_gfp_mask(mapping, GFP_HIGHUSER);
+		mapping_set_gfp_mask(mapping, GFP_HIGHUSER |
+				     __GFP_NORETRY | __GFP_NOWARN);

_NORETRY means the mm does try hard at all to free memory. We've just done this patch in 4.12 and totally regret it, because now gpu tasks run out of memory with plenty of (gpu) memory available that could be reaped.

There's some discussions going on with Michal Hocko and Chris Wilson about possible solutions for this without so much hacks. -Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
maintainer
#4 lore raw

Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter:

Quoted 28 lines · Lucas Stach
Quoted 16 lines

GPU buffers can be quite large, so userspace is expected to deal with allocation failure. Don't trigger the OOM killer when page allocation for the GEM objects fails, as this opens an easy possiblity for unprivileged applications to DOS the system,a s the shmem pages are not fully accounted to the allocating process.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +0 -0
index f0efc5d..4c53508 100644

drivers/gpu/drm/etnaviv/etnaviv_gem.c static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, · line 666

Quoted 6 lines
drivers/gpu/drm/etnaviv/etnaviv_gem.c +2 -1
@@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev,
 		 * going to pin these pages.
 		 */
 		mapping = obj->filp->f_mapping;
-		mapping_set_gfp_mask(mapping, GFP_HIGHUSER);
+		mapping_set_gfp_mask(mapping, GFP_HIGHUSER |
+				     __GFP_NORETRY | __GFP_NOWARN);

_NORETRY means the mm does try hard at all to free memory. We've just done this patch in 4.12 and totally regret it, because now gpu tasks run out of memory with plenty of (gpu) memory available that could be reaped.

Quoted 2 lines

There's some discussions going on with Michal Hocko and Chris Wilson about possible solutions for this without so much hacks.

Thanks for the heads up. In contrast to some of the other drivers Etnaviv hasn't grown a proper shrinker yet, so I still think this patch is a step in the right direction for the time being. Currently with Etnaviv sitting on lots of GPU memory and the MM trying hard to free more memory for GPU usage a simple piglit run might pressure the OOM killer hard enough to bring the system down by killing PID 1.

Regards,
Lucas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#5 lore raw
Quoted 32 lines · Lucas Stach

Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter:

Quoted 28 lines · Lucas Stach
Quoted 16 lines

GPU buffers can be quite large, so userspace is expected to deal with allocation failure. Don't trigger the OOM killer when page allocation for the GEM objects fails, as this opens an easy possiblity for unprivileged applications to DOS the system,a s the shmem pages are not fully accounted to the allocating process.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +0 -0
index f0efc5d..4c53508 100644

drivers/gpu/drm/etnaviv/etnaviv_gem.c static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, · line 666

Quoted 6 lines
drivers/gpu/drm/etnaviv/etnaviv_gem.c +2 -1
@@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev,
 		 * going to pin these pages.
 		 */
 		mapping = obj->filp->f_mapping;
-		mapping_set_gfp_mask(mapping, GFP_HIGHUSER);
+		mapping_set_gfp_mask(mapping, GFP_HIGHUSER |
+				     __GFP_NORETRY | __GFP_NOWARN);

_NORETRY means the mm does try hard at all to free memory. We've just done this patch in 4.12 and totally regret it, because now gpu tasks run out of memory with plenty of (gpu) memory available that could be reaped.

Quoted 2 lines

There's some discussions going on with Michal Hocko and Chris Wilson about possible solutions for this without so much hacks.

Quoted 7 lines

Thanks for the heads up. In contrast to some of the other drivers Etnaviv hasn't grown a proper shrinker yet, so I still think this patch is a step in the right direction for the time being. Currently with Etnaviv sitting on lots of GPU memory and the MM trying hard to free more memory for GPU usage a simple piglit run might pressure the OOM killer hard enough to bring the system down by killing PID 1.

Ah ok, that changes the balance of course. Time to type that shrinker I'd say :-)
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#6 lore raw
Quoted 41 lines · Daniel Vetter
Quoted 32 lines · Lucas Stach

Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter:

Quoted 28 lines · Lucas Stach
Quoted 16 lines

GPU buffers can be quite large, so userspace is expected to deal with allocation failure. Don't trigger the OOM killer when page allocation for the GEM objects fails, as this opens an easy possiblity for unprivileged applications to DOS the system,a s the shmem pages are not fully accounted to the allocating process.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +0 -0
index f0efc5d..4c53508 100644

drivers/gpu/drm/etnaviv/etnaviv_gem.c static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, · line 666

Quoted 6 lines
drivers/gpu/drm/etnaviv/etnaviv_gem.c +2 -1
@@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev,
 		 * going to pin these pages.
 		 */
 		mapping = obj->filp->f_mapping;
-		mapping_set_gfp_mask(mapping, GFP_HIGHUSER);
+		mapping_set_gfp_mask(mapping, GFP_HIGHUSER |
+				     __GFP_NORETRY | __GFP_NOWARN);

_NORETRY means the mm does try hard at all to free memory. We've just done this patch in 4.12 and totally regret it, because now gpu tasks run out of memory with plenty of (gpu) memory available that could be reaped.

Quoted 2 lines

There's some discussions going on with Michal Hocko and Chris Wilson about possible solutions for this without so much hacks.

Quoted 7 lines

Thanks for the heads up. In contrast to some of the other drivers Etnaviv hasn't grown a proper shrinker yet, so I still think this patch is a step in the right direction for the time being. Currently with Etnaviv sitting on lots of GPU memory and the MM trying hard to free more memory for GPU usage a simple piglit run might pressure the OOM killer hard enough to bring the system down by killing PID 1.

Quoted 2 lines

Ah ok, that changes the balance of course. Time to type that shrinker I'd say :-)

Yeah, shrinker sounds like an absolute must. Anyway, I've posted __GFP_RETRY_MAYFAIL patchset [1] so you can use that flag.

[1] http://lkml.kernel.org/r/[email protected]

Signature · 6 lines
Michal Hocko
SUSE Labs
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
maintainer
#7 lore raw

Am Montag, den 26.06.2017, 07:52 +0200 schrieb Michal Hocko:

Quoted 56 lines · Daniel Vetter
Quoted 55 lines · Lucas Stach
Quoted 53 lines

Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter:

Quoted 38 lines · Lucas Stach
Quoted 21 lines

GPU buffers can be quite large, so userspace is expected to deal with
allocation failure. Don't trigger the OOM killer when page allocation for
the GEM objects fails, as this opens an easy possiblity for unprivileged
applications to DOS the system,a s the shmem pages are not fully accounted
to the allocating process.

---
 drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++-

 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.c +0 -0
index f0efc5d..4c53508 100644

drivers/gpu/drm/etnaviv/etnaviv_gem.c static struct drm_gem_object · line 668

Quoted 8 lines
drivers/gpu/drm/etnaviv/etnaviv_gem.c +2 -1
@@ -662,7 +662,8 @@ static struct drm_gem_object
 *__etnaviv_gem_new(struct drm_device *dev,
  		 * going to pin these pages.
  		 */
  		mapping = obj->filp->f_mapping;
-		mapping_set_gfp_mask(mapping, GFP_HIGHUSER);
+		mapping_set_gfp_mask(mapping, GFP_HIGHUSER |
+				     __GFP_NORETRY |
 __GFP_NOWARN);

_NORETRY means the mm does try hard at all to free memory. We've just done
this patch in 4.12 and totally regret it, because now gpu tasks run out of
memory with plenty of (gpu) memory available that could be reaped.

Quoted 3 lines

There's some discussions going on with Michal Hocko and Chris Wilson
about possible solutions for this without so much hacks.

Thanks for the heads up. In contrast to some of the other drivers Etnaviv hasn't grown a proper shrinker yet, so I still think this patch
is a step in the right direction for the time being. Currently with Etnaviv sitting on lots of GPU memory and the MM trying
hard to free more memory for GPU usage a simple piglit run might pressure the OOM killer hard enough to bring the system down by killing
PID 1.

Quoted 8 lines
Quoted 3 lines

Ah ok, that changes the balance of course. Time to type that shrinker I'd
say :-)

Yeah, shrinker sounds like an absolute must. Anyway, I've posted __GFP_RETRY_MAYFAIL patchset [1] so you can use that flag.

[1] http://lkml.kernel.org/r/[email protected]

Thanks, I'll switch etnaviv over to use this flag when the patchset gets applied.

A proper shrinker is on the TODO list, but there are only so much hours in a day...

Regards,
Lucas
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel