Skip to contentSkip to conversation

drm/i915: Clean up associated VMAs on context destruction

PATCH Superseded Tvrtko Ursulin 2 participants 3 messages Sep 11, 2015

Subsystems (2)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915: Clean up associated VMAs on context destruction

From: Tvrtko Ursulin <[email protected]>

Prevent leaking VMAs and PPGTT VMs when objects are imported via flink.

Scenario is that any VMAs created by the importer will be left dangling after the importer exits, or destroys the PPGTT context with which they are associated.

This is caused by object destruction not running when the importer closes the buffer object handle due the reference held by the exporter. This also leaks the VM since the VMA has a reference on it.

In practice these leaks can be observed by stopping and starting the X server on a kernel with fbcon compiled in. Every time X server exits another VMA will be leaked against the fbcon's frame buffer object.

Also on systems where flink buffer sharing is used extensively, like Android, this leak has even more serious consequences.

This version is takes a general approach from the earlier work by Rafael Barabalho (drm/i915: Clean-up PPGTT on context destruction) and tries to incorporate the subsequent discussion between Chris Wilson and Daniel Vetter.

On context destruction a VM is marked as closed and a worker thread scheduled to unbind all inactive VMAs for this VM. At the same time, active VMAs retired on this closed VM are unbound immediately.

Testcase: igt/gem_ppgtt.c
Cc: Daniel Vetter <[email protected]>
Cc: Chris Wilson <[email protected]>
Cc: Rafael Barbalho <[email protected]>
Cc: Michel Thierry <[email protected]>
---

Ok another attempt guys. Does it have any merrit as a stop gap solution until that full GEM VMA rewrite by Chris lands?

---
 drivers/gpu/drm/i915/i915_gem.c         |  8 +++--
 drivers/gpu/drm/i915/i915_gem_context.c | 56 +++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_gtt.c     |  3 ++
 drivers/gpu/drm/i915/i915_gem_gtt.h     |  5 +++
 4 files changed, 70 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +6 -2
index 41263cd4170c..1dc005758a96 100644
@@ -2372,7 +2372,7 @@ i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
 static void
 i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
 {
-	struct i915_vma *vma;
+	struct i915_vma *vma, *next;
 
 	RQ_BUG_ON(obj->last_read_req[ring] == NULL);
 	RQ_BUG_ON(!(obj->active & (1 << ring)));
@@ -2394,9 +2394,13 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
 	list_move_tail(&obj->global_list,
 		       &to_i915(obj->base.dev)->mm.bound_list);
 
-	list_for_each_entry(vma, &obj->vma_list, vma_link) {
+	list_for_each_entry_safe(vma, next, &obj->vma_list, vma_link) {
 		if (!list_empty(&vma->mm_list))
 			list_move_tail(&vma->mm_list, &vma->vm->inactive_list);
+
+		/* Unbind VMAs from contexts closed in the interim. */
+		if (vma->vm->closed)
+			WARN_ON(i915_vma_unbind(vma));
 	}
 
 	i915_gem_request_assign(&obj->last_fenced_req, NULL);
drivers/gpu/drm/i915/i915_gem_context.c +56 -0

68 lines not on this page · raw message

drivers/gpu/drm/i915/i915_gem_gtt.c +3 -0
index 87862813cfde..b8e54ce579a7 100644
@@ -3178,6 +3178,9 @@ __i915_gem_vma_create(struct drm_i915_gem_object *obj,
 	if (WARN_ON(i915_is_ggtt(vm) != !!ggtt_view))
 		return ERR_PTR(-EINVAL);
 
+	if (WARN_ON(vm->closed))
+		return ERR_PTR(-EINVAL);
+
 	vma = kmem_cache_zalloc(to_i915(obj->base.dev)->vmas, GFP_KERNEL);
 	if (vma == NULL)
 		return ERR_PTR(-ENOMEM);
drivers/gpu/drm/i915/i915_gem_gtt.h +5 -0
index 82750073d5b3..aa079ee67c16 100644
@@ -275,6 +275,9 @@ struct i915_address_space {
 	u64 start;		/* Start offset always 0 for dri2 */
 	u64 total;		/* size addr space maps (ex. 2GB for ggtt) */
 
+	/** Address space is being destroyed? */
+	bool closed;
+
 	struct i915_page_scratch *scratch_page;
 	struct i915_page_table *scratch_pt;
 	struct i915_page_directory *scratch_pd;
@@ -371,6 +374,8 @@ struct i915_hw_ppgtt {
 
 	struct drm_i915_file_private *file_priv;
 
+	struct work_struct cleanup_work;
+
 	gen6_pte_t __iomem *pd_addr;
 
 	int (*enable)(struct i915_hw_ppgtt *ppgtt);
Signature · 6 lines
2.5.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw
Quoted 27 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Prevent leaking VMAs and PPGTT VMs when objects are imported via flink.

Scenario is that any VMAs created by the importer will be left dangling after the importer exits, or destroys the PPGTT context with which they are associated.

This is caused by object destruction not running when the importer closes the buffer object handle due the reference held by the exporter. This also leaks the VM since the VMA has a reference on it.

In practice these leaks can be observed by stopping and starting the X server on a kernel with fbcon compiled in. Every time X server exits another VMA will be leaked against the fbcon's frame buffer object.

Also on systems where flink buffer sharing is used extensively, like Android, this leak has even more serious consequences.

This version is takes a general approach from the earlier work by Rafael Barabalho (drm/i915: Clean-up PPGTT on context destruction) and tries to incorporate the subsequent discussion between Chris Wilson and Daniel Vetter.

Quoted 4 lines

On context destruction a VM is marked as closed and a worker thread scheduled to unbind all inactive VMAs for this VM. At the same time, active VMAs retired on this closed VM are unbound immediately.

You don't need a worker, since you just can just drop the vma from the retirement.
http://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=nightly&id=9d4020dce054cca23bd1fea72092d036f0a3ea13

That patch is as old as the test case, just waiting for some review on earlier code.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw
Quoted 28 lines · Chris Wilson
Quoted 27 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Prevent leaking VMAs and PPGTT VMs when objects are imported via flink.

Scenario is that any VMAs created by the importer will be left dangling after the importer exits, or destroys the PPGTT context with which they are associated.

This is caused by object destruction not running when the importer closes the buffer object handle due the reference held by the exporter. This also leaks the VM since the VMA has a reference on it.

In practice these leaks can be observed by stopping and starting the X server on a kernel with fbcon compiled in. Every time X server exits another VMA will be leaked against the fbcon's frame buffer object.

Also on systems where flink buffer sharing is used extensively, like Android, this leak has even more serious consequences.

This version is takes a general approach from the earlier work by Rafael Barabalho (drm/i915: Clean-up PPGTT on context destruction) and tries to incorporate the subsequent discussion between Chris Wilson and Daniel Vetter.

Quoted 7 lines
Quoted 4 lines

On context destruction a VM is marked as closed and a worker thread scheduled to unbind all inactive VMAs for this VM. At the same time, active VMAs retired on this closed VM are unbound immediately.

You don't need a worker, since you just can just drop the vma from the retirement.

I was thinking that retirement does not necessarily happen - maybe both VMAs are already inactive at the time of context destruction. Which is then a question is it OK to wait for the next retirement on the same object to clean it up. I wasn't sure so thought it is safer to clean it up immediately since it is not a lot of code.

Quoted 4 lines

http://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=nightly&id=9d4020dce054cca23bd1fea72092d036f0a3ea13

That patch is as old as the test case, just waiting for some review on earlier code.

Plus my patch doesn't fix flink-and-close-vma-leak, but a new one I also posted, flink-and-exit-vma-leak. The latter is what was affecting Android, and can be seen with X.org and fbcon.

Your cleanup and handle close is more complete but a question is how long will "just waiting for some review on earlier code" take :), especially considering it depends on a bigger rewrite of the core.

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx