Refcounting dma_resv and using that for drm_exec support in TTM
Subsystems (11)
Hi everybody,
The idea of ref-counting dma_resv or ww_mutex came up multiple times from different people, but so far at least I have abandoned that as to complicated to implement considering how widely used that object is.
Thanks to AI I gave the task to refcount dma_resv to Claude Sonet 4 just to check how horrible it would look like.
Well turns out that this is actually a cleanup we should most likely aim for and I'm really wondering why we haven't done it like this in the first place.
Not only resolves it a bunch of issues with dma_resv instances shared by multiple GEM objects (we just recently had a bunch of patches for that on the mailing list), but also allows TTM to implement it's delayed delete handling without any zombie resurrection or similar hacks.
The patch set is totally work in progress and only survives a smoke test with amdgpu, but I still wanted to check if the idea is valuable and should be looked in further.
Please comment and/or tear apart :)
Cheers,
Christian.
PATCH 1/12 dma-buf: Add reference counting to dma_resv
Introduce reference counting for dma_resv objects to better manage their lifecycle. This replaces the previous approach where dma_buf would either embed a dma_resv or allocate it inline using a size trick.
Add three new functions:
- dma_resv_alloc(): allocates and initializes a dma_resv with refcount
- dma_resv_get(): acquires a reference to a dma_resv
- dma_resv_put(): releases a reference, freeing when count reaches zero
Update all callers to use dma_resv_put() instead of dma_resv_fini(), which now becomes an internal cleanup function. The dma_buf export path now explicitly allocates the dma_resv when needed rather than using pointer arithmetic tricks.
This provides clearer ownership semantics and makes the code more maintainable by removing the embedded allocation hack.
--- drivers/dma-buf/dma-buf.c | 24 +++---- drivers/dma-buf/dma-resv.c | 80 +++++++++++++++++++--- drivers/dma-buf/st-dma-resv.c | 10 +-- drivers/gpu/drm/drm_gem.c | 2 +- drivers/gpu/drm/drm_mode_config.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_object.c | 2 +- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 +- drivers/gpu/drm/i915/gt/intel_gtt.c | 2 +- drivers/gpu/drm/nouveau/nouveau_bo.c | 2 +- drivers/gpu/drm/ttm/tests/ttm_bo_test.c | 2 +- drivers/gpu/drm/ttm/ttm_bo_util.c | 2 +- include/linux/dma-resv.h | 22 +++++- 12 files changed, 117 insertions(+), 37 deletions(-)
drivers/dma-buf/dma-buf.c +11 -13
@@ -182,8 +182,7 @@ static void dma_buf_release(struct dentry *dentry) dmabuf->ops->release (dmabuf); -if (dmabuf->resv == (struct dma_resv *)&dmabuf[1 ]) -dma_resv_fini (dmabuf->resv); +dma_resv_put (dmabuf->resv);WARN_ON (!list_empty (&dmabuf->attachments));module_put (dmabuf->owner);
@@ -707,10 +706,9 @@ static struct file *dma_buf_getfile(size_t size, int flags)*/ struct dma_buf *dma_buf_export (const struct dma_buf_export_info *exp_info) { -struct dma_buf *dmabuf;struct dma_resv *resv = exp_info->resv; +struct dma_buf *dmabuf;struct file *file; -size_t alloc_size =sizeof (struct dma_buf );int ret; if (WARN_ON(!exp_info->priv || !exp_info->ops
@@ -731,12 +729,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) goto err_module; } -if (!exp_info->resv) - alloc_size +=sizeof (struct dma_resv ); -else -/* prevent &dma_buf[1] == dma_buf->resv */ - alloc_size +=1 ; - dmabuf =kzalloc (alloc_size,GFP_KERNEL ); + dmabuf =kzalloc_obj (*dmabuf);if (!dmabuf) { ret = -ENOMEM ; goto err_file;
@@ -754,10 +747,13 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)INIT_LIST_HEAD (&dmabuf->attachments);if (!resv) { - dmabuf->resv = (struct dma_resv *)&dmabuf[1 ]; -dma_resv_init (dmabuf->resv); + dmabuf->resv =dma_resv_alloc (); +if (!dmabuf->resv) { + ret = -ENOMEM ; + goto err_dmabuf; + } }else { - dmabuf->resv = resv; + dmabuf->resv =dma_resv_get (resv); } file->private_data = dmabuf;
@@ -770,6 +766,8 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)return dmabuf; +err_dmabuf: +kfree (dmabuf); err_file:fput (file); err_module:
drivers/dma-buf/dma-resv.c +71 -9
103 lines not on this page · raw message
drivers/dma-buf/st-dma-resv.c +5 -5
@@ -59,7 +59,7 @@ static void test_sanitycheck(struct kunit *test)KUNIT_FAIL (test,"Resv locking failed\n" ); else dma_resv_unlock(&resv); -dma_resv_fini (&resv); +dma_resv_put (&resv); }static void test_signaling (struct kunit *test)
@@ -101,7 +101,7 @@ static void test_signaling(struct kunit *test) err_unlock:dma_resv_unlock (&resv); err_free: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }
@@ -160,7 +160,7 @@ static void test_for_each(struct kunit *test) err_unlock:dma_resv_unlock (&resv); err_free: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }
@@ -231,7 +231,7 @@ static void test_for_each_unlocked(struct kunit *test)dma_resv_iter_end (&cursor);dma_fence_signal (f); err_free: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }
@@ -282,7 +282,7 @@ static void test_get_fences(struct kunit *test)dma_fence_put (fences[i]);kfree (fences); err_resv: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }
drivers/gpu/drm/drm_gem.c +1 -1
@@ -249,7 +249,7 @@ void drm_gem_private_object_fini(struct drm_gem_object *obj) {WARN_ON (obj->dma_buf); -dma_resv_fini (&obj->_resv); +dma_resv_put (&obj->_resv);mutex_destroy (&obj->gpuva.lock); }EXPORT_SYMBOL (drm_gem_private_object_fini);
drivers/gpu/drm/drm_mode_config.c +1 -1
@@ -681,7 +681,7 @@ int drmm_mode_config_init(struct drm_device *dev)drm_modeset_drop_locks (&modeset_ctx);drm_modeset_acquire_fini (&modeset_ctx); -dma_resv_fini (&resv); +dma_resv_put (&resv); } return drmm_add_action_or_reset(dev, drm_mode_config_init_release,
drivers/gpu/drm/i915/gem/i915_gem_object.c +1 -1
@@ -144,7 +144,7 @@ void __i915_gem_object_fini(struct drm_i915_gem_object *obj) {mutex_destroy (&obj->mm.get_page.lock);mutex_destroy (&obj->mm.get_dma_page.lock); -dma_resv_fini (&obj->base._resv); +dma_resv_put (&obj->base._resv); }/**
drivers/gpu/drm/i915/gt/intel_ggtt.c +2 -2
@@ -1140,7 +1140,7 @@ void i915_ggtt_driver_late_release(struct drm_i915_private *i915)struct i915_ggtt *ggtt =to_gt (i915)->ggtt;GEM_WARN_ON (kref_read (&ggtt->vm.resv_ref) !=1 ); -dma_resv_fini (&ggtt->vm._resv); +dma_resv_put (&ggtt->vm._resv); }static unsigned int gen6_get_total_gtt_size (u16 snb_gmch_ctl)
@@ -1524,7 +1524,7 @@ static int ggtt_probe_hw(struct i915_ggtt *ggtt, struct intel_gt *gt) ret =intel_ggtt_gmch_probe (ggtt);if (ret) { -dma_resv_fini (&ggtt->vm._resv); +dma_resv_put (&ggtt->vm._resv);return ret; }
drivers/gpu/drm/i915/gt/intel_gtt.c +1 -1
@@ -225,7 +225,7 @@ void i915_vm_resv_release(struct kref *kref)struct i915_address_space *vm =container_of (kref,typeof (*vm), resv_ref); -dma_resv_fini (&vm->_resv); +dma_resv_put (&vm->_resv);mutex_destroy (&vm->mutex);kfree (vm);
drivers/gpu/drm/nouveau/nouveau_bo.c +1 -1
@@ -160,7 +160,7 @@ nouveau_bo_del_ttm(struct ttm_buffer_object *bo)drm_gem_object_release (&bo->base); }else { -dma_resv_fini (&bo->base._resv); +dma_resv_put (&bo->base._resv); }kfree (nvbo);
drivers/gpu/drm/ttm/tests/ttm_bo_test.c +1 -1
@@ -376,7 +376,7 @@ static void ttm_bo_unreserve_bulk(struct kunit *test)ttm_resource_free (bo1, &res1);ttm_resource_free (bo2, &res2); -dma_resv_fini (resv); +dma_resv_put (resv); }static void ttm_bo_fini_basic (struct kunit *test)
drivers/gpu/drm/ttm/ttm_bo_util.c +1 -1
@@ -207,7 +207,7 @@ static void ttm_transfered_destroy(struct ttm_buffer_object *bo)struct ttm_transfer_obj *fbo; fbo = container_of(bo, struct ttm_transfer_obj, base); -dma_resv_fini (&fbo->base.base._resv); +dma_resv_put (&fbo->base.base._resv);ttm_bo_put (fbo->bo);kfree (fbo); }
include/linux/dma-resv.h +21 -1
@@ -44,6 +44,7 @@#include <linux/slab.h> #include <linux/seqlock.h> #include <linux/rcupdate.h> +#include <linux/kref.h> extern struct ww_class reservation_ww_class;
@@ -153,6 +154,23 @@ static inline enum dma_resv_usage dma_resv_usage_rw(bool write)* drm_gem_object with the same scheme. */ struct dma_resv { +/** +* @refcount: +* +* Reference count for this reservation object. The object is freed +* when the reference count reaches zero via dma_resv_put(). +*/ +struct kref refcount; + +/** +* @allocated: +* +* True if this object was allocated by dma_resv_alloc(), false if +* embedded in another structure. Used to determine whether to free +* the object memory in the release function. +*/ +bool allocated; +/** * @lock: *
@@ -465,7 +483,9 @@ static inline void dma_resv_unlock(struct dma_resv *obj) }void dma_resv_init (struct dma_resv *obj); -void dma_resv_fini (struct dma_resv *obj); +struct dma_resv *dma_resv_alloc (void ); +struct dma_resv *dma_resv_get (struct dma_resv *obj); +void dma_resv_put (struct dma_resv *obj);int dma_resv_reserve_fences (struct dma_resv *obj,unsigned int num_fences);void dma_resv_add_fence (struct dma_resv *obj,struct dma_fence *fence,enum dma_resv_usage usage);
Signature · 1 line
2.43.0
Re: [PATCH 01/12] dma-buf: Add reference counting to dma_resv
Quoted 9 lines · Christian König
Introduce reference counting for
dma_resvobjects to better manage their lifecycle. This replaces the previous approach wheredma_bufwould either embed adma_resvor allocate it inline using a size trick.Add three new functions:
-dma_resv_alloc(): allocates and initializes adma_resvwith refcount
-dma_resv_get(): acquires a reference to adma_resv
-dma_resv_put(): releases a reference, freeing when count reaches zero
Quoted 8 lines
Update all callers to use
dma_resv_put()instead ofdma_resv_fini(), which now becomes an internal cleanup function. Thedma_bufexport path now explicitly allocates thedma_resvwhen needed rather than using pointer arithmetic tricks.This provides clearer ownership semantics and makes the code more maintainable by removing the embedded allocation hack.
This looks a lot better to me. In particular, I agree with the last sentence in the commit message.
I'm confident enough in this to say:
However, given that this is a community-wide change, I'd wait for at least one other person to give it their blessing as well.
Matt
Quoted 404 lines
Signed-off-by Christian König <[email protected]>Assisted-by Claude:Sonnet 4--- drivers/dma-buf/dma-buf.c | 24 +++---- drivers/dma-buf/dma-resv.c | 80 +++++++++++++++++++--- drivers/dma-buf/st-dma-resv.c | 10 +-- drivers/gpu/drm/drm_gem.c | 2 +- drivers/gpu/drm/drm_mode_config.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_object.c | 2 +- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 +- drivers/gpu/drm/i915/gt/intel_gtt.c | 2 +- drivers/gpu/drm/nouveau/nouveau_bo.c | 2 +- drivers/gpu/drm/ttm/tests/ttm_bo_test.c | 2 +- drivers/gpu/drm/ttm/ttm_bo_util.c | 2 +- include/linux/dma-resv.h | 22 +++++- 12 files changed, 117 insertions(+), 37 deletions(-)drivers/dma-buf/dma-buf.c +11
-13@@ -182,8 +182,7 @@ static void dma_buf_release(struct dentry *dentry) dmabuf->ops->release (dmabuf); -if (dmabuf->resv == (struct dma_resv *)&dmabuf[1 ]) -dma_resv_fini (dmabuf->resv); +dma_resv_put (dmabuf->resv);WARN_ON (!list_empty (&dmabuf->attachments));module_put (dmabuf->owner);@@ -707,10 +706,9 @@ static struct file *dma_buf_getfile(size_t size, int flags)*/ struct dma_buf *dma_buf_export (const struct dma_buf_export_info *exp_info) { -struct dma_buf *dmabuf;struct dma_resv *resv = exp_info->resv; +struct dma_buf *dmabuf;struct file *file; -size_t alloc_size =sizeof (struct dma_buf );int ret; if (WARN_ON(!exp_info->priv || !exp_info->ops@@ -731,12 +729,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) goto err_module; } -if (!exp_info->resv) - alloc_size +=sizeof (struct dma_resv ); -else -/* prevent &dma_buf[1] == dma_buf->resv */ - alloc_size +=1 ; - dmabuf =kzalloc (alloc_size,GFP_KERNEL ); + dmabuf =kzalloc_obj (*dmabuf);if (!dmabuf) { ret = -ENOMEM ; goto err_file;@@ -754,10 +747,13 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)INIT_LIST_HEAD (&dmabuf->attachments);if (!resv) { - dmabuf->resv = (struct dma_resv *)&dmabuf[1 ]; -dma_resv_init (dmabuf->resv); + dmabuf->resv =dma_resv_alloc (); +if (!dmabuf->resv) { + ret = -ENOMEM ; + goto err_dmabuf; + } }else { - dmabuf->resv = resv; + dmabuf->resv =dma_resv_get (resv); } file->private_data = dmabuf;@@ -770,6 +766,8 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)return dmabuf; +err_dmabuf: +kfree (dmabuf); err_file:fput (file); err_module:drivers/dma-buf/dma-resv.c +71
-9@@ -36,6 +36,7 @@#include <linux/dma-resv.h> #include <linux/dma-fence-array.h> #include <linux/export.h> +#include <linux/kref.h> #include <linux/mm.h> #include <linux/sched/mm.h> #include <linux/mmu_notifier.h> @@ -137,26 +138,87 @@ static void dma_resv_list_free(struct dma_resv_list *list)*/ void dma_resv_init (struct dma_resv *obj) { +kref_init (&obj->refcount); + obj->allocated = false;ww_mutex_init (&obj->lock, &reservation_ww_class);RCU_INIT_POINTER (obj->fences,NULL ); }EXPORT_SYMBOL (dma_resv_init); -/** -* dma_resv_fini - destroys a reservation object -* @obj: the reservation object +/* +* dma_resv_release - release function for kref +* @kref: the kref inside the dma_resv object +* +* This is called when the last reference to a dma_resv object is released. +* Cleans up the object and frees it if it was allocated by dma_resv_alloc(). */ -void dma_resv_fini (struct dma_resv *obj) +static void dma_resv_release (struct kref *kref) { -/* -* This object should be dead and all references must have -* been released to it, so no need to be protected with rcu. -*/ + struct dma_resv *obj = container_of(kref, struct dma_resv, refcount); +dma_resv_list_free (rcu_dereference_protected (obj->fences, true));ww_mutex_destroy (&obj->lock); +if (obj->allocated) +kfree (obj); +} + +/** +* dma_resv_alloc - allocate and initialize a reservation object +* +* Allocates a new dma_resv object, initializes it, and returns it with a +* reference count of 1. The object must be freed with dma_resv_put() when +* no longer needed. +* +* Returns: +* A pointer to the allocated dma_resv object, or NULL on allocation failure. +*/ +struct dma_resv *dma_resv_alloc (void ) +{ +struct dma_resv *obj; + + obj =kzalloc_obj (*obj); +if (!obj) +return NULL ; + +dma_resv_init (obj); + obj->allocated = true; + +return obj; +} +EXPORT_SYMBOL (dma_resv_alloc); + +/** +* dma_resv_get - acquire a reference to a reservation object +* @obj: the reservation object +* +* Increments the reference count on the dma_resv object. +* +* Returns: +* The dma_resv object pointer for convenience. +*/ +struct dma_resv *dma_resv_get (struct dma_resv *obj) +{ +if (obj) +kref_get (&obj->refcount); +return obj; +} +EXPORT_SYMBOL (dma_resv_get); + +/** +* dma_resv_put - release a reference to a reservation object +* @obj: the reservation object +* +* Decrements the reference count on the dma_resv object. When the reference +* count reaches zero, the object is cleaned up with dma_resv_fini() and freed +* if it was allocated by dma_resv_alloc(). +*/ +void dma_resv_put (struct dma_resv *obj) +{ +if (obj) +kref_put (&obj->refcount, dma_resv_release); } -EXPORT_SYMBOL (dma_resv_fini); +EXPORT_SYMBOL (dma_resv_put);/* Dereference the fences while ensuring RCU rules */ static inline struct dma_resv_list *dma_resv_fences_list (struct dma_resv *obj)drivers/dma-buf/st-dma-resv.c +5
-5@@ -59,7 +59,7 @@ static void test_sanitycheck(struct kunit *test)KUNIT_FAIL (test,"Resv locking failed\n" ); else dma_resv_unlock(&resv); -dma_resv_fini (&resv); +dma_resv_put (&resv); }static void test_signaling (struct kunit *test)@@ -101,7 +101,7 @@ static void test_signaling(struct kunit *test) err_unlock:dma_resv_unlock (&resv); err_free: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }@@ -160,7 +160,7 @@ static void test_for_each(struct kunit *test) err_unlock:dma_resv_unlock (&resv); err_free: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }@@ -231,7 +231,7 @@ static void test_for_each_unlocked(struct kunit *test)dma_resv_iter_end (&cursor);dma_fence_signal (f); err_free: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }@@ -282,7 +282,7 @@ static void test_get_fences(struct kunit *test)dma_fence_put (fences[i]);kfree (fences); err_resv: -dma_resv_fini (&resv); +dma_resv_put (&resv);dma_fence_put (f); }drivers/gpu/drm/drm_gem.c +1
-1@@ -249,7 +249,7 @@ void drm_gem_private_object_fini(struct drm_gem_object *obj) {WARN_ON (obj->dma_buf); -dma_resv_fini (&obj->_resv); +dma_resv_put (&obj->_resv);mutex_destroy (&obj->gpuva.lock); }EXPORT_SYMBOL (drm_gem_private_object_fini);drivers/gpu/drm/drm_mode_config.c +1
-1@@ -681,7 +681,7 @@ int drmm_mode_config_init(struct drm_device *dev)drm_modeset_drop_locks (&modeset_ctx);drm_modeset_acquire_fini (&modeset_ctx); -dma_resv_fini (&resv); +dma_resv_put (&resv); } return drmm_add_action_or_reset(dev, drm_mode_config_init_release,drivers/gpu/drm/i915/gem/i915_gem_object.c +1
-1@@ -144,7 +144,7 @@ void __i915_gem_object_fini(struct drm_i915_gem_object *obj) {mutex_destroy (&obj->mm.get_page.lock);mutex_destroy (&obj->mm.get_dma_page.lock); -dma_resv_fini (&obj->base._resv); +dma_resv_put (&obj->base._resv); }/** drivers/gpu/drm/i915/gt/intel_ggtt.c +2
-2@@ -1140,7 +1140,7 @@ void i915_ggtt_driver_late_release(struct drm_i915_private *i915)struct i915_ggtt *ggtt =to_gt (i915)->ggtt;GEM_WARN_ON (kref_read (&ggtt->vm.resv_ref) !=1 ); -dma_resv_fini (&ggtt->vm._resv); +dma_resv_put (&ggtt->vm._resv); }static unsigned int gen6_get_total_gtt_size (u16 snb_gmch_ctl)@@ -1524,7 +1524,7 @@ static int ggtt_probe_hw(struct i915_ggtt *ggtt, struct intel_gt *gt) ret =intel_ggtt_gmch_probe (ggtt);if (ret) { -dma_resv_fini (&ggtt->vm._resv); +dma_resv_put (&ggtt->vm._resv);return ret; }drivers/gpu/drm/i915/gt/intel_gtt.c +1
-1@@ -225,7 +225,7 @@ void i915_vm_resv_release(struct kref *kref)struct i915_address_space *vm =container_of (kref,typeof (*vm), resv_ref); -dma_resv_fini (&vm->_resv); +dma_resv_put (&vm->_resv);mutex_destroy (&vm->mutex);kfree (vm);drivers/gpu/drm/nouveau/nouveau_bo.c +1
-1@@ -160,7 +160,7 @@ nouveau_bo_del_ttm(struct ttm_buffer_object *bo)drm_gem_object_release (&bo->base); }else { -dma_resv_fini (&bo->base._resv); +dma_resv_put (&bo->base._resv); }kfree (nvbo);drivers/gpu/drm/ttm/tests/ttm_bo_test.c +1
-1@@ -376,7 +376,7 @@ static void ttm_bo_unreserve_bulk(struct kunit *test)ttm_resource_free (bo1, &res1);ttm_resource_free (bo2, &res2); -dma_resv_fini (resv); +dma_resv_put (resv); }static void ttm_bo_fini_basic (struct kunit *test)drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1@@ -207,7 +207,7 @@ static void ttm_transfered_destroy(struct ttm_buffer_object *bo)struct ttm_transfer_obj *fbo; fbo = container_of(bo, struct ttm_transfer_obj, base); -dma_resv_fini (&fbo->base.base._resv); +dma_resv_put (&fbo->base.base._resv);ttm_bo_put (fbo->bo);kfree (fbo); }include/linux/dma-resv.h +21
-1@@ -44,6 +44,7 @@#include <linux/slab.h> #include <linux/seqlock.h> #include <linux/rcupdate.h> +#include <linux/kref.h> extern struct ww_class reservation_ww_class;@@ -153,6 +154,23 @@ static inline enum dma_resv_usage dma_resv_usage_rw(bool write)* drm_gem_object with the same scheme. */ struct dma_resv { +/** +* @refcount: +* +* Reference count for this reservation object. The object is freed +* when the reference count reaches zero via dma_resv_put(). +*/ +struct kref refcount; + +/** +* @allocated: +* +* True if this object was allocated by dma_resv_alloc(), false if +* embedded in another structure. Used to determine whether to free +* the object memory in the release function. +*/ +bool allocated; +/** * @lock: * @@ -465,7 +483,9 @@ static inline void dma_resv_unlock(struct dma_resv *obj) }void dma_resv_init (struct dma_resv *obj); -void dma_resv_fini (struct dma_resv *obj); +struct dma_resv *dma_resv_alloc (void ); +struct dma_resv *dma_resv_get (struct dma_resv *obj); +void dma_resv_put (struct dma_resv *obj);int dma_resv_reserve_fences (struct dma_resv *obj,unsigned int num_fences);void dma_resv_add_fence (struct dma_resv *obj,struct dma_fence *fence,enum dma_resv_usage usage);Signature · 1 line
2.43.0
Danilo Krummrich
maintainerRe: [PATCH 01/12] dma-buf: Add reference counting to dma_resv
Quoted 7 lines · Matthew Brost
Quoted 3 lines · Christian König
This provides clearer ownership semantics and makes the code more maintainable by removing the embedded allocation hack.
This looks a lot better to me. In particular, I agree with the last sentence in the commit message.
I have to disagree with this, it is the opposite. As long as the struct dma_resv::allocated fields and the corresponding semantics exists, this does result into less clear ownership semantics.
When the dma_resv is embedded in another object the reference count becomes meaningless. If the object embedding the dma_resv is freed it doesn't matter whether I have a reference count, it would a UAF regardless.
It is misleading (and hence error prone) to have an API where one can obtain a reference count of an object where the underlying memory can be freed regardless of the obtained reference count.
A refernece count represents a shared ownership model, which is undermined if the underlying memory is not owned by the reference count.
That said, I don't mind the reference count, but we can't mix up exclusive ownership (embedding a structure) and shared ownership (reference count).
Quoted 13 lines
Quoted 13 lines
drivers/dma-buf/dma-resv.c +6
-4+static void dma_resv_release (struct kref *kref) { -/* -* This object should be dead and all references must have -* been released to it, so no need to be protected with rcu. -*/ + struct dma_resv *obj = container_of(kref, struct dma_resv, refcount); +dma_resv_list_free (rcu_dereference_protected (obj->fences, true));ww_mutex_destroy (&obj->lock); +if (obj->allocated) +kfree (obj); +}
Matthew Brost
maintainerRe: [PATCH 01/12] dma-buf: Add reference counting to dma_resv
Quoted 10 lines · Danilo Krummrich
Quoted 7 lines · Matthew Brost
Quoted 3 lines · Christian König
This provides clearer ownership semantics and makes the code more maintainable by removing the embedded allocation hack.
This looks a lot better to me. In particular, I agree with the last sentence in the commit message.
I have to disagree with this, it is the opposite. As long as the struct
Daniilo is of course correct here, completely missed this when I looked.
Quoted 7 lines
dma_resv::allocatedfields and the corresponding semantics exists, this does result into less clear ownership semantics.When the
dma_resvis embedded in another object the reference count becomes meaningless. If the object embedding thedma_resvis freed it doesn't matter whether I have a reference count, it would a UAF regardless.
Yes, I agree. The allocated field would need to be dropped to make this viable, and we would disallow embedding a dma-resv object into other objects (which I believe is the suggestion).
This doesn't look too painful, as I can only find two instances of embedding in the kernel: drm_gem_object and i915_address_space and handful of stack variables.
Matt
Quoted 23 lines
It is misleading (and hence error prone) to have an API where one can obtain a reference count of an object where the underlying memory can be freed regardless of the obtained reference count.
A refernece count represents a shared ownership model, which is undermined if the underlying memory is not owned by the reference count.
That said, I don't mind the reference count, but we can't mix up exclusive ownership (embedding a structure) and shared ownership (reference count).
Quoted 13 lines
Quoted 13 lines
drivers/dma-buf/dma-resv.c +6
-4+static void dma_resv_release (struct kref *kref) { -/* -* This object should be dead and all references must have -* been released to it, so no need to be protected with rcu. -*/ + struct dma_resv *obj = container_of(kref, struct dma_resv, refcount); +dma_resv_list_free (rcu_dereference_protected (obj->fences, true));ww_mutex_destroy (&obj->lock); +if (obj->allocated) +kfree (obj); +}
Christian König
maintainerRe: [PATCH 01/12] dma-buf: Add reference counting to dma_resv
Quoted 17 lines · Matthew Brost
Quoted 10 lines · Danilo Krummrich
Quoted 7 lines · Matthew Brost
Quoted 3 lines · Christian König
This provides clearer ownership semantics and makes the code more maintainable by removing the embedded allocation hack.
This looks a lot better to me. In particular, I agree with the last sentence in the commit message.
I have to disagree with this, it is the opposite. As long as the struct
Daniilo is of course correct here, completely missed this when I looked.
Quoted 3 lines
dma_resv::allocatedfields and the corresponding semantics exists, this does result into less clear ownership semantics.
Quoted 8 lines
Quoted 4 lines
When the
dma_resvis embedded in another object the reference count becomes meaningless. If the object embedding thedma_resvis freed it doesn't matter whether I have a reference count, it would a UAF regardless.Yes, I agree. The allocated field would need to be dropped to make this viable, and we would disallow embedding a dma-resv object into other objects (which I believe is the suggestion).
Yeah completely agree as well. This was basically just the first hacky version.
Quoted 3 lines
This doesn't look too painful, as I can only find two instances of embedding in the kernel:
drm_gem_objectandi915_address_spaceand handful of stack variables.
The use case in the TTM BO deletion path is the only really ugly one as far as I can see.
It uses the drm_gem_object embedded reservation object to make sure memory allocation can't fail during deletion.
We need something like a dummy delete_resv allocated for each TTM BO during creation or something like that to avoid this.
But that in turn means potentially means taking a look at all TTM using drivers if/when they use this in their deletion path.
Doable but a bit more work. Probably also a good job for AI.
Question is also who is taking that work? @Natalie can you pick up from here?
Regards,
Christian.
Quoted 26 lines
Matt
Quoted 23 lines
It is misleading (and hence error prone) to have an API where one can obtain a reference count of an object where the underlying memory can be freed regardless of the obtained reference count.
A refernece count represents a shared ownership model, which is undermined if the underlying memory is not owned by the reference count.
That said, I don't mind the reference count, but we can't mix up exclusive ownership (embedding a structure) and shared ownership (reference count).
Quoted 13 lines
Quoted 13 lines
drivers/dma-buf/dma-resv.c +6
-4+static void dma_resv_release (struct kref *kref) { -/* -* This object should be dead and all references must have -* been released to it, so no need to be protected with rcu. -*/ + struct dma_resv *obj = container_of(kref, struct dma_resv, refcount); +dma_resv_list_free (rcu_dereference_protected (obj->fences, true));ww_mutex_destroy (&obj->lock); +if (obj->allocated) +kfree (obj); +}
Matthew Brost
maintainerRe: [PATCH 01/12] dma-buf: Add reference counting to dma_resv
Quoted 29 lines · Christian König
Quoted 17 lines · Matthew Brost
Quoted 10 lines · Danilo Krummrich
Quoted 7 lines · Matthew Brost
Quoted 3 lines · Christian König
This provides clearer ownership semantics and makes the code more maintainable by removing the embedded allocation hack.
This looks a lot better to me. In particular, I agree with the last sentence in the commit message.
I have to disagree with this, it is the opposite. As long as the struct
Daniilo is of course correct here, completely missed this when I looked.
Quoted 3 lines
dma_resv::allocatedfields and the corresponding semantics exists, this does result into less clear ownership semantics.Quoted 8 lines
Quoted 4 lines
When the
dma_resvis embedded in another object the reference count becomes meaningless. If the object embedding thedma_resvis freed it doesn't matter whether I have a reference count, it would a UAF regardless.Yes, I agree. The allocated field would need to be dropped to make this viable, and we would disallow embedding a dma-resv object into other objects (which I believe is the suggestion).
Yeah completely agree as well. This was basically just the first hacky version.
Quoted 8 lines
Quoted 3 lines
This doesn't look too painful, as I can only find two instances of embedding in the kernel:
drm_gem_objectandi915_address_spaceand handful of stack variables.The use case in the TTM BO deletion path is the only really ugly one as far as I can see.
It uses the
drm_gem_objectembedded reservation object to make sure memory allocation can't fail during deletion.
Can't you just preallocate, during gem initialization, what is currently the embedded reservation as a separate pointer, drop the reference to the shared reservation, copy the fences, and then perform a pointer swap during individualization?
Maybe I'm missing some details here, but when I briefly thought about the individualization, it seemed fairly straightforward.
Matt
Quoted 38 lines
We need something like a dummy
delete_resvallocated for each TTM BO during creation or something like that to avoid this.But that in turn means potentially means taking a look at all TTM using drivers if/when they use this in their deletion path.
Doable but a bit more work. Probably also a good job for AI.
Question is also who is taking that work? @Natalie can you pick up from here?
Regards,
Christian.Quoted 26 lines
Matt
Quoted 23 lines
It is misleading (and hence error prone) to have an API where one can obtain a reference count of an object where the underlying memory can be freed regardless of the obtained reference count.
A refernece count represents a shared ownership model, which is undermined if the underlying memory is not owned by the reference count.
That said, I don't mind the reference count, but we can't mix up exclusive ownership (embedding a structure) and shared ownership (reference count).
Quoted 13 lines
Quoted 13 lines
drivers/dma-buf/dma-resv.c +6
-4+static void dma_resv_release (struct kref *kref) { -/* -* This object should be dead and all references must have -* been released to it, so no need to be protected with rcu. -*/ + struct dma_resv *obj = container_of(kref, struct dma_resv, refcount); +dma_resv_list_free (rcu_dereference_protected (obj->fences, true));ww_mutex_destroy (&obj->lock); +if (obj->allocated) +kfree (obj); +}
PATCH 2/12 dma-buf/tests: Convert st-dma-resv tests to use dma_resv_alloc
In reply to Christian KönigUpdate all test cases in st-dma-resv.c to use the new dma_resv_alloc() API instead of stack-allocating dma_resv objects. This demonstrates the proper usage of the new allocation and reference counting interface.
All five test functions now:
- Use dma_resv_alloc() to allocate objects
- Check for allocation failure with KUNIT_ASSERT_NOT_NULL
- Clean up with dma_resv_put() which handles freeing
This change also provides better test coverage for the reference counting implementation.
--- drivers/dma-buf/st-dma-resv.c | 90 +++++++++++++++++++---------------- 1 file changed, 50 insertions(+), 40 deletions(-)
drivers/dma-buf/st-dma-resv.c +50 -40
@@ -41,7 +41,7 @@ static struct dma_fence *alloc_fence(void)static void test_sanitycheck (struct kunit *test) { -struct dma_resv resv; +struct dma_resv *resv;struct dma_fence *f;int r;
@@ -53,20 +53,22 @@ static void test_sanitycheck(struct kunit *test)dma_fence_signal (f);dma_fence_put (f); -dma_resv_init (&resv); - r =dma_resv_lock (&resv,NULL ); + resv =dma_resv_alloc (); +KUNIT_ASSERT_NOT_NULL (test, resv); + + r =dma_resv_lock (resv,NULL );if (r)KUNIT_FAIL (test,"Resv locking failed\n" );else -dma_resv_unlock (&resv); -dma_resv_put (&resv); +dma_resv_unlock (resv); +dma_resv_put (resv); }static void test_signaling (struct kunit *test) {const struct dma_resv_usage_param *param = test->param_value;enum dma_resv_usage usage = param->usage; -struct dma_resv resv; +struct dma_resv *resv;struct dma_fence *f;int r;
@@ -75,33 +77,35 @@ static void test_signaling(struct kunit *test)dma_fence_enable_signaling (f); -dma_resv_init (&resv); - r =dma_resv_lock (&resv,NULL ); + resv =dma_resv_alloc (); +KUNIT_ASSERT_NOT_NULL (test, resv); + + r =dma_resv_lock (resv,NULL );if (r) {KUNIT_FAIL (test,"Resv locking failed" ); goto err_free; } - r =dma_resv_reserve_fences (&resv,1 ); + r =dma_resv_reserve_fences (resv,1 );if (r) {KUNIT_FAIL (test,"Resv shared slot allocation failed" ); goto err_unlock; } -dma_resv_add_fence (&resv, f, usage); -if (dma_resv_test_signaled (&resv, usage)) { +dma_resv_add_fence (resv, f, usage); +if (dma_resv_test_signaled (resv, usage)) {KUNIT_FAIL (test,"Resv unexpectedly signaled" ); goto err_unlock; }dma_fence_signal (f); -if (!dma_resv_test_signaled (&resv, usage)) { +if (!dma_resv_test_signaled (resv, usage)) {KUNIT_FAIL (test,"Resv not reporting signaled" ); goto err_unlock; } err_unlock: -dma_resv_unlock (&resv); +dma_resv_unlock (resv); err_free: -dma_resv_put (&resv); +dma_resv_put (resv);dma_fence_put (f); }
@@ -111,7 +115,7 @@ static void test_for_each(struct kunit *test)enum dma_resv_usage usage = param->usage;struct dma_resv_iter cursor;struct dma_fence *f, *fence; -struct dma_resv resv; +struct dma_resv *resv;int r; f =alloc_fence ();
@@ -119,23 +123,25 @@ static void test_for_each(struct kunit *test)dma_fence_enable_signaling (f); -dma_resv_init (&resv); - r =dma_resv_lock (&resv,NULL ); + resv =dma_resv_alloc (); +KUNIT_ASSERT_NOT_NULL (test, resv); + + r =dma_resv_lock (resv,NULL );if (r) {KUNIT_FAIL (test,"Resv locking failed" ); goto err_free; } - r =dma_resv_reserve_fences (&resv,1 ); + r =dma_resv_reserve_fences (resv,1 );if (r) {KUNIT_FAIL (test,"Resv shared slot allocation failed" ); goto err_unlock; } -dma_resv_add_fence (&resv, f, usage); +dma_resv_add_fence (resv, f, usage); r = -ENOENT ; -dma_resv_for_each_fence (&cursor, &resv, usage, fence) { +dma_resv_for_each_fence (&cursor, resv, usage, fence) {if (!r) {KUNIT_FAIL (test,"More than one fence found" ); goto err_unlock;
@@ -158,9 +164,9 @@ static void test_for_each(struct kunit *test) }dma_fence_signal (f); err_unlock: -dma_resv_unlock (&resv); +dma_resv_unlock (resv); err_free: -dma_resv_put (&resv); +dma_resv_put (resv);dma_fence_put (f); }
@@ -170,7 +176,7 @@ static void test_for_each_unlocked(struct kunit *test)enum dma_resv_usage usage = param->usage;struct dma_resv_iter cursor;struct dma_fence *f, *fence; -struct dma_resv resv; +struct dma_resv *resv;int r; f =alloc_fence ();
@@ -178,25 +184,27 @@ static void test_for_each_unlocked(struct kunit *test)dma_fence_enable_signaling (f); -dma_resv_init (&resv); - r =dma_resv_lock (&resv,NULL ); + resv =dma_resv_alloc (); +KUNIT_ASSERT_NOT_NULL (test, resv); + + r =dma_resv_lock (resv,NULL );if (r) {KUNIT_FAIL (test,"Resv locking failed" ); goto err_free; } - r =dma_resv_reserve_fences (&resv,1 ); + r =dma_resv_reserve_fences (resv,1 );if (r) {KUNIT_FAIL (test,"Resv shared slot allocation failed" ); -dma_resv_unlock (&resv); +dma_resv_unlock (resv); goto err_free; } -dma_resv_add_fence (&resv, f, usage); -dma_resv_unlock (&resv); +dma_resv_add_fence (resv, f, usage); +dma_resv_unlock (resv); r = -ENOENT ; -dma_resv_iter_begin (&cursor, &resv, usage); +dma_resv_iter_begin (&cursor, resv, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) {if (!r) {KUNIT_FAIL (test,"More than one fence found" );
@@ -231,7 +239,7 @@ static void test_for_each_unlocked(struct kunit *test)dma_resv_iter_end (&cursor);dma_fence_signal (f); err_free: -dma_resv_put (&resv); +dma_resv_put (resv);dma_fence_put (f); }
@@ -240,7 +248,7 @@ static void test_get_fences(struct kunit *test)const struct dma_resv_usage_param *param = test->param_value;enum dma_resv_usage usage = param->usage;struct dma_fence *f, **fences =NULL ; -struct dma_resv resv; +struct dma_resv *resv;int r, i; f =alloc_fence ();
@@ -248,24 +256,26 @@ static void test_get_fences(struct kunit *test)dma_fence_enable_signaling (f); -dma_resv_init (&resv); - r =dma_resv_lock (&resv,NULL ); + resv =dma_resv_alloc (); +KUNIT_ASSERT_NOT_NULL (test, resv); + + r =dma_resv_lock (resv,NULL );if (r) {KUNIT_FAIL (test,"Resv locking failed" ); goto err_resv; } - r =dma_resv_reserve_fences (&resv,1 ); + r =dma_resv_reserve_fences (resv,1 );if (r) {KUNIT_FAIL (test,"Resv shared slot allocation failed" ); -dma_resv_unlock (&resv); +dma_resv_unlock (resv); goto err_resv; } -dma_resv_add_fence (&resv, f, usage); -dma_resv_unlock (&resv); +dma_resv_add_fence (resv, f, usage); +dma_resv_unlock (resv); - r =dma_resv_get_fences (&resv, usage, &i, &fences); + r =dma_resv_get_fences (resv, usage, &i, &fences);if (r) {KUNIT_FAIL (test,"get_fences failed" ); goto err_free;
@@ -282,7 +292,7 @@ static void test_get_fences(struct kunit *test)dma_fence_put (fences[i]);kfree (fences); err_resv: -dma_resv_put (&resv); +dma_resv_put (resv);dma_fence_put (f); }
Signature · 1 line
2.43.0
PATCH 3/12 drm/gem: Add helper for drm_gem_object resv assignment
In reply to Christian KönigIntroduce drm_gem_object_set_resv() to safely assign reservation objects to GEM objects with proper reference counting. This helper replaces all direct assignments to drm_gem_object.resv throughout the DRM subsystem.
The function:
- Acquires a reference to the new resv via dma_resv_get()
- Assigns it to obj->resv
- Releases the old resv reference via dma_resv_put()
This ensures proper reference counting when sharing reservation objects between GEM objects or when importing dma-bufs, preventing reference leaks and use-after-free bugs.
Update all drivers to use the new helper:
- drm_prime: dma-buf import path
- drm_gem_shmem_helper: shmem prime import
- i915: dmabuf import
- msm: MSM_BO_NO_SHARE case
- panthor: exclusive_vm case
- virtio: dma-buf import
- xe: dummy object creation
- ttm: external resv assignment
- ttm/tests: test cases
--- drivers/gpu/drm/drm_gem.c | 31 ++++++++++++++++++- drivers/gpu/drm/drm_gem_shmem_helper.c | 2 +- drivers/gpu/drm/drm_prime.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 2 +- drivers/gpu/drm/panthor/panthor_gem.c | 2 +- drivers/gpu/drm/ttm/tests/ttm_bo_test.c | 2 +- drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 7 ++--- drivers/gpu/drm/virtio/virtgpu_prime.c | 2 +- drivers/gpu/drm/xe/xe_dma_buf.c | 2 +- include/drm/drm_gem.h | 2 ++ 12 files changed, 43 insertions(+), 15 deletions(-)
drivers/gpu/drm/drm_gem.c +30 -1
@@ -230,7 +230,7 @@ void drm_gem_private_object_init(struct drm_device *dev,mutex_init (&obj->gpuva.lock);dma_resv_init (&obj->_resv);if (!obj->resv) - obj->resv = &obj->_resv; + obj->resv =dma_resv_get (&obj->_resv);drm_gem_gpuva_init (obj);
@@ -249,11 +249,40 @@ void drm_gem_private_object_fini(struct drm_gem_object *obj) {WARN_ON (obj->dma_buf); +dma_resv_put (obj->resv);dma_resv_put (&obj->_resv);mutex_destroy (&obj->gpuva.lock); }EXPORT_SYMBOL (drm_gem_private_object_fini); +/** +* drm_gem_object_set_resv - Set the reservation object for a GEM object +* @obj: GEM object +* @resv: reservation object to assign +* +* This function safely assigns a new reservation object to a GEM object. +* It releases the old reservation object reference (if any) and acquires +* a reference to the new one. +* +* This should be used when changing the reservation object of an already +* initialized GEM object, for example when importing a dma-buf or sharing +* a reservation object with another object. +* +* Returns: +* The new reservation object pointer for convenience. +*/ +struct dma_resv *drm_gem_object_set_resv (struct drm_gem_object *obj, +struct dma_resv *resv) +{ +struct dma_resv *old_resv = obj->resv; + + obj->resv =dma_resv_get (resv); +dma_resv_put (old_resv); + +return obj->resv; +} +EXPORT_SYMBOL (drm_gem_object_set_resv); +static void drm_gem_object_handle_get (struct drm_gem_object *obj) {struct drm_device *dev = obj->dev;
drivers/gpu/drm/drm_gem_shmem_helper.c +1 -1
@@ -977,7 +977,7 @@ struct drm_gem_object *drm_gem_shmem_prime_import_no_map(struct drm_device *dev,drm_dbg_prime (dev,"size = %zu\n" , size); shmem->base.import_attach = attach; - shmem->base.resv = dma_buf->resv; +drm_gem_object_set_resv (&shmem->base, dma_buf->resv);return &shmem->base;
drivers/gpu/drm/drm_prime.c +1 -1
@@ -1006,7 +1006,7 @@ struct drm_gem_object *drm_gem_prime_import_dev(struct drm_device *dev, } obj->import_attach = attach; - obj->resv = dma_buf->resv; +drm_gem_object_set_resv (obj, dma_buf->resv);return obj;
drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c +1 -1
@@ -320,7 +320,7 @@ struct drm_gem_object *i915_gem_prime_import(struct drm_device *dev,i915_gem_object_init (obj, &i915_gem_object_dmabuf_ops, &lock_class,I915_BO_ALLOC_USER ); obj->base.import_attach = attach; - obj->base.resv = dma_buf->resv; +drm_gem_object_set_resv (&obj->base, dma_buf->resv);/* We use GTT as shorthand for a coherent domain, one that is * neither in the GPU cache nor in the CPU cache, where all
drivers/gpu/drm/msm/msm_gem.c +1 -1
@@ -1151,7 +1151,7 @@ int msm_gem_new_handle(struct drm_device *dev, struct drm_file *file,drm_gem_object_get (r_obj); - obj->resv = r_obj->resv; +drm_gem_object_set_resv (obj, r_obj->resv); } ret =drm_gem_handle_create (file, obj, handle);
drivers/gpu/drm/panthor/panthor_gem.c +1 -1
@@ -1024,7 +1024,7 @@ panthor_gem_create(struct drm_device *dev, size_t size, uint32_t flags,if (exclusive_vm) { bo->exclusive_vm_root_gem =panthor_vm_root_gem (exclusive_vm);drm_gem_object_get (bo->exclusive_vm_root_gem); - bo->base.resv = bo->exclusive_vm_root_gem->resv; +drm_gem_object_set_resv (&bo->base, bo->exclusive_vm_root_gem->resv); }panthor_gem_debugfs_set_usage_flags (bo, usage_flags);
drivers/gpu/drm/ttm/tests/ttm_bo_test.c +1 -1
@@ -461,7 +461,7 @@ static void ttm_bo_fini_shared_resv(struct kunit *test) bo =ttm_bo_kunit_init (test, test->priv,BO_SIZE ,NULL ); bo->type = ttm_bo_type_device; - bo->base.resv = external_resv; +drm_gem_object_set_resv (&bo->base, external_resv);ttm_bo_fini (bo); }
drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c +1 -1
@@ -181,7 +181,7 @@ struct ttm_buffer_object *ttm_bo_kunit_init(struct kunit *test, bo->base = gem_obj;if (obj) - bo->base.resv = obj; +drm_gem_object_set_resv (&bo->base, obj); err =drm_gem_object_init (devs->drm, &bo->base, size);KUNIT_ASSERT_EQ (test, err,0 );
drivers/gpu/drm/ttm/ttm_bo.c +2 -5
@@ -209,7 +209,7 @@ static int ttm_bo_individualize_resv(struct ttm_buffer_object *bo)* the resv object while holding the lru_lock. */ spin_lock (&bo->bdev->lru_lock); - bo->base.resv = &bo->base._resv; +drm_gem_object_set_resv (&bo->base, &bo->base._resv);spin_unlock (&bo->bdev->lru_lock); }
@@ -942,10 +942,7 @@ int ttm_bo_init_reserved(struct ttm_device *bdev, struct ttm_buffer_object *bo, bo->pin_count =0 ; bo->sg = sg; bo->bulk_move =NULL ; -if (resv) - bo->base.resv = resv; -else - bo->base.resv = &bo->base._resv; +drm_gem_object_set_resv (&bo->base, resv ?: &bo->base._resv);atomic_inc (&ttm_glob.bo_count);/*
drivers/gpu/drm/virtio/virtgpu_prime.c +1 -1
@@ -361,7 +361,7 @@ struct drm_gem_object *virtgpu_gem_prime_import(struct drm_device *dev,INIT_LIST_HEAD (&bo->restore_node); obj = &bo->base.base; - obj->resv = buf->resv; +drm_gem_object_set_resv (obj, buf->resv); obj->funcs = &virtgpu_gem_dma_buf_funcs;drm_gem_private_object_init (dev, obj, buf->size);
drivers/gpu/drm/xe/xe_dma_buf.c +1 -1
@@ -293,7 +293,7 @@ xe_dma_buf_create_obj(struct drm_device *dev, struct dma_buf *dma_buf)if (!dummy_obj)return ERR_PTR (-ENOMEM ); - dummy_obj->resv = resv; +drm_gem_object_set_resv (dummy_obj, resv); xe_validation_guard(&ctx, &xe->val, &exec, (struct xe_val_flags) {}, ret) { ret =drm_exec_lock_obj (&exec, dummy_obj);drm_exec_retry_on_contention (&exec);
include/drm/drm_gem.h +2 -0
@@ -529,6 +529,8 @@ int drm_gem_object_init(struct drm_device *dev,void drm_gem_private_object_init (struct drm_device *dev,struct drm_gem_object *obj,size_t size);void drm_gem_private_object_fini (struct drm_gem_object *obj); +struct dma_resv *drm_gem_object_set_resv (struct drm_gem_object *obj, +struct dma_resv *resv);void drm_gem_vm_open (struct vm_area_struct *vma);void drm_gem_vm_close (struct vm_area_struct *vma); int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
Signature · 1 line
2.43.0
PATCH 4/12 drm/ttm: Switch LRU cursor to track dma_resv instead of buffer objects
In reply to Christian KönigRefactor the LRU cursor to hold references to dma_resv directly while locking it rather than the buffer object.
This avoid the need to grab a reference to the BO and so allows handling of BOs with zero reference count.
--- drivers/gpu/drm/ttm/ttm_bo_util.c | 124 +++++++++++++++-------------- drivers/gpu/drm/ttm/ttm_resource.c | 18 +++++ include/drm/ttm/ttm_bo.h | 10 +-- include/drm/ttm/ttm_resource.h | 2 + 4 files changed, 87 insertions(+), 67 deletions(-)
drivers/gpu/drm/ttm/ttm_bo_util.c +63 -61
@@ -819,19 +819,17 @@ int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo) }static bool ttm_lru_walk_trylock (struct ttm_bo_lru_cursor *curs, -struct ttm_buffer_object *bo) +struct dma_resv *resv) {struct ttm_operation_ctx *ctx = curs->arg->ctx; - curs->needs_unlock = false; - -if (dma_resv_trylock (bo->base.resv)) { - curs->needs_unlock = true; +if (dma_resv_trylock (resv)) { + curs->resv =dma_resv_get (resv);return true; } -if (bo->base.resv == ctx->resv && ctx->allow_res_evict) { -dma_resv_assert_held (bo->base.resv); +if (resv == ctx->resv && ctx->allow_res_evict) { +dma_resv_assert_held (resv);return true; }
@@ -839,18 +837,18 @@ static bool ttm_lru_walk_trylock(struct ttm_bo_lru_cursor *curs, }static int ttm_lru_walk_ticketlock (struct ttm_bo_lru_cursor *curs, -struct ttm_buffer_object *bo) +struct dma_resv *resv) {struct ttm_lru_walk_arg *arg = curs->arg;int ret;if (arg->ctx->interruptible) - ret =dma_resv_lock_interruptible (bo->base.resv, arg->ticket); + ret =dma_resv_lock_interruptible (resv, arg->ticket);else - ret =dma_resv_lock (bo->base.resv, arg->ticket); + ret =dma_resv_lock (resv, arg->ticket);if (!ret) { - curs->needs_unlock = true; + curs->resv =dma_resv_get (resv);/* * Only a single ticketlock per loop. Ticketlocks are prone * to return -EDEADLK causing the eviction to fail, so
@@ -920,14 +918,16 @@ s64 ttm_lru_walk_for_evict(struct ttm_lru_walk *walk, struct ttm_device *bdev, }EXPORT_SYMBOL (ttm_lru_walk_for_evict); -static void ttm_bo_lru_cursor_cleanup_bo (struct ttm_bo_lru_cursor *curs) +static void ttm_bo_lru_cursor_cleanup (struct ttm_bo_lru_cursor *curs) { -struct ttm_buffer_object *bo = curs->bo; +if (curs->resv) { +dma_resv_unlock (curs->resv); +dma_resv_put (curs->resv); + curs->resv =NULL ; + } -if (bo) { -if (curs->needs_unlock) -dma_resv_unlock (bo->base.resv); -ttm_bo_put (bo); +if (curs->bo) { +drm_gem_object_put (&curs->bo->base); curs->bo =NULL ; } }
@@ -941,7 +941,7 @@ void ttm_bo_lru_cursor_fini(struct ttm_bo_lru_cursor *curs) {spinlock_t *lru_lock = &curs->res_curs.man->bdev->lru_lock; -ttm_bo_lru_cursor_cleanup_bo (curs); +ttm_bo_lru_cursor_cleanup (curs);spin_lock (lru_lock);ttm_resource_cursor_fini (&curs->res_curs);spin_unlock (lru_lock);
@@ -972,21 +972,18 @@ ttm_bo_lru_cursor_init(struct ttm_bo_lru_cursor *curs,EXPORT_SYMBOL (ttm_bo_lru_cursor_init);static struct ttm_buffer_object * -__ttm_bo_lru_cursor_next (struct ttm_bo_lru_cursor *curs) +__ttm_bo_lru_cursor_iter (struct ttm_bo_lru_cursor *curs,bool first) {spinlock_t *lru_lock = &curs->res_curs.man->bdev->lru_lock; -struct ttm_resource *res =NULL ; -struct ttm_buffer_object *bo;struct ttm_lru_walk_arg *arg = curs->arg; -bool first = !curs->bo; - -ttm_bo_lru_cursor_cleanup_bo (curs); +int ret; -spin_lock (lru_lock);for (;;) { -int mem_type, ret =0 ; -bool bo_locked = false; +struct ttm_resource *res; + +ttm_bo_lru_cursor_cleanup (curs); +spin_lock (lru_lock);if (first) { res =ttm_resource_manager_first (&curs->res_curs); first = false;
@@ -996,43 +993,48 @@ __ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs)if (!res)break ; - bo = res->bo; -if (ttm_lru_walk_trylock (curs, bo)) - bo_locked = true; -else if (!arg->ticket || arg->ctx->no_wait_gpu || arg->trylock_only) -continue ; - -if (!ttm_bo_get_unless_zero (bo)) { -if (curs->needs_unlock) -dma_resv_unlock (bo->base.resv); -continue ; +if (!ttm_lru_walk_trylock (curs, res->bo->base.resv)) { +struct dma_resv *resv; + +if (!arg->ticket || arg->ctx->no_wait_gpu || + arg->trylock_only) { +spin_unlock (lru_lock); +continue ; + } + + resv =dma_resv_get (res->bo->base.resv); +spin_unlock (lru_lock); + + ret =ttm_lru_walk_ticketlock (curs, resv); +if (ret && ret != -EALREADY ) +return ERR_PTR (ret); + +/* +* We need to double check that we still have the same +* dma_resv object. +*/ +spin_lock (lru_lock); + res =ttm_resource_manager_current (&curs->res_curs); +if (ret || !res || res->bo->base.resv != resv) { +spin_unlock (lru_lock); +dma_resv_put (resv); +continue ; + } +dma_resv_put (resv); } - - mem_type = res->mem_type;spin_unlock (lru_lock); -if (!bo_locked) - ret =ttm_lru_walk_ticketlock (curs, bo); + +/* Grab a GEM reference to the BO if it isn't already deleted */ +if (kref_get_unless_zero (&res->bo->base.refcount)) + curs->bo = res->bo;/* -* Note that in between the release of the lru lock and the -* ticketlock, the bo may have switched resource, -* and also memory type, since the resource may have been -* freed and allocated again with a different memory type. -* In that case, just skip it. +* The BO is now locked so it can't be released any more until +* we drop both the lock and the eventual GEM reference. */ - curs->bo = bo; -if (!ret && bo->resource && bo->resource->mem_type == mem_type) -return bo; - -ttm_bo_lru_cursor_cleanup_bo (curs); -if (ret && ret != -EALREADY ) -return ERR_PTR (ret); - -spin_lock (lru_lock); +return res->bo; } - -spin_unlock (lru_lock); -return res ? bo :NULL ; +return NULL ; }/**
@@ -1046,7 +1048,7 @@ __ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs)*/ struct ttm_buffer_object *ttm_bo_lru_cursor_next (struct ttm_bo_lru_cursor *curs) { -return __ttm_bo_lru_cursor_next (curs); +return __ttm_bo_lru_cursor_iter (curs, false); }EXPORT_SYMBOL (ttm_bo_lru_cursor_next);
@@ -1060,8 +1062,8 @@ EXPORT_SYMBOL(ttm_bo_lru_cursor_next);*/ struct ttm_buffer_object *ttm_bo_lru_cursor_first (struct ttm_bo_lru_cursor *curs) { -ttm_bo_lru_cursor_cleanup_bo (curs); -return __ttm_bo_lru_cursor_next (curs); +ttm_bo_lru_cursor_cleanup (curs); +return __ttm_bo_lru_cursor_iter (curs, true); }EXPORT_SYMBOL (ttm_bo_lru_cursor_first);
drivers/gpu/drm/ttm/ttm_resource.c +18 -0
@@ -714,6 +714,24 @@ ttm_resource_manager_next(struct ttm_resource_cursor *cursor)return NULL ; } +/* TODO */ +struct ttm_resource * +ttm_resource_manager_current (struct ttm_resource_cursor *cursor) +{ +struct ttm_resource_manager *man = cursor->man; +struct ttm_lru_item *lru; + +lockdep_assert_held (&man->bdev->lru_lock); + + lru = &cursor->hitch; +list_for_each_entry_continue_reverse (lru, &man->lru[cursor->priority], + link) { +if (ttm_lru_item_is_res (lru)) +return ttm_lru_item_to_res (lru); + } +return NULL ; +} +/** * ttm_lru_first_res_or_null() - Return the first resource on an lru list * @head: The list head of the lru list.
include/drm/ttm/ttm_bo.h +4 -6
@@ -488,15 +488,13 @@ struct ttm_bo_lru_cursor {/** @res_curs: Embedded struct ttm_resource_cursor. */ struct ttm_resource_cursor res_curs;/** -* @bo: Buffer object pointer if a buffer object is refcounted, -* NULL otherwise. +* @resv: reference to the locked dma_resv */ -struct ttm_buffer_object *bo; +struct dma_resv *resv;/** -* @needs_unlock: Valid iff @bo != NULL. The bo resv needs -* unlock before the next iteration or after loop exit. +* @bo: TTM BO with GEM reference, NULL for deleted BOs */ -bool needs_unlock; +struct ttm_buffer_object *bo;/** @arg: Pointer to common BO LRU walk arguments. */ struct ttm_lru_walk_arg *arg; };
include/drm/ttm/ttm_resource.h +2 -0
@@ -488,6 +488,8 @@ struct ttm_resource * ttm_resource_manager_first(struct ttm_resource_cursor *cursor);struct ttm_resource *ttm_resource_manager_next (struct ttm_resource_cursor *cursor); +struct ttm_resource * +ttm_resource_manager_current (struct ttm_resource_cursor *cursor);struct ttm_resource *ttm_lru_first_res_or_null (struct list_head *head);
Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH 04/12] drm/ttm: Switch LRU cursor to track dma_resv instead of buffer objects
Quoted 177 lines · Christian König
Refactor the LRU cursor to hold references to
dma_resvdirectly while locking it rather than the buffer object.This avoid the need to grab a reference to the BO and so allows handling
of BOs with zero reference count.Signed-off-by Christian König <[email protected]>---
drivers/gpu/drm/ttm/ttm_bo_util.c | 124 +++++++++++++++------------ --
drivers/gpu/drm/ttm/ttm_resource.c | 18 +++++ include/drm/ttm/ttm_bo.h | 10 +-- include/drm/ttm/ttm_resource.h | 2 +4 files changed, 87 insertions(+), 67 deletions(-)drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -819,19 +819,17 @@ int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo) }
static bool ttm_lru_walk_trylock(structttm_bo_lru_cursor*curs,drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- struct ttm_buffer_object *bo) + struct dma_resv *resv){
structttm_operation_ctx*ctx =curs->arg->ctx;drivers/gpu/drm/ttm/ttm_bo_util.c +2
-4- curs->needs_unlock = false; - -if (dma_resv_trylock (bo->base.resv)) { - curs->needs_unlock = true; +if (dma_resv_trylock (resv)) { + curs->resv =dma_resv_get (resv);return true;
}drivers/gpu/drm/ttm/ttm_bo_util.c +2
-2-if (bo->base.resv == ctx->resv && ctx->allow_res_evict) { -dma_resv_assert_held (bo->base.resv); +if (resv == ctx->resv && ctx->allow_res_evict) { +dma_resv_assert_held (resv);return true;
}drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -839,18 +837,18 @@ static bool ttm_lru_walk_trylock(struct ttm_bo_lru_cursor *curs, }
static int ttm_lru_walk_ticketlock(structttm_bo_lru_cursor*curs,drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- struct ttm_buffer_object *bo) + struct dma_resv *resv){
structttm_lru_walk_arg*arg =curs->arg;
int ret;
if (arg->ctx->interruptible)drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1- ret = dma_resv_lock_interruptible(bo->base.resv,
arg->ticket);drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+ ret = dma_resv_lock_interruptible(resv, arg-Quoted 1 line
ticket);
else
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- ret =dma_resv_lock (bo->base.resv, arg->ticket); + ret =dma_resv_lock (resv, arg->ticket);
if (!ret) {drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- curs->needs_unlock = true; + curs->resv =dma_resv_get (resv);/*
* Only a single ticketlock per loop. Ticketlocks are prone
* to return-EDEADLKcausing the eviction to fail, sodrivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -920,14 +918,16 @@ s64 ttm_lru_walk_for_evict(struct ttm_lru_walk *walk, struct ttm_device *bdev, } EXPORT_SYMBOL(ttm_lru_walk_for_evict);
-static voidttm_bo_lru_cursor_cleanup_bo(struct ttm_bo_lru_cursor *curs)
+static voidttm_bo_lru_cursor_cleanup(struct ttm_bo_lru_cursor *curs)
{drivers/gpu/drm/ttm/ttm_bo_util.c +5
-1-struct ttm_buffer_object *bo = curs->bo; +if (curs->resv) { +dma_resv_unlock (curs->resv); +dma_resv_put (curs->resv); + curs->resv =NULL ; + }drivers/gpu/drm/ttm/ttm_bo_util.c +2
-4-if (bo) { -if (curs->needs_unlock) -dma_resv_unlock (bo->base.resv); -ttm_bo_put (bo); +if (curs->bo) { +drm_gem_object_put (&curs->bo->base);
curs->bo= NULL;
}
}drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -941,7 +941,7 @@ void ttm_bo_lru_cursor_fini(struct ttm_bo_lru_cursor *curs) { spinlock_t *lru_lock = &curs->res_curs.man->bdev->lru_lock;drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-ttm_bo_lru_cursor_cleanup_bo (curs); +ttm_bo_lru_cursor_cleanup (curs);
spin_lock(lru_lock);ttm_resource_cursor_fini(&curs->res_curs);spin_unlock(lru_lock);drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -972,21 +972,18 @@ ttm_bo_lru_cursor_init(struct ttm_bo_lru_cursor *curs, EXPORT_SYMBOL(ttm_bo_lru_cursor_init);
static structttm_buffer_object*drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-__ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs) +__ttm_bo_lru_cursor_iter(struct ttm_bo_lru_cursor *curs, bool first){
spinlock_t*lru_lock=&curs->res_curs.man->bdev->lru_lock;drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2-struct ttm_resource *res =NULL ; -struct ttm_buffer_object *bo;struct
ttm_lru_walk_arg*arg =curs->arg;drivers/gpu/drm/ttm/ttm_bo_util.c +1
-3-bool first = !curs->bo; - -ttm_bo_lru_cursor_cleanup_bo (curs); +int ret;drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1-spin_lock (lru_lock);for (;;) {
drivers/gpu/drm/ttm/ttm_bo_util.c +3
-2-int mem_type, ret =0 ; -bool bo_locked = false; +struct ttm_resource *res; + +ttm_bo_lru_cursor_cleanup (curs);drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+spin_lock (lru_lock);if (first) {
res = ttm_resource_manager_first(&curs-Quoted 1 line
res_curs);first = false;
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -996,43 +993,48 @@ __ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs) if (!res) break;drivers/gpu/drm/ttm/ttm_bo_util.c +0
-4- bo = res->bo; -if (ttm_lru_walk_trylock (curs, bo)) - bo_locked = true; - else if (!arg->ticket || arg->ctx->no_wait_gpu ||
arg->trylock_only)drivers/gpu/drm/ttm/ttm_bo_util.c +1
-6-continue ; - -if (!ttm_bo_get_unless_zero (bo)) { -if (curs->needs_unlock) -dma_resv_unlock (bo->base.resv); -continue ; +if (!ttm_lru_walk_trylock (curs, res->bo->base.resv)){
drivers/gpu/drm/ttm/ttm_bo_util.c +17
-0+struct dma_resv *resv; + +if (!arg->ticket || arg->ctx->no_wait_gpu || + arg->trylock_only) { +spin_unlock (lru_lock); +continue ; + } + + resv =dma_resv_get (res->bo->base.resv); +spin_unlock (lru_lock); + + ret =ttm_lru_walk_ticketlock (curs, resv); +if (ret && ret != -EALREADY ) +return ERR_PTR (ret); + +/* +* We need to double check that we still have the same
drivers/gpu/drm/ttm/ttm_bo_util.c __ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs) · line 1014
Quoted 1 line
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+* dma_resv object.
While I think this would actually work, we need documentation and asserts of what exactly we're relying on here to keep the bo alive. Otherwise future work will most likely break something.
Thanks,
Thomas
Quoted 154 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +3
-0+*/ +spin_lock (lru_lock); + res = ttm_resource_manager_current(&curs-Quoted 1 line
res_curs);drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+ if (ret || !res || res->bo->base.resv !=resv) {
drivers/gpu/drm/ttm/ttm_bo_util.c +5
-0+spin_unlock (lru_lock); +dma_resv_put (resv); +continue ; + } +dma_resv_put (resv);}
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2- - mem_type = res->mem_type;
spin_unlock(lru_lock);drivers/gpu/drm/ttm/ttm_bo_util.c +2
-2-if (!bo_locked) - ret =ttm_lru_walk_ticketlock (curs, bo); + +/* Grab a GEM reference to the BO if it isn't already deleted */
drivers/gpu/drm/ttm/ttm_bo_util.c +2
-0+if (kref_get_unless_zero (&res->bo->base.refcount)) + curs->bo = res->bo;
/*drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1-* Note that in between the release of the lru lock and the
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2-* ticketlock, the bo may have switched resource, -* and also memory type, since the resource may have been
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1-* freed and allocated again with a different memory type.
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-* In that case, just skip it. +* The BO is now locked so it can't be released any more until
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+* we drop both the lock and the eventual GEM reference.
*/drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2- curs->bo = bo; - if (!ret && bo->resource && bo->resource->mem_type==
mem_type)drivers/gpu/drm/ttm/ttm_bo_util.c +1
-7-return bo; - -ttm_bo_lru_cursor_cleanup_bo (curs); -if (ret && ret != -EALREADY ) -return ERR_PTR (ret); - -spin_lock (lru_lock); +return res->bo;}
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-3- -spin_unlock (lru_lock); -return res ? bo :NULL ; +return NULL ;}
/**drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -1046,7 +1048,7 @@ __ttm_bo_lru_cursor_next(structttm_bo_lru_cursor *curs) */ struct
ttm_buffer_object*ttm_bo_lru_cursor_next(structttm_bo_lru_cursor*curs)
{drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-return __ttm_bo_lru_cursor_next (curs); +return __ttm_bo_lru_cursor_iter (curs, false);}
EXPORT_SYMBOL(ttm_bo_lru_cursor_next);drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -1060,8 +1062,8 @@ EXPORT_SYMBOL(ttm_bo_lru_cursor_next);*/ struct
ttm_buffer_object*ttm_bo_lru_cursor_first(structttm_bo_lru_cursor*curs)
{drivers/gpu/drm/ttm/ttm_bo_util.c +2
-2-ttm_bo_lru_cursor_cleanup_bo (curs); -return __ttm_bo_lru_cursor_next (curs); +ttm_bo_lru_cursor_cleanup (curs); +return __ttm_bo_lru_cursor_iter (curs, true);}
EXPORT_SYMBOL(ttm_bo_lru_cursor_first);drivers/gpu/drm/ttm/ttm_resource.c +0
-0@@ -714,6 +714,24 @@ ttm_resource_manager_next(struct ttm_resource_cursor *cursor) return NULL; }drivers/gpu/drm/ttm/ttm_resource.c +11
-0+/* TODO */ +struct ttm_resource * +ttm_resource_manager_current(struct ttm_resource_cursor *cursor) +{ + struct ttm_resource_manager *man = cursor->man; + struct ttm_lru_item *lru; + + lockdep_assert_held(&man->bdev->lru_lock); + + lru = &cursor->hitch; + list_for_each_entry_continue_reverse(lru, &man->lru[cursor-Quoted 1 line
priority],
drivers/gpu/drm/ttm/ttm_resource.c +7
-0+ link) { +if (ttm_lru_item_is_res (lru)) +return ttm_lru_item_to_res (lru); + } +return NULL ; +} +/**
*ttm_lru_first_res_or_null()- Return the first resource on an lru list
* @head: The list head of the lru list.include/drm/ttm/ttm_bo.h +4
-6@@ -488,15 +488,13 @@ struct ttm_bo_lru_cursor { /** @res_curs: Embedded struct ttm_resource_cursor. */ struct ttm_resource_cursor res_curs; /** -* @bo: Buffer object pointer if a buffer object is refcounted, -* NULL otherwise. +* @resv: reference to the locked dma_resv */ -struct ttm_buffer_object *bo; +struct dma_resv *resv; /** -* @needs_unlock: Valid iff @bo != NULL. The bo resv needs -* unlock before the next iteration or after loop exit. +* @bo: TTM BO with GEM reference, NULL for deleted BOs */ -bool needs_unlock; +struct ttm_buffer_object *bo; /** @arg: Pointer to common BO LRU walk arguments. */ struct ttm_lru_walk_arg *arg;};
include/drm/ttm/ttm_resource.h +0
-0@@ -488,6 +488,8 @@ struct ttm_resource * ttm_resource_manager_first(struct ttm_resource_cursor *cursor);struct
ttm_resource*ttm_resource_manager_next(struct ttm_resource_cursor *cursor);include/drm/ttm/ttm_resource.h +2
-0+struct ttm_resource * +ttm_resource_manager_current (struct ttm_resource_cursor *cursor);
structttm_resource*ttm_lru_first_res_or_null(struct list_head *head);
Matthew Brost
maintainerRe: [PATCH 04/12] drm/ttm: Switch LRU cursor to track dma_resv instead of buffer objects
Quoted 175 lines · Thomas Hellström
Quoted 174 lines · Christian König
Refactor the LRU cursor to hold references to
dma_resvdirectly while locking it rather than the buffer object.This avoid the need to grab a reference to the BO and so allows handling
of BOs with zero reference count.Signed-off-by Christian König <[email protected]>---
drivers/gpu/drm/ttm/ttm_bo_util.c | 124 +++++++++++++++------------ --
drivers/gpu/drm/ttm/ttm_resource.c | 18 +++++ include/drm/ttm/ttm_bo.h | 10 +-- include/drm/ttm/ttm_resource.h | 2 +4 files changed, 87 insertions(+), 67 deletions(-)drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -819,19 +819,17 @@ int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo) }
static bool ttm_lru_walk_trylock(structttm_bo_lru_cursor*curs,drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- struct ttm_buffer_object *bo) + struct dma_resv *resv){
structttm_operation_ctx*ctx =curs->arg->ctx;drivers/gpu/drm/ttm/ttm_bo_util.c +2
-4- curs->needs_unlock = false; - -if (dma_resv_trylock (bo->base.resv)) { - curs->needs_unlock = true; +if (dma_resv_trylock (resv)) { + curs->resv =dma_resv_get (resv);return true;
}drivers/gpu/drm/ttm/ttm_bo_util.c +2
-2-if (bo->base.resv == ctx->resv && ctx->allow_res_evict) { -dma_resv_assert_held (bo->base.resv); +if (resv == ctx->resv && ctx->allow_res_evict) { +dma_resv_assert_held (resv);return true;
}drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -839,18 +837,18 @@ static bool ttm_lru_walk_trylock(struct ttm_bo_lru_cursor *curs, }
static int ttm_lru_walk_ticketlock(structttm_bo_lru_cursor*curs,drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- struct ttm_buffer_object *bo) + struct dma_resv *resv){
structttm_lru_walk_arg*arg =curs->arg;
int ret;
if (arg->ctx->interruptible)drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1- ret = dma_resv_lock_interruptible(bo->base.resv,
arg->ticket);drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+ ret = dma_resv_lock_interruptible(resv, arg-Quoted 1 line
ticket);
else
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- ret =dma_resv_lock (bo->base.resv, arg->ticket); + ret =dma_resv_lock (resv, arg->ticket);
if (!ret) {drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1- curs->needs_unlock = true; + curs->resv =dma_resv_get (resv);/*
* Only a single ticketlock per loop. Ticketlocks are prone
* to return-EDEADLKcausing the eviction to fail, sodrivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -920,14 +918,16 @@ s64 ttm_lru_walk_for_evict(struct ttm_lru_walk *walk, struct ttm_device *bdev, } EXPORT_SYMBOL(ttm_lru_walk_for_evict);
-static voidttm_bo_lru_cursor_cleanup_bo(struct ttm_bo_lru_cursor *curs)
+static voidttm_bo_lru_cursor_cleanup(struct ttm_bo_lru_cursor *curs)
{drivers/gpu/drm/ttm/ttm_bo_util.c +5
-1-struct ttm_buffer_object *bo = curs->bo; +if (curs->resv) { +dma_resv_unlock (curs->resv); +dma_resv_put (curs->resv); + curs->resv =NULL ; + }drivers/gpu/drm/ttm/ttm_bo_util.c +2
-4-if (bo) { -if (curs->needs_unlock) -dma_resv_unlock (bo->base.resv); -ttm_bo_put (bo); +if (curs->bo) { +drm_gem_object_put (&curs->bo->base);
curs->bo= NULL;
}
}drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -941,7 +941,7 @@ void ttm_bo_lru_cursor_fini(struct ttm_bo_lru_cursor *curs) { spinlock_t *lru_lock = &curs->res_curs.man->bdev->lru_lock;drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-ttm_bo_lru_cursor_cleanup_bo (curs); +ttm_bo_lru_cursor_cleanup (curs);
spin_lock(lru_lock);ttm_resource_cursor_fini(&curs->res_curs);spin_unlock(lru_lock);drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -972,21 +972,18 @@ ttm_bo_lru_cursor_init(struct ttm_bo_lru_cursor *curs, EXPORT_SYMBOL(ttm_bo_lru_cursor_init);
static structttm_buffer_object*drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-__ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs) +__ttm_bo_lru_cursor_iter(struct ttm_bo_lru_cursor *curs, bool first){
spinlock_t*lru_lock=&curs->res_curs.man->bdev->lru_lock;drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2-struct ttm_resource *res =NULL ; -struct ttm_buffer_object *bo;struct
ttm_lru_walk_arg*arg =curs->arg;drivers/gpu/drm/ttm/ttm_bo_util.c +1
-3-bool first = !curs->bo; - -ttm_bo_lru_cursor_cleanup_bo (curs); +int ret;drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1-spin_lock (lru_lock);for (;;) {
drivers/gpu/drm/ttm/ttm_bo_util.c +3
-2-int mem_type, ret =0 ; -bool bo_locked = false; +struct ttm_resource *res; + +ttm_bo_lru_cursor_cleanup (curs);drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+spin_lock (lru_lock);if (first) {
res = ttm_resource_manager_first(&curs-Quoted 1 line
res_curs);first = false;
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -996,43 +993,48 @@ __ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs) if (!res) break;drivers/gpu/drm/ttm/ttm_bo_util.c +0
-4- bo = res->bo; -if (ttm_lru_walk_trylock (curs, bo)) - bo_locked = true; - else if (!arg->ticket || arg->ctx->no_wait_gpu ||
arg->trylock_only)drivers/gpu/drm/ttm/ttm_bo_util.c +1
-6-continue ; - -if (!ttm_bo_get_unless_zero (bo)) { -if (curs->needs_unlock) -dma_resv_unlock (bo->base.resv); -continue ; +if (!ttm_lru_walk_trylock (curs, res->bo->base.resv)){
drivers/gpu/drm/ttm/ttm_bo_util.c +15
-0+struct dma_resv *resv; + +if (!arg->ticket || arg->ctx->no_wait_gpu || + arg->trylock_only) { +spin_unlock (lru_lock); +continue ; + } + + resv =dma_resv_get (res->bo->base.resv); +spin_unlock (lru_lock); + + ret =ttm_lru_walk_ticketlock (curs, resv); +if (ret && ret != -EALREADY ) +return ERR_PTR (ret); +
Quoted 9 lines
drivers/gpu/drm/ttm/ttm_bo_util.c __ttm_bo_lru_cursor_next(struct ttm_bo_lru_cursor *curs) · line 1014
Quoted 4 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +2
-0+/* +* We need to double check that we still have the same
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+* dma_resv object. While I think this would actually work, we need documentation and asserts of what exactly we're relying on here to keep the bo alive. Otherwise future work will most likely break something.
As a general comment, we should build some TTM assertion macros akin to xe_assert. TTM has a lot of subtle invariants that make the whole thing work, and many of them are not guarded by assertions. Assertions, of course, help catch bugs at runtime, but they also serve as self-documenting invariants, which leads to a codebase that is easier to understand and maintain.
Matt
Quoted 158 lines
Thanks,
ThomasQuoted 154 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +3
-0+*/ +spin_lock (lru_lock); + res = ttm_resource_manager_current(&curs-Quoted 1 line
res_curs);drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+ if (ret || !res || res->bo->base.resv !=resv) {
drivers/gpu/drm/ttm/ttm_bo_util.c +5
-0+spin_unlock (lru_lock); +dma_resv_put (resv); +continue ; + } +dma_resv_put (resv);}
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2- - mem_type = res->mem_type;
spin_unlock(lru_lock);drivers/gpu/drm/ttm/ttm_bo_util.c +2
-2-if (!bo_locked) - ret =ttm_lru_walk_ticketlock (curs, bo); + +/* Grab a GEM reference to the BO if it isn't already deleted */
drivers/gpu/drm/ttm/ttm_bo_util.c +2
-0+if (kref_get_unless_zero (&res->bo->base.refcount)) + curs->bo = res->bo;
/*drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1-* Note that in between the release of the lru lock and the
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2-* ticketlock, the bo may have switched resource, -* and also memory type, since the resource may have been
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-1-* freed and allocated again with a different memory type.
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-* In that case, just skip it. +* The BO is now locked so it can't be released any more until
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+* we drop both the lock and the eventual GEM reference.
*/drivers/gpu/drm/ttm/ttm_bo_util.c +0
-2- curs->bo = bo; - if (!ret && bo->resource && bo->resource->mem_type==
mem_type)drivers/gpu/drm/ttm/ttm_bo_util.c +1
-7-return bo; - -ttm_bo_lru_cursor_cleanup_bo (curs); -if (ret && ret != -EALREADY ) -return ERR_PTR (ret); - -spin_lock (lru_lock); +return res->bo;}
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-3- -spin_unlock (lru_lock); -return res ? bo :NULL ; +return NULL ;}
/**drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -1046,7 +1048,7 @@ __ttm_bo_lru_cursor_next(structttm_bo_lru_cursor *curs) */ struct
ttm_buffer_object*ttm_bo_lru_cursor_next(structttm_bo_lru_cursor*curs)
{drivers/gpu/drm/ttm/ttm_bo_util.c +1
-1-return __ttm_bo_lru_cursor_next (curs); +return __ttm_bo_lru_cursor_iter (curs, false);}
EXPORT_SYMBOL(ttm_bo_lru_cursor_next);drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -1060,8 +1062,8 @@ EXPORT_SYMBOL(ttm_bo_lru_cursor_next);*/ struct
ttm_buffer_object*ttm_bo_lru_cursor_first(structttm_bo_lru_cursor*curs)
{drivers/gpu/drm/ttm/ttm_bo_util.c +2
-2-ttm_bo_lru_cursor_cleanup_bo (curs); -return __ttm_bo_lru_cursor_next (curs); +ttm_bo_lru_cursor_cleanup (curs); +return __ttm_bo_lru_cursor_iter (curs, true);}
EXPORT_SYMBOL(ttm_bo_lru_cursor_first);drivers/gpu/drm/ttm/ttm_resource.c +0
-0@@ -714,6 +714,24 @@ ttm_resource_manager_next(struct ttm_resource_cursor *cursor) return NULL; }drivers/gpu/drm/ttm/ttm_resource.c +11
-0+/* TODO */ +struct ttm_resource * +ttm_resource_manager_current(struct ttm_resource_cursor *cursor) +{ + struct ttm_resource_manager *man = cursor->man; + struct ttm_lru_item *lru; + + lockdep_assert_held(&man->bdev->lru_lock); + + lru = &cursor->hitch; + list_for_each_entry_continue_reverse(lru, &man->lru[cursor-Quoted 1 line
priority],
drivers/gpu/drm/ttm/ttm_resource.c +7
-0+ link) { +if (ttm_lru_item_is_res (lru)) +return ttm_lru_item_to_res (lru); + } +return NULL ; +} +/**
*ttm_lru_first_res_or_null()- Return the first resource on an lru list
* @head: The list head of the lru list.include/drm/ttm/ttm_bo.h +4
-6@@ -488,15 +488,13 @@ struct ttm_bo_lru_cursor { /** @res_curs: Embedded struct ttm_resource_cursor. */ struct ttm_resource_cursor res_curs; /** -* @bo: Buffer object pointer if a buffer object is refcounted, -* NULL otherwise. +* @resv: reference to the locked dma_resv */ -struct ttm_buffer_object *bo; +struct dma_resv *resv; /** -* @needs_unlock: Valid iff @bo != NULL. The bo resv needs -* unlock before the next iteration or after loop exit. +* @bo: TTM BO with GEM reference, NULL for deleted BOs */ -bool needs_unlock; +struct ttm_buffer_object *bo; /** @arg: Pointer to common BO LRU walk arguments. */ struct ttm_lru_walk_arg *arg;};
include/drm/ttm/ttm_resource.h +0
-0@@ -488,6 +488,8 @@ struct ttm_resource * ttm_resource_manager_first(struct ttm_resource_cursor *cursor);struct
ttm_resource*ttm_resource_manager_next(struct ttm_resource_cursor *cursor);include/drm/ttm/ttm_resource.h +2
-0+struct ttm_resource * +ttm_resource_manager_current (struct ttm_resource_cursor *cursor);
structttm_resource*ttm_lru_first_res_or_null(struct list_head *head);
PATCH 5/12 drm/ttm: switch to ttm_bo_lru_for_each_reserved_guarded for swapout
In reply to Christian KönigInstead of the walker wrapper use the underlying foreach. Saves us quite a bunch of complexity and loc.
--- drivers/gpu/drm/ttm/ttm_bo.c | 64 ++++++-------------------------- drivers/gpu/drm/ttm/ttm_device.c | 19 ++++++++-- include/drm/ttm/ttm_bo.h | 5 +-- 3 files changed, 28 insertions(+), 60 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +11 -53
@@ -1088,25 +1088,18 @@ int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx)EXPORT_SYMBOL (ttm_bo_wait_ctx);/** -* struct ttm_bo_swapout_walk - Parameters for the swapout walk +* ttm_bo_swapout() - Swap out buffer objects on the LRU list to shmem. +* @bo: The buffer to swap out. +* @ctx: The ttm_operation_ctx governing the swapout operation. +* @gfp_flags: The gfp flags used for shmem page allocations. +* +* Return: The number of bytes actually swapped out, or negative error code +* on error. */ -struct ttm_bo_swapout_walk { -/** @walk: The walk base parameters. */ -struct ttm_lru_walk walk; -/** @gfp_flags: The gfp flags to use for ttm_tt_swapout() */ -gfp_t gfp_flags; -/** @hit_low: Whether we should attempt to swap BO's with low watermark threshold */ -/** @evict_low: If we cannot swap a bo when @try_low is false (first pass) */ -bool hit_low, evict_low; -}; - -static s64 -ttm_bo_swapout_cb (struct ttm_lru_walk *walk,struct ttm_buffer_object *bo) +s64 ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, +gfp_t gfp_flags) { -struct ttm_place place = { .mem_type = bo->resource->mem_type }; -struct ttm_bo_swapout_walk *swapout_walk = -container_of (walk,typeof (*swapout_walk), walk); -struct ttm_operation_ctx *ctx = walk->arg.ctx; +struct ttm_place place = {.mem_type = bo->resource->mem_type};struct ttm_device *bdev = bo->bdev;struct ttm_tt *tt = bo->ttm;s64 ret;
@@ -1174,7 +1167,7 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) bdev->funcs->swap_notify (bo);if (ttm_tt_is_populated (tt)) { - ret =ttm_tt_swapout (bdev, tt, swapout_walk->gfp_flags); + ret =ttm_tt_swapout (bdev, tt, gfp_flags);if (!ret) {spin_lock (&bdev->lru_lock);ttm_resource_del_bulk_move_unevictable (bo->resource, bo);
@@ -1191,41 +1184,6 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo)return ret; } -const struct ttm_lru_walk_ops ttm_swap_ops = { - .process_bo = ttm_bo_swapout_cb, -}; - -/** -* ttm_bo_swapout() - Swap out buffer objects on the LRU list to shmem. -* @bdev: The ttm device. -* @ctx: The ttm_operation_ctx governing the swapout operation. -* @man: The resource manager whose resources / buffer objects are -* goint to be swapped out. -* @gfp_flags: The gfp flags used for shmem page allocations. -* @target: The desired number of pages to swap out. -* -* Return: The number of pages actually swapped out, or negative error code -* on error. -*/ -s64 ttm_bo_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, -struct ttm_resource_manager *man,gfp_t gfp_flags, -s64 target) -{ -struct ttm_bo_swapout_walk swapout_walk = { - .walk = { - .ops = &ttm_swap_ops, - .arg = { - .ctx = ctx, - .trylock_only = true, - }, - }, - .gfp_flags = gfp_flags, - }; - -return ttm_lru_walk_for_evict (&swapout_walk.walk, bdev, man, target); -} -EXPORT_SYMBOL_FOR_TESTS_ONLY (ttm_bo_swapout); -void ttm_bo_tt_destroy (struct ttm_buffer_object *bo) {if (bo->ttm ==NULL )
drivers/gpu/drm/ttm/ttm_device.c +15 -4
@@ -171,6 +171,12 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx, gfp_t gfp_flags) {struct ttm_resource_manager *man; +struct ttm_bo_lru_cursor cursor; +struct ttm_buffer_object *bo; +struct ttm_lru_walk_arg arg = { + .ctx = ctx, + .trylock_only = true + };unsigned i;s64 lret;
@@ -179,10 +185,15 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,if (!man || !man->use_tt)continue ; - lret =ttm_bo_swapout (bdev, ctx, man, gfp_flags,1 ); -/* Can be both positive (num_pages) and negative (error) */ -if (lret) -return lret; +ttm_bo_lru_for_each_reserved_guarded (&cursor, man, &arg, bo) { + lret =ttm_bo_swapout (bo, ctx, gfp_flags); +continue ; +/* Can be both positive (num_pages) and negative (error) */ +if (lret && lret != -EBUSY && lret != -EALREADY ) +return lret; + } +if (IS_ERR (bo)) +return PTR_ERR (bo); }return 0 ; }
include/drm/ttm/ttm_bo.h +2 -3
@@ -423,9 +423,8 @@ void *ttm_bo_kmap_try_from_panic(struct ttm_buffer_object *bo, unsigned long pagint ttm_bo_vmap (struct ttm_buffer_object *bo,struct iosys_map *map);void ttm_bo_vunmap (struct ttm_buffer_object *bo,struct iosys_map *map);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo); -s64 ttm_bo_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, -struct ttm_resource_manager *man,gfp_t gfp_flags, -s64 target); +s64 ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, +gfp_t gfp_flags);void ttm_bo_pin (struct ttm_buffer_object *bo);void ttm_bo_unpin (struct ttm_buffer_object *bo); int ttm_bo_evict_first(struct ttm_device *bdev,
Signature · 1 line
2.43.0
PATCH 6/12 drm/ttm: move delete handling into ttm_bo_evict
In reply to Christian KönigBoth callers do the same thing, so we can trivially unify that.
--- drivers/gpu/drm/ttm/ttm_bo.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +9 -15
@@ -363,6 +363,13 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo,struct ttm_place hop;int ret =0 ; +if (bo->deleted) { + ret =ttm_bo_wait_ctx (bo, ctx); +if (!ret) +ttm_bo_cleanup_memtype_use (bo); +return ret; + } +memset (&hop,0 ,sizeof (hop));dma_resv_assert_held (bo->base.resv);
@@ -470,13 +477,7 @@ int ttm_bo_evict_first(struct ttm_device *bdev, struct ttm_resource_manager *manif (!bo->resource || bo->resource->mem_type != mem_type) goto out_bo_moved; -if (bo->deleted) { - ret =ttm_bo_wait_ctx (bo, ctx); -if (!ret) -ttm_bo_cleanup_memtype_use (bo); - }else { - ret =ttm_bo_evict (bo, ctx); - } + ret =ttm_bo_evict (bo, ctx); out_bo_moved:dma_resv_unlock (bo->base.resv); out_no_lock:
@@ -524,14 +525,7 @@ static s64 ttm_bo_evict_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *if (bo->pin_count || !bo->bdev->funcs->eviction_valuable (bo, evict_walk->place))return 0 ; -if (bo->deleted) { - lret =ttm_bo_wait_ctx (bo, walk->arg.ctx); -if (!lret) -ttm_bo_cleanup_memtype_use (bo); - }else { - lret =ttm_bo_evict (bo, walk->arg.ctx); - } - + lret =ttm_bo_evict (bo, walk->arg.ctx);if (lret) goto out;
Signature · 1 line
2.43.0
PATCH 7/12 drm/ttm: use ttm_bo_lru_for_each_reserved_guarded in evict_all
In reply to Christian KönigUse the for_each loop to evict all BOs of an resource manager as well.
Greately simplifying the handling and finally allows us to remove ttm_bo_evict_first().
--- drivers/gpu/drm/ttm/ttm_bo.c | 51 +----------------------------- drivers/gpu/drm/ttm/ttm_resource.c | 22 +++++++++---- include/drm/ttm/ttm_bo.h | 1 + 3 files changed, 17 insertions(+), 57 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +1 -50
@@ -355,8 +355,7 @@ static int ttm_bo_bounce_temp_buffer(struct ttm_buffer_object *bo,return 0 ; } -static int ttm_bo_evict (struct ttm_buffer_object *bo, -struct ttm_operation_ctx *ctx) +int ttm_bo_evict (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx) {struct ttm_resource *evict_mem;struct ttm_placement placement;
@@ -441,54 +440,6 @@ bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, }EXPORT_SYMBOL (ttm_bo_eviction_valuable); -/** -* ttm_bo_evict_first() - Evict the first bo on the manager's LRU list. -* @bdev: The ttm device. -* @man: The manager whose bo to evict. -* @ctx: The TTM operation ctx governing the eviction. -* -* Return: 0 if successful or the resource disappeared. Negative error code on error. -*/ -int ttm_bo_evict_first (struct ttm_device *bdev,struct ttm_resource_manager *man, -struct ttm_operation_ctx *ctx) -{ -struct ttm_resource_cursor cursor; -struct ttm_buffer_object *bo; -struct ttm_resource *res; -unsigned int mem_type; -int ret =0 ; - -spin_lock (&bdev->lru_lock); -ttm_resource_cursor_init (&cursor, man); - res =ttm_resource_manager_first (&cursor); -ttm_resource_cursor_fini (&cursor); -if (!res) { - ret = -ENOENT ; - goto out_no_ref; - } - bo = res->bo; -if (!ttm_bo_get_unless_zero (bo)) - goto out_no_ref; - mem_type = res->mem_type; -spin_unlock (&bdev->lru_lock); - ret =ttm_bo_reserve (bo, ctx->interruptible, ctx->no_wait_gpu,NULL ); -if (ret) - goto out_no_lock; -if (!bo->resource || bo->resource->mem_type != mem_type) - goto out_bo_moved; - - ret =ttm_bo_evict (bo, ctx); -out_bo_moved: -dma_resv_unlock (bo->base.resv); -out_no_lock: -ttm_bo_put (bo); -return ret; - -out_no_ref: -spin_unlock (&bdev->lru_lock); -return ret; -} -/** * struct ttm_bo_evict_walk - Parameters for the evict walk. */
drivers/gpu/drm/ttm/ttm_resource.c +15 -7
@@ -561,17 +561,25 @@ EXPORT_SYMBOL(ttm_resource_manager_init);int ttm_resource_manager_evict_all (struct ttm_device *bdev,struct ttm_resource_manager *man) { -struct ttm_operation_ctx ctx = { }; +struct ttm_bo_lru_cursor cursor; +struct ttm_operation_ctx ctx = { + .interruptible = false, + .no_wait_gpu = false, + }; +struct ttm_lru_walk_arg arg = { + .ctx = &ctx, + .trylock_only = true + }; +struct ttm_buffer_object *bo;struct dma_fence *fence;int ret, i; -do { - ret =ttm_bo_evict_first (bdev, man, &ctx); +ttm_bo_lru_for_each_reserved_guarded (&cursor, man, &arg, bo) { + ret =ttm_bo_evict (bo, &ctx); +if (ret) +return ret;cond_resched (); - }while (!ret); - -if (ret && ret != -ENOENT ) -return ret; + } ret =0 ;
include/drm/ttm/ttm_bo.h +1 -0
@@ -404,6 +404,7 @@ int ttm_bo_validate(struct ttm_buffer_object *bo,void ttm_bo_fini (struct ttm_buffer_object *bo);void ttm_bo_set_bulk_move (struct ttm_buffer_object *bo,struct ttm_lru_bulk_move *bulk); +int ttm_bo_evict (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx);bool ttm_bo_eviction_valuable (struct ttm_buffer_object *bo,const struct ttm_place *place); int ttm_bo_init_reserved(struct ttm_device *bdev, struct ttm_buffer_object *bo,
Signature · 1 line
2.43.0
PATCH 8/12 drm/ttm: use dma_resv reference in ttm_device_clear_lru_dma_mappings
In reply to Christian KönigInstead of trying to grab a BO reference.
Also lock the dma_resv object, that TT unpopulate is done without holding the lock looks extremely suspicious.
Only compile tested!
--- drivers/gpu/drm/ttm/ttm_device.c | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-)
drivers/gpu/drm/ttm/ttm_device.c +17 -7
@@ -293,19 +293,29 @@ static void ttm_device_clear_lru_dma_mappings(struct ttm_device *bdev,spin_lock (&bdev->lru_lock);while ((res =ttm_lru_first_res_or_null (list))) { -struct ttm_buffer_object *bo = res->bo; +struct dma_resv *resv;/* Take ref against racing releases once lru_lock is unlocked */ -if (!ttm_bo_get_unless_zero (bo)) -continue ; + resv =dma_resv_get (res->bo->base.resv); +spin_unlock (&bdev->lru_lock); + +dma_resv_lock (resv,NULL ); -list_del_init (&bo->resource->lru.link); +/* Double check that res and bo is still valid */ +spin_lock (&bdev->lru_lock); +if (res !=ttm_lru_first_res_or_null (list)) { +dma_resv_unlock (resv); +dma_resv_put (resv); +continue ; + } +list_del_init (&res->lru.link);spin_unlock (&bdev->lru_lock); -if (bo->ttm) -ttm_tt_unpopulate (bo->bdev, bo->ttm); +if (res->bo->ttm) +ttm_tt_unpopulate (res->bo->bdev, res->bo->ttm); -ttm_bo_put (bo); +dma_resv_unlock (resv); +dma_resv_put (resv);spin_lock (&bdev->lru_lock); }spin_unlock (&bdev->lru_lock);
Signature · 1 line
2.43.0
PATCH 9/12 drm/ttm: nuke buffer refcounting
In reply to Christian KönigTTMs buffer reference is finally useles.
WIP, add more commit message
--- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 122 ++++++++++-------------- drivers/gpu/drm/ttm/ttm_bo_internal.h | 60 ------------ drivers/gpu/drm/ttm/ttm_bo_util.c | 11 +-- drivers/gpu/drm/ttm/ttm_device.c | 1 - drivers/gpu/drm/xe/xe_bo.c | 2 +- include/drm/ttm/ttm_bo.h | 22 +++-- 7 files changed, 69 insertions(+), 151 deletions(-) delete mode 100644 drivers/gpu/drm/ttm/ttm_bo_internal.h
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -1
@@ -939,7 +939,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)* Don't manipulate the TTM LRUs while in TTM bo destruction. * We're called through i915_ttm_delete_mem_notify(). */ -if (!kref_read (&bo->kref)) +if (ttm_bo_is_deleted (bo))return ;/*
drivers/gpu/drm/ttm/ttm_bo.c +51 -71
192 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_bo_internal.h deleted +0 -60
60 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_bo_util.c +4 -7
@@ -39,8 +39,6 @@#include <drm/drm_cache.h> -#include "ttm_bo_internal.h" -struct ttm_transfer_obj {struct ttm_buffer_object base;struct ttm_buffer_object *bo;
@@ -208,7 +206,7 @@ static void ttm_transfered_destroy(struct ttm_buffer_object *bo) fbo = container_of(bo, struct ttm_transfer_obj, base);dma_resv_put (&fbo->base.base._resv); -ttm_bo_put (fbo->bo); +drm_gem_object_put (&fbo->bo->base);kfree (fbo); }
@@ -247,7 +245,6 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo,atomic_inc (&ttm_glob.bo_count);drm_vma_node_reset (&fbo->base.base.vma_node); -kref_init (&fbo->base.kref); fbo->base.destroy = &ttm_transfered_destroy; fbo->base.pin_count =0 ;if (bo->type != ttm_bo_type_sg)
@@ -273,7 +270,7 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, fbo->base.bulk_move =NULL ; } -ttm_bo_get (bo); +drm_gem_object_get (&bo->base); fbo->bo = bo;ttm_bo_move_to_lru_tail_unlocked (&fbo->base);
@@ -632,7 +629,7 @@ static int ttm_bo_move_to_ghost(struct ttm_buffer_object *bo, bo->ttm =NULL ;dma_resv_unlock (&ghost_obj->base._resv); -ttm_bo_put (ghost_obj); +ttm_bo_fini (ghost_obj);return 0 ; }
@@ -809,7 +806,7 @@ int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo) }dma_resv_unlock (&ghost->base._resv); -ttm_bo_put (ghost); +ttm_bo_fini (ghost); bo->ttm = ttm;return 0 ;
drivers/gpu/drm/ttm/ttm_device.c +0 -1
@@ -38,7 +38,6 @@#include <drm/ttm/ttm_placement.h> #include "ttm_module.h" -#include "ttm_bo_internal.h" /* * ttm_global_mutex - protecting the global state
drivers/gpu/drm/xe/xe_bo.c +1 -1
@@ -1651,7 +1651,7 @@ static bool xe_ttm_bo_lock_in_destructor(struct ttm_buffer_object *ttm_bo)struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev);bool locked; -xe_assert (xe, !kref_read (&ttm_bo->kref)); +xe_assert (xe,ttm_bo_is_deleted (ttm_bo));/* * We can typically only race with TTM trylocking under the
include/drm/ttm/ttm_bo.h +12 -10
@@ -33,7 +33,6 @@#include <drm/drm_gem.h> -#include <linux/kref.h> #include <linux/list.h> #include "ttm_device.h"
@@ -78,11 +77,8 @@ enum ttm_bo_type {* @type: The bo type. * @page_alignment: Page alignment. * @destroy: Destruction function. If NULL, kfree is used. -* @kref: Reference count of this buffer object. When this refcount reaches -* zero, the object is destroyed or put on the delayed delete list. * @resource: structure describing current placement. * @ttm: TTM structure holding system pages. -* @deleted: True if the object is only a zombie and already deleted. * @bulk_move: The bulk move object. * @priority: Priority for LRU, BOs with lower priority are evicted first. * @pin_count: Pin count.
@@ -109,17 +105,11 @@ struct ttm_buffer_object {uint32_t page_alignment;void (*destroy) (struct ttm_buffer_object *); -/* -* Members not needing protection. -*/ -struct kref kref; -/* * Members protected by the bo::resv::reserved lock. */ struct ttm_resource *resource;struct ttm_tt *ttm; -bool deleted;struct ttm_lru_bulk_move *bulk_move;unsigned priority;unsigned pin_count;
@@ -378,6 +368,18 @@ static inline void ttm_bo_unreserve(struct ttm_buffer_object *bo)dma_resv_unlock (bo->base.resv); } +/** +* ttm_bo_is_deleted - test if buffer is already deleted +* @bo: the struct ttm_buffer_object to test +* +* Returns true if the buffer is already deleted and only waiting for +* destruction. +*/ +static inline bool ttm_bo_is_deleted (struct ttm_buffer_object *bo) +{ +return !kref_read (&bo->base.refcount); +} +/** * ttm_kmap_obj_virtual *
Signature · 1 line
2.43.0
Danilo Krummrich
maintainerRe: [PATCH 09/12] drm/ttm: nuke buffer refcounting
include/drm/ttm/ttm_bo.h static inline void ttm_bo_unreserve(struct ttm_buffer_object *bo) · line 381
Quoted 11 lines · On Fri Jul 10, 2026 at 8:52 PM CEST, =?UTF-8?q?Christian=20K=C3=B6nig?= wrote:
include/drm/ttm/ttm_bo.h +11
-0+/** +* ttm_bo_is_deleted - test if buffer is already deleted +* @bo: the struct ttm_buffer_object to test +* +* Returns true if the buffer is already deleted and only waiting for +* destruction. +*/ +static inline bool ttm_bo_is_deleted (struct ttm_buffer_object *bo) +{ +return !kref_read (&bo->base.refcount); +}
This is an anti-pattern, branch decisions cannot rely on an atomic reference count unless additional invariants are upheld that otherwise guarantee that the read values does not instantly become meaningless.
Something like this should at least document in which context and under which conditions it is valid to rely on the returned value.
Matthew Brost
maintainerRe: [PATCH 09/12] drm/ttm: nuke buffer refcounting
Quoted 16 lines · Danilo Krummrich
include/drm/ttm/ttm_bo.h static inline void ttm_bo_unreserve(struct ttm_buffer_object *bo) · line 381
Quoted 11 lines · On Fri Jul 10, 2026 at 8:52 PM CEST, =?UTF-8?q?Christian=20K=C3=B6nig?= wrote:
include/drm/ttm/ttm_bo.h +11
-0+/** +* ttm_bo_is_deleted - test if buffer is already deleted +* @bo: the struct ttm_buffer_object to test +* +* Returns true if the buffer is already deleted and only waiting for +* destruction. +*/ +static inline bool ttm_bo_is_deleted (struct ttm_buffer_object *bo) +{ +return !kref_read (&bo->base.refcount); +}This is an anti-pattern, branch decisions cannot rely on an atomic reference count unless additional invariants are upheld that otherwise guarantee that the read values does not instantly become meaningless.
So I believe that means the TTM refcount reaching zero is a terminal state; i.e., reinitialization (0 -> 1) is not allowed. I believe I agree with that or if reinit is allowed the check needs to change.
Matt
Quoted 3 lines
Something like this should at least document in which context and under which conditions it is valid to rely on the returned value.
PATCH 10/12 drm/exec: add drm_exec_lock_resv function
In reply to Christian KönigRestructure the drm_exec object to work with dma_resv references instead of GEM object references.
Add the new function dma_exec_lock_resv() to lock individual dma_resv objects and so allow higher level implementations to handle contention purely on dma_resv objects.
WIP! Don't commit like that!
--- drivers/gpu/drm/drm_exec.c | 75 ++++++++++++++++++++++---------------- drivers/gpu/drm/drm_gem.c | 2 + include/drm/drm_exec.h | 9 +++-- 3 files changed, 50 insertions(+), 36 deletions(-)
drivers/gpu/drm/drm_exec.c +43 -32
@@ -58,8 +58,11 @@ static void drm_exec_unlock_all(struct drm_exec *exec)drm_gem_object_put (obj); } -drm_gem_object_put (exec->prelocked); - exec->prelocked =NULL ; +if (exec->prelocked) { +dma_resv_unlock (exec->prelocked); +dma_resv_put (exec->prelocked); + exec->prelocked =NULL ; + } }/**
@@ -101,7 +104,7 @@ void drm_exec_fini(struct drm_exec *exec)drm_exec_unlock_all (exec);kvfree (exec->objects);if (exec->contended !=DRM_EXEC_DUMMY ) { -drm_gem_object_put (exec->contended); +dma_resv_put (exec->contended);ww_acquire_fini (&exec->ticket); } }
@@ -158,50 +161,41 @@ static int drm_exec_obj_locked(struct drm_exec *exec,/* Make sure the contended object is locked first */ static int drm_exec_lock_contended (struct drm_exec *exec) { -struct drm_gem_object *obj = exec->contended; +struct dma_resv *resv = exec->contended;int ret; -if (likely (!obj)) +if (likely (!resv))return 0 ;/* Always cleanup the contention so that error handling can kick in */ exec->contended =NULL ;if (exec->flags &DRM_EXEC_INTERRUPTIBLE_WAIT ) { - ret =dma_resv_lock_slow_interruptible (obj->resv, - &exec->ticket); + ret =dma_resv_lock_slow_interruptible (resv, &exec->ticket);if (unlikely (ret)) goto error_dropref; }else { -dma_resv_lock_slow (obj->resv, &exec->ticket); +dma_resv_lock_slow (resv, &exec->ticket); } - ret =drm_exec_obj_locked (exec, obj); -if (unlikely (ret)) - goto error_unlock; - - exec->prelocked = obj; + exec->prelocked = resv;return 0 ; -error_unlock: -dma_resv_unlock (obj->resv); - error_dropref: -drm_gem_object_put (obj); +dma_resv_put (resv);return ret; }/** -* drm_exec_lock_obj - lock a GEM object for use +* drm_exec_lock_resv - lock a dma_resv object * @exec: the drm_exec object with the state -* @obj: the GEM object to lock +* @resv: the dma_resv object to lock * -* Lock a GEM object for use and grab a reference to it. +* Lock a dma_resv object for use or grab a reference to it on contention. * * Returns: -EDEADLK if a contention is detected, -EALREADY when object is -* already locked (can be suppressed by setting the DRM_EXEC_IGNORE_DUPLICATES -* flag), -ENOMEM when memory allocation failed and zero for success. +* already locked, -ENOMEM when memory allocation failed and zero for success. */ -int drm_exec_lock_obj (struct drm_exec *exec,struct drm_gem_object *obj) +int drm_exec_lock_resv (struct drm_exec *exec,struct dma_resv *resv) {int ret;
@@ -209,22 +203,39 @@ int drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj)if (unlikely (ret))return ret; -if (exec->prelocked == obj) { -drm_gem_object_put (exec->prelocked); +if (exec->prelocked == resv) { +dma_resv_put (exec->prelocked); exec->prelocked =NULL ;return 0 ; }if (exec->flags &DRM_EXEC_INTERRUPTIBLE_WAIT ) - ret =dma_resv_lock_interruptible (obj->resv, &exec->ticket); + ret =dma_resv_lock_interruptible (resv, &exec->ticket);else - ret =dma_resv_lock (obj->resv, &exec->ticket); + ret =dma_resv_lock (resv, &exec->ticket); -if (unlikely (ret == -EDEADLK )) { -drm_gem_object_get (obj); - exec->contended = obj; -return -EDEADLK ; - } +if (unlikely (ret == -EDEADLK )) + exec->contended =dma_resv_get (resv); +return ret; +} +EXPORT_SYMBOL (drm_exec_lock_resv); + +/** +* drm_exec_lock_obj - lock a GEM object for use +* @exec: the drm_exec object with the state +* @obj: the GEM object to lock +* +* Lock a GEM object for use and grab a reference to it. +* +* Returns: -EDEADLK if a contention is detected, -EALREADY when object is +* already locked (can be suppressed by setting the DRM_EXEC_IGNORE_DUPLICATES +* flag), -ENOMEM when memory allocation failed and zero for success. +*/ +int drm_exec_lock_obj (struct drm_exec *exec,struct drm_gem_object *obj) +{ +int ret; + + ret =drm_exec_lock_resv (exec, obj->resv);if (unlikely (ret == -EALREADY ) && exec->flags &DRM_EXEC_IGNORE_DUPLICATES )
drivers/gpu/drm/drm_gem.c +2 -0
@@ -229,6 +229,8 @@ void drm_gem_private_object_init(struct drm_device *dev, obj->size = size;mutex_init (&obj->gpuva.lock);dma_resv_init (&obj->_resv); + +/* TODO: This needs to go away for drm_exec to work correctly!!! */ if (!obj->resv) obj->resv =dma_resv_get (&obj->_resv);
include/drm/drm_exec.h +5 -4
@@ -47,14 +47,14 @@ struct drm_exec {struct drm_gem_object **objects;/** -* @contended: contended GEM object we backed off for +* @contended: contended dma_resv object we backed off for */ -struct drm_gem_object *contended; +struct dma_resv *contended;/** -* @prelocked: already locked GEM object due to contention +* @prelocked: already locked dma_resv object due to contention */ -struct drm_gem_object *prelocked; +struct dma_resv *prelocked; };/**
@@ -175,6 +175,7 @@ static inline struct ww_acquire_ctx *drm_exec_ticket(struct drm_exec *exec)void drm_exec_init (struct drm_exec *exec,u32 flags,unsigned nr);void drm_exec_fini (struct drm_exec *exec);bool drm_exec_cleanup (struct drm_exec *exec); +int drm_exec_lock_resv (struct drm_exec *exec,struct dma_resv *resv);int drm_exec_lock_obj (struct drm_exec *exec,struct drm_gem_object *obj);void drm_exec_unlock_obj (struct drm_exec *exec,struct drm_gem_object *obj); int drm_exec_prepare_obj(struct drm_exec *exec, struct drm_gem_object *obj,
Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH 10/12] drm/exec: add drm_exec_lock_resv function
Quoted 4 lines · Christian König
Restructure the
drm_execobject to work withdma_resvreferences instead
of GEM object references.
Quoted 8 lines
Add the new function
dma_exec_lock_resv()to lock individualdma_resvobjects and so allow higher level implementations to handle contention
purely ondma_resvobjects.WIP! Don't commit like that!
Signed-off-by Christian König <[email protected]>
Here, IMO we should move a dma-resv based implementation to dma-buf to facilitate passing it also through dma_buf_map(),
And if wanting to avoid rewriting all users of drm_exec, Make drm_exec a thin wrapper on top.
Thanks,
Thomas
Quoted 215 lines
---
drivers/gpu/drm/drm_exec.c | 75 ++++++++++++++++++++++-------------- --
drivers/gpu/drm/drm_gem.c | 2 +
include/drm/drm_exec.h | 9 +++--3 files changed, 50 insertions(+), 36 deletions(-)drivers/gpu/drm/drm_exec.c +0
-0@@ -58,8 +58,11 @@ static void drm_exec_unlock_all(struct drm_exec *exec) drm_gem_object_put(obj); }drivers/gpu/drm/drm_exec.c +5
-2-drm_gem_object_put (exec->prelocked); - exec->prelocked =NULL ; +if (exec->prelocked) { +dma_resv_unlock (exec->prelocked); +dma_resv_put (exec->prelocked); + exec->prelocked =NULL ; + }}
/**drivers/gpu/drm/drm_exec.c +2
-2@@ -101,7 +104,7 @@ void drm_exec_fini(struct drm_exec *exec) drm_exec_unlock_all(exec); kvfree(exec->objects); if (exec->contended != DRM_EXEC_DUMMY) { -drm_gem_object_put (exec->contended); +dma_resv_put (exec->contended); ww_acquire_fini(&exec->ticket); } }@@ -158,50 +161,41 @@ static int drm_exec_obj_locked(struct drm_exec *exec, /* Make sure the contended object is locked first */ static int drm_exec_lock_contended(struct drm_exec *exec) { -struct drm_gem_object *obj = exec->contended; +struct dma_resv *resv = exec->contended; int ret;drivers/gpu/drm/drm_exec.c +1
-1-if (likely (!obj)) +if (likely (!resv))return 0;
/* Always cleanup the contention so that error handling can kick in */
exec->contended= NULL;
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT) {drivers/gpu/drm/drm_exec.c +0
-2- ret = dma_resv_lock_slow_interruptible(obj->resv, - &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_slow_interruptible(resv, &exec-Quoted 1 line
ticket);
if (unlikely(ret))
gotoerror_dropref;
} else {drivers/gpu/drm/drm_exec.c +1
-1-dma_resv_lock_slow (obj->resv, &exec->ticket); +dma_resv_lock_slow (resv, &exec->ticket);}
drivers/gpu/drm/drm_exec.c +1
-5- ret =drm_exec_obj_locked (exec, obj); -if (unlikely (ret)) - goto error_unlock; - - exec->prelocked = obj; + exec->prelocked = resv;return 0;
drivers/gpu/drm/drm_exec.c +0
-3-error_unlock: -dma_resv_unlock (obj->resv); -error_dropref:
drivers/gpu/drm/drm_exec.c +1
-1-drm_gem_object_put (obj); +dma_resv_put (resv);return ret;
}
/**
- *drm_exec_lock_obj- lock a GEM object for use
+ *drm_exec_lock_resv- lock adma_resvobject
* @exec: thedrm_execobject with the state
- * @obj: the GEM object to lock
+ * @resv: thedma_resvobject to lock
*
- * Lock a GEM object for use and grab a reference to it.
+ * Lock adma_resvobject for use or grab a reference to it on contention.
*
* Returns:-EDEADLKif a contention is detected,-EALREADYwhen object is
- * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
- * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ * already locked,-ENOMEMwhen memory allocation failed and zero for success.
*/
-intdrm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj)
+intdrm_exec_lock_resv(struct drm_exec *exec, struct dma_resv *resv){
int ret;drivers/gpu/drm/drm_exec.c +0
-0@@ -209,22 +203,39 @@ int drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj) if (unlikely(ret)) return ret;drivers/gpu/drm/drm_exec.c +2
-2-if (exec->prelocked == obj) { -drm_gem_object_put (exec->prelocked); +if (exec->prelocked == resv) { +dma_resv_put (exec->prelocked);
exec->prelocked= NULL; return 0;
}
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT)drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_interruptible(obj->resv, &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_interruptible(resv, &exec-Quoted 1 line
ticket);
else
drivers/gpu/drm/drm_exec.c +1
-1- ret =dma_resv_lock (obj->resv, &exec->ticket); + ret =dma_resv_lock (resv, &exec->ticket);drivers/gpu/drm/drm_exec.c +14
-5-if (unlikely (ret == -EDEADLK )) { -drm_gem_object_get (obj); - exec->contended = obj; -return -EDEADLK ; - } +if (unlikely (ret == -EDEADLK )) + exec->contended =dma_resv_get (resv); +return ret; +} +EXPORT_SYMBOL (drm_exec_lock_resv); + +/** +* drm_exec_lock_obj - lock a GEM object for use +* @exec: the drm_exec object with the state +* @obj: the GEM object to lock +* +* Lock a GEM object for use and grab a reference to it. +* +* Returns: -EDEADLK if a contention is detected, -EALREADY when object is
+ * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
+ * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ */
+intdrm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj)drivers/gpu/drm/drm_exec.c +4
-0+{ +int ret; + + ret =drm_exec_lock_resv (exec, obj->resv);
if (unlikely(ret ==-EALREADY) &&exec->flags&DRM_EXEC_IGNORE_DUPLICATES)drivers/gpu/drm/drm_gem.c +2
-0@@ -229,6 +229,8 @@ void drm_gem_private_object_init(structdrm_device *dev, obj->size = size; mutex_init(&obj->gpuva.lock); dma_resv_init(&obj->_resv); + +/* TODO: This needs to go away for drm_exec to work correctly!!! */
if (!obj->resv)
obj->resv=dma_resv_get(&obj->_resv);include/drm/drm_exec.h +0
-0@@ -47,14 +47,14 @@ struct drm_exec { struct drm_gem_object **objects;
/**include/drm/drm_exec.h +1
-1-* @contended: contended GEM object we backed off for +* @contended: contended dma_resv object we backed off for */
include/drm/drm_exec.h +1
-1-struct drm_gem_object *contended; +struct dma_resv *contended;
/**include/drm/drm_exec.h +1
-1-* @prelocked: already locked GEM object due to contention +* @prelocked: already locked dma_resv object due to contention
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *prelocked; +struct dma_resv *prelocked;};
/**include/drm/drm_exec.h +1
-0@@ -175,6 +175,7 @@ static inline struct ww_acquire_ctx *drm_exec_ticket(struct drm_exec *exec) void drm_exec_init(struct drm_exec *exec, u32 flags, unsigned nr); void drm_exec_fini(struct drm_exec *exec); bool drm_exec_cleanup(struct drm_exec *exec); +int drm_exec_lock_resv (struct drm_exec *exec,struct dma_resv *resv);int
drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj);
voiddrm_exec_unlock_obj(struct drm_exec *exec, struct drm_gem_object *obj);
int drm_exec_prepare_obj(structdrm_exec*exec, structdrm_gem_object*obj,
Christian König
maintainerRe: [PATCH 10/12] drm/exec: add drm_exec_lock_resv function
Quoted 10 lines · Thomas Hellström
Quoted 9 lines · Christian König
Restructure the
drm_execobject to work withdma_resvreferences instead
of GEM object references.Add the new function
dma_exec_lock_resv()to lock individualdma_resvobjects and so allow higher level implementations to handle contention
purely ondma_resvobjects.
Quoted 6 lines
Quoted 3 lines
WIP! Don't commit like that!
Signed-off-by Christian König <[email protected]>Here, IMO we should move a dma-resv based implementation to dma-buf to facilitate passing it also through
dma_buf_map(),
I still don't see why that would be necessary?
dma_buf_map() just maps the current location of the buffer, it has no requirement to force the buffer into VRAM.
At least on amdgpu we always validate buffer during dma_buf_map() with VRAM|GTT, so we never cause any eviction at all.
Quoted 2 lines
And if wanting to avoid rewriting all users of
drm_exec, Makedrm_execa thin wrapper on top.
DMA-buf looks like the wrong place for this since it only works on exported buffers and that should be the absolute minority.
We could have a dma-resv contention tracking helper, but I still don't see for what that would be good for?
Regards,
Christian.
Quoted 221 lines
Thanks,
ThomasQuoted 215 lines
---
drivers/gpu/drm/drm_exec.c | 75 ++++++++++++++++++++++-------------- --
drivers/gpu/drm/drm_gem.c | 2 +
include/drm/drm_exec.h | 9 +++--3 files changed, 50 insertions(+), 36 deletions(-)drivers/gpu/drm/drm_exec.c +0
-0@@ -58,8 +58,11 @@ static void drm_exec_unlock_all(struct drm_exec *exec) drm_gem_object_put(obj); }drivers/gpu/drm/drm_exec.c +5
-2-drm_gem_object_put (exec->prelocked); - exec->prelocked =NULL ; +if (exec->prelocked) { +dma_resv_unlock (exec->prelocked); +dma_resv_put (exec->prelocked); + exec->prelocked =NULL ; + }}
/**drivers/gpu/drm/drm_exec.c +2
-2@@ -101,7 +104,7 @@ void drm_exec_fini(struct drm_exec *exec) drm_exec_unlock_all(exec); kvfree(exec->objects); if (exec->contended != DRM_EXEC_DUMMY) { -drm_gem_object_put (exec->contended); +dma_resv_put (exec->contended); ww_acquire_fini(&exec->ticket); } }@@ -158,50 +161,41 @@ static int drm_exec_obj_locked(struct drm_exec *exec, /* Make sure the contended object is locked first */ static int drm_exec_lock_contended(struct drm_exec *exec) { -struct drm_gem_object *obj = exec->contended; +struct dma_resv *resv = exec->contended; int ret;drivers/gpu/drm/drm_exec.c +1
-1-if (likely (!obj)) +if (likely (!resv))return 0;
/* Always cleanup the contention so that error handling can kick in */
exec->contended= NULL;
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT) {drivers/gpu/drm/drm_exec.c +0
-2- ret = dma_resv_lock_slow_interruptible(obj->resv, - &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_slow_interruptible(resv, &exec-Quoted 1 line
ticket);
if (unlikely(ret))
gotoerror_dropref;
} else {drivers/gpu/drm/drm_exec.c +1
-1-dma_resv_lock_slow (obj->resv, &exec->ticket); +dma_resv_lock_slow (resv, &exec->ticket);}
drivers/gpu/drm/drm_exec.c +1
-5- ret =drm_exec_obj_locked (exec, obj); -if (unlikely (ret)) - goto error_unlock; - - exec->prelocked = obj; + exec->prelocked = resv;return 0;
drivers/gpu/drm/drm_exec.c +0
-3-error_unlock: -dma_resv_unlock (obj->resv); -error_dropref:
drivers/gpu/drm/drm_exec.c +1
-1-drm_gem_object_put (obj); +dma_resv_put (resv);return ret;
}
/**
- *drm_exec_lock_obj- lock a GEM object for use
+ *drm_exec_lock_resv- lock adma_resvobject
* @exec: thedrm_execobject with the state
- * @obj: the GEM object to lock
+ * @resv: thedma_resvobject to lock
*
- * Lock a GEM object for use and grab a reference to it.
+ * Lock adma_resvobject for use or grab a reference to it on contention.
*
* Returns:-EDEADLKif a contention is detected,-EALREADYwhen object is
- * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
- * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ * already locked,-ENOMEMwhen memory allocation failed and zero for success.
*/
-intdrm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj)
+intdrm_exec_lock_resv(struct drm_exec *exec, struct dma_resv *resv){
int ret;drivers/gpu/drm/drm_exec.c +0
-0@@ -209,22 +203,39 @@ int drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj) if (unlikely(ret)) return ret;drivers/gpu/drm/drm_exec.c +2
-2-if (exec->prelocked == obj) { -drm_gem_object_put (exec->prelocked); +if (exec->prelocked == resv) { +dma_resv_put (exec->prelocked);
exec->prelocked= NULL; return 0;
}
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT)drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_interruptible(obj->resv, &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_interruptible(resv, &exec-Quoted 1 line
ticket);
else
drivers/gpu/drm/drm_exec.c +1
-1- ret =dma_resv_lock (obj->resv, &exec->ticket); + ret =dma_resv_lock (resv, &exec->ticket);drivers/gpu/drm/drm_exec.c +14
-5-if (unlikely (ret == -EDEADLK )) { -drm_gem_object_get (obj); - exec->contended = obj; -return -EDEADLK ; - } +if (unlikely (ret == -EDEADLK )) + exec->contended =dma_resv_get (resv); +return ret; +} +EXPORT_SYMBOL (drm_exec_lock_resv); + +/** +* drm_exec_lock_obj - lock a GEM object for use +* @exec: the drm_exec object with the state +* @obj: the GEM object to lock +* +* Lock a GEM object for use and grab a reference to it. +* +* Returns: -EDEADLK if a contention is detected, -EALREADY when object is
+ * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
+ * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ */
+intdrm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj)drivers/gpu/drm/drm_exec.c +4
-0+{ +int ret; + + ret =drm_exec_lock_resv (exec, obj->resv);
if (unlikely(ret ==-EALREADY) &&exec->flags&DRM_EXEC_IGNORE_DUPLICATES)drivers/gpu/drm/drm_gem.c +2
-0@@ -229,6 +229,8 @@ void drm_gem_private_object_init(structdrm_device *dev, obj->size = size; mutex_init(&obj->gpuva.lock); dma_resv_init(&obj->_resv); + +/* TODO: This needs to go away for drm_exec to work correctly!!! */
if (!obj->resv)
obj->resv=dma_resv_get(&obj->_resv);include/drm/drm_exec.h +0
-0@@ -47,14 +47,14 @@ struct drm_exec { struct drm_gem_object **objects;
/**include/drm/drm_exec.h +1
-1-* @contended: contended GEM object we backed off for +* @contended: contended dma_resv object we backed off for */
include/drm/drm_exec.h +1
-1-struct drm_gem_object *contended; +struct dma_resv *contended;
/**include/drm/drm_exec.h +1
-1-* @prelocked: already locked GEM object due to contention +* @prelocked: already locked dma_resv object due to contention
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *prelocked; +struct dma_resv *prelocked;};
/**include/drm/drm_exec.h +1
-0@@ -175,6 +175,7 @@ static inline struct ww_acquire_ctx *drm_exec_ticket(struct drm_exec *exec) void drm_exec_init(struct drm_exec *exec, u32 flags, unsigned nr); void drm_exec_fini(struct drm_exec *exec); bool drm_exec_cleanup(struct drm_exec *exec); +int drm_exec_lock_resv (struct drm_exec *exec,struct dma_resv *resv);int
drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj);
voiddrm_exec_unlock_obj(struct drm_exec *exec, struct drm_gem_object *obj);
int drm_exec_prepare_obj(structdrm_exec*exec, structdrm_gem_object*obj,
Thomas Hellström
maintainerRe: [PATCH 10/12] drm/exec: add drm_exec_lock_resv function
Quoted 20 lines · Christian König
Quoted 11 lines · Thomas Hellström
Quoted 10 lines · Christian König
Restructure the
drm_execobject to work withdma_resvreferences instead
of GEM object references.Add the new function
dma_exec_lock_resv()to lock individualdma_resv
objects and so allow higher level implementations to handle contention
purely ondma_resvobjects.Quoted 7 lines
Quoted 3 lines
WIP! Don't commit like that!
Signed-off-by Christian König <[email protected]>Here, IMO we should move a dma-resv based implementation to dma-buf to
facilitate passing it also throughdma_buf_map(),
Quoted 7 lines
I still don't see why that would be necessary?
dma_buf_map()just maps the current location of the buffer, it has no requirement to force the buffer into VRAM.At least on amdgpu we always validate buffer during
dma_buf_map()with VRAM|GTT, so we never cause any eviction at all.
For fast interconnects xe wants to avoid pinning in VRAM and needs a more aggressive validation. If you have a bunch of processes using WW transactions to lock out others from VRAM allocation with one process not participating that wouldn't work out well.
Quoted 10 lines
Quoted 3 lines
And if wanting to avoid rewriting all users of
drm_exec, Makedrm_exec
a thin wrapper on top.DMA-buf looks like the wrong place for this since it only works on exported buffers and that should be the absolute minority.
We could have a dma-resv contention tracking helper, but I still don't see for what that would be good for?
See the above. The problem is that the WW transaction always starts with a drm_exec on the importer side and it needs to be the same structure that holds the contended lock on rollback.
What are the issues you are seeing?
/Thomas
Quoted 243 lines
Regards,
Christian.Quoted 239 lines
Thanks,
ThomasQuoted 233 lines
---
drivers/gpu/drm/drm_exec.c | 75 ++++++++++++++++++++++----------
----
--
drivers/gpu/drm/drm_gem.c | 2 + include/drm/drm_exec.h | 9 +++--3 files changed, 50 insertions(+), 36 deletions(-)drivers/gpu/drm/drm_exec.c +0
-0@@ -58,8 +58,11 @@ static void drm_exec_unlock_all(struct
drm_exec
*exec)
drm_gem_object_put(obj);
}drivers/gpu/drm/drm_exec.c +5
-2-drm_gem_object_put (exec->prelocked); - exec->prelocked =NULL ; +if (exec->prelocked) { +dma_resv_unlock (exec->prelocked); +dma_resv_put (exec->prelocked); + exec->prelocked =NULL ; + }}
/**drivers/gpu/drm/drm_exec.c +1
-1@@ -101,7 +104,7 @@ void drm_exec_fini(struct drm_exec *exec) drm_exec_unlock_all(exec); kvfree(exec->objects); if (exec->contended != DRM_EXEC_DUMMY) { -drm_gem_object_put (exec->contended); +dma_resv_put (exec->contended); ww_acquire_fini(&exec->ticket); } }@@ -158,50 +161,41 @@ static int drm_exec_obj_locked(struct
drm_exec
*exec,
/* Make sure the contended object is locked first */ static intdrm_exec_lock_contended(struct drm_exec *exec){drivers/gpu/drm/drm_exec.c +1
-1-struct drm_gem_object *obj = exec->contended; +struct dma_resv *resv = exec->contended;int ret;
drivers/gpu/drm/drm_exec.c +1
-1-if (likely (!obj)) +if (likely (!resv))return 0;
/* Always cleanup the contention so that error handling can
kick in */
exec->contended= NULL;
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT) {drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_slow_interruptible(obj-Quoted 1 line
resv,
drivers/gpu/drm/drm_exec.c +0
-1- &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_slow_interruptible(resv,&exec-
Quoted 1 line
ticket);
if (unlikely(ret))
gotoerror_dropref;
} else {drivers/gpu/drm/drm_exec.c +1
-1-dma_resv_lock_slow (obj->resv, &exec->ticket); +dma_resv_lock_slow (resv, &exec->ticket);}
drivers/gpu/drm/drm_exec.c +1
-5- ret =drm_exec_obj_locked (exec, obj); -if (unlikely (ret)) - goto error_unlock; - - exec->prelocked = obj; + exec->prelocked = resv;return 0;
drivers/gpu/drm/drm_exec.c +0
-3-error_unlock: -dma_resv_unlock (obj->resv); -error_dropref:
drivers/gpu/drm/drm_exec.c +1
-1-drm_gem_object_put (obj); +dma_resv_put (resv);return ret;
}
/**
- *drm_exec_lock_obj- lock a GEM object for use
+ *drm_exec_lock_resv- lock adma_resvobject
* @exec: thedrm_execobject with the state
- * @obj: the GEM object to lock
+ * @resv: thedma_resvobject to lock
*
- * Lock a GEM object for use and grab a reference to it.
+ * Lock adma_resvobject for use or grab a reference to it on contention.
*
* Returns:-EDEADLKif a contention is detected,-EALREADYwhen object is
- * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
- * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ * already locked,-ENOMEMwhen memory allocation failed and zero
for success.
*/
-int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj)
+intdrm_exec_lock_resv(struct drm_exec *exec, struct dma_resv *resv)
{
int ret;drivers/gpu/drm/drm_exec.c +0
-0@@ -209,22 +203,39 @@ int drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj) if (unlikely(ret)) return ret;drivers/gpu/drm/drm_exec.c +2
-2-if (exec->prelocked == obj) { -drm_gem_object_put (exec->prelocked); +if (exec->prelocked == resv) { +dma_resv_put (exec->prelocked);
exec->prelocked= NULL; return 0;
}
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT)drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_interruptible(obj->resv,&exec-
Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_interruptible(resv, &exec-Quoted 1 line
ticket);
else
drivers/gpu/drm/drm_exec.c +1
-1- ret =dma_resv_lock (obj->resv, &exec->ticket); + ret =dma_resv_lock (resv, &exec->ticket);drivers/gpu/drm/drm_exec.c +14
-5-if (unlikely (ret == -EDEADLK )) { -drm_gem_object_get (obj); - exec->contended = obj; -return -EDEADLK ; - } +if (unlikely (ret == -EDEADLK )) + exec->contended =dma_resv_get (resv); +return ret; +} +EXPORT_SYMBOL (drm_exec_lock_resv); + +/** +* drm_exec_lock_obj - lock a GEM object for use +* @exec: the drm_exec object with the state +* @obj: the GEM object to lock +* +* Lock a GEM object for use and grab a reference to it. +* +* Returns: -EDEADLK if a contention is detected, -EALREADY when object is
+ * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
+ * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ */
+int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj)drivers/gpu/drm/drm_exec.c +4
-0+{ +int ret; + + ret =drm_exec_lock_resv (exec, obj->resv);
if (unlikely(ret ==-EALREADY) &&exec->flags&DRM_EXEC_IGNORE_DUPLICATES)drivers/gpu/drm/drm_gem.c +2
-0@@ -229,6 +229,8 @@ void drm_gem_private_object_init(structdrm_device *dev, obj->size = size; mutex_init(&obj->gpuva.lock); dma_resv_init(&obj->_resv); + +/* TODO: This needs to go away for drm_exec to work correctly!!! */
if (!obj->resv)
obj->resv=dma_resv_get(&obj->_resv);include/drm/drm_exec.h +0
-0@@ -47,14 +47,14 @@ struct drm_exec { struct drm_gem_object **objects;
/**include/drm/drm_exec.h +1
-1-* @contended: contended GEM object we backed off for +* @contended: contended dma_resv object we backed off for
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *contended; +struct dma_resv *contended;
/**include/drm/drm_exec.h +0
-1-* @prelocked: already locked GEM object due to contention
include/drm/drm_exec.h +1
-0+* @prelocked: already locked dma_resv object due to contention
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *prelocked; +struct dma_resv *prelocked;};
/**include/drm/drm_exec.h +0
-0@@ -175,6 +175,7 @@ static inline struct ww_acquire_ctx *drm_exec_ticket(struct drm_exec *exec)void
drm_exec_init(struct drm_exec *exec, u32 flags, unsigned nr);
voiddrm_exec_fini(struct drm_exec *exec);
booldrm_exec_cleanup(struct drm_exec *exec);
+intdrm_exec_lock_resv(struct drm_exec *exec, struct dma_resv *resv);
int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj);
voiddrm_exec_unlock_obj(struct drm_exec *exec, struct drm_gem_object *obj);
int drm_exec_prepare_obj(structdrm_exec*exec, structdrm_gem_object*obj,
Christian König
maintainerRe: [PATCH 10/12] drm/exec: add drm_exec_lock_resv function
Quoted 42 lines · Thomas Hellström
Quoted 20 lines · Christian König
Quoted 11 lines · Thomas Hellström
Quoted 10 lines · Christian König
Restructure the
drm_execobject to work withdma_resvreferences instead
of GEM object references.Add the new function
dma_exec_lock_resv()to lock individualdma_resv
objects and so allow higher level implementations to handle contention
purely ondma_resvobjects.Quoted 7 lines
Quoted 3 lines
WIP! Don't commit like that!
Signed-off-by Christian König <[email protected]>Here, IMO we should move a dma-resv based implementation to dma-buf to
facilitate passing it also throughdma_buf_map(),Quoted 7 lines
I still don't see why that would be necessary?
dma_buf_map()just maps the current location of the buffer, it has no requirement to force the buffer into VRAM.At least on amdgpu we always validate buffer during
dma_buf_map()with VRAM|GTT, so we never cause any eviction at all.For fast interconnects xe wants to avoid pinning in VRAM and needs a more aggressive validation. If you have a bunch of processes using WW transactions to lock out others from VRAM allocation with one process not participating that wouldn't work out well.
Quoted 8 lines
Quoted 3 lines
And if wanting to avoid rewriting all users of
drm_exec, Makedrm_exec
a thin wrapper on top.DMA-buf looks like the wrong place for this since it only works on exported buffers and that should be the absolute minority.
Quoted 8 lines
Quoted 2 lines
We could have a dma-resv contention tracking helper, but I still don't see for what that would be good for?
See the above. The problem is that the WW transaction always starts with a
drm_execon the importer side and it needs to be the same structure that holds the contended lock on rollback.What are the issues you are seeing?
Mhm, there had been quite some discussion on that topic in the past and so far the result was usually that this shouldn't be necessary.
On the other hand with a reference counted dma_resv object it should be technically doable and the past discussions have already been years ago with quite a bunch of different requirements.
For dma_buf_map() I still don't see the use case, cause that function is really "give me access to the current location of the buffer" it should not move the buffer. If the exporter needs the buffer in a different location later on it can still do so.
But for dma_buf_pin() it might be perfectly valid to kick out other buffers and then I agree that definately needs to ww locking context.
Regards,
Christian.
Quoted 247 lines
/Thomas
Quoted 243 lines
Regards,
Christian.Quoted 239 lines
Thanks,
ThomasQuoted 233 lines
---
drivers/gpu/drm/drm_exec.c | 75 ++++++++++++++++++++++----------
----
--
drivers/gpu/drm/drm_gem.c | 2 + include/drm/drm_exec.h | 9 +++--3 files changed, 50 insertions(+), 36 deletions(-)drivers/gpu/drm/drm_exec.c +0
-0@@ -58,8 +58,11 @@ static void drm_exec_unlock_all(struct
drm_exec
*exec)
drm_gem_object_put(obj);
}drivers/gpu/drm/drm_exec.c +5
-2-drm_gem_object_put (exec->prelocked); - exec->prelocked =NULL ; +if (exec->prelocked) { +dma_resv_unlock (exec->prelocked); +dma_resv_put (exec->prelocked); + exec->prelocked =NULL ; + }}
/**drivers/gpu/drm/drm_exec.c +1
-1@@ -101,7 +104,7 @@ void drm_exec_fini(struct drm_exec *exec) drm_exec_unlock_all(exec); kvfree(exec->objects); if (exec->contended != DRM_EXEC_DUMMY) { -drm_gem_object_put (exec->contended); +dma_resv_put (exec->contended); ww_acquire_fini(&exec->ticket); } }@@ -158,50 +161,41 @@ static int drm_exec_obj_locked(struct
drm_exec
*exec,
/* Make sure the contended object is locked first */ static intdrm_exec_lock_contended(struct drm_exec *exec){drivers/gpu/drm/drm_exec.c +1
-1-struct drm_gem_object *obj = exec->contended; +struct dma_resv *resv = exec->contended;int ret;
drivers/gpu/drm/drm_exec.c +1
-1-if (likely (!obj)) +if (likely (!resv))return 0;
/* Always cleanup the contention so that error handling can
kick in */
exec->contended= NULL;
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT) {drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_slow_interruptible(obj-Quoted 1 line
resv,
drivers/gpu/drm/drm_exec.c +0
-1- &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_slow_interruptible(resv,&exec-
Quoted 1 line
ticket);
if (unlikely(ret))
gotoerror_dropref;
} else {drivers/gpu/drm/drm_exec.c +1
-1-dma_resv_lock_slow (obj->resv, &exec->ticket); +dma_resv_lock_slow (resv, &exec->ticket);}
drivers/gpu/drm/drm_exec.c +1
-5- ret =drm_exec_obj_locked (exec, obj); -if (unlikely (ret)) - goto error_unlock; - - exec->prelocked = obj; + exec->prelocked = resv;return 0;
drivers/gpu/drm/drm_exec.c +0
-3-error_unlock: -dma_resv_unlock (obj->resv); -error_dropref:
drivers/gpu/drm/drm_exec.c +1
-1-drm_gem_object_put (obj); +dma_resv_put (resv);return ret;
}
/**
- *drm_exec_lock_obj- lock a GEM object for use
+ *drm_exec_lock_resv- lock adma_resvobject
* @exec: thedrm_execobject with the state
- * @obj: the GEM object to lock
+ * @resv: thedma_resvobject to lock
*
- * Lock a GEM object for use and grab a reference to it.
+ * Lock adma_resvobject for use or grab a reference to it on contention.
*
* Returns:-EDEADLKif a contention is detected,-EALREADYwhen object is
- * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
- * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ * already locked,-ENOMEMwhen memory allocation failed and zero
for success.
*/
-int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj)
+intdrm_exec_lock_resv(struct drm_exec *exec, struct dma_resv *resv)
{
int ret;drivers/gpu/drm/drm_exec.c +0
-0@@ -209,22 +203,39 @@ int drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj) if (unlikely(ret)) return ret;drivers/gpu/drm/drm_exec.c +2
-2-if (exec->prelocked == obj) { -drm_gem_object_put (exec->prelocked); +if (exec->prelocked == resv) { +dma_resv_put (exec->prelocked);
exec->prelocked= NULL; return 0;
}
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT)drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_interruptible(obj->resv,&exec-
Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_interruptible(resv, &exec-Quoted 1 line
ticket);
else
drivers/gpu/drm/drm_exec.c +1
-1- ret =dma_resv_lock (obj->resv, &exec->ticket); + ret =dma_resv_lock (resv, &exec->ticket);drivers/gpu/drm/drm_exec.c +14
-5-if (unlikely (ret == -EDEADLK )) { -drm_gem_object_get (obj); - exec->contended = obj; -return -EDEADLK ; - } +if (unlikely (ret == -EDEADLK )) + exec->contended =dma_resv_get (resv); +return ret; +} +EXPORT_SYMBOL (drm_exec_lock_resv); + +/** +* drm_exec_lock_obj - lock a GEM object for use +* @exec: the drm_exec object with the state +* @obj: the GEM object to lock +* +* Lock a GEM object for use and grab a reference to it. +* +* Returns: -EDEADLK if a contention is detected, -EALREADY when object is
+ * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
+ * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ */
+int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj)drivers/gpu/drm/drm_exec.c +4
-0+{ +int ret; + + ret =drm_exec_lock_resv (exec, obj->resv);
if (unlikely(ret ==-EALREADY) &&exec->flags&DRM_EXEC_IGNORE_DUPLICATES)drivers/gpu/drm/drm_gem.c +2
-0@@ -229,6 +229,8 @@ void drm_gem_private_object_init(structdrm_device *dev, obj->size = size; mutex_init(&obj->gpuva.lock); dma_resv_init(&obj->_resv); + +/* TODO: This needs to go away for drm_exec to work correctly!!! */
if (!obj->resv)
obj->resv=dma_resv_get(&obj->_resv);include/drm/drm_exec.h +0
-0@@ -47,14 +47,14 @@ struct drm_exec { struct drm_gem_object **objects;
/**include/drm/drm_exec.h +1
-1-* @contended: contended GEM object we backed off for +* @contended: contended dma_resv object we backed off for
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *contended; +struct dma_resv *contended;
/**include/drm/drm_exec.h +0
-1-* @prelocked: already locked GEM object due to contention
include/drm/drm_exec.h +1
-0+* @prelocked: already locked dma_resv object due to contention
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *prelocked; +struct dma_resv *prelocked;};
/**include/drm/drm_exec.h +0
-0@@ -175,6 +175,7 @@ static inline struct ww_acquire_ctx *drm_exec_ticket(struct drm_exec *exec)void
drm_exec_init(struct drm_exec *exec, u32 flags, unsigned nr);
voiddrm_exec_fini(struct drm_exec *exec);
booldrm_exec_cleanup(struct drm_exec *exec);
+intdrm_exec_lock_resv(struct drm_exec *exec, struct dma_resv *resv);
int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj);
voiddrm_exec_unlock_obj(struct drm_exec *exec, struct drm_gem_object *obj);
int drm_exec_prepare_obj(structdrm_exec*exec, structdrm_gem_object*obj,
Thomas Hellström
maintainerRe: [PATCH 10/12] drm/exec: add drm_exec_lock_resv function
Quoted 57 lines · Christian König
Quoted 56 lines · Thomas Hellström
Quoted 22 lines · Christian König
Quoted 12 lines · Thomas Hellström
Quoted 11 lines · Christian König
Restructure the
drm_execobject to work withdma_resvreferences
instead
of GEM object references.Add the new function
dma_exec_lock_resv()to lock individualdma_resv
objects and so allow higher level implementations to handle contention
purely ondma_resvobjects.Quoted 8 lines
Quoted 3 lines
WIP! Don't commit like that!
Signed-off-by Christian König <[email protected]>Here, IMO we should move a dma-resv based implementation to dma-buf
to
facilitate passing it also throughdma_buf_map(),Quoted 8 lines
I still don't see why that would be necessary?
dma_buf_map()just maps the current location of the buffer, it has no
requirement to force the buffer into VRAM.At least on amdgpu we always validate buffer during
dma_buf_map()with VRAM|GTT, so we never cause any eviction at all.For fast interconnects xe wants to avoid pinning in VRAM and needs a
more aggressive validation. If you have a bunch of processes using WW
transactions to lock out others from VRAM allocation with one process
not participating that wouldn't work out well.Quoted 11 lines
Quoted 3 lines
And if wanting to avoid rewriting all users of
drm_exec, Makedrm_exec
a thin wrapper on top.DMA-buf looks like the wrong place for this since it only works on
exported buffers and that should be the absolute minority.We could have a dma-resv contention tracking helper, but I still don't see for what that would be good for?
See the above. The problem is that the WW transaction always starts with a
drm_execon the importer side and it needs to be the same structure that holds the contended lock on rollback.
Quoted 8 lines
Quoted 1 line
What are the issues you are seeing?
Mhm, there had been quite some discussion on that topic in the past and so far the result was usually that this shouldn't be necessary.
On the other hand with a reference counted
dma_resvobject it should be technically doable and the past discussions have already been years ago with quite a bunch of different requirements.
Yeah, IIRC the main argument was that for various reasons neither the WW mutex nor the dma_resv had a refcount, so either one had to use an ops (like my rfc) or attach a refcount to the drm_resv or ww_mutex, like the series under discussion.
Quoted 8 lines
For
dma_buf_map()I still don't see the use case, cause that function is really "give me access to the current location of the buffer" it should not move the buffer. If the exporter needs the buffer in a different location later on it can still do so.But for
dma_buf_pin()it might be perfectly valid to kick out other buffers and then I agree that definately needs to ww locking context.
OK, so xe has always seen the dma_buf_map() as the dma-buf equivalent of ttm_bo_validate() + dma_mapping. The main reason for that is the pairing with what was previously known as move_notify. If the map was invalidated, you'd release it and when you needed it again, you called dma_buf_map(), and if p2p was requested, the exporter moved the dma_buf to vram if all attachments supported it.
Pinning would create the awkwardness of having to pin and then immediately unpin again.
But regardless of what dma_buf interface we use for this, or perhaps agree on a new interface, the functionality will be needed, and since with refcounted dma_resv, this is more of a move than a reimplementation, I think we should do it now or agree to do it as a follow-up.
Thanks,
Thomas
Quoted 265 lines
Regards,
Christian.Quoted 261 lines
/Thomas
Quoted 257 lines
Regards,
Christian.Quoted 253 lines
Thanks,
ThomasQuoted 247 lines
---
drivers/gpu/drm/drm_exec.c | 75 ++++++++++++++++++++++------
----
----
--
drivers/gpu/drm/drm_gem.c | 2 + include/drm/drm_exec.h | 9 +++--3 files changed, 50 insertions(+), 36 deletions(-)drivers/gpu/drm/drm_exec.c +0
-0@@ -58,8 +58,11 @@ static void drm_exec_unlock_all(struct
drm_exec
*exec)
drm_gem_object_put(obj);
}drivers/gpu/drm/drm_exec.c +5
-2-drm_gem_object_put (exec->prelocked); - exec->prelocked =NULL ; +if (exec->prelocked) { +dma_resv_unlock (exec->prelocked); +dma_resv_put (exec->prelocked); + exec->prelocked =NULL ; + }}
/**drivers/gpu/drm/drm_exec.c +1
-1@@ -101,7 +104,7 @@ void drm_exec_fini(struct drm_exec *exec) drm_exec_unlock_all(exec); kvfree(exec->objects); if (exec->contended != DRM_EXEC_DUMMY) { -drm_gem_object_put (exec->contended); +dma_resv_put (exec->contended); ww_acquire_fini(&exec->ticket); } }@@ -158,50 +161,41 @@ static int drm_exec_obj_locked(struct
drm_exec
*exec,
/* Make sure the contended object is locked first */ static intdrm_exec_lock_contended(struct drm_exec *exec){drivers/gpu/drm/drm_exec.c +1
-1-struct drm_gem_object *obj = exec->contended; +struct dma_resv *resv = exec->contended;int ret;
drivers/gpu/drm/drm_exec.c +1
-1-if (likely (!obj)) +if (likely (!resv))return 0;
/* Always cleanup the contention so that error handling
can
kick in */
exec->contended= NULL;
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT) {drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_slow_interruptible(obj-Quoted 1 line
resv,
drivers/gpu/drm/drm_exec.c +0
-1-&exec-
Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_slow_interruptible(resv,&exec-
Quoted 1 line
ticket);
if (unlikely(ret))
gotoerror_dropref;
} else {drivers/gpu/drm/drm_exec.c +0
-1- dma_resv_lock_slow(obj->resv, &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+dma_resv_lock_slow (resv, &exec->ticket);}
drivers/gpu/drm/drm_exec.c +1
-5- ret =drm_exec_obj_locked (exec, obj); -if (unlikely (ret)) - goto error_unlock; - - exec->prelocked = obj; + exec->prelocked = resv;return 0;
drivers/gpu/drm/drm_exec.c +0
-3-error_unlock: -dma_resv_unlock (obj->resv); -error_dropref:
drivers/gpu/drm/drm_exec.c +1
-1-drm_gem_object_put (obj); +dma_resv_put (resv);return ret;
}
/**
- *drm_exec_lock_obj- lock a GEM object for use
+ *drm_exec_lock_resv- lock adma_resvobject
* @exec: thedrm_execobject with the state
- * @obj: the GEM object to lock
+ * @resv: thedma_resvobject to lock
*
- * Lock a GEM object for use and grab a reference to it.
+ * Lock adma_resvobject for use or grab a reference to it on
contention.
*
* Returns:-EDEADLKif a contention is detected,-EALREADYwhen
object is
- * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
- * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ * already locked,-ENOMEMwhen memory allocation failed and zero
for success.
*/
-int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj)
+int drm_exec_lock_resv(structdrm_exec*exec, structdma_resv
*resv)
{
int ret;drivers/gpu/drm/drm_exec.c +0
-0@@ -209,22 +203,39 @@ int drm_exec_lock_obj(struct drm_exec *exec, struct drm_gem_object *obj) if (unlikely(ret)) return ret;drivers/gpu/drm/drm_exec.c +2
-2-if (exec->prelocked == obj) { -drm_gem_object_put (exec->prelocked); +if (exec->prelocked == resv) { +dma_resv_put (exec->prelocked);
exec->prelocked= NULL; return 0;
}
if (exec->flags&DRM_EXEC_INTERRUPTIBLE_WAIT)drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock_interruptible(obj->resv,&exec-
Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret = dma_resv_lock_interruptible(resv,&exec-
Quoted 1 line
ticket);
else
drivers/gpu/drm/drm_exec.c +0
-1- ret = dma_resv_lock(obj->resv, &exec-Quoted 1 line
ticket);
drivers/gpu/drm/drm_exec.c +1
-0+ ret =dma_resv_lock (resv, &exec->ticket);drivers/gpu/drm/drm_exec.c +14
-5-if (unlikely (ret == -EDEADLK )) { -drm_gem_object_get (obj); - exec->contended = obj; -return -EDEADLK ; - } +if (unlikely (ret == -EDEADLK )) + exec->contended =dma_resv_get (resv); +return ret; +} +EXPORT_SYMBOL (drm_exec_lock_resv); + +/** +* drm_exec_lock_obj - lock a GEM object for use +* @exec: the drm_exec object with the state +* @obj: the GEM object to lock +* +* Lock a GEM object for use and grab a reference to it. +* +* Returns: -EDEADLK if a contention is detected, -EALREADY when
object is
+ * already locked (can be suppressed by setting theDRM_EXEC_IGNORE_DUPLICATES
+ * flag),-ENOMEMwhen memory allocation failed and zero for success.
+ */
+int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj)drivers/gpu/drm/drm_exec.c +4
-0+{ +int ret; + + ret =drm_exec_lock_resv (exec, obj->resv);
if (unlikely(ret ==-EALREADY) &&exec->flags&DRM_EXEC_IGNORE_DUPLICATES)drivers/gpu/drm/drm_gem.c +2
-0@@ -229,6 +229,8 @@ void drm_gem_private_object_init(structdrm_device *dev, obj->size = size; mutex_init(&obj->gpuva.lock); dma_resv_init(&obj->_resv); + +/* TODO: This needs to go away for drm_exec to work correctly!!! */
if (!obj->resv)
obj->resv=dma_resv_get(&obj->_resv);include/drm/drm_exec.h +0
-0@@ -47,14 +47,14 @@ struct drm_exec { struct drm_gem_object **objects;
/**include/drm/drm_exec.h +0
-1-* @contended: contended GEM object we backed off for
include/drm/drm_exec.h +1
-0+* @contended: contended dma_resv object we backed off
for
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *contended; +struct dma_resv *contended;
/**include/drm/drm_exec.h +0
-1-* @prelocked: already locked GEM object due to contention
include/drm/drm_exec.h +1
-0+* @prelocked: already locked dma_resv object due to contention
*/include/drm/drm_exec.h +1
-1-struct drm_gem_object *prelocked; +struct dma_resv *prelocked;};
/**include/drm/drm_exec.h +0
-0@@ -175,6 +175,7 @@ static inline struct ww_acquire_ctx *drm_exec_ticket(struct drm_exec *exec) void drm_exec_init(struct drm_exec *exec, u32 flags,unsigned
nr);
voiddrm_exec_fini(struct drm_exec *exec); booldrm_exec_cleanup(struct drm_exec *exec); +int drm_exec_lock_resv(structdrm_exec*exec, structdma_resv
*resv);
int drm_exec_lock_obj(structdrm_exec*exec, structdrm_gem_object
*obj);
voiddrm_exec_unlock_obj(struct drm_exec *exec, struct drm_gem_object *obj);
int drm_exec_prepare_obj(structdrm_exec*exec, structdrm_gem_object*obj,
PATCH 11/12 drm/ttm: support using drm_exec during eviction v4
In reply to Christian KönigAllow specifying a drm_exec object in TTMs operation context which is used to lock objects during eviction.
This allows to handle deadlocks much more gracefully and with that avoid returning -ENOMEM on heavily contended domains.
v2: rebased on top of Thomas work
v3: rebased again
v4: adjust to dma_resv changes
--- drivers/gpu/drm/ttm/ttm_bo_util.c | 12 ++++++++---- include/drm/ttm/ttm_bo.h | 5 +++++ 2 files changed, 13 insertions(+), 4 deletions(-)
drivers/gpu/drm/ttm/ttm_bo_util.c +8 -4
@@ -38,6 +38,7 @@#include <drm/ttm/ttm_tt.h> #include <drm/drm_cache.h> +#include <drm/drm_exec.h> struct ttm_transfer_obj {struct ttm_buffer_object base;
@@ -839,7 +840,9 @@ static int ttm_lru_walk_ticketlock(struct ttm_bo_lru_cursor *curs,struct ttm_lru_walk_arg *arg = curs->arg;int ret; -if (arg->ctx->interruptible) +if (arg->ctx->exec) + ret =drm_exec_lock_resv (arg->ctx->exec, resv); +else if (arg->ctx->interruptible) ret =dma_resv_lock_interruptible (resv, arg->ticket);else ret =dma_resv_lock (resv, arg->ticket);
@@ -853,7 +856,8 @@ static int ttm_lru_walk_ticketlock(struct ttm_bo_lru_cursor *curs,* trylocking for this walk. */ arg->ticket =NULL ; - }else if (ret == -EDEADLK ) { + + }else if (!arg->ctx->exec && ret == -EDEADLK ) {/* Caller needs to exit the ww transaction. */ ret = -ENOSPC ; }
@@ -993,8 +997,8 @@ __ttm_bo_lru_cursor_iter(struct ttm_bo_lru_cursor *curs, bool first)if (!ttm_lru_walk_trylock (curs, res->bo->base.resv)) {struct dma_resv *resv; -if (!arg->ticket || arg->ctx->no_wait_gpu || - arg->trylock_only) { +if ((!arg->ticket || arg->ctx->no_wait_gpu || + arg->trylock_only) && !arg->ctx->exec) {spin_unlock (lru_lock);continue ; }
include/drm/ttm/ttm_bo.h +5 -0
@@ -186,6 +186,11 @@ struct ttm_operation_ctx {* @bytes_moved: Statistics on how many bytes have been moved. */ uint64_t bytes_moved; +/* +* @exec: optional drm_exec object to use for locking and contention +* handling +*/ +struct drm_exec *exec; };struct ttm_lru_walk ;
Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH 11/12] drm/ttm: support using drm_exec during eviction v4
Quoted 11 lines · Christian König
Allow specifying a
drm_execobject in TTMs operation context which is used to lock objects during eviction.This allows to handle deadlocks much more gracefully and with that avoid returning
-ENOMEMon heavily contended domains.v2: rebased on top of Thomas work
v3: rebased again
v4: adjust todma_resvchangesSigned-off-by Christian König <[email protected]>
Same thing here. Use a ww context at the dma-buf level.
Thanks,
Thomas
Quoted 75 lines
---
drivers/gpu/drm/ttm/ttm_bo_util.c | 12 ++++++++---- include/drm/ttm/ttm_bo.h | 5 +++++2 files changed, 13 insertions(+), 4 deletions(-)drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -38,6 +38,7 @@ #include <drm/ttm/ttm_tt.h>
#include <drm/drm_cache.h>
+#include <drm/drm_exec.h>
structttm_transfer_obj{
structttm_buffer_objectbase;drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -839,7 +840,9 @@ static int ttm_lru_walk_ticketlock(struct ttm_bo_lru_cursor *curs, struct ttm_lru_walk_arg *arg = curs->arg; int ret;drivers/gpu/drm/ttm/ttm_bo_util.c +3
-1-if (arg->ctx->interruptible) +if (arg->ctx->exec) + ret =drm_exec_lock_resv (arg->ctx->exec, resv); +else if (arg->ctx->interruptible)ret = dma_resv_lock_interruptible(resv, arg-
Quoted 1 line
ticket);
else
ret =dma_resv_lock(resv, arg->ticket);drivers/gpu/drm/ttm/ttm_bo_util.c +2
-1@@ -853,7 +856,8 @@ static int ttm_lru_walk_ticketlock(structttm_bo_lru_cursor *curs, * trylocking for this walk. */ arg->ticket = NULL; - }else if (ret == -EDEADLK ) { + + }else if (!arg->ctx->exec && ret == -EDEADLK ) { /* Caller needs to exit the ww transaction. */ ret = -ENOSPC;}
drivers/gpu/drm/ttm/ttm_bo_util.c +0
-0@@ -993,8 +997,8 @@ __ttm_bo_lru_cursor_iter(struct ttm_bo_lru_cursor *curs, bool first) if (!ttm_lru_walk_trylock(curs, res->bo->base.resv)) { struct dma_resv *resv;drivers/gpu/drm/ttm/ttm_bo_util.c +1
-2-if (!arg->ticket || arg->ctx->no_wait_gpu || - arg->trylock_only) { + if ((!arg->ticket || arg->ctx->no_wait_gpu||
drivers/gpu/drm/ttm/ttm_bo_util.c +1
-0+ arg->trylock_only) && !arg->ctx->exec){
spin_unlock(lru_lock); continue;
}include/drm/ttm/ttm_bo.h +0
-0@@ -186,6 +186,11 @@ struct ttm_operation_ctx { * @bytes_moved: Statistics on how many bytes have beenmoved.
*/
uint64_tbytes_moved;include/drm/ttm/ttm_bo.h +2
-0+/* +* @exec: optional drm_exec object to use for locking and contention
include/drm/ttm/ttm_bo.h +3
-0+* handling +*/ +struct drm_exec *exec;};
structttm_lru_walk;
PATCH 12/12 drm/amdgpu: use drm_exec during BO validation
In reply to Christian KönigThis allows to detect deadlocks happening because of resource constraints.
Especially submissions which want to use all of GDS doesn't result in sporadic -ENOMEM any more.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 82 ++++++++++++++------------ 1 file changed, 44 insertions(+), 38 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +44 -38
@@ -793,7 +793,7 @@ static int amdgpu_cs_bo_validate(void *param, struct amdgpu_bo *bo)struct ttm_operation_ctx ctx = { .interruptible = true, .no_wait_gpu = false, - .resv = bo->tbo.base.resv + .exec = &p->exec, };uint32_t domain;int r;
@@ -845,7 +845,10 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, union drm_amdgpu_cs *cs) {struct amdgpu_fpriv *fpriv = p->filp->driver_priv; -struct ttm_operation_ctx ctx = { true, false }; +struct ttm_operation_ctx ctx = { + .interruptible =true, + .exec = &p->exec + };struct amdgpu_vm *vm = &fpriv->vm;struct amdgpu_bo_list_entry *e;struct drm_gem_object *obj;
@@ -899,6 +902,11 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, e->user_invalidated = userpage_invalidated; } +amdgpu_cs_get_threshold_for_moves (p->adev, &p->bytes_moved_threshold, + &p->bytes_moved_vis_threshold); + p->bytes_moved =0 ; + p->bytes_moved_vis =0 ; + drm_exec_until_all_locked(&p->exec) { r =amdgpu_vm_lock_pd (&fpriv->vm, &p->exec,1 + p->gang_size);drm_exec_retry_on_contention (&p->exec);
@@ -922,47 +930,48 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,if (unlikely (r)) goto out_free_user_pages; } - } -amdgpu_bo_list_for_each_userptr_entry (e, p->bo_list) { -struct mm_struct *usermm; +amdgpu_bo_list_for_each_userptr_entry (e, p->bo_list) { +struct mm_struct *usermm; - usermm =amdgpu_ttm_tt_get_usermm (e->bo->tbo.ttm); -if (usermm && usermm != current->mm) { - r = -EPERM ; - goto out_free_user_pages; - } - -if (amdgpu_ttm_tt_is_userptr (e->bo->tbo.ttm) && - e->user_invalidated) { -amdgpu_bo_placement_from_domain (e->bo, -AMDGPU_GEM_DOMAIN_CPU ); - r =ttm_bo_validate (&e->bo->tbo, &e->bo->placement, - &ctx); -if (r) + usermm =amdgpu_ttm_tt_get_usermm (e->bo->tbo.ttm); +if (usermm && usermm != current->mm) { + r = -EPERM ; goto out_free_user_pages; + } -amdgpu_ttm_tt_set_user_pages (e->bo->tbo.ttm, - e->range); +if (amdgpu_ttm_tt_is_userptr (e->bo->tbo.ttm) && + e->user_invalidated) { +amdgpu_bo_placement_from_domain (e->bo, +AMDGPU_GEM_DOMAIN_CPU ); + r =ttm_bo_validate (&e->bo->tbo, &e->bo->placement, + &ctx); +drm_exec_retry_on_contention (&p->exec); +if (r) + goto out_free_user_pages; + +amdgpu_ttm_tt_set_user_pages (e->bo->tbo.ttm, + e->range); + } } - } -amdgpu_cs_get_threshold_for_moves (p->adev, &p->bytes_moved_threshold, - &p->bytes_moved_vis_threshold); - p->bytes_moved =0 ; - p->bytes_moved_vis =0 ; + r =amdgpu_vm_validate (p->adev, &fpriv->vm,NULL , + amdgpu_cs_bo_validate, p); +drm_exec_retry_on_contention (&p->exec); +if (r) { +drm_err (adev_to_drm (p->adev),"amdgpu_vm_validate() failed.\n" ); + goto out_free_user_pages; + } - r =amdgpu_vm_validate (p->adev, &fpriv->vm,NULL , - amdgpu_cs_bo_validate, p); -if (r) { -drm_err (adev_to_drm (p->adev),"amdgpu_vm_validate() failed.\n" ); - goto out_free_user_pages; - } +drm_exec_for_each_locked_object (&p->exec, obj) { + r =amdgpu_cs_bo_validate (p,gem_to_amdgpu_bo (obj)); +drm_exec_retry_on_contention (&p->exec); +if (unlikely (r)) + goto out_free_user_pages; -drm_exec_for_each_locked_object (&p->exec, obj) { - r =amdgpu_cs_bo_validate (p,gem_to_amdgpu_bo (obj)); -if (unlikely (r)) - goto out_free_user_pages; +amdgpu_cs_report_moved_bytes (p->adev, p->bytes_moved, + p->bytes_moved_vis); + } }if (p->uf_bo) {
@@ -973,9 +982,6 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, p->gang_leader->uf_addr +=amdgpu_bo_gpu_offset (p->uf_bo); } -amdgpu_cs_report_moved_bytes (p->adev, p->bytes_moved, - p->bytes_moved_vis); -for (i =0 ; i < p->gang_size; ++i) amdgpu_job_set_resources(p->jobs[i], p->bo_list->gds_obj, p->bo_list->gws_obj,
Signature · 1 line
2.43.0
Patchwork
CI✗ Fi.CI.BUILD: failure for series starting with [01/12] dma-buf: Add reference counting to dma_resv
In reply to Christian König== Series Details ==
== Series Details ==
Series: series starting with [01/12] dma-buf: Add reference counting to dma_resv
URL : https://patchwork.freedesktop.org/series/170208/
State : failure
== Summary ==
Error: patch https://patchwork.freedesktop.org/api/1.0/series/170208/revisions/1/mbox/ not applied
Applying: dma-buf: Add reference counting to dma_resv
Applying: dma-buf/tests: Convert st-dma-resv tests to use dma_resv_alloc
Applying: drm/gem: Add helper for drm_gem_object resv assignment
Applying: drm/ttm: Switch LRU cursor to track dma_resv instead of buffer objects
Applying: drm/ttm: switch to ttm_bo_lru_for_each_reserved_guarded for swapout
Applying: drm/ttm: move delete handling into ttm_bo_evict
Applying: drm/ttm: use ttm_bo_lru_for_each_reserved_guarded in evict_all
Applying: drm/ttm: use dma_resv reference in ttm_device_clear_lru_dma_mappings
Applying: drm/ttm: nuke buffer refcounting
Applying: drm/exec: add drm_exec_lock_resv function
Applying: drm/ttm: support using drm_exec during eviction v4
Applying: drm/amdgpu: use drm_exec during BO validation
Using index info to reconstruct a base tree...
M drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
Falling back to patching base and 3-way merge...
Auto-merging drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
CONFLICT (content): Merge conflict in drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0012 drm/amdgpu: use drm_exec during BO validation
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Build failed, no error log produced
Thomas Hellström
maintainerHi, Christian,
Quoted 24 lines · Christian König
Hi everybody,
The idea of ref-counting
dma_resvorww_mutexcame up multiple times from
different people, but so far at least I have abandoned that as to complicated to implement considering how widely used that object is.Thanks to AI I gave the task to refcount
dma_resvto Claude Sonet 4 just
to check how horrible it would look like.Well turns out that this is actually a cleanup we should most likely aim
for and I'm really wondering why we haven't done it like this in the first place.Not only resolves it a bunch of issues with
dma_resvinstances shared by
multiple GEM objects (we just recently had a bunch of patches for that on
the mailing list), but also allows TTM to implement it's delayed delete
handling without any zombie resurrection or similar hacks.
Quoted 6 lines
The patch set is totally work in progress and only survives a smoke test
with amdgpu, but I still wanted to check if the idea is valuable and should be looked in further.Please comment and/or tear apart :)
Before starting to look at this, in the spirit of trying to aligning on a "final" solution for ww transactions used in TTM, could we agree on how we are supposed to pass the importer's ww transaction context to the exporter's dma_map_attachment() operation to handle exporter evictions while mapping, because that decides what abstaction we should use for WW transations within TTM.
drm_exec_init()xe_bo_validate() // IMporterdma_buf_map() // importer;map_attachment() // callback in exporter xe_bo_validate() // exporter tries to move exportert bo to VRAM but runs out of memory.ttm_bo_evict() // exporter's evict needs importer's drm_exec
As mentioned a couple of times before, we can't pass a drm_exec to the exporter because that would make dma-buf depend on drm.
So with this series it would be sufficient to move drm_exec to the dma- buf layer and rename it, but I still think we should have a plan.
Thanks,
Thomas
Quoted 3 lines
Cheers,
Christian.
Matthew Brost
maintainerQuoted 49 lines · Thomas Hellström
Hi, Christian,
Quoted 24 lines · Christian König
Hi everybody,
The idea of ref-counting
dma_resvorww_mutexcame up multiple times from
different people, but so far at least I have abandoned that as to complicated to implement considering how widely used that object is.Thanks to AI I gave the task to refcount
dma_resvto Claude Sonet 4 just
to check how horrible it would look like.Well turns out that this is actually a cleanup we should most likely aim
for and I'm really wondering why we haven't done it like this in the first place.Not only resolves it a bunch of issues with
dma_resvinstances shared by
multiple GEM objects (we just recently had a bunch of patches for that on
the mailing list), but also allows TTM to implement it's delayed delete
handling without any zombie resurrection or similar hacks.Quoted 6 lines
The patch set is totally work in progress and only survives a smoke test
with amdgpu, but I still wanted to check if the idea is valuable and should be looked in further.Please comment and/or tear apart :)
Before starting to look at this, in the spirit of trying to aligning on a "final" solution for ww transactions used in TTM, could we agree on how we are supposed to pass the importer's ww transaction context to the exporter's
dma_map_attachment()operation to handle exporter evictions while mapping, because that decides what abstaction we should use for WW transations within TTM.
drm_exec_init()xe_bo_validate()// IMporterdma_buf_map()// importer;map_attachment()// callback in exporterxe_bo_validate()// exporter tries to move exportert bo to VRAM but runs out of memory.ttm_bo_evict()// exporter's evict needs importer'sdrm_exec
Quoted 5 lines
As mentioned a couple of times before, we can't pass a
drm_execto the exporter because that would make dma-buf depend on drm.So with this series it would be sufficient to move
drm_execto the dma- buf layer and rename it, but I still think we should have a plan.
+1 to Thomas's thinking here — drm_exec should be moved to the dma-buf layer.
I think that would involve dropping the GEM references from the base layer and instead operating on dma-resv objects. Of course, if we want a drm_exec layer that operates on GEM objects, it could be built on top of the base dma-buf mechanism.
Matt
Quoted 10 lines
Thanks,
ThomasQuoted 3 lines
Cheers,
Christian.