completely rework the dma_resv semantic
Subsystems (22)
Christian König
reviewerHi guys,
as discussed before this set of patches completely rework the dma_resv semantic and spreads the new handling over all the existing drivers and users.
First of all this drops the DAG approach because it requires that every single driver implements those relatively complicated rules correctly and any violation of that immediately leads to either corruption of freed memory or even more severe security problems.
Instead we just keep all fences around all the time until they are signaled. Only fences with the same context are assumed to be signaled in the correct order since this is exercised elsewhere as well. Replacing fences is now only supported for hardware mechanism like VM page table updates where the hardware can guarantee that the resource can't be accessed any more.
Then the concept of a single exclusive fence and multiple shared fences is dropped as well.
Instead the dma_resv object is now just a container for dma_fence objects where each fence has associated usage flags. Those use flags describe how the operation represented by the dma_fence object is using the resource protected by the dma_resv object. This allows us to add multiple fences for each usage type.
Additionally to the existing WRITE/READ usages this patch set also adds the new KERNEL and OTHER usages. The KERNEL usages is used in cases where the kernel needs to do some operation with the resource protected by the dma_resv object, like copies or clears. Those are mandatory to wait for when dynamic memory management is used.
The OTHER usage is for cases where we don't want that the operation represented by the dma_fence object participate in any implicit sync but needs to be respected by the kernel memory management. Examples for those are VM page table updates and preemption fences.
While doing this the new implementation cleans up existing workarounds all over the place, but especially amdgpu and TTM. Surprisingly I also found two use cases for the KERNEL/OTHER usage in i915 and Nouveau, those might need more thoughts.
In general the existing functionality should been preserved, the only downside is that we now always need to reserve a slot before adding a fence. The newly added call to the reservation function can probably use some more cleanup.
TODOs: Testing, testing, testing, doublechecking the newly added kerneldoc for any typos.
Please review and/or comment,
Christian.
Christian König
reviewerPATCH 1/26 drm/amdgpu: partially revert "svm bo enable_signal call condition"
Partially revert commit 5f319c5c21b5909abb43d8aadc92a8aa549ee443.
First of all this is illegal use of RCU to call dma_fence_enable_sw_signaling() since we don't hold a reference to the fence in question and can crash badly.
Then the code doesn't seem to have the intended effect since only the exclusive fence is handled, but the KFD fences are always added as shared fence.
Only keep the handling to throw away the content of SVM BOs.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 9 --------- 1 file changed, 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0 -9
@@ -116,17 +116,8 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, abo =ttm_to_amdgpu_bo (bo);if (abo->flags &AMDGPU_AMDKFD_CREATE_SVM_BO ) { -struct dma_fence *fence; -struct dma_resv *resv = &bo->base._resv; - -rcu_read_lock (); - fence =rcu_dereference (resv->fence_excl); -if (fence && !fence->ops->signaled) -dma_fence_enable_sw_signaling (fence); - placement->num_placement =0 ; placement->num_busy_placement =0 ; -rcu_read_unlock ();return ; }
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 2/26 dma-buf: cleanup pruning of dma_resv objects
In reply to Christian KönigThe i915 driver implements a prune function which is called when it is very likely that the fences inside the dma_resv object can be removed because they are all signaled. TTM does something similar after waiting for the dma_resv to be idle.
Move those functions into the dma-resv.c code since the behavior of pruning fences is something internal to the object.
--- drivers/dma-buf/dma-resv.c | 32 ++++++++++++++++++++ drivers/gpu/drm/i915/Makefile | 1 - drivers/gpu/drm/i915/dma_resv_utils.c | 17 ----------- drivers/gpu/drm/i915/dma_resv_utils.h | 13 -------- drivers/gpu/drm/i915/gem/i915_gem_shrinker.c | 3 +- drivers/gpu/drm/i915/gem/i915_gem_wait.c | 3 +- drivers/gpu/drm/ttm/ttm_bo.c | 2 +- include/linux/dma-resv.h | 2 ++ 8 files changed, 37 insertions(+), 36 deletions(-) delete mode 100644 drivers/gpu/drm/i915/dma_resv_utils.c delete mode 100644 drivers/gpu/drm/i915/dma_resv_utils.h
drivers/dma-buf/dma-resv.c +32 -0
@@ -324,6 +324,38 @@ void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence) }EXPORT_SYMBOL (dma_resv_add_excl_fence); +/** +* dma_resv_prune - remove signaled fences +* @obj: The dma_resv object to prune +* +* Remove all the signaled fences from the dma_resv object. +*/ +void dma_resv_prune (struct dma_resv *obj) +{ +dma_resv_assert_held (obj); + +if (dma_resv_test_signaled (obj, true)) +dma_resv_add_excl_fence (obj,NULL ); +} +EXPORT_SYMBOL (dma_resv_prune_unlocked); + +/** +* dma_resv_prune_unlocked - try to remove signaled fences +* @obj: The dma_resv object to prune +* +* Try to lock the object, test if it is signaled and if yes then remove all the +* signaled fences. +*/ +void dma_resv_prune_unlocked (struct dma_resv *obj) +{ +if (!dma_resv_trylock (obj)) +return ; + +dma_resv_prune (obj); +dma_resv_unlock (obj); +} +EXPORT_SYMBOL (dma_resv_prune); +/** * dma_resv_iter_restart_unlocked - restart the unlocked iterator * @cursor: The dma_resv_iter object to restart
drivers/gpu/drm/i915/Makefile +0 -1
@@ -60,7 +60,6 @@ i915-y += i915_drv.o \ # core library code i915-y += \ - dma_resv_utils.o \ i915_memcpy.o \ i915_mm.o \ i915_sw_fence.o \
drivers/gpu/drm/i915/dma_resv_utils.c deleted +0 -17
@@ -1,17 +0,0 @@ -// SPDX-License-Identifier: MIT -/* -* Copyright © 2020 Intel Corporation -*/ - -#include <linux/dma-resv.h> - -#include "dma_resv_utils.h" - -void dma_resv_prune (struct dma_resv *resv) -{ -if (dma_resv_trylock (resv)) { -if (dma_resv_test_signaled (resv, true)) -dma_resv_add_excl_fence (resv,NULL ); -dma_resv_unlock (resv); - } -}
drivers/gpu/drm/i915/dma_resv_utils.h deleted +0 -13
@@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: MIT */ -/* -* Copyright © 2020 Intel Corporation -*/ - -#ifndef DMA_RESV_UTILS_H -#define DMA_RESV_UTILS_H - -struct dma_resv ; - -void dma_resv_prune (struct dma_resv *resv); - -#endif /* DMA_RESV_UTILS_H */
drivers/gpu/drm/i915/gem/i915_gem_shrinker.c +1 -2
@@ -15,7 +15,6 @@#include "gt/intel_gt_requests.h" -#include "dma_resv_utils.h" #include "i915_trace.h" static bool swap_available (void )
@@ -229,7 +228,7 @@ i915_gem_shrink(struct i915_gem_ww_ctx *ww,i915_gem_object_unlock (obj); } -dma_resv_prune (obj->base.resv); +dma_resv_prune_unlocked (obj->base.resv); scanned += obj->base.size >>PAGE_SHIFT ; skip:
drivers/gpu/drm/i915/gem/i915_gem_wait.c +1 -2
@@ -10,7 +10,6 @@#include "gt/intel_engine.h" -#include "dma_resv_utils.h" #include "i915_gem_ioctls.h" #include "i915_gem_object.h"
@@ -57,7 +56,7 @@ i915_gem_object_wait_reservation(struct dma_resv *resv,* signaled. */ if (timeout >0 ) -dma_resv_prune (resv); +dma_resv_prune_unlocked (resv);return ret; }
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
@@ -1086,7 +1086,7 @@ int ttm_bo_wait(struct ttm_buffer_object *bo,if (timeout ==0 )return -EBUSY ; -dma_resv_add_excl_fence (bo->base.resv,NULL ); +dma_resv_prune (bo->base.resv);return 0 ; }EXPORT_SYMBOL (ttm_bo_wait);
include/linux/dma-resv.h +2 -0
@@ -458,6 +458,8 @@ void dma_resv_fini(struct dma_resv *obj);int dma_resv_reserve_shared (struct dma_resv *obj,unsigned int num_fences);void dma_resv_add_shared_fence (struct dma_resv *obj,struct dma_fence *fence);void dma_resv_add_excl_fence (struct dma_resv *obj,struct dma_fence *fence); +void dma_resv_prune (struct dma_resv *obj); +void dma_resv_prune_unlocked (struct dma_resv *obj);int dma_resv_get_fences (struct dma_resv *obj,struct dma_fence **pfence_excl,unsigned *pshared_count,struct dma_fence ***pshared);int dma_resv_copy_fences (struct dma_resv *dst,struct dma_resv *src);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 3/26 dma-buf: make fence mandatory for dma_resv_add_excl_fence
In reply to Christian KönigCalling dma_resv_add_excl_fence() with the fence as NULL and expecting that that this frees up the fences is simply abuse of the internals of the dma_resv object.
Rework how pruning fences works and make the fence parameter mandatory.
--- drivers/dma-buf/dma-resv.c | 39 ++++++++++++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 4 deletions(-)
drivers/dma-buf/dma-resv.c +35 -4
@@ -96,6 +96,34 @@ static void dma_resv_list_free(struct dma_resv_list *list)kfree_rcu (list, rcu); } +/** +* dma_resv_list_prune - drop all signaled fences +* @list: list to check for signaled fences +* @obj: dma_resv object for lockdep +* +* Replace all the signaled fences with the stub fence to free them up. +*/ +static void dma_resv_list_prune (struct dma_resv_list *list, +struct dma_resv *obj) +{ +unsigned int i; + +if (!list) +return ; + +for (i =0 ; i < list->shared_count; ++i) { +struct dma_fence *fence; + + fence =rcu_dereference_protected (list->shared[i], +dma_resv_held (obj)); +if (!dma_fence_is_signaled (fence)) +continue ; + +RCU_INIT_POINTER (list->shared[i],dma_fence_get_stub ()); +dma_fence_put (fence); + } +} +/** * dma_resv_init - initialize a reservation object * @obj: the reservation object
@@ -305,8 +333,7 @@ void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence)if (old) i = old->shared_count; -if (fence) -dma_fence_get (fence); +dma_fence_get (fence);write_seqcount_begin (&obj->seq);/* write_seqcount_begin provides the necessary memory barrier */
@@ -334,8 +361,12 @@ void dma_resv_prune(struct dma_resv *obj) {dma_resv_assert_held (obj); -if (dma_resv_test_signaled (obj, true)) -dma_resv_add_excl_fence (obj,NULL ); +write_seqcount_begin (&obj->seq); +if (obj->fence_excl &&dma_fence_is_signaled (obj->fence_excl)) +dma_fence_put (rcu_replace_pointer (obj->fence_excl,NULL , +dma_resv_held (obj))); +dma_resv_list_prune (dma_resv_shared_list (obj), obj); +write_seqcount_end (&obj->seq); }EXPORT_SYMBOL (dma_resv_prune_unlocked);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 4/26 drm/qxl: use iterator instead of dma_resv_shared_list
In reply to Christian KönigI'm not sure why it is useful to know the number of fences in the reservation object, but we try to avoid exposing the dma_resv_shared_list() function.
So use the iterator instead. If more information is desired we could use dma_resv_describe() as well.
--- drivers/gpu/drm/qxl/qxl_debugfs.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
drivers/gpu/drm/qxl/qxl_debugfs.c +10 -7
@@ -57,13 +57,16 @@ qxl_debugfs_buffers_info(struct seq_file *m, void *data)struct qxl_bo *bo;list_for_each_entry (bo, &qdev->gem.objects, list) { -struct dma_resv_list *fobj; -int rel; - -rcu_read_lock (); - fobj =dma_resv_shared_list (bo->tbo.base.resv); - rel = fobj ? fobj->shared_count :0 ; -rcu_read_unlock (); +struct dma_resv_iter cursor; +struct dma_fence *fence; +int rel =0 ; + +dma_resv_iter_begin (&cursor, bo->tbo.base.resv, true); +dma_resv_for_each_fence_unlocked (&cursor, fence) { +if (dma_resv_iter_is_restarted (&cursor)) + rel =0 ; + ++rel; + } seq_printf(m, "size %ld, pc %d, num releases %d\n", (unsigned long)bo->tbo.base.size,
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 5/26 dma-buf: add dma_resv_replace_fences
In reply to Christian KönigThis function allows to replace fences from the shared fence list when we can gurantee that the operation represented by the original fence has finished or no accesses to the resources protected by the dma_resv object any more when the new fence finishes.
Then use this function in the amdkfd code when BOs are unmapped from the process.
--- drivers/dma-buf/dma-resv.c | 43 ++++++++++++++++ .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 49 +++---------------- include/linux/dma-resv.h | 2 + 3 files changed, 52 insertions(+), 42 deletions(-)
drivers/dma-buf/dma-resv.c +43 -0
@@ -312,6 +312,49 @@ void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence) }EXPORT_SYMBOL (dma_resv_add_shared_fence); +/** +* dma_resv_replace_fences - replace fences in the dma_resv obj +* @obj: the reservation object +* @context: the context of the fences to replace +* @replacement: the new fence to use instead +* +* Replace fences with a specified context with a new fence. Only valid if the +* operation represented by the original fences is completed or has no longer +* access to the resources protected by the dma_resv object when the new fence +* completes. +*/ +void dma_resv_replace_fences (struct dma_resv *obj,uint64_t context, +struct dma_fence *replacement) +{ +struct dma_resv_list *list; +struct dma_fence *old; +unsigned int i; + +dma_resv_assert_held (obj); + +write_seqcount_begin (&obj->seq); + + old =dma_resv_excl_fence (obj); +if (old->context == context) { +RCU_INIT_POINTER (obj->fence_excl,dma_fence_get (replacement)); +dma_fence_put (old); + } + + list =dma_resv_shared_list (obj); +for (i =0 ; list && i < list->shared_count; ++i) { + old =rcu_dereference_protected (list->shared[i], +dma_resv_held (obj)); +if (old->context != context) +continue ; + +rcu_assign_pointer (list->shared[i],dma_fence_get (replacement)); +dma_fence_put (old); + } + +write_seqcount_end (&obj->seq); +} +EXPORT_SYMBOL (dma_resv_replace_fences); +/** * dma_resv_add_excl_fence - Add an exclusive fence. * @obj: the reservation object
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +7 -42
60 lines not on this page · raw message
include/linux/dma-resv.h +2 -0
@@ -457,6 +457,8 @@ void dma_resv_init(struct dma_resv *obj);void dma_resv_fini (struct dma_resv *obj);int dma_resv_reserve_shared (struct dma_resv *obj,unsigned int num_fences);void dma_resv_add_shared_fence (struct dma_resv *obj,struct dma_fence *fence); +void dma_resv_replace_fences (struct dma_resv *obj,uint64_t context, +struct dma_fence *fence);void dma_resv_add_excl_fence (struct dma_resv *obj,struct dma_fence *fence);void dma_resv_prune (struct dma_resv *obj);void dma_resv_prune_unlocked (struct dma_resv *obj);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 6/26 dma-buf: finally make the dma_resv_list private
In reply to Christian KönigDrivers should never touch this directly.
--- drivers/dma-buf/dma-resv.c | 26 ++++++++++++++++++++++++++ include/linux/dma-resv.h | 26 +------------------------- 2 files changed, 27 insertions(+), 25 deletions(-)
drivers/dma-buf/dma-resv.c +26 -0
@@ -56,6 +56,19 @@DEFINE_WD_CLASS (reservation_ww_class);EXPORT_SYMBOL (reservation_ww_class); +/** +* struct dma_resv_list - a list of shared fences +* @rcu: for internal use +* @shared_count: table of shared fences +* @shared_max: for growing shared fence table +* @shared: shared fence table +*/ +struct dma_resv_list { +struct rcu_head rcu; +u32 shared_count, shared_max; +struct dma_fence __rcu *shared[]; +}; +/** * dma_resv_list_alloc - allocate fence list * @shared_max: number of fences we need space for
@@ -161,6 +174,19 @@ void dma_resv_fini(struct dma_resv *obj) }EXPORT_SYMBOL (dma_resv_fini); +/** +* dma_resv_shared_list - get the reservation object's shared fence list +* @obj: the reservation object +* +* Returns the shared fence list. Caller must either hold the objects +* through dma_resv_lock() or the RCU read side lock through rcu_read_lock(), +* or one of the variants of each +*/ +static inline struct dma_resv_list *dma_resv_shared_list (struct dma_resv *obj) +{ +return rcu_dereference_check (obj->fence,dma_resv_held (obj)); +} +/** * dma_resv_reserve_shared - Reserve space to add shared fences to * a dma_resv.
include/linux/dma-resv.h +1 -25
@@ -47,18 +47,7 @@extern struct ww_class reservation_ww_class; -/** -* struct dma_resv_list - a list of shared fences -* @rcu: for internal use -* @shared_count: table of shared fences -* @shared_max: for growing shared fence table -* @shared: shared fence table -*/ -struct dma_resv_list { -struct rcu_head rcu; -u32 shared_count, shared_max; -struct dma_fence __rcu *shared[]; -}; +struct dma_resv_list ;/** * struct dma_resv - a reservation object manages fences for a buffer
@@ -440,19 +429,6 @@ dma_resv_excl_fence(struct dma_resv *obj)return rcu_dereference_check (obj->fence_excl,dma_resv_held (obj)); } -/** -* dma_resv_shared_list - get the reservation object's shared fence list -* @obj: the reservation object -* -* Returns the shared fence list. Caller must either hold the objects -* through dma_resv_lock() or the RCU read side lock through rcu_read_lock(), -* or one of the variants of each -*/ -static inline struct dma_resv_list *dma_resv_shared_list (struct dma_resv *obj) -{ -return rcu_dereference_check (obj->fence,dma_resv_held (obj)); -} -void dma_resv_init (struct dma_resv *obj);void dma_resv_fini (struct dma_resv *obj);int dma_resv_reserve_shared (struct dma_resv *obj,unsigned int num_fences);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 7/26 dma-buf: drop excl_fence parameter from dma_resv_get_fences
In reply to Christian KönigReturning the exclusive fence separately is no longer used.
Instead add a write parameter to indicate the use case.
--- drivers/dma-buf/dma-resv.c | 48 ++++++++------------ drivers/dma-buf/st-dma-resv.c | 26 ++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_display.c | 6 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 2 +- drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 3 +- include/linux/dma-resv.h | 4 +- 6 files changed, 31 insertions(+), 58 deletions(-)
drivers/dma-buf/dma-resv.c +18 -30
@@ -675,57 +675,45 @@ EXPORT_SYMBOL(dma_resv_copy_fences);* dma_resv_get_fences - Get an object's shared and exclusive * fences without update side lock held * @obj: the reservation object -* @fence_excl: the returned exclusive fence (or NULL) -* @shared_count: the number of shared fences returned -* @shared: the array of shared fence ptrs returned (array is krealloc'd to -* the required size, and must be freed by caller) -* -* Retrieve all fences from the reservation object. If the pointer for the -* exclusive fence is not specified the fence is put into the array of the -* shared fences as well. Returns either zero or -ENOMEM. +* @write: true if we should return all fences +* @num_fences: the number of fences returned +* @fences: the array of fence ptrs returned (array is krealloc'd to the +* required size, and must be freed by caller) +* +* Retrieve all fences from the reservation object. +* Returns either zero or -ENOMEM. */ -int dma_resv_get_fences (struct dma_resv *obj,struct dma_fence **fence_excl, -unsigned int *shared_count,struct dma_fence ***shared) +int dma_resv_get_fences (struct dma_resv *obj,bool write, +unsigned int *num_fences,struct dma_fence ***fences) {struct dma_resv_iter cursor;struct dma_fence *fence; - *shared_count =0 ; - *shared =NULL ; - -if (fence_excl) - *fence_excl =NULL ; + *num_fences =0 ; + *fences =NULL ; -dma_resv_iter_begin (&cursor, obj, true); +dma_resv_iter_begin (&cursor, obj, write);dma_resv_for_each_fence_unlocked (&cursor, fence) {if (dma_resv_iter_is_restarted (&cursor)) {unsigned int count; -while (*shared_count) -dma_fence_put ((*shared)[--(*shared_count)]); +while (*num_fences) +dma_fence_put ((*fences)[--(*num_fences)]); -if (fence_excl) -dma_fence_put (*fence_excl); - - count = cursor.shared_count; - count += fence_excl ?0 :1 ; + count = cursor.shared_count +1 ;/* Eventually re-allocate the array */ - *shared =krealloc_array (*shared, count, + *fences =krealloc_array (*fences, count,sizeof (void *),GFP_KERNEL ); -if (count && !*shared) { +if (count && !*fences) {dma_resv_iter_end (&cursor);return -ENOMEM ; } } -dma_fence_get (fence); -if (dma_resv_iter_is_exclusive (&cursor) && fence_excl) - *fence_excl = fence; -else - (*shared)[(*shared_count)++] = fence; + (*fences)[(*num_fences)++] =dma_fence_get (fence); }dma_resv_iter_end (&cursor);
drivers/dma-buf/st-dma-resv.c +5 -21
@@ -275,7 +275,7 @@ static int test_shared_for_each_unlocked(void *arg)static int test_get_fences (void *arg,bool shared) { -struct dma_fence *f, *excl =NULL , **fences =NULL ; +struct dma_fence *f, **fences =NULL ;struct dma_resv resv;int r, i;
@@ -304,35 +304,19 @@ static int test_get_fences(void *arg, bool shared) }dma_resv_unlock (&resv); - r =dma_resv_get_fences (&resv, &excl, &i, &fences); + r =dma_resv_get_fences (&resv, shared, &i, &fences);if (r) {pr_err ("get_fences failed\n" ); goto err_free; } -if (shared) { -if (excl !=NULL ) { -pr_err ("get_fences returned unexpected excl fence\n" ); - goto err_free; - } -if (i !=1 || fences[0 ] != f) { -pr_err ("get_fences returned unexpected shared fence\n" ); - goto err_free; - } - }else { -if (excl != f) { -pr_err ("get_fences returned unexpected excl fence\n" ); - goto err_free; - } -if (i !=0 ) { -pr_err ("get_fences returned unexpected shared fence\n" ); - goto err_free; - } +if (i !=1 || fences[0 ] != f) { +pr_err ("get_fences returned unexpected fence\n" ); + goto err_free; }dma_fence_signal (f); err_free: -dma_fence_put (excl);while (i--)dma_fence_put (fences[i]);kfree (fences);
drivers/gpu/drm/amd/amdgpu/amdgpu_display.c +4 -2
@@ -200,8 +200,10 @@ int amdgpu_display_crtc_page_flip_target(struct drm_crtc *crtc, goto unpin; } - r =dma_resv_get_fences (new_abo->tbo.base.resv,NULL , - &work->shared_count, &work->shared); +/* TODO: Unify this with other drivers */ + r =dma_resv_get_fences (new_abo->tbo.base.resv, true, + &work->shared_count, + &work->shared);if (unlikely (r !=0 )) {DRM_ERROR ("failed to get fences for buffer\n" ); goto unpin;
drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +1 -1
@@ -112,7 +112,7 @@ void amdgpu_pasid_free_delayed(struct dma_resv *resv,unsigned count;int r; - r =dma_resv_get_fences (resv,NULL , &count, &fences); + r =dma_resv_get_fences (resv, true, &count, &fences);if (r) goto fallback;
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +1 -2
@@ -189,8 +189,7 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)continue ;if (bo->flags &ETNA_SUBMIT_BO_WRITE ) { - ret =dma_resv_get_fences (robj,NULL , - &bo->nr_shared, + ret =dma_resv_get_fences (robj, true, &bo->nr_shared, &bo->shared);if (ret)return ret;
include/linux/dma-resv.h +2 -2
@@ -438,8 +438,8 @@ void dma_resv_replace_fences(struct dma_resv *obj, uint64_t context,void dma_resv_add_excl_fence (struct dma_resv *obj,struct dma_fence *fence);void dma_resv_prune (struct dma_resv *obj);void dma_resv_prune_unlocked (struct dma_resv *obj); -int dma_resv_get_fences (struct dma_resv *obj,struct dma_fence **pfence_excl, -unsigned *pshared_count,struct dma_fence ***pshared); +int dma_resv_get_fences (struct dma_resv *obj,bool write, +unsigned int *num_fences,struct dma_fence ***fences);int dma_resv_copy_fences (struct dma_resv *dst,struct dma_resv *src);long dma_resv_wait_timeout (struct dma_resv *obj,bool wait_all,bool intr,unsigned long timeout);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 8/26 dma-buf: add dma_resv_get_singleton
In reply to Christian KönigAdd a function to simplify getting a single fence for all the fences in the dma_resv object.
--- drivers/dma-buf/dma-resv.c | 50 ++++++++++++++++++++++++++++++++++++++ include/linux/dma-resv.h | 2 ++ 2 files changed, 52 insertions(+)
drivers/dma-buf/dma-resv.c +50 -0
@@ -34,6 +34,7 @@*/ #include <linux/dma-resv.h> +#include <linux/dma-fence-array.h> #include <linux/export.h> #include <linux/mm.h> #include <linux/sched/mm.h>
@@ -721,6 +722,55 @@ int dma_resv_get_fences(struct dma_resv *obj, bool write, }EXPORT_SYMBOL_GPL (dma_resv_get_fences); +/** +* dma_resv_get_singleton - Get a single fence for all the fences +* @obj: the reservation object +* @write: true if we should return all fences +* @fence: the resulting fence +* +* Get a single fence representing all the fences inside the resv object. +* Returns either 0 for success or -ENOMEM. +* +* Warning: This can't be used like this when adding the fence back to the resv +* object since that can lead to stack corruption when finalizing the +* dma_fence_array. +*/ +int dma_resv_get_singleton (struct dma_resv *obj,bool write, +struct dma_fence **fence) +{ +struct dma_fence_array *array; +struct dma_fence **fences; +unsigned count; +int r; + + r =dma_resv_get_fences (obj, write, &count, &fences); +if (r) +return r; + +if (count ==0 ) { + *fence =NULL ; +return 0 ; + } + +if (count ==1 ) { + *fence = fences[0 ]; +kfree (fences); +return 0 ; + } + + array =dma_fence_array_create (count, fences, +dma_fence_context_alloc (1 ), +1 , false); +if (!array) { +kfree (fences); +return -ENOMEM ; + } + + *fence = &array->base; +return 0 ; +} +EXPORT_SYMBOL_GPL (dma_resv_get_singleton); +/** * dma_resv_wait_timeout - Wait on reservation's objects * shared and/or exclusive fences.
include/linux/dma-resv.h +2 -0
@@ -440,6 +440,8 @@ void dma_resv_prune(struct dma_resv *obj);void dma_resv_prune_unlocked (struct dma_resv *obj);int dma_resv_get_fences (struct dma_resv *obj,bool write,unsigned int *num_fences,struct dma_fence ***fences); +int dma_resv_get_singleton (struct dma_resv *obj,bool write, +struct dma_fence **fence);int dma_resv_copy_fences (struct dma_resv *dst,struct dma_resv *src);long dma_resv_wait_timeout (struct dma_resv *obj,bool wait_all,bool intr,unsigned long timeout);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 9/26 RDMA: use dma_resv_wait() instead of extracting the fence
In reply to Christian KönigUse dma_resv_wait() instead of extracting the exclusive fence and waiting on it manually.
--- drivers/infiniband/core/umem_dmabuf.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
drivers/infiniband/core/umem_dmabuf.c +2 -6
@@ -16,7 +16,6 @@ int ib_umem_dmabuf_map_pages(struct ib_umem_dmabuf *umem_dmabuf) {struct sg_table *sgt;struct scatterlist *sg; -struct dma_fence *fence;unsigned long start, end, cur =0 ;unsigned int nmap =0 ;int i;
@@ -68,11 +67,8 @@ int ib_umem_dmabuf_map_pages(struct ib_umem_dmabuf *umem_dmabuf)* may be not up-to-date. Wait for the exporter to finish * the migration. */ - fence =dma_resv_excl_fence (umem_dmabuf->attach->dmabuf->resv); -if (fence) -return dma_fence_wait (fence, false); - -return 0 ; +return dma_resv_wait_timeout (umem_dmabuf->attach->dmabuf->resv, false, + false,MAX_SCHEDULE_TIMEOUT ); }EXPORT_SYMBOL (ib_umem_dmabuf_map_pages);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 10/26 drm/etnaviv: stop using dma_resv_excl_fence
In reply to Christian KönigWe can get the excl fence together with the shared ones as well.
--- drivers/gpu/drm/etnaviv/etnaviv_gem.h | 1 - drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 14 +++++--------- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 10 ---------- 3 files changed, 5 insertions(+), 20 deletions(-)
drivers/gpu/drm/etnaviv/etnaviv_gem.h +0 -1
@@ -80,7 +80,6 @@ struct etnaviv_gem_submit_bo {u64 va;struct etnaviv_gem_object *obj;struct etnaviv_vram_mapping *mapping; -struct dma_fence *excl;unsigned int nr_shared;struct dma_fence **shared; };
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +5 -9
@@ -188,15 +188,11 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)if (submit->flags &ETNA_SUBMIT_NO_IMPLICIT )continue ; -if (bo->flags &ETNA_SUBMIT_BO_WRITE ) { - ret =dma_resv_get_fences (robj, true, &bo->nr_shared, - &bo->shared); -if (ret) -return ret; - }else { - bo->excl =dma_fence_get (dma_resv_excl_fence (robj)); - } - + ret =dma_resv_get_fences (robj, + !!(bo->flags &ETNA_SUBMIT_BO_WRITE ), + &bo->nr_shared, &bo->shared); +if (ret) +return ret; }return ret;
drivers/gpu/drm/etnaviv/etnaviv_sched.c +0 -10
@@ -39,16 +39,6 @@ etnaviv_sched_dependency(struct drm_sched_job *sched_job,struct etnaviv_gem_submit_bo *bo = &submit->bos[i];int j; -if (bo->excl) { - fence = bo->excl; - bo->excl =NULL ; - -if (!dma_fence_is_signaled (fence)) -return fence; - -dma_fence_put (fence); - } -for (j =0 ; j < bo->nr_shared; j++) {if (!bo->shared[j])continue ;
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 11/26 drm/nouveau: stop using dma_resv_excl_fence
In reply to Christian KönigInstead use the new dma_resv_get_singleton function.
--- drivers/gpu/drm/nouveau/nouveau_bo.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/nouveau/nouveau_bo.c +8 -1
@@ -959,7 +959,14 @@ nouveau_bo_vm_cleanup(struct ttm_buffer_object *bo, {struct nouveau_drm *drm =nouveau_bdev (bo->bdev);struct drm_device *dev = drm->dev; -struct dma_fence *fence =dma_resv_excl_fence (bo->base.resv); +struct dma_fence *fence; +int ret; + +/* TODO: This is actually a memory management dependency */ + ret =dma_resv_get_singleton (bo->base.resv, false, &fence); +if (ret) +dma_resv_wait_timeout (bo->base.resv, false, false, +MAX_SCHEDULE_TIMEOUT );nv10_bo_put_tile_region (dev, *old_tile, fence); *old_tile = new_tile;
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 12/26 drm/vmwgfx: stop using dma_resv_excl_fence
In reply to Christian KönigInstead use the new dma_resv_get_singleton function.
--- drivers/gpu/drm/vmwgfx/vmwgfx_resource.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/vmwgfx/vmwgfx_resource.c +4 -2
@@ -1168,8 +1168,10 @@ int vmw_resources_clean(struct vmw_buffer_object *vbo, pgoff_t start,vmw_bo_fence_single (bo,NULL );if (bo->moving)dma_fence_put (bo->moving); - bo->moving =dma_fence_get - (dma_resv_excl_fence (bo->base.resv)); + +/* TODO: This is actually a memory management dependency */ +return dma_resv_get_singleton (bo->base.resv, false, + &bo->moving); }return 0 ;
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 13/26 drm/radeon: stop using dma_resv_excl_fence
In reply to Christian KönigInstead use the new dma_resv_get_singleton function.
--- drivers/gpu/drm/radeon/radeon_display.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/gpu/drm/radeon/radeon_display.c +6 -1
@@ -533,7 +533,12 @@ static int radeon_crtc_page_flip_target(struct drm_crtc *crtc,DRM_ERROR ("failed to pin new rbo buffer before flip\n" ); goto cleanup; } - work->fence =dma_fence_get (dma_resv_excl_fence (new_rbo->tbo.base.resv)); + r =dma_resv_get_singleton (new_rbo->tbo.base.resv, false, &work->fence); +if (r) { +radeon_bo_unreserve (new_rbo); +DRM_ERROR ("failed to get new rbo buffer fences\n" ); + goto cleanup; + }radeon_bo_get_tiling_flags (new_rbo, &tiling_flags,NULL );radeon_bo_unreserve (new_rbo);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 14/26 drm/amdgpu: remove excl as shared workarounds
In reply to Christian KönigThis was added because of the now dropped shared on excl dependency.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 5 +---- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 6 ------ 2 files changed, 1 insertion(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -4
@@ -1275,14 +1275,11 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,/* * Work around dma_resv shortcommings by wrapping up the * submission in a dma_fence_chain and add it as exclusive -* fence, but first add the submission as shared fence to make -* sure that shared fences never signal before the exclusive -* one. +* fence. */ dma_fence_chain_init (chain,dma_resv_excl_fence (resv),dma_fence_get (p->fence),1 ); -dma_resv_add_shared_fence (resv, p->fence);rcu_assign_pointer (resv->fence_excl, &chain->base); e->chain =NULL ; }
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +0 -6
@@ -226,12 +226,6 @@ static void amdgpu_gem_object_close(struct drm_gem_object *obj,if (!amdgpu_vm_ready (vm)) goto out_unlock; - fence =dma_resv_excl_fence (bo->tbo.base.resv); -if (fence) { -amdgpu_bo_fence (bo, fence, true); - fence =NULL ; - } - r =amdgpu_vm_clear_freed (adev, vm, &fence);if (r || !fence) goto out_unlock;
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 15/26 drm/amdgpu: use dma_resv_for_each_fence for CS workaround
In reply to Christian KönigGet the write fence using dma_resv_for_each_fence instead of accessing it manually.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +6 -3
@@ -1268,6 +1268,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,amdgpu_bo_list_for_each_entry (e, p->bo_list) {struct dma_resv *resv = e->tv.bo->base.resv;struct dma_fence_chain *chain = e->chain; +struct dma_resv_iter cursor; +struct dma_fence *fence;if (!chain)continue ;
@@ -1277,9 +1279,10 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,* submission in a dma_fence_chain and add it as exclusive * fence. */ -dma_fence_chain_init (chain,dma_resv_excl_fence (resv), -dma_fence_get (p->fence),1 ); - +dma_resv_for_each_fence (&cursor, resv, false, fence) { +break ; + } +dma_fence_chain_init (chain, fence,dma_fence_get (p->fence),1 );rcu_assign_pointer (resv->fence_excl, &chain->base); e->chain =NULL ; }
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 16/26 dma-buf: finally make dma_resv_excl_fence private
In reply to Christian KönigDrivers should never touch this directly.
--- drivers/dma-buf/dma-resv.c | 17 +++++++++++++++++ include/linux/dma-resv.h | 17 ----------------- 2 files changed, 17 insertions(+), 17 deletions(-)
drivers/dma-buf/dma-resv.c +17 -0
@@ -175,6 +175,23 @@ void dma_resv_fini(struct dma_resv *obj) }EXPORT_SYMBOL (dma_resv_fini); +/** +* dma_resv_excl_fence - return the object's exclusive fence +* @obj: the reservation object +* +* Returns the exclusive fence (if any). Caller must either hold the objects +* through dma_resv_lock() or the RCU read side lock through rcu_read_lock(), +* or one of the variants of each +* +* RETURNS +* The exclusive fence or NULL +*/ +static inline struct dma_fence * +dma_resv_excl_fence (struct dma_resv *obj) +{ +return rcu_dereference_check (obj->fence_excl,dma_resv_held (obj)); +} +/** * dma_resv_shared_list - get the reservation object's shared fence list * @obj: the reservation object
include/linux/dma-resv.h +0 -17
@@ -412,23 +412,6 @@ static inline void dma_resv_unlock(struct dma_resv *obj)ww_mutex_unlock (&obj->lock); } -/** -* dma_resv_excl_fence - return the object's exclusive fence -* @obj: the reservation object -* -* Returns the exclusive fence (if any). Caller must either hold the objects -* through dma_resv_lock() or the RCU read side lock through rcu_read_lock(), -* or one of the variants of each -* -* RETURNS -* The exclusive fence or NULL -*/ -static inline struct dma_fence * -dma_resv_excl_fence (struct dma_resv *obj) -{ -return rcu_dereference_check (obj->fence_excl,dma_resv_held (obj)); -} -void dma_resv_init (struct dma_resv *obj);void dma_resv_fini (struct dma_resv *obj);int dma_resv_reserve_shared (struct dma_resv *obj,unsigned int num_fences);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 17/26 dma-buf: drop the DAG approach for the dma_resv object
In reply to Christian KönigSo far we had the approach of using a directed acyclic graph with the dma_resv obj.
This turned out to have many downsides, especially it means that every single driver and user of this interface needs to be aware of this restriction when adding fences. If the rules for the DAG are not followed then we end up with potential hard to debug memory corruption, information leaks or even elephant big security holes because we allow userspace to access freed up memory.
Since we already took a step back from that by always looking at all fences we now go a step further and stop dropping the shared fences when a new exclusive one is added.
--- drivers/dma-buf/dma-resv.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-)
drivers/dma-buf/dma-resv.c +1 -13
@@ -411,29 +411,17 @@ EXPORT_SYMBOL(dma_resv_replace_fences);void dma_resv_add_excl_fence (struct dma_resv *obj,struct dma_fence *fence) {struct dma_fence *old_fence =dma_resv_excl_fence (obj); -struct dma_resv_list *old; -u32 i =0 ;dma_resv_assert_held (obj); - old =dma_resv_shared_list (obj); -if (old) - i = old->shared_count; -dma_fence_get (fence);write_seqcount_begin (&obj->seq);/* write_seqcount_begin provides the necessary memory barrier */ RCU_INIT_POINTER (obj->fence_excl, fence); -if (old) - old->shared_count =0 ; +dma_resv_list_prune (dma_resv_shared_list (obj), obj);write_seqcount_end (&obj->seq); -/* inplace update, no shared fences */ -while (i--) -dma_fence_put (rcu_dereference_protected (old->shared[i], -dma_resv_held (obj))); -dma_fence_put (old_fence); }EXPORT_SYMBOL (dma_resv_add_excl_fence);
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 18/26 dma-buf/drivers: make reserving a shared slot mandatory
In reply to Christian KönigAudit all the users of dma_resv_add_excl_fence() and make sure they reserve a shared slot also when only trying to add an exclusive fence.
This is the next step towards handling the exclusive fence like a shared one.
--- drivers/dma-buf/st-dma-resv.c | 64 +++++++++---------- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 8 +++ drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 8 +-- drivers/gpu/drm/i915/gem/i915_gem_clflush.c | 3 +- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 8 +-- .../drm/i915/gem/selftests/i915_gem_migrate.c | 5 +- drivers/gpu/drm/i915/i915_vma.c | 6 ++ .../drm/i915/selftests/intel_memory_region.c | 7 ++ drivers/gpu/drm/lima/lima_gem.c | 10 ++- drivers/gpu/drm/msm/msm_gem_submit.c | 18 +++--- drivers/gpu/drm/nouveau/nouveau_fence.c | 9 +-- drivers/gpu/drm/panfrost/panfrost_job.c | 4 ++ drivers/gpu/drm/ttm/ttm_bo_util.c | 12 +++- drivers/gpu/drm/ttm/ttm_execbuf_util.c | 11 ++-- drivers/gpu/drm/v3d/v3d_gem.c | 15 +++-- drivers/gpu/drm/vgem/vgem_fence.c | 12 ++-- drivers/gpu/drm/virtio/virtgpu_gem.c | 9 +++ drivers/gpu/drm/vmwgfx/vmwgfx_bo.c | 16 +++-- 18 files changed, 133 insertions(+), 92 deletions(-)
drivers/dma-buf/st-dma-resv.c +30 -34
@@ -75,17 +75,16 @@ static int test_signaling(void *arg, bool shared) goto err_free; } -if (shared) { - r =dma_resv_reserve_shared (&resv,1 ); -if (r) { -pr_err ("Resv shared slot allocation failed\n" ); - goto err_unlock; - } + r =dma_resv_reserve_shared (&resv,1 ); +if (r) { +pr_err ("Resv shared slot allocation failed\n" ); + goto err_unlock; + } +if (shared)dma_resv_add_shared_fence (&resv, f); - }else { +else dma_resv_add_excl_fence (&resv, f); - }if (dma_resv_test_signaled (&resv, shared)) {pr_err ("Resv unexpectedly signaled\n" );
@@ -134,17 +133,16 @@ static int test_for_each(void *arg, bool shared) goto err_free; } -if (shared) { - r =dma_resv_reserve_shared (&resv,1 ); -if (r) { -pr_err ("Resv shared slot allocation failed\n" ); - goto err_unlock; - } + r =dma_resv_reserve_shared (&resv,1 ); +if (r) { +pr_err ("Resv shared slot allocation failed\n" ); + goto err_unlock; + } +if (shared)dma_resv_add_shared_fence (&resv, f); - }else { +else dma_resv_add_excl_fence (&resv, f); - } r = -ENOENT ;dma_resv_for_each_fence (&cursor, &resv, shared, fence) {
@@ -206,18 +204,17 @@ static int test_for_each_unlocked(void *arg, bool shared) goto err_free; } -if (shared) { - r =dma_resv_reserve_shared (&resv,1 ); -if (r) { -pr_err ("Resv shared slot allocation failed\n" ); -dma_resv_unlock (&resv); - goto err_free; - } + r =dma_resv_reserve_shared (&resv,1 ); +if (r) { +pr_err ("Resv shared slot allocation failed\n" ); +dma_resv_unlock (&resv); + goto err_free; + } +if (shared)dma_resv_add_shared_fence (&resv, f); - }else { +else dma_resv_add_excl_fence (&resv, f); - }dma_resv_unlock (&resv); r = -ENOENT ;
@@ -290,18 +287,17 @@ static int test_get_fences(void *arg, bool shared) goto err_resv; } -if (shared) { - r =dma_resv_reserve_shared (&resv,1 ); -if (r) { -pr_err ("Resv shared slot allocation failed\n" ); -dma_resv_unlock (&resv); - goto err_resv; - } + r =dma_resv_reserve_shared (&resv,1 ); +if (r) { +pr_err ("Resv shared slot allocation failed\n" ); +dma_resv_unlock (&resv); + goto err_resv; + } +if (shared)dma_resv_add_shared_fence (&resv, f); - }else { +else dma_resv_add_excl_fence (&resv, f); - }dma_resv_unlock (&resv); r =dma_resv_get_fences (&resv, shared, &i, &fences);
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +8 -0
@@ -1367,6 +1367,14 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence, bool shared) {struct dma_resv *resv = bo->tbo.base.resv; +int r; + + r =dma_resv_reserve_fences (resv,1 ); +if (r) { +/* As last resort on OOM we block for the fence */ +dma_fence_wait (fence, false); +return ; + }if (shared)dma_resv_add_shared_fence (resv, fence);
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +3 -5
@@ -179,11 +179,9 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)struct etnaviv_gem_submit_bo *bo = &submit->bos[i];struct dma_resv *robj = bo->obj->base.resv; -if (!(bo->flags &ETNA_SUBMIT_BO_WRITE )) { - ret =dma_resv_reserve_shared (robj,1 ); -if (ret) -return ret; - } + ret =dma_resv_reserve_shared (robj,1 ); +if (ret) +return ret;if (submit->flags &ETNA_SUBMIT_NO_IMPLICIT )continue ;
drivers/gpu/drm/i915/gem/i915_gem_clflush.c +2 -1
@@ -100,7 +100,8 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj,trace_i915_gem_object_clflush (obj); clflush =NULL ; -if (!(flags &I915_CLFLUSH_SYNC )) +if (!(flags &I915_CLFLUSH_SYNC ) && +dma_resv_reserve_shared (obj->base.resv,1 ) ==0 ) clflush =clflush_work_create (obj);if (clflush) { i915_sw_fence_await_reservation(&clflush->base.chain,
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +3 -5
@@ -989,11 +989,9 @@ static int eb_validate_vmas(struct i915_execbuffer *eb) } } -if (!(ev->flags &EXEC_OBJECT_WRITE )) { - err =dma_resv_reserve_shared (vma->resv,1 ); -if (err) -return err; - } + err =dma_resv_reserve_shared (vma->resv,1 ); +if (err) +return err;GEM_BUG_ON (drm_mm_node_allocated (&vma->node) &&eb_vma_misplaced (&eb->exec[i], vma, ev->flags));
drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c +4 -1
@@ -179,7 +179,10 @@ static int igt_lmem_pages_migrate(void *arg)i915_gem_object_is_lmem (obj), 0xdeadbeaf, &rq);if (rq) { -dma_resv_add_excl_fence (obj->base.resv, &rq->fence); + err =dma_resv_reserve_shared (obj->base.resv,1 ); +if (!err) +dma_resv_add_excl_fence (obj->base.resv, + &rq->fence);i915_request_put (rq); }if (err)
drivers/gpu/drm/i915/i915_vma.c +6 -0
@@ -1255,6 +1255,12 @@ int _i915_vma_move_to_active(struct i915_vma *vma,intel_frontbuffer_put (front); } +if (!(flags & __EXEC_OBJECT_NO_RESERVE)) { + err =dma_resv_reserve_shared (vma->resv,1 ); +if (unlikely (err)) +return err; + } +if (fence) {dma_resv_add_excl_fence (vma->resv, fence); obj->write_domain =I915_GEM_DOMAIN_RENDER ;
drivers/gpu/drm/i915/selftests/intel_memory_region.c +7 -0
@@ -894,6 +894,13 @@ static int igt_lmem_write_cpu(void *arg) }i915_gem_object_lock (obj,NULL ); + + err =dma_resv_reserve_shared (obj->base.resv,1 ); +if (err) { +i915_gem_object_unlock (obj); + goto out_put; + } +/* Put the pages into a known state -- from the gpu for added fun */ intel_engine_pm_get (engine); err = intel_context_migrate_clear(engine->gt->migrate.context, NULL,
drivers/gpu/drm/lima/lima_gem.c +4 -6
@@ -255,13 +255,11 @@ int lima_gem_get_info(struct drm_file *file, u32 handle, u32 *va, u64 *offset)static int lima_gem_sync_bo (struct lima_sched_task *task,struct lima_bo *bo,bool write,bool explicit) { -int err =0 ; +int err; -if (!write) { - err =dma_resv_reserve_shared (lima_bo_resv (bo),1 ); -if (err) -return err; - } + err =dma_resv_reserve_shared (lima_bo_resv (bo),1 ); +if (err) +return err;/* explicit sync use user passed dep fence */ if (explicit)
drivers/gpu/drm/msm/msm_gem_submit.c +8 -10
@@ -320,16 +320,14 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)struct drm_gem_object *obj = &submit->bos[i].obj->base;bool write = submit->bos[i].flags &MSM_SUBMIT_BO_WRITE ; -if (!write) { -/* NOTE: _reserve_shared() must happen before -* _add_shared_fence(), which makes this a slightly -* strange place to call it. OTOH this is a -* convenient can-fail point to hook it in. -*/ - ret =dma_resv_reserve_shared (obj->resv,1 ); -if (ret) -return ret; - } +/* NOTE: _reserve_shared() must happen before +* _add_shared_fence(), which makes this a slightly +* strange place to call it. OTOH this is a +* convenient can-fail point to hook it in. +*/ + ret =dma_resv_reserve_shared (obj->resv,1 ); +if (ret) +return ret;/* exclusive fences must be ordered */ if (no_implicit && !write)
drivers/gpu/drm/nouveau/nouveau_fence.c +3 -6
@@ -349,12 +349,9 @@ nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan,struct nouveau_fence *f;int ret; -if (!exclusive) { - ret =dma_resv_reserve_shared (resv,1 ); - -if (ret) -return ret; - } + ret =dma_resv_reserve_shared (resv,1 ); +if (ret) +return ret;dma_resv_for_each_fence (&cursor, resv, exclusive, fence) {struct nouveau_channel *prev =NULL ;
drivers/gpu/drm/panfrost/panfrost_job.c +4 -0
@@ -247,6 +247,10 @@ static int panfrost_acquire_object_fences(struct drm_gem_object **bos,int i, ret;for (i =0 ; i < bo_count; i++) { + ret =dma_resv_reserve_shared (bos[i]->resv,1 ); +if (ret) +return ret; +/* panfrost always uses write mode in its current uapi */ ret =drm_sched_job_add_implicit_dependencies (job, bos[i], true);
drivers/gpu/drm/ttm/ttm_bo_util.c +9 -3
@@ -221,9 +221,6 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, fbo->base = *bo; -ttm_bo_get (bo); - fbo->bo = bo; -/** * Fix up members that we shouldn't copy directly: * TODO: Explicit member copy would probably be better here.
@@ -246,6 +243,15 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, ret =dma_resv_trylock (&fbo->base.base._resv);WARN_ON (!ret); + ret =dma_resv_reserve_shared (&fbo->base.base._resv,1 ); +if (ret) { +kfree (fbo); +return ret; + } + +ttm_bo_get (bo); + fbo->bo = bo; +ttm_bo_move_to_lru_tail_unlocked (&fbo->base); *new_obj = &fbo->base;
drivers/gpu/drm/ttm/ttm_execbuf_util.c +5 -6
@@ -90,6 +90,7 @@ int ttm_eu_reserve_buffers(struct ww_acquire_ctx *ticket,list_for_each_entry (entry, list, head) {struct ttm_buffer_object *bo = entry->bo; +unsigned int num_fences; ret =ttm_bo_reserve (bo, intr, (ticket ==NULL ), ticket);if (ret == -EALREADY && dups) {
@@ -100,12 +101,10 @@ int ttm_eu_reserve_buffers(struct ww_acquire_ctx *ticket,continue ; } + num_fences =min (entry->num_shared,1u );if (!ret) { -if (!entry->num_shared) -continue ; - ret =dma_resv_reserve_shared (bo->base.resv, - entry->num_shared); + num_fences);if (!ret)continue ; }
@@ -120,9 +119,9 @@ int ttm_eu_reserve_buffers(struct ww_acquire_ctx *ticket, ret =ttm_bo_reserve_slowpath (bo, intr, ticket); } -if (!ret && entry->num_shared) +if (!ret) ret =dma_resv_reserve_shared (bo->base.resv, - entry->num_shared); + num_fences);if (unlikely (ret !=0 )) {if (ticket) {
drivers/gpu/drm/v3d/v3d_gem.c +10 -5
@@ -259,16 +259,21 @@ v3d_lock_bo_reservations(struct v3d_job *job,return ret;for (i =0 ; i < job->bo_count; i++) { + ret =dma_resv_reserve_shared (job->bo[i]->resv,1 ); +if (ret) + goto fail; + ret =drm_sched_job_add_implicit_dependencies (&job->base, job->bo[i], true); -if (ret) { -drm_gem_unlock_reservations (job->bo, job->bo_count, - acquire_ctx); -return ret; - } +if (ret) + goto fail; }return 0 ; + +fail: +drm_gem_unlock_reservations (job->bo, job->bo_count, acquire_ctx); +return ret; }/**
drivers/gpu/drm/vgem/vgem_fence.c +7 -5
@@ -157,12 +157,14 @@ int vgem_fence_attach_ioctl(struct drm_device *dev, }/* Expose the fence via the dma-buf */ - ret =0 ;dma_resv_lock (resv,NULL ); -if (arg->flags &VGEM_FENCE_WRITE ) -dma_resv_add_excl_fence (resv, fence); -else if ((ret =dma_resv_reserve_shared (resv,1 )) ==0 ) -dma_resv_add_shared_fence (resv, fence); + ret =dma_resv_reserve_shared (resv,1 ); +if (!ret) { +if (arg->flags &VGEM_FENCE_WRITE ) +dma_resv_add_excl_fence (resv, fence); +else +dma_resv_add_shared_fence (resv, fence); + }dma_resv_unlock (resv);/* Record the fence in our idr for later signaling */
drivers/gpu/drm/virtio/virtgpu_gem.c +9 -0
@@ -214,6 +214,7 @@ void virtio_gpu_array_add_obj(struct virtio_gpu_object_array *objs,int virtio_gpu_array_lock_resv (struct virtio_gpu_object_array *objs) { +unsigned int i;int ret;if (objs->nents ==1 ) {
@@ -222,6 +223,14 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs) ret =drm_gem_lock_reservations (objs->objs, objs->nents, &objs->ticket); } +if (ret) +return ret; + +for (i =0 ; i < objs->nents; ++i) { + ret =dma_resv_reserve_shared (objs->objs[i]->resv,1 ); +if (ret) +return ret; + }return ret; }
drivers/gpu/drm/vmwgfx/vmwgfx_bo.c +11 -5
@@ -1053,16 +1053,22 @@ void vmw_bo_fence_single(struct ttm_buffer_object *bo, struct vmw_fence_obj *fence) {struct ttm_device *bdev = bo->bdev; -struct vmw_private *dev_priv = container_of(bdev, struct vmw_private, bdev); +int ret; -if (fence ==NULL ) { +if (fence ==NULL )vmw_execbuf_fence_commands (NULL , dev_priv, &fence,NULL ); +else +dma_fence_get (&fence->base); + + ret =dma_resv_reserve_shared (bo->base.resv,1 ); +if (!ret)dma_resv_add_excl_fence (bo->base.resv, &fence->base); -dma_fence_put (&fence->base); - }else -dma_resv_add_excl_fence (bo->base.resv, &fence->base); +else +/* Last resort fallback when we are OOM */ +dma_fence_wait (&fence->base, false); +dma_fence_put (&fence->base); }
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 19/26 drm: support more than one write fence in drm_gem_plane_helper_prepare_fb
In reply to Christian KönigUse dma_resv_get_singleton() here to eventually get more than one write fence as single fence.
--- drivers/gpu/drm/drm_gem_atomic_helper.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)
drivers/gpu/drm/drm_gem_atomic_helper.c +7 -11
@@ -143,25 +143,21 @@*/ int drm_gem_plane_helper_prepare_fb (struct drm_plane *plane,struct drm_plane_state *state) { -struct dma_resv_iter cursor;struct drm_gem_object *obj;struct dma_fence *fence; +int ret;if (!state->fb)return 0 ; obj =drm_gem_fb_get_obj (state->fb,0 ); -dma_resv_iter_begin (&cursor, obj->resv, false); -dma_resv_for_each_fence_unlocked (&cursor, fence) { -/* TODO: Currently there should be only one write fence, so this -* here works fine. But drm_atomic_set_fence_for_plane() should -* be changed to be able to handle more fences in general for -* multiple BOs per fb anyway. */ -dma_fence_get (fence); -break ; - } -dma_resv_iter_end (&cursor); + ret =dma_resv_get_singleton (obj->resv, false, &fence); +if (ret) +return ret; +/* TODO: drm_atomic_set_fence_for_plane() should be changed to be able +* to handle more fences in general for multiple BOs per fb. +*/ drm_atomic_set_fence_for_plane (state, fence);return 0 ; }
Signature · 1 line
2.25.1
Re: [PATCH 19/26] drm: support more than one write fence in drm_gem_plane_helper_prepare_fb
Quoted 2 lines · Christian König
Use
dma_resv_get_singleton()here to eventually get more than one write fence as single fence.
Yeah this is nice, atomic commit helpers not supporting multiple write fences was really my main worry in this entire endeavour. Otherwise looks all rather reasonable.
I'll try to find some review bandwidth, but would be really if you can volunteer others too (especially making sure ttm drivers set the KERNEL fences correctly in all cases).
-Daniel
Quoted 46 lines
Signed-off-by Christian König <[email protected]>--- drivers/gpu/drm/drm_gem_atomic_helper.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)drivers/gpu/drm/drm_gem_atomic_helper.c +7
-11@@ -143,25 +143,21 @@*/ int drm_gem_plane_helper_prepare_fb (struct drm_plane *plane,struct drm_plane_state *state) { -struct dma_resv_iter cursor;struct drm_gem_object *obj;struct dma_fence *fence; +int ret;if (!state->fb)return 0 ; obj =drm_gem_fb_get_obj (state->fb,0 ); -dma_resv_iter_begin (&cursor, obj->resv, false); -dma_resv_for_each_fence_unlocked (&cursor, fence) { -/* TODO: Currently there should be only one write fence, so this -* here works fine. But drm_atomic_set_fence_for_plane() should -* be changed to be able to handle more fences in general for -* multiple BOs per fb anyway. */ -dma_fence_get (fence); -break ; - } -dma_resv_iter_end (&cursor); + ret =dma_resv_get_singleton (obj->resv, false, &fence); +if (ret) +return ret; +/* TODO: drm_atomic_set_fence_for_plane() should be changed to be able +* to handle more fences in general for multiple BOs per fb. +*/ drm_atomic_set_fence_for_plane (state, fence);return 0 ; }Signature · 1 line
2.25.1
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Christian König
reviewerRe: [PATCH 19/26] drm: support more than one write fence in drm_gem_plane_helper_prepare_fb
Am 25.11.21 um 16:47 schrieb Daniel Vetter:
Quoted 10 lines · Christian König
Quoted 2 lines
Use
dma_resv_get_singleton()here to eventually get more than one write fence as single fence.Yeah this is nice, atomic commit helpers not supporting multiple write fences was really my main worry in this entire endeavour. Otherwise looks all rather reasonable.
I'll try to find some review bandwidth, but would be really if you can volunteer others too (especially making sure ttm drivers set the KERNEL fences correctly in all cases).
Maybe I should split that up into switching over to adding the enum and then switching to kernel/bookkeep(previously other) for some use cases.
It would be good if I could get an rb on the trivial driver cleanups first. I can send those out individually if that helps.
Thanks,
Christian.
Quoted 48 lines
-Daniel
Quoted 45 lines
Signed-off-by Christian König <[email protected]>--- drivers/gpu/drm/drm_gem_atomic_helper.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)drivers/gpu/drm/drm_gem_atomic_helper.c +7
-11@@ -143,25 +143,21 @@*/ int drm_gem_plane_helper_prepare_fb (struct drm_plane *plane,struct drm_plane_state *state) { -struct dma_resv_iter cursor;struct drm_gem_object *obj;struct dma_fence *fence; +int ret;if (!state->fb)return 0 ; obj =drm_gem_fb_get_obj (state->fb,0 ); -dma_resv_iter_begin (&cursor, obj->resv, false); -dma_resv_for_each_fence_unlocked (&cursor, fence) { -/* TODO: Currently there should be only one write fence, so this -* here works fine. But drm_atomic_set_fence_for_plane() should -* be changed to be able to handle more fences in general for -* multiple BOs per fb anyway. */ -dma_fence_get (fence); -break ; - } -dma_resv_iter_end (&cursor); + ret =dma_resv_get_singleton (obj->resv, false, &fence); +if (ret) +return ret; +/* TODO: drm_atomic_set_fence_for_plane() should be changed to be able +* to handle more fences in general for multiple BOs per fb. +*/ drm_atomic_set_fence_for_plane (state, fence);return 0 ; }Signature · 1 line
2.25.1
Re: [PATCH 19/26] drm: support more than one write fence in drm_gem_plane_helper_prepare_fb
Quoted 12 lines · Christian König
Am 25.11.21 um 16:47 schrieb Daniel Vetter:
Quoted 10 lines · Christian König
Quoted 2 lines
Use
dma_resv_get_singleton()here to eventually get more than one write fence as single fence.Yeah this is nice, atomic commit helpers not supporting multiple write fences was really my main worry in this entire endeavour. Otherwise looks all rather reasonable.
I'll try to find some review bandwidth, but would be really if you can volunteer others too (especially making sure ttm drivers set the KERNEL fences correctly in all cases).
Quoted 5 lines
Maybe I should split that up into switching over to adding the enum and then switching to kernel/bookkeep(previously other) for some use cases.
It would be good if I could get an rb on the trivial driver cleanups first. I can send those out individually if that helps.
Yeah some of the conversion patches might make sense to be split a bit more. Especially when there's functional changes hiding, but I tought you've split them out? But didn't read them in detail.
Either way for stuff like this I think it's always best to split the mechanical stuff from the concept intro/docs and functional changes, except when it's really very obvious what's going on and just as small patch.
-Daniel
Quoted 50 lines
Thanks,
Christian.Quoted 45 lines
-Daniel
Quoted 42 lines
Signed-off-by Christian König <[email protected]>--- drivers/gpu/drm/drm_gem_atomic_helper.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)drivers/gpu/drm/drm_gem_atomic_helper.c +7
-11@@ -143,25 +143,21 @@*/ int drm_gem_plane_helper_prepare_fb (struct drm_plane *plane,struct drm_plane_state *state) { -struct dma_resv_iter cursor;struct drm_gem_object *obj;struct dma_fence *fence; +int ret;if (!state->fb)return 0 ; obj =drm_gem_fb_get_obj (state->fb,0 ); -dma_resv_iter_begin (&cursor, obj->resv, false); -dma_resv_for_each_fence_unlocked (&cursor, fence) { -/* TODO: Currently there should be only one write fence, so this -* here works fine. But drm_atomic_set_fence_for_plane() should -* be changed to be able to handle more fences in general for -* multiple BOs per fb anyway. */ -dma_fence_get (fence); -break ; - } -dma_resv_iter_end (&cursor); + ret =dma_resv_get_singleton (obj->resv, false, &fence); +if (ret) +return ret; +/* TODO: drm_atomic_set_fence_for_plane() should be changed to be able +* to handle more fences in general for multiple BOs per fb. +*/ drm_atomic_set_fence_for_plane (state, fence);return 0 ; }Signature · 1 line
2.25.1
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Christian König
reviewerPATCH 20/26 drm/nouveau: support more than one write fence in fenv50_wndw_prepare_fb
In reply to Christian KönigUse dma_resv_get_singleton() here to eventually get more than one write fence as single fence.
--- drivers/gpu/drm/nouveau/dispnv50/wndw.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)
drivers/gpu/drm/nouveau/dispnv50/wndw.c +5 -9
@@ -536,8 +536,6 @@ nv50_wndw_prepare_fb(struct drm_plane *plane, struct drm_plane_state *state)struct nouveau_bo *nvbo;struct nv50_head_atom *asyh;struct nv50_wndw_ctxdma *ctxdma; -struct dma_resv_iter cursor; -struct dma_fence *fence;int ret;NV_ATOMIC (drm,"%s prepare: %p\n" , plane->name, fb);
@@ -560,13 +558,11 @@ nv50_wndw_prepare_fb(struct drm_plane *plane, struct drm_plane_state *state) asyw->image.handle[0 ] = ctxdma->object.handle; } -dma_resv_iter_begin (&cursor, nvbo->bo.base.resv, false); -dma_resv_for_each_fence_unlocked (&cursor, fence) { -/* TODO: We only use the first writer here */ - asyw->state.fence =dma_fence_get (fence); -break ; - } -dma_resv_iter_end (&cursor); + ret =dma_resv_get_singleton (nvbo->bo.base.resv, false, + &asyw->state.fence); +if (ret) +return ret; + asyw->image.offset[0 ] = nvbo->offset;if (wndw->func->prepare) {
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 21/26 drm/amdgpu: use dma_resv_get_singleton in amdgpu_pasid_free_cb
In reply to Christian KönigMakes the code a bit more simpler.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +3 -20
@@ -107,36 +107,19 @@ static void amdgpu_pasid_free_cb(struct dma_fence *fence,void amdgpu_pasid_free_delayed (struct dma_resv *resv,u32 pasid) { -struct dma_fence *fence, **fences;struct amdgpu_pasid_cb *cb; -unsigned count; +struct dma_fence *fence;int r; - r =dma_resv_get_fences (resv, true, &count, &fences); + r =dma_resv_get_singleton (resv, true, &fence);if (r) goto fallback; -if (count ==0 ) { +if (!fence) {amdgpu_pasid_free (pasid);return ; } -if (count ==1 ) { - fence = fences[0 ]; -kfree (fences); - }else { -uint64_t context =dma_fence_context_alloc (1 ); -struct dma_fence_array *array; - - array =dma_fence_array_create (count, fences, context, -1 , false); -if (!array) { -kfree (fences); - goto fallback; - } - fence = &array->base; - } - cb =kmalloc (sizeof (*cb),GFP_KERNEL );if (!cb) {/* Last resort when we are OOM */
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 22/26 dma-buf: add enum dma_resv_usage
In reply to Christian KönigThis change adds the dma_resv_usage enum and allows us to specify why a dma_resv object is queried for its containing fences.
Additional to that a dma_resv_usage_rw() helper function is added to aid retrieving the fences for a read or write userspace submission.
This is then deployed to the different query functions of the dma_resv object and all of their users.
--- drivers/dma-buf/dma-buf.c | 3 +- drivers/dma-buf/dma-resv.c | 33 +++--- drivers/dma-buf/st-dma-resv.c | 48 ++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_display.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 3 +- drivers/gpu/drm/drm_gem.c | 6 +- drivers/gpu/drm/drm_gem_atomic_helper.c | 2 +- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 6 +- drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 10 +- .../gpu/drm/i915/display/intel_atomic_plane.c | 3 +- drivers/gpu/drm/i915/gem/i915_gem_busy.c | 4 +- drivers/gpu/drm/i915/gem/i915_gem_lmem.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_wait.c | 6 +- .../drm/i915/gem/selftests/i915_gem_dmabuf.c | 3 +- drivers/gpu/drm/i915/i915_request.c | 3 +- drivers/gpu/drm/i915/i915_sw_fence.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 3 +- drivers/gpu/drm/nouveau/dispnv50/wndw.c | 3 +- drivers/gpu/drm/nouveau/nouveau_bo.c | 8 +- drivers/gpu/drm/nouveau/nouveau_fence.c | 3 +- drivers/gpu/drm/nouveau/nouveau_gem.c | 3 +- drivers/gpu/drm/panfrost/panfrost_drv.c | 3 +- drivers/gpu/drm/qxl/qxl_debugfs.c | 3 +- drivers/gpu/drm/radeon/radeon_display.c | 3 +- drivers/gpu/drm/radeon/radeon_gem.c | 9 +- drivers/gpu/drm/radeon/radeon_mn.c | 4 +- drivers/gpu/drm/radeon/radeon_sync.c | 2 +- drivers/gpu/drm/radeon/radeon_uvd.c | 4 +- drivers/gpu/drm/scheduler/sched_main.c | 3 +- drivers/gpu/drm/ttm/ttm_bo.c | 18 +-- drivers/gpu/drm/vgem/vgem_fence.c | 4 +- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +- drivers/gpu/drm/vmwgfx/vmwgfx_bo.c | 4 +- drivers/gpu/drm/vmwgfx/vmwgfx_resource.c | 4 +- drivers/infiniband/core/umem_dmabuf.c | 3 +- include/linux/dma-resv.h | 106 +++++++++++++++--- 46 files changed, 244 insertions(+), 126 deletions(-)
drivers/dma-buf/dma-buf.c +2 -1
@@ -1124,7 +1124,8 @@ static int __dma_buf_begin_cpu_access(struct dma_buf *dmabuf,long ret;/* Wait on any implicit rendering fences */ - ret =dma_resv_wait_timeout (resv, write, true,MAX_SCHEDULE_TIMEOUT ); + ret =dma_resv_wait_timeout (resv,dma_resv_usage_rw (write), + true,MAX_SCHEDULE_TIMEOUT );if (ret <0 )return ret;
drivers/dma-buf/dma-resv.c +16 -17
@@ -473,7 +473,7 @@ static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor) cursor->seq =read_seqcount_begin (&cursor->obj->seq); cursor->index =-1 ; cursor->shared_count =0 ; -if (cursor->all_fences) { +if (cursor->usage >=DMA_RESV_USAGE_READ ) { cursor->fences =dma_resv_shared_list (cursor->obj);if (cursor->fences) cursor->shared_count = cursor->fences->shared_count;
@@ -580,7 +580,7 @@ struct dma_fence *dma_resv_iter_first(struct dma_resv_iter *cursor)dma_resv_assert_held (cursor->obj); cursor->index =0 ; -if (cursor->all_fences) +if (cursor->usage >=DMA_RESV_USAGE_READ ) cursor->fences =dma_resv_shared_list (cursor->obj);else cursor->fences =NULL ;
@@ -635,7 +635,7 @@ int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv *src) list =NULL ; excl =NULL ; -dma_resv_iter_begin (&cursor, src, true); +dma_resv_iter_begin (&cursor, src,DMA_RESV_USAGE_OTHER );dma_resv_for_each_fence_unlocked (&cursor, f) {if (dma_resv_iter_is_restarted (&cursor)) {
@@ -681,7 +681,7 @@ EXPORT_SYMBOL(dma_resv_copy_fences);* dma_resv_get_fences - Get an object's shared and exclusive * fences without update side lock held * @obj: the reservation object -* @write: true if we should return all fences +* @usage: controls which fences to include * @num_fences: the number of fences returned * @fences: the array of fence ptrs returned (array is krealloc'd to the * required size, and must be freed by caller)
@@ -689,7 +689,7 @@ EXPORT_SYMBOL(dma_resv_copy_fences);* Retrieve all fences from the reservation object. * Returns either zero or -ENOMEM. */ -int dma_resv_get_fences (struct dma_resv *obj,bool write, +int dma_resv_get_fences (struct dma_resv *obj,enum dma_resv_usage usage,unsigned int *num_fences,struct dma_fence ***fences) {struct dma_resv_iter cursor;
@@ -698,7 +698,7 @@ int dma_resv_get_fences(struct dma_resv *obj, bool write, *num_fences =0 ; *fences =NULL ; -dma_resv_iter_begin (&cursor, obj, write); +dma_resv_iter_begin (&cursor, obj, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) {if (dma_resv_iter_is_restarted (&cursor)) {
@@ -730,7 +730,7 @@ EXPORT_SYMBOL_GPL(dma_resv_get_fences);/** * dma_resv_get_singleton - Get a single fence for all the fences * @obj: the reservation object -* @write: true if we should return all fences +* @usage: controls which fences to include * @fence: the resulting fence * * Get a single fence representing all the fences inside the resv object.
@@ -740,7 +740,7 @@ EXPORT_SYMBOL_GPL(dma_resv_get_fences);* object since that can lead to stack corruption when finalizing the * dma_fence_array. */ -int dma_resv_get_singleton (struct dma_resv *obj,bool write, +int dma_resv_get_singleton (struct dma_resv *obj,enum dma_resv_usage usage,struct dma_fence **fence) {struct dma_fence_array *array;
@@ -748,7 +748,7 @@ int dma_resv_get_singleton(struct dma_resv *obj, bool write,unsigned count;int r; - r =dma_resv_get_fences (obj, write, &count, &fences); + r =dma_resv_get_fences (obj, usage, &count, &fences);if (r)return r;
@@ -780,7 +780,7 @@ EXPORT_SYMBOL_GPL(dma_resv_get_singleton);* dma_resv_wait_timeout - Wait on reservation's objects * shared and/or exclusive fences. * @obj: the reservation object -* @wait_all: if true, wait on all fences, else wait on just exclusive fence +* @usage: controls which fences to include in the wait * @intr: if true, do interruptible wait * @timeout: timeout value in jiffies or zero to return immediately *
@@ -790,14 +790,14 @@ EXPORT_SYMBOL_GPL(dma_resv_get_singleton);* Returns -ERESTARTSYS if interrupted, 0 if the wait timed out, or * greater than zer on success. */ -long dma_resv_wait_timeout (struct dma_resv *obj,bool wait_all,bool intr, -unsigned long timeout) +long dma_resv_wait_timeout (struct dma_resv *obj,enum dma_resv_usage usage, +bool intr,unsigned long timeout) {long ret = timeout ? timeout :1 ;struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, obj, wait_all); +dma_resv_iter_begin (&cursor, obj, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) { ret =dma_fence_wait_timeout (fence, intr, ret);
@@ -817,8 +817,7 @@ EXPORT_SYMBOL_GPL(dma_resv_wait_timeout);* dma_resv_test_signaled - Test if a reservation object's fences have been * signaled. * @obj: the reservation object -* @test_all: if true, test all fences, otherwise only test the exclusive -* fence +* @usage: controls which fences to include in the test * * Callers are not required to hold specific locks, but maybe hold * dma_resv_lock() already.
@@ -827,12 +826,12 @@ EXPORT_SYMBOL_GPL(dma_resv_wait_timeout);* * True if all fences signaled, else false. */ -bool dma_resv_test_signaled (struct dma_resv *obj,bool test_all) +bool dma_resv_test_signaled (struct dma_resv *obj,enum dma_resv_usage usage) {struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, obj, test_all); +dma_resv_iter_begin (&cursor, obj, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) {dma_resv_iter_end (&cursor);return false;
drivers/dma-buf/st-dma-resv.c +25 -23
161 lines not on this page · raw message
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +3 -1
@@ -1279,7 +1279,9 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,* submission in a dma_fence_chain and add it as exclusive * fence. */ -dma_resv_for_each_fence (&cursor, resv, false, fence) { +dma_resv_for_each_fence (&cursor, resv, +DMA_RESV_USAGE_WRITE , + fence) {break ; }dma_fence_chain_init (chain, fence,dma_fence_get (p->fence),1 );
drivers/gpu/drm/amd/amdgpu/amdgpu_display.c +1 -2
@@ -200,8 +200,7 @@ int amdgpu_display_crtc_page_flip_target(struct drm_crtc *crtc, goto unpin; } -/* TODO: Unify this with other drivers */ - r =dma_resv_get_fences (new_abo->tbo.base.resv, true, + r =dma_resv_get_fences (new_abo->tbo.base.resv,DMA_RESV_USAGE_WRITE , &work->shared_count, &work->shared);if (unlikely (r !=0 )) {
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +2 -1
@@ -526,7 +526,8 @@ int amdgpu_gem_wait_idle_ioctl(struct drm_device *dev, void *data,return -ENOENT ; } robj =gem_to_amdgpu_bo (gobj); - ret =dma_resv_wait_timeout (robj->tbo.base.resv, true, true, timeout); + ret =dma_resv_wait_timeout (robj->tbo.base.resv,DMA_RESV_USAGE_READ , + true, timeout);/* ret == 0 means not signaled, * ret > 0 means signaled
drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +3 -2
@@ -111,7 +111,7 @@ void amdgpu_pasid_free_delayed(struct dma_resv *resv,struct dma_fence *fence;int r; - r =dma_resv_get_singleton (resv, true, &fence); + r =dma_resv_get_singleton (resv,DMA_RESV_USAGE_OTHER , &fence);if (r) goto fallback;
@@ -139,7 +139,8 @@ void amdgpu_pasid_free_delayed(struct dma_resv *resv,/* Not enough memory for the delayed delete, as last resort * block for all the fences to complete. */ -dma_resv_wait_timeout (resv, true, false,MAX_SCHEDULE_TIMEOUT ); +dma_resv_wait_timeout (resv,DMA_RESV_USAGE_OTHER , + false,MAX_SCHEDULE_TIMEOUT );amdgpu_pasid_free (pasid); }
drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +2 -2
@@ -75,8 +75,8 @@ static bool amdgpu_mn_invalidate_gfx(struct mmu_interval_notifier *mni,mmu_interval_set_seq (mni, cur_seq); - r =dma_resv_wait_timeout (bo->tbo.base.resv, true, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_OTHER , + false,MAX_SCHEDULE_TIMEOUT );mutex_unlock (&adev->notifier_lock);if (r <=0 )DRM_ERROR ("(%ld) failed to wait for user bo\n" , r);
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -2
@@ -764,8 +764,8 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr)return 0 ; } - r =dma_resv_wait_timeout (bo->tbo.base.resv, false, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_KERNEL , + false,MAX_SCHEDULE_TIMEOUT );if (r <0 )return r;
drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c +2 -1
@@ -259,7 +259,8 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct amdgpu_sync *sync,if (resv ==NULL )return -EINVAL ; -dma_resv_for_each_fence (&cursor, resv, true, f) { +/* TODO: Use DMA_RESV_USAGE_READ here */ +dma_resv_for_each_fence (&cursor, resv,DMA_RESV_USAGE_OTHER , f) {dma_fence_chain_for_each (f, f) {struct dma_fence_chain *chain =to_dma_fence_chain (f);
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +2 -1
@@ -1360,7 +1360,8 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,* If true, then return false as any KFD process needs all its BOs to * be resident to run successfully */ -dma_resv_for_each_fence (&resv_cursor, bo->base.resv, true, f) { +dma_resv_for_each_fence (&resv_cursor, bo->base.resv, +DMA_RESV_USAGE_OTHER , f) {if (amdkfd_fence_check_mm (f, current->mm))return false; }
drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +2 -1
@@ -1162,7 +1162,8 @@ static int amdgpu_uvd_send_msg(struct amdgpu_ring *ring, struct amdgpu_bo *bo, ib->length_dw =16 ;if (direct) { - r =dma_resv_wait_timeout (bo->tbo.base.resv, true, false, + r =dma_resv_wait_timeout (bo->tbo.base.resv, +DMA_RESV_USAGE_KERNEL , false,msecs_to_jiffies (10 ));if (r ==0 ) r = -ETIMEDOUT ;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -3
@@ -2105,7 +2105,7 @@ static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_for_each_fence (&cursor, resv, true, fence) { +dma_resv_for_each_fence (&cursor, resv,DMA_RESV_USAGE_OTHER , fence) {/* Add a callback for each fence in the reservation object */ amdgpu_vm_prt_get (adev);amdgpu_vm_add_prt_cb (adev, fence);
@@ -2707,7 +2707,7 @@ bool amdgpu_vm_evictable(struct amdgpu_bo *bo)return true;/* Don't evict VM page tables while they are busy */ -if (!dma_resv_test_signaled (bo->tbo.base.resv, true)) +if (!dma_resv_test_signaled (bo->tbo.base.resv,DMA_RESV_USAGE_OTHER ))return false;/* Try to block ongoing updates */
@@ -2887,7 +2887,8 @@ void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint32_t min_vm_size,*/ long amdgpu_vm_wait_idle (struct amdgpu_vm *vm,long timeout) { - timeout =dma_resv_wait_timeout (vm->root.bo->tbo.base.resv, true, + timeout =dma_resv_wait_timeout (vm->root.bo->tbo.base.resv, +DMA_RESV_USAGE_OTHER , true, timeout);if (timeout <=0 )return timeout;
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +2 -1
@@ -9048,7 +9048,8 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state,* deadlock during GPU reset when this fence will not signal * but we hold reservation lock for the BO. */ - r =dma_resv_wait_timeout (abo->tbo.base.resv, true, false, + r =dma_resv_wait_timeout (abo->tbo.base.resv, +DMA_RESV_USAGE_WRITE , false,msecs_to_jiffies (5000 ));if (unlikely (r <=0 ))DRM_ERROR ("Waiting for fences timed out!" );
drivers/gpu/drm/drm_gem.c +4 -2
@@ -770,7 +770,8 @@ long drm_gem_dma_resv_wait(struct drm_file *filep, u32 handle,return -EINVAL ; } - ret =dma_resv_wait_timeout (obj->resv, wait_all, true, timeout); + ret =dma_resv_wait_timeout (obj->resv,dma_resv_usage_rw (wait_all), + true, timeout);if (ret ==0 ) ret = -ETIME ;else if (ret >0 )
@@ -1344,7 +1345,8 @@ int drm_gem_fence_array_add_implicit(struct xarray *fence_array,struct dma_fence *fence;int ret =0 ; -dma_resv_for_each_fence (&cursor, obj->resv, write, fence) { +dma_resv_for_each_fence (&cursor, obj->resv,dma_resv_usage_rw (write), + fence) { ret =drm_gem_fence_array_add (fence_array, fence);if (ret)break ;
drivers/gpu/drm/drm_gem_atomic_helper.c +1 -1
@@ -151,7 +151,7 @@ int drm_gem_plane_helper_prepare_fb(struct drm_plane *plane, struct drm_plane_streturn 0 ; obj =drm_gem_fb_get_obj (state->fb,0 ); - ret =dma_resv_get_singleton (obj->resv, false, &fence); + ret =dma_resv_get_singleton (obj->resv,DMA_RESV_USAGE_WRITE , &fence);if (ret)return ret;
drivers/gpu/drm/etnaviv/etnaviv_gem.c +4 -2
@@ -380,12 +380,14 @@ int etnaviv_gem_cpu_prep(struct drm_gem_object *obj, u32 op, }if (op &ETNA_PREP_NOSYNC ) { -if (!dma_resv_test_signaled (obj->resv, write)) +if (!dma_resv_test_signaled (obj->resv, +dma_resv_usage_rw (write)))return -EBUSY ; }else {unsigned long remain =etnaviv_timeout_to_jiffies (timeout); - ret =dma_resv_wait_timeout (obj->resv, write, true, remain); + ret =dma_resv_wait_timeout (obj->resv,dma_resv_usage_rw (write), + true, remain);if (ret <=0 )return ret ==0 ? -ETIMEDOUT : ret; }
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +6 -4
@@ -178,17 +178,19 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)for (i =0 ; i < submit->nr_bos; i++) {struct etnaviv_gem_submit_bo *bo = &submit->bos[i];struct dma_resv *robj = bo->obj->base.resv; +enum dma_resv_usage usage; ret =dma_resv_reserve_shared (robj,1 );if (ret)return ret;if (submit->flags &ETNA_SUBMIT_NO_IMPLICIT ) -continue ; + usage =DMA_RESV_USAGE_KERNEL ; +else + usage =dma_resv_usage_rw (bo->flags &ETNA_SUBMIT_BO_WRITE ); - ret =dma_resv_get_fences (robj, - !!(bo->flags &ETNA_SUBMIT_BO_WRITE ), - &bo->nr_shared, &bo->shared); + ret =dma_resv_get_fences (robj, usage, &bo->nr_shared, + &bo->shared);if (ret)return ret; }
drivers/gpu/drm/i915/display/intel_atomic_plane.c +2 -1
@@ -749,7 +749,8 @@ intel_prepare_plane_fb(struct drm_plane *_plane,if (ret <0 ) goto unpin_fb; -dma_resv_iter_begin (&cursor, obj->base.resv, false); +dma_resv_iter_begin (&cursor, obj->base.resv, +DMA_RESV_USAGE_WRITE );dma_resv_for_each_fence_unlocked (&cursor, fence) {add_rps_boost_after_vblank (new_plane_state->hw.crtc, fence);
drivers/gpu/drm/i915/gem/i915_gem_busy.c +2 -2
@@ -138,12 +138,12 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,* Alternatively, we can trade that extra information on read/write * activity with * args->busy = -* !dma_resv_test_signaled(obj->resv, true); +* !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ); * to report the overall busyness. This is what the wait-ioctl does. * */ args->busy =0 ; -dma_resv_iter_begin (&cursor, obj->base.resv, true); +dma_resv_iter_begin (&cursor, obj->base.resv,DMA_RESV_USAGE_READ );dma_resv_for_each_fence_unlocked (&cursor, fence) {if (dma_resv_iter_is_restarted (&cursor)) args->busy =0 ;
drivers/gpu/drm/i915/gem/i915_gem_lmem.c +1 -1
@@ -66,7 +66,7 @@ bool __i915_gem_object_is_lmem(struct drm_i915_gem_object *obj)struct intel_memory_region *mr =READ_ONCE (obj->mm.region);#ifdef CONFIG_LOCKDEP -GEM_WARN_ON (dma_resv_test_signaled (obj->base.resv, true) && +GEM_WARN_ON (dma_resv_test_signaled (obj->base.resv,DMA_RESV_USAGE_OTHER ) &&i915_gem_object_evictable (obj));#endif return mr && (mr->type == INTEL_MEMORY_LOCAL ||
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +1 -1
@@ -85,7 +85,7 @@ static bool i915_gem_userptr_invalidate(struct mmu_interval_notifier *mni,return true;/* we will unbind on next submission, still have userptr pins */ - r =dma_resv_wait_timeout (obj->base.resv, true, false, + r =dma_resv_wait_timeout (obj->base.resv,DMA_RESV_USAGE_OTHER , false,MAX_SCHEDULE_TIMEOUT );if (r <=0 )drm_err (&i915->drm,"(%ld) failed to wait for idle\n" , r);
drivers/gpu/drm/i915/gem/i915_gem_wait.c +4 -2
@@ -40,7 +40,8 @@ i915_gem_object_wait_reservation(struct dma_resv *resv,struct dma_fence *fence;long ret = timeout ?:1 ; -dma_resv_iter_begin (&cursor, resv, flags &I915_WAIT_ALL ); +dma_resv_iter_begin (&cursor, resv, +dma_resv_usage_rw (flags &I915_WAIT_ALL ));dma_resv_for_each_fence_unlocked (&cursor, fence) { ret =i915_gem_object_wait_fence (fence, flags, timeout);if (ret <=0 )
@@ -124,7 +125,8 @@ i915_gem_object_wait_priority(struct drm_i915_gem_object *obj,struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, obj->base.resv, flags &I915_WAIT_ALL ); +dma_resv_iter_begin (&cursor, obj->base.resv, +dma_resv_usage_rw (flags &I915_WAIT_ALL ));dma_resv_for_each_fence_unlocked (&cursor, fence)i915_gem_fence_wait_priority (fence, attr);dma_resv_iter_end (&cursor);
drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c +2 -1
@@ -219,7 +219,8 @@ static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915, goto out_detach; } - timeout =dma_resv_wait_timeout (dmabuf->resv, false, true,5 *HZ ); + timeout =dma_resv_wait_timeout (dmabuf->resv,DMA_RESV_USAGE_WRITE , + true,5 *HZ );if (!timeout) {pr_err ("dmabuf wait for exclusive fence timed out.\n" ); timeout = -ETIME ;
drivers/gpu/drm/i915/i915_request.c +2 -1
@@ -1541,7 +1541,8 @@ i915_request_await_object(struct i915_request *to,struct dma_fence *fence;int ret =0 ; -dma_resv_for_each_fence (&cursor, obj->base.resv, write, fence) { +dma_resv_for_each_fence (&cursor, obj->base.resv, +dma_resv_usage_rw (write), fence) { ret =i915_request_await_dma_fence (to, fence);if (ret)break ;
drivers/gpu/drm/i915/i915_sw_fence.c +1 -1
@@ -579,7 +579,7 @@ int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,debug_fence_assert (fence);might_sleep_if (gfpflags_allow_blocking (gfp)); -dma_resv_iter_begin (&cursor, resv, write); +dma_resv_iter_begin (&cursor, resv,dma_resv_usage_rw (write));dma_resv_for_each_fence_unlocked (&cursor, f) { pending =i915_sw_fence_await_dma_fence (fence, f, timeout, gfp);
drivers/gpu/drm/msm/msm_gem.c +2 -1
@@ -848,7 +848,8 @@ int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout) op &MSM_PREP_NOSYNC ?0 :timeout_to_jiffies (timeout);long ret; - ret =dma_resv_wait_timeout (obj->resv, write, true, remain); + ret =dma_resv_wait_timeout (obj->resv,dma_resv_usage_rw (write), + true, remain);if (ret ==0 )return remain ==0 ? -EBUSY : -ETIMEDOUT ;else if (ret <0 )
drivers/gpu/drm/nouveau/dispnv50/wndw.c +2 -1
@@ -558,7 +558,8 @@ nv50_wndw_prepare_fb(struct drm_plane *plane, struct drm_plane_state *state) asyw->image.handle[0 ] = ctxdma->object.handle; } - ret =dma_resv_get_singleton (nvbo->bo.base.resv, false, + ret =dma_resv_get_singleton (nvbo->bo.base.resv, +DMA_RESV_USAGE_WRITE , &asyw->state.fence);if (ret)return ret;
drivers/gpu/drm/nouveau/nouveau_bo.c +4 -4
@@ -962,11 +962,11 @@ nouveau_bo_vm_cleanup(struct ttm_buffer_object *bo,struct dma_fence *fence;int ret; -/* TODO: This is actually a memory management dependency */ - ret =dma_resv_get_singleton (bo->base.resv, false, &fence); + ret =dma_resv_get_singleton (bo->base.resv,DMA_RESV_USAGE_KERNEL , + &fence);if (ret) -dma_resv_wait_timeout (bo->base.resv, false, false, -MAX_SCHEDULE_TIMEOUT ); +dma_resv_wait_timeout (bo->base.resv,DMA_RESV_USAGE_KERNEL , + false,MAX_SCHEDULE_TIMEOUT );nv10_bo_put_tile_region (dev, *old_tile, fence); *old_tile = new_tile;
drivers/gpu/drm/nouveau/nouveau_fence.c +2 -1
@@ -353,7 +353,8 @@ nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan,if (ret)return ret; -dma_resv_for_each_fence (&cursor, resv, exclusive, fence) { +dma_resv_for_each_fence (&cursor, resv,dma_resv_usage_rw (exclusive), + fence) {struct nouveau_channel *prev =NULL ;bool must_wait = true;
drivers/gpu/drm/nouveau/nouveau_gem.c +2 -1
@@ -962,7 +962,8 @@ nouveau_gem_ioctl_cpu_prep(struct drm_device *dev, void *data,return -ENOENT ; nvbo =nouveau_gem_object (gem); - lret =dma_resv_wait_timeout (nvbo->bo.base.resv, write, true, + lret =dma_resv_wait_timeout (nvbo->bo.base.resv, +dma_resv_usage_rw (write), true, no_wait ?0 :30 *HZ );if (!lret) ret = -EBUSY ;
drivers/gpu/drm/panfrost/panfrost_drv.c +2 -1
@@ -316,7 +316,8 @@ panfrost_ioctl_wait_bo(struct drm_device *dev, void *data,if (!gem_obj)return -ENOENT ; - ret =dma_resv_wait_timeout (gem_obj->resv, true, true, timeout); + ret =dma_resv_wait_timeout (gem_obj->resv,DMA_RESV_USAGE_READ , + true, timeout);if (!ret) ret = timeout ? -ETIMEDOUT : -EBUSY ;
drivers/gpu/drm/qxl/qxl_debugfs.c +2 -1
@@ -61,7 +61,8 @@ qxl_debugfs_buffers_info(struct seq_file *m, void *data)struct dma_fence *fence;int rel =0 ; -dma_resv_iter_begin (&cursor, bo->tbo.base.resv, true); +dma_resv_iter_begin (&cursor, bo->tbo.base.resv, +DMA_RESV_USAGE_OTHER );dma_resv_for_each_fence_unlocked (&cursor, fence) {if (dma_resv_iter_is_restarted (&cursor)) rel =0 ;
drivers/gpu/drm/radeon/radeon_display.c +2 -1
@@ -533,7 +533,8 @@ static int radeon_crtc_page_flip_target(struct drm_crtc *crtc,DRM_ERROR ("failed to pin new rbo buffer before flip\n" ); goto cleanup; } - r =dma_resv_get_singleton (new_rbo->tbo.base.resv, false, &work->fence); + r =dma_resv_get_singleton (new_rbo->tbo.base.resv,DMA_RESV_USAGE_WRITE , + &work->fence);if (r) {radeon_bo_unreserve (new_rbo);DRM_ERROR ("failed to get new rbo buffer fences\n" );
drivers/gpu/drm/radeon/radeon_gem.c +6 -3
@@ -161,7 +161,9 @@ static int radeon_gem_set_domain(struct drm_gem_object *gobj, }if (domain ==RADEON_GEM_DOMAIN_CPU ) {/* Asking for cpu access wait for object idle */ - r =dma_resv_wait_timeout (robj->tbo.base.resv, true, true,30 *HZ ); + r =dma_resv_wait_timeout (robj->tbo.base.resv, +DMA_RESV_USAGE_OTHER , + true,30 *HZ );if (!r) r = -EBUSY ;
@@ -523,7 +525,7 @@ int radeon_gem_busy_ioctl(struct drm_device *dev, void *data, } robj =gem_to_radeon_bo (gobj); - r =dma_resv_test_signaled (robj->tbo.base.resv, true); + r =dma_resv_test_signaled (robj->tbo.base.resv,DMA_RESV_USAGE_READ );if (r ==0 ) r = -EBUSY ;else
@@ -552,7 +554,8 @@ int radeon_gem_wait_idle_ioctl(struct drm_device *dev, void *data, } robj =gem_to_radeon_bo (gobj); - ret =dma_resv_wait_timeout (robj->tbo.base.resv, true, true,30 *HZ ); + ret =dma_resv_wait_timeout (robj->tbo.base.resv,DMA_RESV_USAGE_READ , + true,30 *HZ );if (ret ==0 ) r = -EBUSY ;else if (ret <0 )
drivers/gpu/drm/radeon/radeon_mn.c +2 -2
@@ -66,8 +66,8 @@ static bool radeon_mn_invalidate(struct mmu_interval_notifier *mn,return true; } - r =dma_resv_wait_timeout (bo->tbo.base.resv, true, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_OTHER , + false,MAX_SCHEDULE_TIMEOUT );if (r <=0 )DRM_ERROR ("(%ld) failed to wait for user bo\n" , r);
drivers/gpu/drm/radeon/radeon_sync.c +1 -1
@@ -96,7 +96,7 @@ int radeon_sync_resv(struct radeon_device *rdev,struct dma_fence *f;int r =0 ; -dma_resv_for_each_fence (&cursor, resv, shared, f) { +dma_resv_for_each_fence (&cursor, resv,dma_resv_usage_rw (shared), f) { fence =to_radeon_fence (f);if (fence && fence->rdev == rdev)radeon_sync_fence (sync, fence);
drivers/gpu/drm/radeon/radeon_uvd.c +2 -2
@@ -478,8 +478,8 @@ static int radeon_uvd_cs_msg(struct radeon_cs_parser *p, struct radeon_bo *bo,return -EINVAL ; } - r =dma_resv_wait_timeout (bo->tbo.base.resv, false, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_KERNEL , + false,MAX_SCHEDULE_TIMEOUT );if (r <=0 ) {DRM_ERROR ("Failed waiting for UVD message (%d)!\n" , r);return r ? r : -ETIME ;
drivers/gpu/drm/scheduler/sched_main.c +2 -1
@@ -703,7 +703,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,struct dma_fence *fence;int ret; -dma_resv_for_each_fence (&cursor, obj->resv, write, fence) { +dma_resv_for_each_fence (&cursor, obj->resv,dma_resv_usage_rw (write), + fence) { ret =drm_sched_job_add_dependency (job, fence);if (ret)return ret;
drivers/gpu/drm/ttm/ttm_bo.c +10 -8
@@ -272,7 +272,7 @@ static void ttm_bo_flush_all_fences(struct ttm_buffer_object *bo)struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, resv, true); +dma_resv_iter_begin (&cursor, resv,DMA_RESV_USAGE_OTHER );dma_resv_for_each_fence_unlocked (&cursor, fence) {if (!fence->ops->signaled)dma_fence_enable_sw_signaling (fence);
@@ -301,7 +301,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,struct dma_resv *resv = &bo->base._resv;int ret; -if (dma_resv_test_signaled (resv, true)) +if (dma_resv_test_signaled (resv,DMA_RESV_USAGE_OTHER )) ret =0 ;else ret = -EBUSY ;
@@ -313,7 +313,8 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,dma_resv_unlock (bo->base.resv);spin_unlock (&bo->bdev->lru_lock); - lret =dma_resv_wait_timeout (resv, true, interruptible, + lret =dma_resv_wait_timeout (resv,DMA_RESV_USAGE_OTHER , + interruptible,30 *HZ );if (lret <0 )
@@ -416,7 +417,8 @@ static void ttm_bo_release(struct kref *kref)/* Last resort, if we fail to allocate memory for the * fences block for the BO to become idle */ -dma_resv_wait_timeout (bo->base.resv, true, false, +dma_resv_wait_timeout (bo->base.resv, +DMA_RESV_USAGE_OTHER , false,30 *HZ ); }
@@ -427,7 +429,7 @@ static void ttm_bo_release(struct kref *kref)ttm_mem_io_free (bdev, bo->resource); } -if (!dma_resv_test_signaled (bo->base.resv, true) || +if (!dma_resv_test_signaled (bo->base.resv,DMA_RESV_USAGE_OTHER ) || !dma_resv_trylock (bo->base.resv)) {/* The BO is not idle, resurrect it for delayed destroy */ ttm_bo_flush_all_fences (bo);
@@ -1072,14 +1074,14 @@ int ttm_bo_wait(struct ttm_buffer_object *bo,long timeout =15 *HZ ;if (no_wait) { -if (dma_resv_test_signaled (bo->base.resv, true)) +if (dma_resv_test_signaled (bo->base.resv,DMA_RESV_USAGE_OTHER ))return 0 ;else return -EBUSY ; } - timeout =dma_resv_wait_timeout (bo->base.resv, true, interruptible, - timeout); + timeout =dma_resv_wait_timeout (bo->base.resv,DMA_RESV_USAGE_OTHER , + interruptible, timeout);if (timeout <0 )return timeout;
drivers/gpu/drm/vgem/vgem_fence.c +3 -1
@@ -130,6 +130,7 @@ int vgem_fence_attach_ioctl(struct drm_device *dev,struct vgem_file *vfile = file->driver_priv;struct dma_resv *resv;struct drm_gem_object *obj; +enum dma_resv_usage usage;struct dma_fence *fence;int ret;
@@ -151,7 +152,8 @@ int vgem_fence_attach_ioctl(struct drm_device *dev,/* Check for a conflicting fence */ resv = obj->resv; -if (!dma_resv_test_signaled (resv, arg->flags &VGEM_FENCE_WRITE )) { + usage =dma_resv_usage_rw (arg->flags &VGEM_FENCE_WRITE ); +if (!dma_resv_test_signaled (resv, usage)) { ret = -EBUSY ; goto err_fence; }
drivers/gpu/drm/virtio/virtgpu_ioctl.c +3 -2
@@ -518,9 +518,10 @@ static int virtio_gpu_wait_ioctl(struct drm_device *dev, void *data,return -ENOENT ;if (args->flags &VIRTGPU_WAIT_NOWAIT ) { - ret =dma_resv_test_signaled (obj->resv, true); + ret =dma_resv_test_signaled (obj->resv,DMA_RESV_USAGE_READ ); }else { - ret =dma_resv_wait_timeout (obj->resv, true, true, timeout); + ret =dma_resv_wait_timeout (obj->resv,DMA_RESV_USAGE_READ , + true, timeout); }if (ret ==0 ) ret = -EBUSY ;
drivers/gpu/drm/vmwgfx/vmwgfx_bo.c +2 -2
@@ -739,8 +739,8 @@ static int vmw_user_bo_synccpu_grab(struct vmw_user_buffer_object *user_bo,if (flags & drm_vmw_synccpu_allow_cs) {long lret; - lret =dma_resv_wait_timeout (bo->base.resv, true, true, - nonblock ?0 : + lret =dma_resv_wait_timeout (bo->base.resv,DMA_RESV_USAGE_READ , + true, nonblock ?0 :MAX_SCHEDULE_TIMEOUT );if (!lret)return -EBUSY ;
drivers/gpu/drm/vmwgfx/vmwgfx_resource.c +2 -2
@@ -1169,8 +1169,8 @@ int vmw_resources_clean(struct vmw_buffer_object *vbo, pgoff_t start,if (bo->moving)dma_fence_put (bo->moving); -/* TODO: This is actually a memory management dependency */ -return dma_resv_get_singleton (bo->base.resv, false, +return dma_resv_get_singleton (bo->base.resv, +DMA_RESV_USAGE_KERNEL , &bo->moving); }
drivers/infiniband/core/umem_dmabuf.c +2 -1
@@ -67,7 +67,8 @@ int ib_umem_dmabuf_map_pages(struct ib_umem_dmabuf *umem_dmabuf)* may be not up-to-date. Wait for the exporter to finish * the migration. */ -return dma_resv_wait_timeout (umem_dmabuf->attach->dmabuf->resv, false, +return dma_resv_wait_timeout (umem_dmabuf->attach->dmabuf->resv, +DMA_RESV_USAGE_KERNEL , false,MAX_SCHEDULE_TIMEOUT ); }EXPORT_SYMBOL (ib_umem_dmabuf_map_pages);
include/linux/dma-resv.h +93 -13
150 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH 22/26] dma-buf: add enum dma_resv_usage
Quoted 8 lines · Christian König
This change adds the
dma_resv_usageenum and allows us to specify why adma_resvobject is queried for its containing fences.Additional to that a
dma_resv_usage_rw()helper function is added to aid retrieving the fences for a read or write userspace submission.This is then deployed to the different query functions of the
dma_resvobject and all of their users.
I think this is entirely mechanical (otherwise it needs some splitting), so would be good to list in the commit message here what all the exact transformations you've applied are.
-Daniel
Quoted 1,296 lines
Signed-off-by Christian König <[email protected]>--- drivers/dma-buf/dma-buf.c | 3 +- drivers/dma-buf/dma-resv.c | 33 +++--- drivers/dma-buf/st-dma-resv.c | 48 ++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_display.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 3 +- drivers/gpu/drm/drm_gem.c | 6 +- drivers/gpu/drm/drm_gem_atomic_helper.c | 2 +- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 6 +- drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 10 +- .../gpu/drm/i915/display/intel_atomic_plane.c | 3 +- drivers/gpu/drm/i915/gem/i915_gem_busy.c | 4 +- drivers/gpu/drm/i915/gem/i915_gem_lmem.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_wait.c | 6 +- .../drm/i915/gem/selftests/i915_gem_dmabuf.c | 3 +- drivers/gpu/drm/i915/i915_request.c | 3 +- drivers/gpu/drm/i915/i915_sw_fence.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 3 +- drivers/gpu/drm/nouveau/dispnv50/wndw.c | 3 +- drivers/gpu/drm/nouveau/nouveau_bo.c | 8 +- drivers/gpu/drm/nouveau/nouveau_fence.c | 3 +- drivers/gpu/drm/nouveau/nouveau_gem.c | 3 +- drivers/gpu/drm/panfrost/panfrost_drv.c | 3 +- drivers/gpu/drm/qxl/qxl_debugfs.c | 3 +- drivers/gpu/drm/radeon/radeon_display.c | 3 +- drivers/gpu/drm/radeon/radeon_gem.c | 9 +- drivers/gpu/drm/radeon/radeon_mn.c | 4 +- drivers/gpu/drm/radeon/radeon_sync.c | 2 +- drivers/gpu/drm/radeon/radeon_uvd.c | 4 +- drivers/gpu/drm/scheduler/sched_main.c | 3 +- drivers/gpu/drm/ttm/ttm_bo.c | 18 +-- drivers/gpu/drm/vgem/vgem_fence.c | 4 +- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +- drivers/gpu/drm/vmwgfx/vmwgfx_bo.c | 4 +- drivers/gpu/drm/vmwgfx/vmwgfx_resource.c | 4 +- drivers/infiniband/core/umem_dmabuf.c | 3 +- include/linux/dma-resv.h | 106 +++++++++++++++--- 46 files changed, 244 insertions(+), 126 deletions(-)drivers/dma-buf/dma-buf.c +2
-1@@ -1124,7 +1124,8 @@ static int __dma_buf_begin_cpu_access(struct dma_buf *dmabuf,long ret;/* Wait on any implicit rendering fences */ - ret =dma_resv_wait_timeout (resv, write, true,MAX_SCHEDULE_TIMEOUT ); + ret =dma_resv_wait_timeout (resv,dma_resv_usage_rw (write), + true,MAX_SCHEDULE_TIMEOUT );if (ret <0 )return ret;drivers/dma-buf/dma-resv.c +16
-17@@ -473,7 +473,7 @@ static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor) cursor->seq =read_seqcount_begin (&cursor->obj->seq); cursor->index =-1 ; cursor->shared_count =0 ; -if (cursor->all_fences) { +if (cursor->usage >=DMA_RESV_USAGE_READ ) { cursor->fences =dma_resv_shared_list (cursor->obj);if (cursor->fences) cursor->shared_count = cursor->fences->shared_count;@@ -580,7 +580,7 @@ struct dma_fence *dma_resv_iter_first(struct dma_resv_iter *cursor)dma_resv_assert_held (cursor->obj); cursor->index =0 ; -if (cursor->all_fences) +if (cursor->usage >=DMA_RESV_USAGE_READ ) cursor->fences =dma_resv_shared_list (cursor->obj);else cursor->fences =NULL ;@@ -635,7 +635,7 @@ int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv *src) list =NULL ; excl =NULL ; -dma_resv_iter_begin (&cursor, src, true); +dma_resv_iter_begin (&cursor, src,DMA_RESV_USAGE_OTHER );dma_resv_for_each_fence_unlocked (&cursor, f) {if (dma_resv_iter_is_restarted (&cursor)) {@@ -681,7 +681,7 @@ EXPORT_SYMBOL(dma_resv_copy_fences);* dma_resv_get_fences - Get an object's shared and exclusive * fences without update side lock held * @obj: the reservation object -* @write: true if we should return all fences +* @usage: controls which fences to include * @num_fences: the number of fences returned * @fences: the array of fence ptrs returned (array is krealloc'd to the * required size, and must be freed by caller) @@ -689,7 +689,7 @@ EXPORT_SYMBOL(dma_resv_copy_fences);* Retrieve all fences from the reservation object. * Returns either zero or -ENOMEM. */ -int dma_resv_get_fences (struct dma_resv *obj,bool write, +int dma_resv_get_fences (struct dma_resv *obj,enum dma_resv_usage usage,unsigned int *num_fences,struct dma_fence ***fences) {struct dma_resv_iter cursor;@@ -698,7 +698,7 @@ int dma_resv_get_fences(struct dma_resv *obj, bool write, *num_fences =0 ; *fences =NULL ; -dma_resv_iter_begin (&cursor, obj, write); +dma_resv_iter_begin (&cursor, obj, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) {if (dma_resv_iter_is_restarted (&cursor)) {@@ -730,7 +730,7 @@ EXPORT_SYMBOL_GPL(dma_resv_get_fences);/** * dma_resv_get_singleton - Get a single fence for all the fences * @obj: the reservation object -* @write: true if we should return all fences +* @usage: controls which fences to include * @fence: the resulting fence * * Get a single fence representing all the fences inside the resv object. @@ -740,7 +740,7 @@ EXPORT_SYMBOL_GPL(dma_resv_get_fences);* object since that can lead to stack corruption when finalizing the * dma_fence_array. */ -int dma_resv_get_singleton (struct dma_resv *obj,bool write, +int dma_resv_get_singleton (struct dma_resv *obj,enum dma_resv_usage usage,struct dma_fence **fence) {struct dma_fence_array *array;@@ -748,7 +748,7 @@ int dma_resv_get_singleton(struct dma_resv *obj, bool write,unsigned count;int r; - r =dma_resv_get_fences (obj, write, &count, &fences); + r =dma_resv_get_fences (obj, usage, &count, &fences);if (r)return r;@@ -780,7 +780,7 @@ EXPORT_SYMBOL_GPL(dma_resv_get_singleton);* dma_resv_wait_timeout - Wait on reservation's objects * shared and/or exclusive fences. * @obj: the reservation object -* @wait_all: if true, wait on all fences, else wait on just exclusive fence +* @usage: controls which fences to include in the wait * @intr: if true, do interruptible wait * @timeout: timeout value in jiffies or zero to return immediately * @@ -790,14 +790,14 @@ EXPORT_SYMBOL_GPL(dma_resv_get_singleton);* Returns -ERESTARTSYS if interrupted, 0 if the wait timed out, or * greater than zer on success. */ -long dma_resv_wait_timeout (struct dma_resv *obj,bool wait_all,bool intr, -unsigned long timeout) +long dma_resv_wait_timeout (struct dma_resv *obj,enum dma_resv_usage usage, +bool intr,unsigned long timeout) {long ret = timeout ? timeout :1 ;struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, obj, wait_all); +dma_resv_iter_begin (&cursor, obj, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) { ret =dma_fence_wait_timeout (fence, intr, ret);@@ -817,8 +817,7 @@ EXPORT_SYMBOL_GPL(dma_resv_wait_timeout);* dma_resv_test_signaled - Test if a reservation object's fences have been * signaled. * @obj: the reservation object -* @test_all: if true, test all fences, otherwise only test the exclusive -* fence +* @usage: controls which fences to include in the test * * Callers are not required to hold specific locks, but maybe hold * dma_resv_lock() already. @@ -827,12 +826,12 @@ EXPORT_SYMBOL_GPL(dma_resv_wait_timeout);* * True if all fences signaled, else false. */ -bool dma_resv_test_signaled (struct dma_resv *obj,bool test_all) +bool dma_resv_test_signaled (struct dma_resv *obj,enum dma_resv_usage usage) {struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, obj, test_all); +dma_resv_iter_begin (&cursor, obj, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) {dma_resv_iter_end (&cursor);return false;drivers/dma-buf/st-dma-resv.c +25
-23@@ -58,7 +58,7 @@ static int sanitycheck(void *arg)return r; } -static int test_signaling (void *arg,bool shared) +static int test_signaling (void *arg,enum dma_resv_usage usage) {struct dma_resv resv;struct dma_fence *f;@@ -81,18 +81,18 @@ static int test_signaling(void *arg, bool shared) goto err_unlock; } -if (shared) +if (usage >=DMA_RESV_USAGE_READ )dma_resv_add_shared_fence (&resv, f);else dma_resv_add_excl_fence (&resv, f); -if (dma_resv_test_signaled (&resv, shared)) { +if (dma_resv_test_signaled (&resv, usage)) {pr_err ("Resv unexpectedly signaled\n" ); r = -EINVAL ; goto err_unlock; }dma_fence_signal (f); -if (!dma_resv_test_signaled (&resv, shared)) { +if (!dma_resv_test_signaled (&resv, usage)) {pr_err ("Resv not reporting signaled\n" ); r = -EINVAL ; goto err_unlock;@@ -107,15 +107,15 @@ static int test_signaling(void *arg, bool shared)static int test_excl_signaling (void *arg) { -return test_signaling (arg, false); +return test_signaling (arg,DMA_RESV_USAGE_WRITE ); }static int test_shared_signaling (void *arg) { -return test_signaling (arg, true); +return test_signaling (arg,DMA_RESV_USAGE_READ ); } -static int test_for_each (void *arg,bool shared) +static int test_for_each (void *arg,enum dma_resv_usage usage) {struct dma_resv_iter cursor;struct dma_fence *f, *fence;@@ -139,13 +139,13 @@ static int test_for_each(void *arg, bool shared) goto err_unlock; } -if (shared) +if (usage >=DMA_RESV_USAGE_READ )dma_resv_add_shared_fence (&resv, f);else dma_resv_add_excl_fence (&resv, f); r = -ENOENT ; -dma_resv_for_each_fence (&cursor, &resv, shared, fence) { +dma_resv_for_each_fence (&cursor, &resv, usage, fence) {if (!r) {pr_err ("More than one fence found\n" ); r = -EINVAL ;@@ -156,7 +156,8 @@ static int test_for_each(void *arg, bool shared) r = -EINVAL ; goto err_unlock; } -if (dma_resv_iter_is_exclusive (&cursor) != !shared) { +if (dma_resv_iter_is_exclusive (&cursor) != + (usage >=DMA_RESV_USAGE_READ )) {pr_err ("Unexpected fence usage\n" ); r = -EINVAL ; goto err_unlock;@@ -178,15 +179,15 @@ static int test_for_each(void *arg, bool shared)static int test_excl_for_each (void *arg) { -return test_for_each (arg, false); +return test_for_each (arg,DMA_RESV_USAGE_WRITE ); }static int test_shared_for_each (void *arg) { -return test_for_each (arg, true); +return test_for_each (arg,DMA_RESV_USAGE_READ ); } -static int test_for_each_unlocked (void *arg,bool shared) +static int test_for_each_unlocked (void *arg,enum dma_resv_usage usage) {struct dma_resv_iter cursor;struct dma_fence *f, *fence;@@ -211,14 +212,14 @@ static int test_for_each_unlocked(void *arg, bool shared) goto err_free; } -if (shared) +if (usage >=DMA_RESV_USAGE_READ )dma_resv_add_shared_fence (&resv, f);else dma_resv_add_excl_fence (&resv, f);dma_resv_unlock (&resv); r = -ENOENT ; -dma_resv_iter_begin (&cursor, &resv, shared); +dma_resv_iter_begin (&cursor, &resv, usage);dma_resv_for_each_fence_unlocked (&cursor, fence) {if (!r) {pr_err ("More than one fence found\n" );@@ -234,7 +235,8 @@ static int test_for_each_unlocked(void *arg, bool shared) r = -EINVAL ; goto err_iter_end; } -if (dma_resv_iter_is_exclusive (&cursor) != !shared) { +if (dma_resv_iter_is_exclusive (&cursor) != + (usage >=DMA_RESV_USAGE_READ )) {pr_err ("Unexpected fence usage\n" ); r = -EINVAL ; goto err_iter_end;@@ -262,15 +264,15 @@ static int test_for_each_unlocked(void *arg, bool shared)static int test_excl_for_each_unlocked (void *arg) { -return test_for_each_unlocked (arg, false); +return test_for_each_unlocked (arg,DMA_RESV_USAGE_WRITE ); }static int test_shared_for_each_unlocked (void *arg) { -return test_for_each_unlocked (arg, true); +return test_for_each_unlocked (arg,DMA_RESV_USAGE_READ ); } -static int test_get_fences (void *arg,bool shared) +static int test_get_fences (void *arg,enum dma_resv_usage usage) {struct dma_fence *f, **fences =NULL ;struct dma_resv resv;@@ -294,13 +296,13 @@ static int test_get_fences(void *arg, bool shared) goto err_resv; } -if (shared) +if (usage >=DMA_RESV_USAGE_READ )dma_resv_add_shared_fence (&resv, f);else dma_resv_add_excl_fence (&resv, f);dma_resv_unlock (&resv); - r =dma_resv_get_fences (&resv, shared, &i, &fences); + r =dma_resv_get_fences (&resv, usage, &i, &fences);if (r) {pr_err ("get_fences failed\n" ); goto err_free;@@ -324,12 +326,12 @@ static int test_get_fences(void *arg, bool shared)static int test_excl_get_fences (void *arg) { -return test_get_fences (arg, false); +return test_get_fences (arg,DMA_RESV_USAGE_WRITE ); }static int test_shared_get_fences (void *arg) { -return test_get_fences (arg, true); +return test_get_fences (arg,DMA_RESV_USAGE_READ ); }int dma_resv (void )drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +3
-1@@ -1279,7 +1279,9 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,* submission in a dma_fence_chain and add it as exclusive * fence. */ -dma_resv_for_each_fence (&cursor, resv, false, fence) { +dma_resv_for_each_fence (&cursor, resv, +DMA_RESV_USAGE_WRITE , + fence) {break ; }dma_fence_chain_init (chain, fence,dma_fence_get (p->fence),1 );drivers/gpu/drm/amd/amdgpu/amdgpu_display.c +1
-2@@ -200,8 +200,7 @@ int amdgpu_display_crtc_page_flip_target(struct drm_crtc *crtc, goto unpin; } -/* TODO: Unify this with other drivers */ - r =dma_resv_get_fences (new_abo->tbo.base.resv, true, + r =dma_resv_get_fences (new_abo->tbo.base.resv,DMA_RESV_USAGE_WRITE , &work->shared_count, &work->shared);if (unlikely (r !=0 )) {drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +2
-1@@ -526,7 +526,8 @@ int amdgpu_gem_wait_idle_ioctl(struct drm_device *dev, void *data,return -ENOENT ; } robj =gem_to_amdgpu_bo (gobj); - ret =dma_resv_wait_timeout (robj->tbo.base.resv, true, true, timeout); + ret =dma_resv_wait_timeout (robj->tbo.base.resv,DMA_RESV_USAGE_READ , + true, timeout);/* ret == 0 means not signaled, * ret > 0 means signaled drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +3
-2@@ -111,7 +111,7 @@ void amdgpu_pasid_free_delayed(struct dma_resv *resv,struct dma_fence *fence;int r; - r =dma_resv_get_singleton (resv, true, &fence); + r =dma_resv_get_singleton (resv,DMA_RESV_USAGE_OTHER , &fence);if (r) goto fallback;@@ -139,7 +139,8 @@ void amdgpu_pasid_free_delayed(struct dma_resv *resv,/* Not enough memory for the delayed delete, as last resort * block for all the fences to complete. */ -dma_resv_wait_timeout (resv, true, false,MAX_SCHEDULE_TIMEOUT ); +dma_resv_wait_timeout (resv,DMA_RESV_USAGE_OTHER , + false,MAX_SCHEDULE_TIMEOUT );amdgpu_pasid_free (pasid); }drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +2
-2@@ -75,8 +75,8 @@ static bool amdgpu_mn_invalidate_gfx(struct mmu_interval_notifier *mni,mmu_interval_set_seq (mni, cur_seq); - r =dma_resv_wait_timeout (bo->tbo.base.resv, true, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_OTHER , + false,MAX_SCHEDULE_TIMEOUT );mutex_unlock (&adev->notifier_lock);if (r <=0 )DRM_ERROR ("(%ld) failed to wait for user bo\n" , r);drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2
-2@@ -764,8 +764,8 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr)return 0 ; } - r =dma_resv_wait_timeout (bo->tbo.base.resv, false, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_KERNEL , + false,MAX_SCHEDULE_TIMEOUT );if (r <0 )return r;drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c +2
-1@@ -259,7 +259,8 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct amdgpu_sync *sync,if (resv ==NULL )return -EINVAL ; -dma_resv_for_each_fence (&cursor, resv, true, f) { +/* TODO: Use DMA_RESV_USAGE_READ here */ +dma_resv_for_each_fence (&cursor, resv,DMA_RESV_USAGE_OTHER , f) {dma_fence_chain_for_each (f, f) {struct dma_fence_chain *chain =to_dma_fence_chain (f);drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +2
-1@@ -1360,7 +1360,8 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,* If true, then return false as any KFD process needs all its BOs to * be resident to run successfully */ -dma_resv_for_each_fence (&resv_cursor, bo->base.resv, true, f) { +dma_resv_for_each_fence (&resv_cursor, bo->base.resv, +DMA_RESV_USAGE_OTHER , f) {if (amdkfd_fence_check_mm (f, current->mm))return false; }drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +2
-1@@ -1162,7 +1162,8 @@ static int amdgpu_uvd_send_msg(struct amdgpu_ring *ring, struct amdgpu_bo *bo, ib->length_dw =16 ;if (direct) { - r =dma_resv_wait_timeout (bo->tbo.base.resv, true, false, + r =dma_resv_wait_timeout (bo->tbo.base.resv, +DMA_RESV_USAGE_KERNEL , false,msecs_to_jiffies (10 ));if (r ==0 ) r = -ETIMEDOUT ;drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4
-3@@ -2105,7 +2105,7 @@ static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_for_each_fence (&cursor, resv, true, fence) { +dma_resv_for_each_fence (&cursor, resv,DMA_RESV_USAGE_OTHER , fence) {/* Add a callback for each fence in the reservation object */ amdgpu_vm_prt_get (adev);amdgpu_vm_add_prt_cb (adev, fence);@@ -2707,7 +2707,7 @@ bool amdgpu_vm_evictable(struct amdgpu_bo *bo)return true;/* Don't evict VM page tables while they are busy */ -if (!dma_resv_test_signaled (bo->tbo.base.resv, true)) +if (!dma_resv_test_signaled (bo->tbo.base.resv,DMA_RESV_USAGE_OTHER ))return false;/* Try to block ongoing updates */ @@ -2887,7 +2887,8 @@ void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint32_t min_vm_size,*/ long amdgpu_vm_wait_idle (struct amdgpu_vm *vm,long timeout) { - timeout =dma_resv_wait_timeout (vm->root.bo->tbo.base.resv, true, + timeout =dma_resv_wait_timeout (vm->root.bo->tbo.base.resv, +DMA_RESV_USAGE_OTHER , true, timeout);if (timeout <=0 )return timeout;drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +2
-1@@ -9048,7 +9048,8 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state,* deadlock during GPU reset when this fence will not signal * but we hold reservation lock for the BO. */ - r =dma_resv_wait_timeout (abo->tbo.base.resv, true, false, + r =dma_resv_wait_timeout (abo->tbo.base.resv, +DMA_RESV_USAGE_WRITE , false,msecs_to_jiffies (5000 ));if (unlikely (r <=0 ))DRM_ERROR ("Waiting for fences timed out!" );drivers/gpu/drm/drm_gem.c +4
-2@@ -770,7 +770,8 @@ long drm_gem_dma_resv_wait(struct drm_file *filep, u32 handle,return -EINVAL ; } - ret =dma_resv_wait_timeout (obj->resv, wait_all, true, timeout); + ret =dma_resv_wait_timeout (obj->resv,dma_resv_usage_rw (wait_all), + true, timeout);if (ret ==0 ) ret = -ETIME ;else if (ret >0 )@@ -1344,7 +1345,8 @@ int drm_gem_fence_array_add_implicit(struct xarray *fence_array,struct dma_fence *fence;int ret =0 ; -dma_resv_for_each_fence (&cursor, obj->resv, write, fence) { +dma_resv_for_each_fence (&cursor, obj->resv,dma_resv_usage_rw (write), + fence) { ret =drm_gem_fence_array_add (fence_array, fence);if (ret)break ;drivers/gpu/drm/drm_gem_atomic_helper.c +1
-1@@ -151,7 +151,7 @@ int drm_gem_plane_helper_prepare_fb(struct drm_plane *plane, struct drm_plane_streturn 0 ; obj =drm_gem_fb_get_obj (state->fb,0 ); - ret =dma_resv_get_singleton (obj->resv, false, &fence); + ret =dma_resv_get_singleton (obj->resv,DMA_RESV_USAGE_WRITE , &fence);if (ret)return ret;drivers/gpu/drm/etnaviv/etnaviv_gem.c +4
-2@@ -380,12 +380,14 @@ int etnaviv_gem_cpu_prep(struct drm_gem_object *obj, u32 op, }if (op &ETNA_PREP_NOSYNC ) { -if (!dma_resv_test_signaled (obj->resv, write)) +if (!dma_resv_test_signaled (obj->resv, +dma_resv_usage_rw (write)))return -EBUSY ; }else {unsigned long remain =etnaviv_timeout_to_jiffies (timeout); - ret =dma_resv_wait_timeout (obj->resv, write, true, remain); + ret =dma_resv_wait_timeout (obj->resv,dma_resv_usage_rw (write), + true, remain);if (ret <=0 )return ret ==0 ? -ETIMEDOUT : ret; }drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +6
-4@@ -178,17 +178,19 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)for (i =0 ; i < submit->nr_bos; i++) {struct etnaviv_gem_submit_bo *bo = &submit->bos[i];struct dma_resv *robj = bo->obj->base.resv; +enum dma_resv_usage usage; ret =dma_resv_reserve_shared (robj,1 );if (ret)return ret;if (submit->flags &ETNA_SUBMIT_NO_IMPLICIT ) -continue ; + usage =DMA_RESV_USAGE_KERNEL ; +else + usage =dma_resv_usage_rw (bo->flags &ETNA_SUBMIT_BO_WRITE ); - ret =dma_resv_get_fences (robj, - !!(bo->flags &ETNA_SUBMIT_BO_WRITE ), - &bo->nr_shared, &bo->shared); + ret =dma_resv_get_fences (robj, usage, &bo->nr_shared, + &bo->shared);if (ret)return ret; }drivers/gpu/drm/i915/display/intel_atomic_plane.c +2
-1@@ -749,7 +749,8 @@ intel_prepare_plane_fb(struct drm_plane *_plane,if (ret <0 ) goto unpin_fb; -dma_resv_iter_begin (&cursor, obj->base.resv, false); +dma_resv_iter_begin (&cursor, obj->base.resv, +DMA_RESV_USAGE_WRITE );dma_resv_for_each_fence_unlocked (&cursor, fence) {add_rps_boost_after_vblank (new_plane_state->hw.crtc, fence);drivers/gpu/drm/i915/gem/i915_gem_busy.c +2
-2@@ -138,12 +138,12 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,* Alternatively, we can trade that extra information on read/write * activity with * args->busy = -* !dma_resv_test_signaled(obj->resv, true); +* !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ); * to report the overall busyness. This is what the wait-ioctl does. * */ args->busy =0 ; -dma_resv_iter_begin (&cursor, obj->base.resv, true); +dma_resv_iter_begin (&cursor, obj->base.resv,DMA_RESV_USAGE_READ );dma_resv_for_each_fence_unlocked (&cursor, fence) {if (dma_resv_iter_is_restarted (&cursor)) args->busy =0 ;drivers/gpu/drm/i915/gem/i915_gem_lmem.c +1
-1@@ -66,7 +66,7 @@ bool __i915_gem_object_is_lmem(struct drm_i915_gem_object *obj)struct intel_memory_region *mr =READ_ONCE (obj->mm.region);#ifdef CONFIG_LOCKDEP -GEM_WARN_ON (dma_resv_test_signaled (obj->base.resv, true) && +GEM_WARN_ON (dma_resv_test_signaled (obj->base.resv,DMA_RESV_USAGE_OTHER ) &&i915_gem_object_evictable (obj));#endif return mr && (mr->type == INTEL_MEMORY_LOCAL ||drivers/gpu/drm/i915/gem/i915_gem_userptr.c +1
-1@@ -85,7 +85,7 @@ static bool i915_gem_userptr_invalidate(struct mmu_interval_notifier *mni,return true;/* we will unbind on next submission, still have userptr pins */ - r =dma_resv_wait_timeout (obj->base.resv, true, false, + r =dma_resv_wait_timeout (obj->base.resv,DMA_RESV_USAGE_OTHER , false,MAX_SCHEDULE_TIMEOUT );if (r <=0 )drm_err (&i915->drm,"(%ld) failed to wait for idle\n" , r);drivers/gpu/drm/i915/gem/i915_gem_wait.c +4
-2@@ -40,7 +40,8 @@ i915_gem_object_wait_reservation(struct dma_resv *resv,struct dma_fence *fence;long ret = timeout ?:1 ; -dma_resv_iter_begin (&cursor, resv, flags &I915_WAIT_ALL ); +dma_resv_iter_begin (&cursor, resv, +dma_resv_usage_rw (flags &I915_WAIT_ALL ));dma_resv_for_each_fence_unlocked (&cursor, fence) { ret =i915_gem_object_wait_fence (fence, flags, timeout);if (ret <=0 )@@ -124,7 +125,8 @@ i915_gem_object_wait_priority(struct drm_i915_gem_object *obj,struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, obj->base.resv, flags &I915_WAIT_ALL ); +dma_resv_iter_begin (&cursor, obj->base.resv, +dma_resv_usage_rw (flags &I915_WAIT_ALL ));dma_resv_for_each_fence_unlocked (&cursor, fence)i915_gem_fence_wait_priority (fence, attr);dma_resv_iter_end (&cursor);drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c +2
-1@@ -219,7 +219,8 @@ static int igt_dmabuf_import_same_driver(struct drm_i915_private *i915, goto out_detach; } - timeout =dma_resv_wait_timeout (dmabuf->resv, false, true,5 *HZ ); + timeout =dma_resv_wait_timeout (dmabuf->resv,DMA_RESV_USAGE_WRITE , + true,5 *HZ );if (!timeout) {pr_err ("dmabuf wait for exclusive fence timed out.\n" ); timeout = -ETIME ;drivers/gpu/drm/i915/i915_request.c +2
-1@@ -1541,7 +1541,8 @@ i915_request_await_object(struct i915_request *to,struct dma_fence *fence;int ret =0 ; -dma_resv_for_each_fence (&cursor, obj->base.resv, write, fence) { +dma_resv_for_each_fence (&cursor, obj->base.resv, +dma_resv_usage_rw (write), fence) { ret =i915_request_await_dma_fence (to, fence);if (ret)break ;drivers/gpu/drm/i915/i915_sw_fence.c +1
-1@@ -579,7 +579,7 @@ int i915_sw_fence_await_reservation(struct i915_sw_fence *fence,debug_fence_assert (fence);might_sleep_if (gfpflags_allow_blocking (gfp)); -dma_resv_iter_begin (&cursor, resv, write); +dma_resv_iter_begin (&cursor, resv,dma_resv_usage_rw (write));dma_resv_for_each_fence_unlocked (&cursor, f) { pending =i915_sw_fence_await_dma_fence (fence, f, timeout, gfp);drivers/gpu/drm/msm/msm_gem.c +2
-1@@ -848,7 +848,8 @@ int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout) op &MSM_PREP_NOSYNC ?0 :timeout_to_jiffies (timeout);long ret; - ret =dma_resv_wait_timeout (obj->resv, write, true, remain); + ret =dma_resv_wait_timeout (obj->resv,dma_resv_usage_rw (write), + true, remain);if (ret ==0 )return remain ==0 ? -EBUSY : -ETIMEDOUT ;else if (ret <0 )drivers/gpu/drm/nouveau/dispnv50/wndw.c +2
-1@@ -558,7 +558,8 @@ nv50_wndw_prepare_fb(struct drm_plane *plane, struct drm_plane_state *state) asyw->image.handle[0 ] = ctxdma->object.handle; } - ret =dma_resv_get_singleton (nvbo->bo.base.resv, false, + ret =dma_resv_get_singleton (nvbo->bo.base.resv, +DMA_RESV_USAGE_WRITE , &asyw->state.fence);if (ret)return ret;drivers/gpu/drm/nouveau/nouveau_bo.c +4
-4@@ -962,11 +962,11 @@ nouveau_bo_vm_cleanup(struct ttm_buffer_object *bo,struct dma_fence *fence;int ret; -/* TODO: This is actually a memory management dependency */ - ret =dma_resv_get_singleton (bo->base.resv, false, &fence); + ret =dma_resv_get_singleton (bo->base.resv,DMA_RESV_USAGE_KERNEL , + &fence);if (ret) -dma_resv_wait_timeout (bo->base.resv, false, false, -MAX_SCHEDULE_TIMEOUT ); +dma_resv_wait_timeout (bo->base.resv,DMA_RESV_USAGE_KERNEL , + false,MAX_SCHEDULE_TIMEOUT );nv10_bo_put_tile_region (dev, *old_tile, fence); *old_tile = new_tile;drivers/gpu/drm/nouveau/nouveau_fence.c +2
-1@@ -353,7 +353,8 @@ nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan,if (ret)return ret; -dma_resv_for_each_fence (&cursor, resv, exclusive, fence) { +dma_resv_for_each_fence (&cursor, resv,dma_resv_usage_rw (exclusive), + fence) {struct nouveau_channel *prev =NULL ;bool must_wait = true;drivers/gpu/drm/nouveau/nouveau_gem.c +2
-1@@ -962,7 +962,8 @@ nouveau_gem_ioctl_cpu_prep(struct drm_device *dev, void *data,return -ENOENT ; nvbo =nouveau_gem_object (gem); - lret =dma_resv_wait_timeout (nvbo->bo.base.resv, write, true, + lret =dma_resv_wait_timeout (nvbo->bo.base.resv, +dma_resv_usage_rw (write), true, no_wait ?0 :30 *HZ );if (!lret) ret = -EBUSY ;drivers/gpu/drm/panfrost/panfrost_drv.c +2
-1@@ -316,7 +316,8 @@ panfrost_ioctl_wait_bo(struct drm_device *dev, void *data,if (!gem_obj)return -ENOENT ; - ret =dma_resv_wait_timeout (gem_obj->resv, true, true, timeout); + ret =dma_resv_wait_timeout (gem_obj->resv,DMA_RESV_USAGE_READ , + true, timeout);if (!ret) ret = timeout ? -ETIMEDOUT : -EBUSY ;drivers/gpu/drm/qxl/qxl_debugfs.c +2
-1@@ -61,7 +61,8 @@ qxl_debugfs_buffers_info(struct seq_file *m, void *data)struct dma_fence *fence;int rel =0 ; -dma_resv_iter_begin (&cursor, bo->tbo.base.resv, true); +dma_resv_iter_begin (&cursor, bo->tbo.base.resv, +DMA_RESV_USAGE_OTHER );dma_resv_for_each_fence_unlocked (&cursor, fence) {if (dma_resv_iter_is_restarted (&cursor)) rel =0 ;drivers/gpu/drm/radeon/radeon_display.c +2
-1@@ -533,7 +533,8 @@ static int radeon_crtc_page_flip_target(struct drm_crtc *crtc,DRM_ERROR ("failed to pin new rbo buffer before flip\n" ); goto cleanup; } - r =dma_resv_get_singleton (new_rbo->tbo.base.resv, false, &work->fence); + r =dma_resv_get_singleton (new_rbo->tbo.base.resv,DMA_RESV_USAGE_WRITE , + &work->fence);if (r) {radeon_bo_unreserve (new_rbo);DRM_ERROR ("failed to get new rbo buffer fences\n" );drivers/gpu/drm/radeon/radeon_gem.c +6
-3@@ -161,7 +161,9 @@ static int radeon_gem_set_domain(struct drm_gem_object *gobj, }if (domain ==RADEON_GEM_DOMAIN_CPU ) {/* Asking for cpu access wait for object idle */ - r =dma_resv_wait_timeout (robj->tbo.base.resv, true, true,30 *HZ ); + r =dma_resv_wait_timeout (robj->tbo.base.resv, +DMA_RESV_USAGE_OTHER , + true,30 *HZ );if (!r) r = -EBUSY ;@@ -523,7 +525,7 @@ int radeon_gem_busy_ioctl(struct drm_device *dev, void *data, } robj =gem_to_radeon_bo (gobj); - r =dma_resv_test_signaled (robj->tbo.base.resv, true); + r =dma_resv_test_signaled (robj->tbo.base.resv,DMA_RESV_USAGE_READ );if (r ==0 ) r = -EBUSY ;else @@ -552,7 +554,8 @@ int radeon_gem_wait_idle_ioctl(struct drm_device *dev, void *data, } robj =gem_to_radeon_bo (gobj); - ret =dma_resv_wait_timeout (robj->tbo.base.resv, true, true,30 *HZ ); + ret =dma_resv_wait_timeout (robj->tbo.base.resv,DMA_RESV_USAGE_READ , + true,30 *HZ );if (ret ==0 ) r = -EBUSY ;else if (ret <0 )drivers/gpu/drm/radeon/radeon_mn.c +2
-2@@ -66,8 +66,8 @@ static bool radeon_mn_invalidate(struct mmu_interval_notifier *mn,return true; } - r =dma_resv_wait_timeout (bo->tbo.base.resv, true, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_OTHER , + false,MAX_SCHEDULE_TIMEOUT );if (r <=0 )DRM_ERROR ("(%ld) failed to wait for user bo\n" , r);drivers/gpu/drm/radeon/radeon_sync.c +1
-1@@ -96,7 +96,7 @@ int radeon_sync_resv(struct radeon_device *rdev,struct dma_fence *f;int r =0 ; -dma_resv_for_each_fence (&cursor, resv, shared, f) { +dma_resv_for_each_fence (&cursor, resv,dma_resv_usage_rw (shared), f) { fence =to_radeon_fence (f);if (fence && fence->rdev == rdev)radeon_sync_fence (sync, fence);drivers/gpu/drm/radeon/radeon_uvd.c +2
-2@@ -478,8 +478,8 @@ static int radeon_uvd_cs_msg(struct radeon_cs_parser *p, struct radeon_bo *bo,return -EINVAL ; } - r =dma_resv_wait_timeout (bo->tbo.base.resv, false, false, -MAX_SCHEDULE_TIMEOUT ); + r =dma_resv_wait_timeout (bo->tbo.base.resv,DMA_RESV_USAGE_KERNEL , + false,MAX_SCHEDULE_TIMEOUT );if (r <=0 ) {DRM_ERROR ("Failed waiting for UVD message (%d)!\n" , r);return r ? r : -ETIME ;drivers/gpu/drm/scheduler/sched_main.c +2
-1@@ -703,7 +703,8 @@ int drm_sched_job_add_implicit_dependencies(struct drm_sched_job *job,struct dma_fence *fence;int ret; -dma_resv_for_each_fence (&cursor, obj->resv, write, fence) { +dma_resv_for_each_fence (&cursor, obj->resv,dma_resv_usage_rw (write), + fence) { ret =drm_sched_job_add_dependency (job, fence);if (ret)return ret;drivers/gpu/drm/ttm/ttm_bo.c +10
-8@@ -272,7 +272,7 @@ static void ttm_bo_flush_all_fences(struct ttm_buffer_object *bo)struct dma_resv_iter cursor;struct dma_fence *fence; -dma_resv_iter_begin (&cursor, resv, true); +dma_resv_iter_begin (&cursor, resv,DMA_RESV_USAGE_OTHER );dma_resv_for_each_fence_unlocked (&cursor, fence) {if (!fence->ops->signaled)dma_fence_enable_sw_signaling (fence);@@ -301,7 +301,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,struct dma_resv *resv = &bo->base._resv;int ret; -if (dma_resv_test_signaled (resv, true)) +if (dma_resv_test_signaled (resv,DMA_RESV_USAGE_OTHER )) ret =0 ;else ret = -EBUSY ;@@ -313,7 +313,8 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,dma_resv_unlock (bo->base.resv);spin_unlock (&bo->bdev->lru_lock); - lret =dma_resv_wait_timeout (resv, true, interruptible, + lret =dma_resv_wait_timeout (resv,DMA_RESV_USAGE_OTHER , + interruptible,30 *HZ );if (lret <0 )@@ -416,7 +417,8 @@ static void ttm_bo_release(struct kref *kref)/* Last resort, if we fail to allocate memory for the * fences block for the BO to become idle */ -dma_resv_wait_timeout (bo->base.resv, true, false, +dma_resv_wait_timeout (bo->base.resv, +DMA_RESV_USAGE_OTHER , false,30 *HZ ); }@@ -427,7 +429,7 @@ static void ttm_bo_release(struct kref *kref)ttm_mem_io_free (bdev, bo->resource); } -if (!dma_resv_test_signaled (bo->base.resv, true) || +if (!dma_resv_test_signaled (bo->base.resv,DMA_RESV_USAGE_OTHER ) || !dma_resv_trylock (bo->base.resv)) {/* The BO is not idle, resurrect it for delayed destroy */ ttm_bo_flush_all_fences (bo);@@ -1072,14 +1074,14 @@ int ttm_bo_wait(struct ttm_buffer_object *bo,long timeout =15 *HZ ;if (no_wait) { -if (dma_resv_test_signaled (bo->base.resv, true)) +if (dma_resv_test_signaled (bo->base.resv,DMA_RESV_USAGE_OTHER ))return 0 ;else return -EBUSY ; } - timeout =dma_resv_wait_timeout (bo->base.resv, true, interruptible, - timeout); + timeout =dma_resv_wait_timeout (bo->base.resv,DMA_RESV_USAGE_OTHER , + interruptible, timeout);if (timeout <0 )return timeout;drivers/gpu/drm/vgem/vgem_fence.c +3
-1@@ -130,6 +130,7 @@ int vgem_fence_attach_ioctl(struct drm_device *dev,struct vgem_file *vfile = file->driver_priv;struct dma_resv *resv;struct drm_gem_object *obj; +enum dma_resv_usage usage;struct dma_fence *fence;int ret;@@ -151,7 +152,8 @@ int vgem_fence_attach_ioctl(struct drm_device *dev,/* Check for a conflicting fence */ resv = obj->resv; -if (!dma_resv_test_signaled (resv, arg->flags &VGEM_FENCE_WRITE )) { + usage =dma_resv_usage_rw (arg->flags &VGEM_FENCE_WRITE ); +if (!dma_resv_test_signaled (resv, usage)) { ret = -EBUSY ; goto err_fence; }drivers/gpu/drm/virtio/virtgpu_ioctl.c +3
-2@@ -518,9 +518,10 @@ static int virtio_gpu_wait_ioctl(struct drm_device *dev, void *data,return -ENOENT ;if (args->flags &VIRTGPU_WAIT_NOWAIT ) { - ret =dma_resv_test_signaled (obj->resv, true); + ret =dma_resv_test_signaled (obj->resv,DMA_RESV_USAGE_READ ); }else { - ret =dma_resv_wait_timeout (obj->resv, true, true, timeout); + ret =dma_resv_wait_timeout (obj->resv,DMA_RESV_USAGE_READ , + true, timeout); }if (ret ==0 ) ret = -EBUSY ;drivers/gpu/drm/vmwgfx/vmwgfx_bo.c +2
-2@@ -739,8 +739,8 @@ static int vmw_user_bo_synccpu_grab(struct vmw_user_buffer_object *user_bo,if (flags & drm_vmw_synccpu_allow_cs) {long lret; - lret =dma_resv_wait_timeout (bo->base.resv, true, true, - nonblock ?0 : + lret =dma_resv_wait_timeout (bo->base.resv,DMA_RESV_USAGE_READ , + true, nonblock ?0 :MAX_SCHEDULE_TIMEOUT );if (!lret)return -EBUSY ;drivers/gpu/drm/vmwgfx/vmwgfx_resource.c +2
-2@@ -1169,8 +1169,8 @@ int vmw_resources_clean(struct vmw_buffer_object *vbo, pgoff_t start,if (bo->moving)dma_fence_put (bo->moving); -/* TODO: This is actually a memory management dependency */ -return dma_resv_get_singleton (bo->base.resv, false, +return dma_resv_get_singleton (bo->base.resv, +DMA_RESV_USAGE_KERNEL , &bo->moving); }drivers/infiniband/core/umem_dmabuf.c +2
-1@@ -67,7 +67,8 @@ int ib_umem_dmabuf_map_pages(struct ib_umem_dmabuf *umem_dmabuf)* may be not up-to-date. Wait for the exporter to finish * the migration. */ -return dma_resv_wait_timeout (umem_dmabuf->attach->dmabuf->resv, false, +return dma_resv_wait_timeout (umem_dmabuf->attach->dmabuf->resv, +DMA_RESV_USAGE_KERNEL , false,MAX_SCHEDULE_TIMEOUT ); }EXPORT_SYMBOL (ib_umem_dmabuf_map_pages);include/linux/dma-resv.h +93
-13@@ -49,6 +49,86 @@ extern struct ww_class reservation_ww_class;struct dma_resv_list ; +/** +* enum dma_resv_usage - how the fences from a dma_resv obj are used +* +* This enum describes the different use cases for a dma_resv object and +* controls which fences are returned when queried. +* +* An important fact is that there is the order KERNEL<WRITE<READ<OTHER and +* when the dma_resv object is asked for fences for one use case the fences +* for the lower use case are returned as well. +* +* For example when asking for WRITE fences then the KERNEL fences are returned +* as well. Similar when asked for READ fences then both WRITE and KERNEL +* fences are returned as well. +*/ +enum dma_resv_usage { +/** +* @DMA_RESV_USAGE_KERNEL: For in kernel memory management only. +* +* This should only be used for things like copying or clearing memory +* with a DMA hardware engine for the purpose of kernel memory +* management. +* +* Drivers *always* need to wait for those fences before accessing the +* resource protected by the dma_resv object. The only exception for +* that is when the resource is known to be locked down in place by +* pinning it previously. +*/ +DMA_RESV_USAGE_KERNEL , + +/** +* @DMA_RESV_USAGE_WRITE: Implicit write synchronization. +* +* This should only be used for userspace command submissions which add +* an implicit write dependency. +*/ +DMA_RESV_USAGE_WRITE , + +/** +* @DMA_RESV_USAGE_READ: Implicit read synchronization. +* +* This should only be used for userspace command submissions which add +* an implicit read dependency. +*/ +DMA_RESV_USAGE_READ , + +/** +* @DMA_RESV_USAGE_OTHER: No implicit sync. +* +* This should be used for operations which don't want to add an +* implicit dependency at all, but still have a dependency on memory +* management. +* +* This might include things like preemption fences as well as device +* page table updates or even userspace command submissions. +* +* The kernel memory management *always* need to wait for those fences +* before moving or freeing the resource protected by the dma_resv +* object. +*/ +DMA_RESV_USAGE_OTHER +}; + +/** +* dma_resv_usage_rw - helper for implicit sync +* @write: true if we create a new implicit sync write +* +* This returns the implicit synchronization usage for write or read accesses. +*/ +static inline enum dma_resv_usage dma_resv_usage_rw (bool write) +{ +/* This looks confusing at first sight, but is indeed correct. +* +* The rational is that new write operations needs to wait for the +* existing read and write operations to finish. +* But a new read operation only needs to wait for the existing write +* operations to finish. +*/ +return write ?DMA_RESV_USAGE_READ :DMA_RESV_USAGE_WRITE ; +} +/** * struct dma_resv - a reservation object manages fences for a buffer * @@ -147,8 +227,8 @@ struct dma_resv_iter {/** @obj: The dma_resv object we iterate over */ struct dma_resv *obj; -/** @all_fences: If all fences should be returned */ -bool all_fences; +/** @usage: Controls which fences are returned */ +enum dma_resv_usage usage;/** @fence: the currently handled fence */ struct dma_fence *fence;@@ -178,14 +258,14 @@ struct dma_fence *dma_resv_iter_next(struct dma_resv_iter *cursor);* dma_resv_iter_begin - initialize a dma_resv_iter object * @cursor: The dma_resv_iter object to initialize * @obj: The dma_resv object which we want to iterate over -* @all_fences: If all fences should be returned or just the exclusive one +* @usage: controls which fences to return */ static inline void dma_resv_iter_begin (struct dma_resv_iter *cursor,struct dma_resv *obj, -bool all_fences) +enum dma_resv_usage usage) { cursor->obj = obj; - cursor->all_fences = all_fences; + cursor->usage = usage; cursor->fence =NULL ; }@@ -242,7 +322,7 @@ static inline bool dma_resv_iter_is_restarted(struct dma_resv_iter *cursor)* dma_resv_for_each_fence - fence iterator * @cursor: a struct dma_resv_iter pointer * @obj: a dma_resv object pointer -* @all_fences: true if all fences should be returned +* @usage: controls which fences to return * @fence: the current fence * * Iterate over the fences in a struct dma_resv object while holding the @@ -251,8 +331,8 @@ static inline bool dma_resv_iter_is_restarted(struct dma_resv_iter *cursor)* valid as long as the lock is held and so no extra reference to the fence is * taken. */ -#define dma_resv_for_each_fence (cursor, obj, all_fences, fence) \ - for (dma_resv_iter_begin(cursor, obj, all_fences), \ +#define dma_resv_for_each_fence (cursor, obj, usage, fence) \ + for (dma_resv_iter_begin(cursor, obj, usage), \ fence = dma_resv_iter_first(cursor); fence; \ fence = dma_resv_iter_next(cursor))@@ -421,14 +501,14 @@ void dma_resv_replace_fences(struct dma_resv *obj, uint64_t context,void dma_resv_add_excl_fence (struct dma_resv *obj,struct dma_fence *fence);void dma_resv_prune (struct dma_resv *obj);void dma_resv_prune_unlocked (struct dma_resv *obj); -int dma_resv_get_fences (struct dma_resv *obj,bool write, +int dma_resv_get_fences (struct dma_resv *obj,enum dma_resv_usage usage,unsigned int *num_fences,struct dma_fence ***fences); -int dma_resv_get_singleton (struct dma_resv *obj,bool write, +int dma_resv_get_singleton (struct dma_resv *obj,enum dma_resv_usage usage,struct dma_fence **fence);int dma_resv_copy_fences (struct dma_resv *dst,struct dma_resv *src); -long dma_resv_wait_timeout (struct dma_resv *obj,bool wait_all,bool intr, -unsigned long timeout); -bool dma_resv_test_signaled (struct dma_resv *obj,bool test_all); +long dma_resv_wait_timeout (struct dma_resv *obj,enum dma_resv_usage usage, +bool intr,unsigned long timeout); +bool dma_resv_test_signaled (struct dma_resv *obj,enum dma_resv_usage usage);void dma_resv_describe (struct dma_resv *obj,struct seq_file *seq);#endif /* _LINUX_RESERVATION_H */ Signature · 1 line
2.25.1
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [PATCH 22/26] dma-buf: add enum dma_resv_usage
In reply to Christian KönigQuoted 10 lines · Christian König
This change adds the
dma_resv_usageenum and allows us to specify why adma_resvobject is queried for its containing fences.Additional to that a
dma_resv_usage_rw()helper function is added to aid retrieving the fences for a read or write userspace submission.This is then deployed to the different query functions of the
dma_resvobject and all of their users.Signed-off-by Christian König <[email protected]>
Just a few comments on the kenreldoc while I scroll through.
Quoted 8 lines
EXPORT_SYMBOL(ib_umem_dmabuf_map_pages);include/linux/dma-resv.h +0
-0@@ -49,6 +49,86 @@ extern struct ww_class reservation_ww_class;struct dma_resv_list ;
include/linux/dma-resv.h extern struct ww_class reservation_ww_class; · line 60
Quoted 10 lines
include/linux/dma-resv.h +9
-0+/** +* enum dma_resv_usage - how the fences from a dma_resv obj are used +* +* This enum describes the different use cases for a dma_resv object and +* controls which fences are returned when queried. +* +* An important fact is that there is the order KERNEL<WRITE<READ<OTHER and +* when the dma_resv object is asked for fences for one use case the fences +* for the lower use case are returned as well.
Might be good to replicate this to all functions that take a dma_resv_usage flag, and then also add a "See enum dma_resv_usage for more information." so we get a clickable hyperlink too.
Quoted 7 lines
include/linux/dma-resv.h +7
-0+* +* For example when asking for WRITE fences then the KERNEL fences are returned +* as well. Similar when asked for READ fences then both WRITE and KERNEL +* fences are returned as well. +*/ +enum dma_resv_usage { +/**
include/linux/dma-resv.h extern struct ww_class reservation_ww_class; · line 77
Quoted 10 lines
include/linux/dma-resv.h +10
-0+* @DMA_RESV_USAGE_KERNEL: For in kernel memory management only. +* +* This should only be used for things like copying or clearing memory +* with a DMA hardware engine for the purpose of kernel memory +* management. +* +* Drivers *always* need to wait for those fences before accessing the +* resource protected by the dma_resv object. The only exception for +* that is when the resource is known to be locked down in place by +* pinning it previously.
Should dma_buf_pin also do the wait for kernel fences? I think that would also ba e bit clearer semantics in the dma-buf patch which does these waits for us.
Or should dma_buf_pin be pipelined and it's up to callers to wait? For kms that's definitely the semantics we want, but it's a bit playing with fire situation, so maybe dma-buf should get the more idiot proof semantics?
Quoted 18 lines
include/linux/dma-resv.h +18
-0+*/ +DMA_RESV_USAGE_KERNEL , + +/** +* @DMA_RESV_USAGE_WRITE: Implicit write synchronization. +* +* This should only be used for userspace command submissions which add +* an implicit write dependency. +*/ +DMA_RESV_USAGE_WRITE , + +/** +* @DMA_RESV_USAGE_READ: Implicit read synchronization. +* +* This should only be used for userspace command submissions which add +* an implicit read dependency. +*/ + DMA_RESV_USAGE_READ,
include/linux/dma-resv.h extern struct ww_class reservation_ww_class; · line 105
Quoted 10 lines
include/linux/dma-resv.h +10
-0+ +/** +* @DMA_RESV_USAGE_OTHER: No implicit sync. +* +* This should be used for operations which don't want to add an +* implicit dependency at all, but still have a dependency on memory +* management. +* +* This might include things like preemption fences as well as device +* page table updates or even userspace command submissions.
I think we should highlight a bit more that for explicitly synchronized userspace like vk OTHER is the normal case. So really not an exception. Ofc aside from amdkgf there's currently no driver doing this, but really we should have lots of them ...
include/linux/dma-resv.h extern struct ww_class reservation_ww_class; · line 118
Quoted 13 lines
include/linux/dma-resv.h +13
-0+* +* The kernel memory management *always* need to wait for those fences +* before moving or freeing the resource protected by the dma_resv +* object. +*/ + DMA_RESV_USAGE_OTHER +}; + +/** +* dma_resv_usage_rw - helper for implicit sync +* @write: true if we create a new implicit sync write +* +* This returns the implicit synchronization usage for write or read accesses.
Pls add "See enum dma_resv_usage for more details." or so. Never hurts to be plentiful with links :-)
Quoted 28 lines
include/linux/dma-resv.h +15
-2+*/ +static inline enum dma_resv_usage dma_resv_usage_rw (bool write) +{ +/* This looks confusing at first sight, but is indeed correct. +* +* The rational is that new write operations needs to wait for the +* existing read and write operations to finish. +* But a new read operation only needs to wait for the existing write +* operations to finish. +*/ +return write ?DMA_RESV_USAGE_READ :DMA_RESV_USAGE_WRITE ; +} +/** * struct dma_resv - a reservation object manages fences for a buffer * @@ -147,8 +227,8 @@ struct dma_resv_iter {/** @obj: The dma_resv object we iterate over */ struct dma_resv *obj; -/** @all_fences: If all fences should be returned */ -bool all_fences; +/** @usage: Controls which fences are returned */ +enum dma_resv_usage usage;/** @fence: the currently handled fence */ struct dma_fence *fence;
include/linux/dma-resv.h struct dma_fence *dma_resv_iter_next(struct dma_resv_iter *cursor); · line 261
Quoted 5 lines
include/linux/dma-resv.h +1
-1@@ -178,14 +258,14 @@ struct dma_fence *dma_resv_iter_next(struct dma_resv_iter *cursor);* dma_resv_iter_begin - initialize a dma_resv_iter object * @cursor: The dma_resv_iter object to initialize * @obj: The dma_resv object which we want to iterate over -* @all_fences: If all fences should be returned or just the exclusive one +* @usage: controls which fences to return
Please add the blurb here I mentioned above. Maybe adjust the text to use the neatly highlighted @usage.
Quoted 13 lines
include/linux/dma-resv.h +2
-2*/ static inline void dma_resv_iter_begin (struct dma_resv_iter *cursor,struct dma_resv *obj, -bool all_fences) +enum dma_resv_usage usage) { cursor->obj = obj; - cursor->all_fences = all_fences; + cursor->usage = usage; cursor->fence =NULL ; }
include/linux/dma-resv.h static inline bool dma_resv_iter_is_restarted(struct dma_resv_iter *cursor) · line 327
Quoted 7 lines
include/linux/dma-resv.h +1
-1@@ -242,7 +322,7 @@ static inline bool dma_resv_iter_is_restarted(struct dma_resv_iter *cursor)* dma_resv_for_each_fence - fence iterator * @cursor: a struct dma_resv_iter pointer * @obj: a dma_resv object pointer -* @all_fences: true if all fences should be returned +* @usage: controls which fences to return * @fence: the current fence *
Same, another place that needs the @usage clarification.
Quoted 20 lines
include/linux/dma-resv.h +3
-4* Iterate over the fences in a struct dma_resv object while holding the @@ -251,8 +331,8 @@ static inline bool dma_resv_iter_is_restarted(struct dma_resv_iter *cursor)* valid as long as the lock is held and so no extra reference to the fence is * taken. */ -#define dma_resv_for_each_fence (cursor, obj, all_fences, fence) \ - for (dma_resv_iter_begin(cursor, obj, all_fences), \ +#define dma_resv_for_each_fence (cursor, obj, usage, fence) \ + for (dma_resv_iter_begin(cursor, obj, usage), \ fence = dma_resv_iter_first(cursor); fence; \ fence = dma_resv_iter_next(cursor))@@ -421,14 +501,14 @@ void dma_resv_replace_fences(struct dma_resv *obj, uint64_t context,void dma_resv_add_excl_fence (struct dma_resv *obj,struct dma_fence *fence);void dma_resv_prune (struct dma_resv *obj);void dma_resv_prune_unlocked (struct dma_resv *obj); -int dma_resv_get_fences (struct dma_resv *obj,bool write, +int dma_resv_get_fences (struct dma_resv *obj,enum dma_resv_usage usage,unsigned int *num_fences,struct dma_fence ***fences); -int dma_resv_get_singleton(struct dma_resv *obj, bool write,
include/linux/dma-resv.h void dma_resv_replace_fences(struct dma_resv *obj, uint64_t context, · line 512
Quoted 10 lines
include/linux/dma-resv.h +4
-3+int dma_resv_get_singleton (struct dma_resv *obj,enum dma_resv_usage usage,struct dma_fence **fence);int dma_resv_copy_fences (struct dma_resv *dst,struct dma_resv *src); -long dma_resv_wait_timeout (struct dma_resv *obj,bool wait_all,bool intr, -unsigned long timeout); -bool dma_resv_test_signaled (struct dma_resv *obj,bool test_all); +long dma_resv_wait_timeout (struct dma_resv *obj,enum dma_resv_usage usage, +bool intr,unsigned long timeout); +bool dma_resv_test_signaled (struct dma_resv *obj,enum dma_resv_usage usage);void dma_resv_describe (struct dma_resv *obj,struct seq_file *seq);
I took endless amounts of discussions, but I think we're arriving at something really neat and tiny here now finally. Both semantics, and how drivers use them.
-Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Christian König
reviewerRe: [PATCH 22/26] dma-buf: add enum dma_resv_usage
Am 25.11.21 um 16:59 schrieb Daniel Vetter:
Quoted 22 lines
[SNIP]
Quoted 7 lines
+ *
+ * For example when asking for WRITE fences then the KERNEL fences are returned
+ * as well. Similar when asked for READ fences then both WRITE and KERNEL
+ * fences are returned as well.
+ */include/linux/dma-resv.h +2
-0+enum dma_resv_usage { +/** include/linux/dma-resv.h extern struct ww_class reservation_ww_class; · line 77
Quoted 10 lines
include/linux/dma-resv.h +10
-0+* @DMA_RESV_USAGE_KERNEL: For in kernel memory management only. +* +* This should only be used for things like copying or clearing memory +* with a DMA hardware engine for the purpose of kernel memory +* management. +* +* Drivers *always* need to wait for those fences before accessing the +* resource protected by the dma_resv object. The only exception for +* that is when the resource is known to be locked down in place by +* pinning it previously. Should
dma_buf_pinalso do the wait for kernel fences? I think that would also ba e bit clearer semantics in the dma-buf patch which does these waits for us.
Quoted 3 lines
Or should
dma_buf_pinbe pipelined and it's up to callers to wait? For kms that's definitely the semantics we want, but it's a bit playing with fire situation, so maybe dma-buf should get the more idiot proof semantics?
Yeah, good question. I've already added a wait after mapping an attachment for static importers.
But for dynamic importers I'm not sure what we want to do. On the one hand waiting for moves to finish is certainly the more defensive approach on the other hand when you have a dynamic importer you absolutely need to handle those dependencies correctly anyway.
Regards,
Christian.
Re: [PATCH 22/26] dma-buf: add enum dma_resv_usage
Quoted 27 lines · Christian König
Am 25.11.21 um 16:59 schrieb Daniel Vetter:
Quoted 22 lines
[SNIP]
Quoted 7 lines
+ *
+ * For example when asking for WRITE fences then the KERNEL fences are returned
+ * as well. Similar when asked for READ fences then both WRITE and KERNEL
+ * fences are returned as well.
+ */include/linux/dma-resv.h +2
-0+enum dma_resv_usage { +/** include/linux/dma-resv.h extern struct ww_class reservation_ww_class; · line 77
Quoted 10 lines
include/linux/dma-resv.h +10
-0+* @DMA_RESV_USAGE_KERNEL: For in kernel memory management only. +* +* This should only be used for things like copying or clearing memory +* with a DMA hardware engine for the purpose of kernel memory +* management. +* +* Drivers *always* need to wait for those fences before accessing the +* resource protected by the dma_resv object. The only exception for +* that is when the resource is known to be locked down in place by +* pinning it previously. Should
dma_buf_pinalso do the wait for kernel fences? I think that would also ba e bit clearer semantics in the dma-buf patch which does these waits for us.Quoted 3 lines
Or should
dma_buf_pinbe pipelined and it's up to callers to wait? For kms that's definitely the semantics we want, but it's a bit playing with fire situation, so maybe dma-buf should get the more idiot proof semantics?
Quoted 7 lines
Yeah, good question. I've already added a wait after mapping an attachment for static importers.
But for dynamic importers I'm not sure what we want to do. On the one hand waiting for moves to finish is certainly the more defensive approach on the other hand when you have a dynamic importer you absolutely need to handle those dependencies correctly anyway.
Hm yeah only doing it for non-dynamic attachments sounds fine to me (with kerneldoc in dma_buf_pin() ofc).
-Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Christian König
reviewerPATCH 23/26 dma-buf: specify usage while adding fences to dma_resv obj
In reply to Christian KönigInstead of distingting between shared and exclusive fences specify the fence usage while adding fences.
Rework all drivers to use this interface instead and deprecate the old one.
--- drivers/dma-buf/dma-resv.c | 389 ++++++++---------- drivers/dma-buf/st-dma-resv.c | 109 ++--- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 2 +- drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 12 +- drivers/gpu/drm/i915/gem/i915_gem_busy.c | 13 +- drivers/gpu/drm/i915/gem/i915_gem_clflush.c | 5 +- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 4 +- .../drm/i915/gem/selftests/i915_gem_migrate.c | 2 +- drivers/gpu/drm/i915/i915_vma.c | 10 +- .../drm/i915/selftests/intel_memory_region.c | 2 +- drivers/gpu/drm/lima/lima_gem.c | 4 +- drivers/gpu/drm/msm/msm_gem_submit.c | 4 +- drivers/gpu/drm/nouveau/nouveau_bo.c | 9 +- drivers/gpu/drm/nouveau/nouveau_fence.c | 2 +- drivers/gpu/drm/panfrost/panfrost_job.c | 2 +- drivers/gpu/drm/qxl/qxl_release.c | 5 +- drivers/gpu/drm/radeon/radeon_object.c | 6 +- drivers/gpu/drm/radeon/radeon_vm.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 6 +- drivers/gpu/drm/ttm/ttm_bo_util.c | 7 +- drivers/gpu/drm/ttm/ttm_execbuf_util.c | 10 +- drivers/gpu/drm/v3d/v3d_gem.c | 6 +- drivers/gpu/drm/vc4/vc4_gem.c | 4 +- drivers/gpu/drm/vgem/vgem_fence.c | 11 +- drivers/gpu/drm/virtio/virtgpu_gem.c | 5 +- drivers/gpu/drm/vmwgfx/vmwgfx_bo.c | 5 +- include/linux/dma-resv.h | 88 ++-- 31 files changed, 312 insertions(+), 434 deletions(-)
drivers/dma-buf/dma-resv.c +175 -214
@@ -44,12 +44,12 @@/** * DOC: Reservation Object Overview * -* The reservation object provides a mechanism to manage shared and -* exclusive fences associated with a buffer. A reservation object -* can have attached one exclusive fence (normally associated with -* write operations) or N shared fences (read operations). The RCU -* mechanism is used to protect read access to fences from locked -* write-side updates. +* The reservation object provides a mechanism to manage a container of +* dma_fence object associated with a resource. A reservation object +* can have any number of fences attaches to it. Each fence carring an usage +* parameter determining how the operation represented by the fence is using the +* resource. The RCU mechanism is used to protect read access to fences from +* locked write-side updates. * * See struct dma_resv for more details. */
@@ -57,36 +57,80 @@DEFINE_WD_CLASS (reservation_ww_class);EXPORT_SYMBOL (reservation_ww_class); +/* Mask for the lower fence pointer bits */ +#define DMA_RESV_LIST_MASK 0x3 +/** -* struct dma_resv_list - a list of shared fences +* struct dma_resv_list - an array of fences * @rcu: for internal use -* @shared_count: table of shared fences -* @shared_max: for growing shared fence table -* @shared: shared fence table +* @num_fences: table of fences +* @max_fences: for growing fence table +* @table: fence table */ struct dma_resv_list {struct rcu_head rcu; -u32 shared_count, shared_max; -struct dma_fence __rcu *shared[]; +u32 num_fences, max_fences; +struct dma_fence __rcu *table[]; }; +/** +* dma_resv_list_entry - extract fence and usage from a list entry +* @list: the list to extract and entry from +* @index: which entry we want +* @check: lockdep check that the access is allowed +* @fence: the resulting fence +* @usage: the resulting usage +* +* Extract the fence and usage flags from an RCU protected entry in the list. +*/ +static void dma_resv_list_entry (struct dma_resv_list *list,unsigned int index, +bool check,struct dma_fence **fence, +enum dma_resv_usage *usage) +{ +long tmp; + + tmp = (long )rcu_dereference_check (list->table[index], check); + *fence = (struct dma_fence *)(tmp & ~DMA_RESV_LIST_MASK ); +if (usage) + *usage = tmp &DMA_RESV_LIST_MASK ; +} + +/** +* dma_resv_list_set - set fence and usage at a specific index +* @list: the list to modify +* @index: where to make the change +* @fence: the fence to set +* @usage: the usage to set +* +* Set the fence and usage flags at the specific index in the list. +*/ +static void dma_resv_list_set (struct dma_resv_list *list, +unsigned int index, +struct dma_fence *fence, +enum dma_resv_usage usage) +{ +long tmp = ((long )fence) | usage; + +RCU_INIT_POINTER (list->table[index], (struct dma_fence *)tmp); +} +/** * dma_resv_list_alloc - allocate fence list -* @shared_max: number of fences we need space for +* @max_fences: number of fences we need space for * * Allocate a new dma_resv_list and make sure to correctly initialize -* shared_max. +* max_fences. */ -static struct dma_resv_list *dma_resv_list_alloc (unsigned int shared_max) +static struct dma_resv_list *dma_resv_list_alloc (unsigned int max_fences) {struct dma_resv_list *list; - list =kmalloc (struct_size (list, shared, shared_max),GFP_KERNEL ); + list =kmalloc (struct_size (list, table, max_fences),GFP_KERNEL );if (!list)return NULL ; - list->shared_max = (ksize(list) - offsetof(typeof(*list), shared)) / -sizeof (*list->shared); + list->max_fences = (ksize(list) - offsetof(typeof(*list), table)) / +sizeof (*list->table);return list; }
@@ -104,9 +148,12 @@ static void dma_resv_list_free(struct dma_resv_list *list)if (!list)return ; -for (i =0 ; i < list->shared_count; ++i) -dma_fence_put (rcu_dereference_protected (list->shared[i], true)); +for (i =0 ; i < list->num_fences; ++i) { +struct dma_fence *fence; +dma_resv_list_entry (list, i, true, &fence,NULL ); +dma_fence_put (fence); + }kfree_rcu (list, rcu); }
@@ -125,15 +172,15 @@ static void dma_resv_list_prune(struct dma_resv_list *list,if (!list)return ; -for (i =0 ; i < list->shared_count; ++i) { +for (i =0 ; i < list->num_fences; ++i) {struct dma_fence *fence; - fence =rcu_dereference_protected (list->shared[i], -dma_resv_held (obj)); +dma_resv_list_entry (list, i, true, &fence,NULL );if (!dma_fence_is_signaled (fence))continue ; -RCU_INIT_POINTER (list->shared[i],dma_fence_get_stub ()); +dma_resv_list_set (list, i,dma_fence_get_stub (), +DMA_RESV_USAGE_OTHER );dma_fence_put (fence); } }
@@ -147,8 +194,7 @@ void dma_resv_init(struct dma_resv *obj)ww_mutex_init (&obj->lock, &reservation_ww_class);seqcount_ww_mutex_init (&obj->seq, &obj->lock); -RCU_INIT_POINTER (obj->fence,NULL ); -RCU_INIT_POINTER (obj->fence_excl,NULL ); +RCU_INIT_POINTER (obj->fences,NULL ); }EXPORT_SYMBOL (dma_resv_init);
@@ -158,81 +204,54 @@ EXPORT_SYMBOL(dma_resv_init);*/ void dma_resv_fini (struct dma_resv *obj) { -struct dma_resv_list *fobj; -struct dma_fence *excl; -/* * This object should be dead and all references must have * been released to it, so no need to be protected with rcu. */ - excl =rcu_dereference_protected (obj->fence_excl,1 ); -if (excl) -dma_fence_put (excl); - - fobj =rcu_dereference_protected (obj->fence,1 ); -dma_resv_list_free (fobj); +dma_resv_list_free (rcu_dereference_protected (obj->fences, true));ww_mutex_destroy (&obj->lock); }EXPORT_SYMBOL (dma_resv_fini);/** -* dma_resv_excl_fence - return the object's exclusive fence -* @obj: the reservation object -* -* Returns the exclusive fence (if any). Caller must either hold the objects -* through dma_resv_lock() or the RCU read side lock through rcu_read_lock(), -* or one of the variants of each -* -* RETURNS -* The exclusive fence or NULL -*/ -static inline struct dma_fence * -dma_resv_excl_fence (struct dma_resv *obj) -{ -return rcu_dereference_check (obj->fence_excl,dma_resv_held (obj)); -} - -/** -* dma_resv_shared_list - get the reservation object's shared fence list +* dma_resv_fences_list - return the dma_resv object's fence list * @obj: the reservation object * -* Returns the shared fence list. Caller must either hold the objects -* through dma_resv_lock() or the RCU read side lock through rcu_read_lock(), -* or one of the variants of each +* Returns the fence list. Caller must either hold the objects through +* dma_resv_lock() or the RCU read side lock through rcu_read_lock(). */ -static inline struct dma_resv_list *dma_resv_shared_list (struct dma_resv *obj) +static inline struct dma_resv_list *dma_resv_fences_list (struct dma_resv *obj) { -return rcu_dereference_check (obj->fence,dma_resv_held (obj)); +return rcu_dereference_check (obj->fences,dma_resv_held (obj)); }/** -* dma_resv_reserve_shared - Reserve space to add shared fences to -* a dma_resv. +* dma_resv_reserve_fences - Reserve space to add fences to a dma_resv object. * @obj: reservation object * @num_fences: number of fences we want to add * -* Should be called before dma_resv_add_shared_fence(). Must -* be called with @obj locked through dma_resv_lock(). +* Should be called before dma_resv_add_fence(). Must be called with @obj +* locked through dma_resv_lock(). * * Note that the preallocated slots need to be re-reserved if @obj is unlocked -* at any time before calling dma_resv_add_shared_fence(). This is validated -* when CONFIG_DEBUG_MUTEXES is enabled. +* at any time before calling dma_resv_add_fence(). This is validated when +* CONFIG_DEBUG_MUTEXES is enabled. * * RETURNS * Zero for success, or -errno */ -int dma_resv_reserve_shared (struct dma_resv *obj,unsigned int num_fences) +int dma_resv_reserve_fences (struct dma_resv *obj,unsigned int num_fences) {struct dma_resv_list *old, *new;unsigned int i, j, k, max;dma_resv_assert_held (obj); - old =dma_resv_shared_list (obj); -if (old && old->shared_max) { -if ((old->shared_count + num_fences) <= old->shared_max) + old =dma_resv_fences_list (obj); +if (old && old->max_fences) { +if ((old->num_fences + num_fences) <= old->max_fences)return 0 ; - max =max (old->shared_count + num_fences, old->shared_max *2 ); + max =max (old->num_fences + num_fences, old->max_fences *2 ); }else { max =max (4ul ,roundup_pow_of_two (num_fences)); }
@@ -247,27 +266,27 @@ int dma_resv_reserve_shared(struct dma_resv *obj, unsigned int num_fences)* references from the old struct are carried over to * the new. */ -for (i =0 , j =0 , k = max; i < (old ? old->shared_count :0 ); ++i) { +for (i =0 , j =0 , k = max; i < (old ? old->num_fences :0 ); ++i) { +enum dma_resv_usage usage;struct dma_fence *fence; - fence =rcu_dereference_protected (old->shared[i], -dma_resv_held (obj)); +dma_resv_list_entry (old, i,dma_resv_held (obj), &fence, &usage);if (dma_fence_is_signaled (fence)) -RCU_INIT_POINTER (new->shared[--k], fence); +RCU_INIT_POINTER (new->table[--k], fence);else -RCU_INIT_POINTER (new->shared[j++], fence); +dma_resv_list_set (new, j++, fence, usage); } - new->shared_count = j; + new->num_fences = j;/* * We are not changing the effective set of fences here so can * merely update the pointer to the new array; both existing * readers and new readers will see exactly the same set of -* active (unsignaled) shared fences. Individual fences and the +* active (unsignaled) fences. Individual fences and the * old array are protected by RCU and so will not vanish under * the gaze of the rcu_read_lock() readers. */ -rcu_assign_pointer (obj->fence, new); +rcu_assign_pointer (obj->fences, new);if (!old)return 0 ;
@@ -276,7 +295,7 @@ int dma_resv_reserve_shared(struct dma_resv *obj, unsigned int num_fences)for (i = k; i < max; ++i) {struct dma_fence *fence; - fence =rcu_dereference_protected (new->shared[i], + fence =rcu_dereference_protected (new->table[i],dma_resv_held (obj));dma_fence_put (fence); }
@@ -284,41 +303,44 @@ int dma_resv_reserve_shared(struct dma_resv *obj, unsigned int num_fences)return 0 ; } -EXPORT_SYMBOL (dma_resv_reserve_shared); +EXPORT_SYMBOL (dma_resv_reserve_fences);#ifdef CONFIG_DEBUG_MUTEXES /** -* dma_resv_reset_shared_max - reset shared fences for debugging +* dma_resv_reset_max_fences - reset fences for debugging * @obj: the dma_resv object to reset * -* Reset the number of pre-reserved shared slots to test that drivers do -* correct slot allocation using dma_resv_reserve_shared(). See also -* &dma_resv_list.shared_max. +* Reset the number of pre-reserved fence slots to test that drivers do +* correct slot allocation using dma_resv_reserve_fences(). See also +* &dma_resv_list.max_fences. */ -void dma_resv_reset_shared_max (struct dma_resv *obj) +void dma_resv_reset_max_fences (struct dma_resv *obj) { -struct dma_resv_list *fences =dma_resv_shared_list (obj); +struct dma_resv_list *fences =dma_resv_fences_list (obj);dma_resv_assert_held (obj); -/* Test shared fence slot reservation */ +/* Test fence slot reservation */ if (fences) - fences->shared_max = fences->shared_count; + fences->max_fences = fences->num_fences; } -EXPORT_SYMBOL (dma_resv_reset_shared_max); +EXPORT_SYMBOL (dma_resv_reset_max_fences);#endif /** -* dma_resv_add_shared_fence - Add a fence to a shared slot +* dma_resv_add_fence - Add a fence to the dma_resv obj * @obj: the reservation object -* @fence: the shared fence to add +* @fence: the fence to add +* @usage: how the fence is used with the resource protected by the dma_resv +* obj. * -* Add a fence to a shared slot, @obj must be locked with dma_resv_lock(), and -* dma_resv_reserve_shared() has been called. +* Add a fence to a slot, @obj must be locked with dma_resv_lock(), and +* dma_resv_reserve_fences() has been called. * * See also &dma_resv.fence for a discussion of the semantics. */ -void dma_resv_add_shared_fence (struct dma_resv *obj,struct dma_fence *fence) +void dma_resv_add_fence (struct dma_resv *obj,struct dma_fence *fence, +enum dma_resv_usage usage) {struct dma_resv_list *fobj;struct dma_fence *old;
@@ -328,39 +350,41 @@ void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence)dma_resv_assert_held (obj); - fobj =dma_resv_shared_list (obj); - count = fobj->shared_count; + fobj =dma_resv_fences_list (obj); + count = fobj->num_fences;write_seqcount_begin (&obj->seq);for (i =0 ; i < count; ++i) { +enum dma_resv_usage old_usage; - old =rcu_dereference_protected (fobj->shared[i], -dma_resv_held (obj)); -if (old->context == fence->context || +dma_resv_list_entry (fobj, i,dma_resv_held (obj), + &old, &old_usage); +if ((old->context == fence->context && old_usage >= usage) ||dma_fence_is_signaled (old)) goto replace; } -BUG_ON (fobj->shared_count >= fobj->shared_max); +BUG_ON (fobj->num_fences >= fobj->max_fences); old =NULL ; count++; replace: -RCU_INIT_POINTER (fobj->shared[i], fence); -/* pointer update must be visible before we extend the shared_count */ -smp_store_mb (fobj->shared_count, count); +dma_resv_list_set (fobj, i, fence, usage); +/* pointer update must be visible before we extend the num_fences */ +smp_store_mb (fobj->num_fences, count);write_seqcount_end (&obj->seq);dma_fence_put (old); } -EXPORT_SYMBOL (dma_resv_add_shared_fence); +EXPORT_SYMBOL (dma_resv_add_fence);/** * dma_resv_replace_fences - replace fences in the dma_resv obj * @obj: the reservation object * @context: the context of the fences to replace * @replacement: the new fence to use instead +* @usage: how the fence is used * * Replace fences with a specified context with a new fence. Only valid if the * operation represented by the original fences is completed or has no longer
@@ -368,64 +392,30 @@ EXPORT_SYMBOL(dma_resv_add_shared_fence);* completes. */ void dma_resv_replace_fences (struct dma_resv *obj,uint64_t context, -struct dma_fence *replacement) +struct dma_fence *replacement, +enum dma_resv_usage usage) {struct dma_resv_list *list; -struct dma_fence *old;unsigned int i;dma_resv_assert_held (obj); + list =dma_resv_fences_list (obj);write_seqcount_begin (&obj->seq); +for (i =0 ; list && i < list->num_fences; ++i) { +struct dma_fence *old; - old =dma_resv_excl_fence (obj); -if (old->context == context) { -RCU_INIT_POINTER (obj->fence_excl,dma_fence_get (replacement)); -dma_fence_put (old); - } - - list =dma_resv_shared_list (obj); -for (i =0 ; list && i < list->shared_count; ++i) { - old =rcu_dereference_protected (list->shared[i], -dma_resv_held (obj)); +dma_resv_list_entry (list, i,dma_resv_held (obj), &old,NULL );if (old->context != context)continue ; -rcu_assign_pointer (list->shared[i],dma_fence_get (replacement)); +dma_resv_list_set (list, i, replacement, usage);dma_fence_put (old); } -write_seqcount_end (&obj->seq); }EXPORT_SYMBOL (dma_resv_replace_fences); -/** -* dma_resv_add_excl_fence - Add an exclusive fence. -* @obj: the reservation object -* @fence: the exclusive fence to add -* -* Add a fence to the exclusive slot. @obj must be locked with dma_resv_lock(). -* Note that this function replaces all fences attached to @obj, see also -* &dma_resv.fence_excl for a discussion of the semantics. -*/ -void dma_resv_add_excl_fence (struct dma_resv *obj,struct dma_fence *fence) -{ -struct dma_fence *old_fence =dma_resv_excl_fence (obj); - -dma_resv_assert_held (obj); - -dma_fence_get (fence); - -write_seqcount_begin (&obj->seq); -/* write_seqcount_begin provides the necessary memory barrier */ -RCU_INIT_POINTER (obj->fence_excl, fence); -dma_resv_list_prune (dma_resv_shared_list (obj), obj); -write_seqcount_end (&obj->seq); - -dma_fence_put (old_fence); -} -EXPORT_SYMBOL (dma_resv_add_excl_fence); -/** * dma_resv_prune - remove signaled fences * @obj: The dma_resv object to prune
@@ -437,10 +427,7 @@ void dma_resv_prune(struct dma_resv *obj)dma_resv_assert_held (obj);write_seqcount_begin (&obj->seq); -if (obj->fence_excl &&dma_fence_is_signaled (obj->fence_excl)) -dma_fence_put (rcu_replace_pointer (obj->fence_excl,NULL , -dma_resv_held (obj))); -dma_resv_list_prune (dma_resv_shared_list (obj), obj); +dma_resv_list_prune (dma_resv_fences_list (obj), obj);write_seqcount_end (&obj->seq); }EXPORT_SYMBOL (dma_resv_prune_unlocked);
@@ -471,15 +458,11 @@ EXPORT_SYMBOL(dma_resv_prune);static void dma_resv_iter_restart_unlocked (struct dma_resv_iter *cursor) { cursor->seq =read_seqcount_begin (&cursor->obj->seq); - cursor->index =-1 ; - cursor->shared_count =0 ; -if (cursor->usage >=DMA_RESV_USAGE_READ ) { - cursor->fences =dma_resv_shared_list (cursor->obj); -if (cursor->fences) - cursor->shared_count = cursor->fences->shared_count; - }else { - cursor->fences =NULL ; - } + cursor->index =0 ; + cursor->num_fences =0 ; + cursor->fences =dma_resv_fences_list (cursor->obj); +if (cursor->fences) + cursor->num_fences = cursor->fences->num_fences; cursor->is_restarted = true; }
@@ -493,31 +476,29 @@ static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor)*/ static void dma_resv_iter_walk_unlocked (struct dma_resv_iter *cursor) { -struct dma_resv *obj = cursor->obj; +if (!cursor->fences) +return ;do {/* Drop the reference from the previous round */ dma_fence_put (cursor->fence); -if (cursor->index ==-1 ) { - cursor->fence =dma_resv_excl_fence (obj); - cursor->index++; -if (!cursor->fence) -continue ; - - }else if (!cursor->fences || - cursor->index >= cursor->shared_count) { +if (cursor->index >= cursor->num_fences) { cursor->fence =NULL ;break ; - }else { -struct dma_resv_list *fences = cursor->fences; -unsigned int idx = cursor->index++; - - cursor->fence =rcu_dereference (fences->shared[idx]); } + +dma_resv_list_entry (cursor->fences, cursor->index++, +dma_resv_held (cursor->obj), + &cursor->fence, + &cursor->fence_usage); cursor->fence =dma_fence_get_rcu (cursor->fence); -if (!cursor->fence || !dma_fence_is_signaled (cursor->fence)) +if (!cursor->fence) +break ; + +if (!dma_fence_is_signaled (cursor->fence) && + cursor->usage >= cursor->fence_usage)break ; }while (true); }
@@ -580,15 +561,9 @@ struct dma_fence *dma_resv_iter_first(struct dma_resv_iter *cursor)dma_resv_assert_held (cursor->obj); cursor->index =0 ; -if (cursor->usage >=DMA_RESV_USAGE_READ ) - cursor->fences =dma_resv_shared_list (cursor->obj); -else - cursor->fences =NULL ; - - fence =dma_resv_excl_fence (cursor->obj); -if (!fence) - fence =dma_resv_iter_next (cursor); + cursor->fences =dma_resv_fences_list (cursor->obj); + fence =dma_resv_iter_next (cursor); cursor->is_restarted = true;return fence; }
@@ -603,17 +578,18 @@ EXPORT_SYMBOL_GPL(dma_resv_iter_first);*/ struct dma_fence *dma_resv_iter_next (struct dma_resv_iter *cursor) { -unsigned int idx; +struct dma_fence *fence;dma_resv_assert_held (cursor->obj); cursor->is_restarted = false; -if (!cursor->fences || cursor->index >= cursor->fences->shared_count) +if (!cursor->fences || cursor->index >= cursor->fences->num_fences)return NULL ; - idx = cursor->index++; -return rcu_dereference_protected (cursor->fences->shared[idx], -dma_resv_held (cursor->obj)); +dma_resv_list_entry (cursor->fences, cursor->index++, +dma_resv_held (cursor->obj), + &fence, &cursor->fence_usage); +return fence; }EXPORT_SYMBOL_GPL (dma_resv_iter_next);
@@ -628,57 +604,43 @@ int dma_resv_copy_fences(struct dma_resv *dst, struct dma_resv *src) {struct dma_resv_iter cursor;struct dma_resv_list *list; -struct dma_fence *f, *excl; +struct dma_fence *f;dma_resv_assert_held (dst); list =NULL ; - excl =NULL ;dma_resv_iter_begin (&cursor, src,DMA_RESV_USAGE_OTHER );dma_resv_for_each_fence_unlocked (&cursor, f) {if (dma_resv_iter_is_restarted (&cursor)) {dma_resv_list_free (list); -dma_fence_put (excl); - -if (cursor.shared_count) { - list =dma_resv_list_alloc (cursor.shared_count); -if (!list) { -dma_resv_iter_end (&cursor); -return -ENOMEM ; - } - - list->shared_count =0 ; - }else { - list =NULL ; + list =dma_resv_list_alloc (cursor.num_fences); +if (!list) { +dma_resv_iter_end (&cursor); +return -ENOMEM ; } - excl =NULL ; + list->num_fences =0 ; }dma_fence_get (f); -if (dma_resv_iter_is_exclusive (&cursor)) - excl = f; -else -RCU_INIT_POINTER (list->shared[list->shared_count++], f); +dma_resv_list_set (list, list->num_fences++, f, +dma_resv_iter_usage (&cursor)); }dma_resv_iter_end (&cursor);write_seqcount_begin (&dst->seq); - excl =rcu_replace_pointer (dst->fence_excl, excl,dma_resv_held (dst)); - list =rcu_replace_pointer (dst->fence, list,dma_resv_held (dst)); + list =rcu_replace_pointer (dst->fences, list,dma_resv_held (dst));write_seqcount_end (&dst->seq);dma_resv_list_free (list); -dma_fence_put (excl); -return 0 ; }EXPORT_SYMBOL (dma_resv_copy_fences);/** -* dma_resv_get_fences - Get an object's shared and exclusive +* dma_resv_get_fences - Get an object's fences * fences without update side lock held * @obj: the reservation object * @usage: controls which fences to include
@@ -707,7 +669,7 @@ int dma_resv_get_fences(struct dma_resv *obj, enum dma_resv_usage usage,while (*num_fences)dma_fence_put ((*fences)[--(*num_fences)]); - count = cursor.shared_count +1 ; + count = cursor.num_fences +1 ;/* Eventually re-allocate the array */ *fences = krealloc_array(*fences, count,
@@ -777,8 +739,7 @@ int dma_resv_get_singleton(struct dma_resv *obj, enum dma_resv_usage usage,EXPORT_SYMBOL_GPL (dma_resv_get_singleton);/** -* dma_resv_wait_timeout - Wait on reservation's objects -* shared and/or exclusive fences. +* dma_resv_wait_timeout - Wait on reservation's objects fences * @obj: the reservation object * @usage: controls which fences to include in the wait * @intr: if true, do interruptible wait
@@ -851,13 +812,13 @@ EXPORT_SYMBOL_GPL(dma_resv_test_signaled);*/ void dma_resv_describe (struct dma_resv *obj,struct seq_file *seq) { +static const char *usage[] = {"kernel" ,"write" ,"read" ,"other" };struct dma_resv_iter cursor;struct dma_fence *fence;dma_resv_for_each_fence (&cursor, obj, true, fence) {seq_printf (seq,"\t%s fence:" , -dma_resv_iter_is_exclusive (&cursor) ? -"Exclusive" :"Shared" ); + usage[dma_resv_iter_usage (&cursor)]);dma_fence_describe (fence, seq); } }
drivers/dma-buf/st-dma-resv.c +31 -78
206 lines not on this page · raw message
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +3 -3
@@ -246,7 +246,7 @@ static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo,*/ replacement =dma_fence_get_stub ();dma_resv_replace_fences (bo->tbo.base.resv, ef->base.context, - replacement); + replacement,DMA_RESV_USAGE_OTHER );dma_fence_put (replacement);return 0 ; }
@@ -1207,7 +1207,7 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info, AMDGPU_FENCE_OWNER_KFD, false);if (ret) goto wait_pd_fail; - ret =dma_resv_reserve_shared (vm->root.bo->tbo.base.resv,1 ); + ret =dma_resv_reserve_fences (vm->root.bo->tbo.base.resv,1 );if (ret) goto reserve_shared_fail; amdgpu_bo_fence(vm->root.bo,
@@ -2454,7 +2454,7 @@ int amdgpu_amdkfd_add_gws_to_process(void *info, void *gws, struct kgd_mem **mem* Add process eviction fence to bo so they can * evict each other. */ - ret =dma_resv_reserve_shared (gws_bo->tbo.base.resv,1 ); + ret =dma_resv_reserve_fences (gws_bo->tbo.base.resv,1 );if (ret) goto reserve_shared_fail;amdgpu_bo_fence (gws_bo, &process_info->eviction_fence->base, true);
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +3 -3
@@ -54,8 +54,8 @@ static int amdgpu_cs_user_fence_chunk(struct amdgpu_cs_parser *p, bo =amdgpu_bo_ref (gem_to_amdgpu_bo (gobj)); p->uf_entry.priority =0 ; p->uf_entry.tv.bo = &bo->tbo; -/* One for TTM and one for the CS job */ - p->uf_entry.tv.num_shared =2 ; +/* One for TTM and two for the CS job */ + p->uf_entry.tv.num_shared =3 ;drm_gem_object_put (gobj);
@@ -1285,7 +1285,7 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,break ; }dma_fence_chain_init (chain, fence,dma_fence_get (p->fence),1 ); -rcu_assign_pointer (resv->fence_excl, &chain->base); +dma_resv_add_fence (resv, &chain->base,DMA_RESV_USAGE_WRITE ); e->chain =NULL ; }
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -4
@@ -1376,10 +1376,8 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence,return ; } -if (shared) -dma_resv_add_shared_fence (resv, fence); -else -dma_resv_add_excl_fence (resv, fence); +dma_resv_add_fence (resv, fence, shared ?DMA_RESV_USAGE_READ : +DMA_RESV_USAGE_WRITE ); }/**
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -2
@@ -2969,7 +2969,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm)if (r) goto error_free_root; - r =dma_resv_reserve_shared (root_bo->tbo.base.resv,1 ); + r =dma_resv_reserve_fences (root_bo->tbo.base.resv,1 );if (r) goto error_unreserve;
@@ -3412,7 +3412,7 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, value =0 ; } - r =dma_resv_reserve_shared (root->tbo.base.resv,1 ); + r =dma_resv_reserve_fences (root->tbo.base.resv,1 );if (r) {pr_debug ("failed %d to reserve fence slot\n" , r); goto error_unlock;
drivers/gpu/drm/amd/amdkfd/kfd_svm.c +1 -1
@@ -524,7 +524,7 @@ svm_range_vram_node_new(struct amdgpu_device *adev, struct svm_range *prange, goto reserve_bo_failed; } - r =dma_resv_reserve_shared (bo->tbo.base.resv,1 ); + r =dma_resv_reserve_fences (bo->tbo.base.resv,1 );if (r) {pr_debug ("failed %d to reserve bo\n" , r);amdgpu_bo_unreserve (bo);
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c +4 -8
@@ -180,7 +180,7 @@ static int submit_fence_sync(struct etnaviv_gem_submit *submit)struct dma_resv *robj = bo->obj->base.resv;enum dma_resv_usage usage; - ret =dma_resv_reserve_shared (robj,1 ); + ret =dma_resv_reserve_fences (robj,1 );if (ret)return ret;
@@ -204,14 +204,10 @@ static void submit_attach_object_fences(struct etnaviv_gem_submit *submit)for (i =0 ; i < submit->nr_bos; i++) {struct drm_gem_object *obj = &submit->bos[i].obj->base; +bool write = submit->bos[i].flags &ETNA_SUBMIT_BO_WRITE ; -if (submit->bos[i].flags &ETNA_SUBMIT_BO_WRITE ) -dma_resv_add_excl_fence (obj->resv, - submit->out_fence); -else -dma_resv_add_shared_fence (obj->resv, - submit->out_fence); - +dma_resv_add_fence (obj->resv, submit->out_fence, write ? +DMA_RESV_USAGE_WRITE :DMA_RESV_USAGE_READ );submit_unlock_object (submit, i); } }
drivers/gpu/drm/i915/gem/i915_gem_busy.c +7 -6
@@ -148,12 +148,13 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,if (dma_resv_iter_is_restarted (&cursor)) args->busy =0 ; -if (dma_resv_iter_is_exclusive (&cursor)) -/* Translate the exclusive fence to the READ *and* WRITE engine */ - args->busy |=busy_check_writer (fence); -else -/* Translate shared fences to READ set of engines */ - args->busy |=busy_check_reader (fence); +/* Translate read fences to READ set of engines */ + args->busy |=busy_check_reader (fence); + } +dma_resv_iter_begin (&cursor, obj->base.resv,DMA_RESV_USAGE_WRITE ); +dma_resv_for_each_fence_unlocked (&cursor, fence) { +/* Translate the write fences to the READ *and* WRITE engine */ + args->busy |=busy_check_writer (fence); }dma_resv_iter_end (&cursor);
drivers/gpu/drm/i915/gem/i915_gem_clflush.c +3 -2
@@ -101,14 +101,15 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj, clflush =NULL ;if (!(flags &I915_CLFLUSH_SYNC ) && -dma_resv_reserve_shared (obj->base.resv,1 ) ==0 ) +dma_resv_reserve_fences (obj->base.resv,1 ) ==0 ) clflush =clflush_work_create (obj);if (clflush) {i915_sw_fence_await_reservation (&clflush->base.chain, obj->base.resv,NULL , true,i915_fence_timeout (to_i915 (obj->base.dev)),I915_FENCE_GFP ); -dma_resv_add_excl_fence (obj->base.resv, &clflush->base.dma); +dma_resv_add_fence (obj->base.resv, &clflush->base.dma, +DMA_RESV_USAGE_KERNEL );dma_fence_work_commit (&clflush->base); }else if (obj->mm.pages) {__do_clflush (obj);
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +2 -2
@@ -989,7 +989,7 @@ static int eb_validate_vmas(struct i915_execbuffer *eb) } } - err =dma_resv_reserve_shared (vma->resv,1 ); + err =dma_resv_reserve_fences (vma->resv,1 );if (err)return err;
@@ -2162,7 +2162,7 @@ static int eb_parse(struct i915_execbuffer *eb) goto err_trampoline; } - err =dma_resv_reserve_shared (shadow->resv,1 ); + err =dma_resv_reserve_fences (shadow->resv,1 );if (err) goto err_trampoline;
drivers/gpu/drm/i915/gem/selftests/i915_gem_migrate.c +1 -1
@@ -179,7 +179,7 @@ static int igt_lmem_pages_migrate(void *arg)i915_gem_object_is_lmem (obj), 0xdeadbeaf, &rq);if (rq) { - err =dma_resv_reserve_shared (obj->base.resv,1 ); + err =dma_resv_reserve_fences (obj->base.resv,1 );if (!err)dma_resv_add_excl_fence (obj->base.resv, &rq->fence);
drivers/gpu/drm/i915/i915_vma.c +6 -4
@@ -1256,25 +1256,27 @@ int _i915_vma_move_to_active(struct i915_vma *vma, }if (!(flags & __EXEC_OBJECT_NO_RESERVE)) { - err =dma_resv_reserve_shared (vma->resv,1 ); + err =dma_resv_reserve_fences (vma->resv,1 );if (unlikely (err))return err; }if (fence) { -dma_resv_add_excl_fence (vma->resv, fence); +dma_resv_add_fence (vma->resv, fence, +DMA_RESV_USAGE_WRITE ); obj->write_domain =I915_GEM_DOMAIN_RENDER ; obj->read_domains =0 ; } }else {if (!(flags & __EXEC_OBJECT_NO_RESERVE)) { - err =dma_resv_reserve_shared (vma->resv,1 ); + err =dma_resv_reserve_fences (vma->resv,1 );if (unlikely (err))return err; }if (fence) { -dma_resv_add_shared_fence (vma->resv, fence); +dma_resv_add_fence (vma->resv, fence, +DMA_RESV_USAGE_READ ); obj->write_domain =0 ; } }
drivers/gpu/drm/i915/selftests/intel_memory_region.c +1 -1
@@ -895,7 +895,7 @@ static int igt_lmem_write_cpu(void *arg)i915_gem_object_lock (obj,NULL ); - err =dma_resv_reserve_shared (obj->base.resv,1 ); + err =dma_resv_reserve_fences (obj->base.resv,1 );if (err) {i915_gem_object_unlock (obj); goto out_put;
drivers/gpu/drm/lima/lima_gem.c +2 -2
@@ -257,7 +257,7 @@ static int lima_gem_sync_bo(struct lima_sched_task *task, struct lima_bo *bo, {int err; - err =dma_resv_reserve_shared (lima_bo_resv (bo),1 ); + err =dma_resv_reserve_fences (lima_bo_resv (bo),1 );if (err)return err;
@@ -365,7 +365,7 @@ int lima_gem_submit(struct drm_file *file, struct lima_submit *submit)if (submit->bos[i].flags &LIMA_SUBMIT_BO_WRITE )dma_resv_add_excl_fence (lima_bo_resv (bos[i]), fence);else -dma_resv_add_shared_fence (lima_bo_resv (bos[i]), fence); +dma_resv_add_fence (lima_bo_resv (bos[i]), fence); } drm_gem_unlock_reservations((struct drm_gem_object **)bos,
drivers/gpu/drm/msm/msm_gem_submit.c +2 -2
@@ -325,7 +325,7 @@ static int submit_fence_sync(struct msm_gem_submit *submit, bool no_implicit)* strange place to call it. OTOH this is a * convenient can-fail point to hook it in. */ - ret =dma_resv_reserve_shared (obj->resv,1 ); + ret =dma_resv_reserve_fences (obj->resv,1 );if (ret)return ret;
@@ -397,7 +397,7 @@ static void submit_attach_object_fences(struct msm_gem_submit *submit)if (submit->bos[i].flags &MSM_SUBMIT_BO_WRITE )dma_resv_add_excl_fence (obj->resv, submit->user_fence);else if (submit->bos[i].flags &MSM_SUBMIT_BO_READ ) -dma_resv_add_shared_fence (obj->resv, submit->user_fence); +dma_resv_add_fence (obj->resv, submit->user_fence); } }
drivers/gpu/drm/nouveau/nouveau_bo.c +5 -4
@@ -1308,10 +1308,11 @@ nouveau_bo_fence(struct nouveau_bo *nvbo, struct nouveau_fence *fence, bool excl {struct dma_resv *resv = nvbo->bo.base.resv; -if (exclusive) -dma_resv_add_excl_fence (resv, &fence->base); -else if (fence) -dma_resv_add_shared_fence (resv, &fence->base); +if (!fence) +return ; + +dma_resv_add_fence (resv, &fence->base, exclusive ? +DMA_RESV_USAGE_WRITE :DMA_RESV_USAGE_READ ); } static void
drivers/gpu/drm/nouveau/nouveau_fence.c +1 -1
@@ -349,7 +349,7 @@ nouveau_fence_sync(struct nouveau_bo *nvbo, struct nouveau_channel *chan,struct nouveau_fence *f;int ret; - ret =dma_resv_reserve_shared (resv,1 ); + ret =dma_resv_reserve_fences (resv,1 );if (ret)return ret;
drivers/gpu/drm/panfrost/panfrost_job.c +1 -1
@@ -247,7 +247,7 @@ static int panfrost_acquire_object_fences(struct drm_gem_object **bos,int i, ret;for (i =0 ; i < bo_count; i++) { - ret =dma_resv_reserve_shared (bos[i]->resv,1 ); + ret =dma_resv_reserve_fences (bos[i]->resv,1 );if (ret)return ret;
drivers/gpu/drm/qxl/qxl_release.c +3 -2
@@ -200,7 +200,7 @@ static int qxl_release_validate_bo(struct qxl_bo *bo)return ret; } - ret =dma_resv_reserve_shared (bo->tbo.base.resv,1 ); + ret =dma_resv_reserve_fences (bo->tbo.base.resv,1 );if (ret)return ret;
@@ -429,7 +429,8 @@ void qxl_release_fence_buffer_objects(struct qxl_release *release)list_for_each_entry (entry, &release->bos, head) { bo = entry->bo; -dma_resv_add_shared_fence (bo->base.resv, &release->base); +dma_resv_add_fence (bo->base.resv, &release->base, +DMA_RESV_USAGE_READ );ttm_bo_move_to_lru_tail_unlocked (bo);dma_resv_unlock (bo->base.resv); }
drivers/gpu/drm/radeon/radeon_object.c +2 -4
@@ -809,8 +809,6 @@ void radeon_bo_fence(struct radeon_bo *bo, struct radeon_fence *fence, {struct dma_resv *resv = bo->tbo.base.resv; -if (shared) -dma_resv_add_shared_fence (resv, &fence->base); -else -dma_resv_add_excl_fence (resv, &fence->base); +dma_resv_add_fence (resv, &fence->base, shared ? +DMA_RESV_USAGE_READ :DMA_RESV_USAGE_WRITE ); }
drivers/gpu/drm/radeon/radeon_vm.c +1 -1
@@ -831,7 +831,7 @@ static int radeon_vm_update_ptes(struct radeon_device *rdev,int r;radeon_sync_resv (rdev, &ib->sync, pt->tbo.base.resv, true); - r =dma_resv_reserve_shared (pt->tbo.base.resv,1 ); + r =dma_resv_reserve_fences (pt->tbo.base.resv,1 );if (r)return r;
drivers/gpu/drm/ttm/ttm_bo.c +3 -3
@@ -762,9 +762,9 @@ static int ttm_bo_add_move_fence(struct ttm_buffer_object *bo,return ret; } -dma_resv_add_shared_fence (bo->base.resv, fence); +dma_resv_add_fence (bo->base.resv, fence,DMA_RESV_USAGE_KERNEL ); - ret =dma_resv_reserve_shared (bo->base.resv,1 ); + ret =dma_resv_reserve_fences (bo->base.resv,1 );if (unlikely (ret)) {dma_fence_put (fence);return ret;
@@ -823,7 +823,7 @@ int ttm_bo_mem_space(struct ttm_buffer_object *bo,bool type_found = false;int i, ret; - ret =dma_resv_reserve_shared (bo->base.resv,1 ); + ret =dma_resv_reserve_fences (bo->base.resv,1 );if (unlikely (ret))return ret;
drivers/gpu/drm/ttm/ttm_bo_util.c +4 -3
@@ -243,7 +243,7 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, ret =dma_resv_trylock (&fbo->base.base._resv);WARN_ON (!ret); - ret =dma_resv_reserve_shared (&fbo->base.base._resv,1 ); + ret =dma_resv_reserve_fences (&fbo->base.base._resv,1 );if (ret) {kfree (fbo);return ret;
@@ -503,7 +503,8 @@ static int ttm_bo_move_to_ghost(struct ttm_buffer_object *bo,if (ret)return ret; -dma_resv_add_excl_fence (&ghost_obj->base._resv, fence); +dma_resv_add_fence (&ghost_obj->base._resv, fence, +DMA_RESV_USAGE_KERNEL );/** * If we're not moving to fixed memory, the TTM object
@@ -558,7 +559,7 @@ int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo,struct ttm_resource_manager *man =ttm_manager_type (bdev, new_mem->mem_type);int ret =0 ; -dma_resv_add_excl_fence (bo->base.resv, fence); +dma_resv_add_fence (bo->base.resv, fence,DMA_RESV_USAGE_KERNEL );if (!evict) ret =ttm_bo_move_to_ghost (bo, fence, man->use_tt);else if (!from->use_tt && pipeline)
drivers/gpu/drm/ttm/ttm_execbuf_util.c +4 -6
@@ -103,7 +103,7 @@ int ttm_eu_reserve_buffers(struct ww_acquire_ctx *ticket, num_fences =min (entry->num_shared,1u );if (!ret) { - ret =dma_resv_reserve_shared (bo->base.resv, + ret =dma_resv_reserve_fences (bo->base.resv, num_fences);if (!ret)continue ;
@@ -120,7 +120,7 @@ int ttm_eu_reserve_buffers(struct ww_acquire_ctx *ticket, }if (!ret) - ret =dma_resv_reserve_shared (bo->base.resv, + ret =dma_resv_reserve_fences (bo->base.resv, num_fences);if (unlikely (ret !=0 )) {
@@ -154,10 +154,8 @@ void ttm_eu_fence_buffer_objects(struct ww_acquire_ctx *ticket,list_for_each_entry (entry, list, head) {struct ttm_buffer_object *bo = entry->bo; -if (entry->num_shared) -dma_resv_add_shared_fence (bo->base.resv, fence); -else -dma_resv_add_excl_fence (bo->base.resv, fence); +dma_resv_add_fence (bo->base.resv, fence, entry->num_shared ? +DMA_RESV_USAGE_READ :DMA_RESV_USAGE_WRITE );ttm_bo_move_to_lru_tail_unlocked (bo);dma_resv_unlock (bo->base.resv); }
drivers/gpu/drm/v3d/v3d_gem.c +3 -3
@@ -259,7 +259,7 @@ v3d_lock_bo_reservations(struct v3d_job *job,return ret;for (i =0 ; i < job->bo_count; i++) { - ret =dma_resv_reserve_shared (job->bo[i]->resv,1 ); + ret =dma_resv_reserve_fences (job->bo[i]->resv,1 );if (ret) goto fail;
@@ -550,8 +550,8 @@ v3d_attach_fences_and_unlock_reservation(struct drm_file *file_priv,for (i =0 ; i < job->bo_count; i++) {/* XXX: Use shared fences for read-only objects. */ -dma_resv_add_excl_fence (job->bo[i]->resv, - job->done_fence); +dma_resv_add_fence (job->bo[i]->resv, job->done_fence, +DMA_RESV_USAGE_WRITE ); }drm_gem_unlock_reservations (job->bo, job->bo_count, acquire_ctx);
drivers/gpu/drm/vc4/vc4_gem.c +2 -2
@@ -543,7 +543,7 @@ vc4_update_bo_seqnos(struct vc4_exec_info *exec, uint64_t seqno) bo =to_vc4_bo (&exec->bo[i]->base); bo->seqno = seqno; -dma_resv_add_shared_fence (bo->base.base.resv, exec->fence); +dma_resv_add_fence (bo->base.base.resv, exec->fence); }list_for_each_entry (bo, &exec->unref_list, unref_head) {
@@ -641,7 +641,7 @@ vc4_lock_bo_reservations(struct drm_device *dev,for (i =0 ; i < exec->bo_count; i++) { bo = &exec->bo[i]->base; - ret =dma_resv_reserve_shared (bo->resv,1 ); + ret =dma_resv_reserve_fences (bo->resv,1 );if (ret) {vc4_unlock_bo_reservations (dev, exec, acquire_ctx);return ret;
drivers/gpu/drm/vgem/vgem_fence.c +4 -7
@@ -160,13 +160,10 @@ int vgem_fence_attach_ioctl(struct drm_device *dev,/* Expose the fence via the dma-buf */ dma_resv_lock (resv,NULL ); - ret =dma_resv_reserve_shared (resv,1 ); -if (!ret) { -if (arg->flags &VGEM_FENCE_WRITE ) -dma_resv_add_excl_fence (resv, fence); -else -dma_resv_add_shared_fence (resv, fence); - } + ret =dma_resv_reserve_fences (resv,1 ); +if (!ret) +dma_resv_add_fence (resv, fence, arg->flags &VGEM_FENCE_WRITE ? +DMA_RESV_USAGE_WRITE :DMA_RESV_USAGE_READ );dma_resv_unlock (resv);/* Record the fence in our idr for later signaling */
drivers/gpu/drm/virtio/virtgpu_gem.c +3 -2
@@ -227,7 +227,7 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs)return ret;for (i =0 ; i < objs->nents; ++i) { - ret =dma_resv_reserve_shared (objs->objs[i]->resv,1 ); + ret =dma_resv_reserve_fences (objs->objs[i]->resv,1 );if (ret)return ret; }
@@ -250,7 +250,8 @@ void virtio_gpu_array_add_fence(struct virtio_gpu_object_array *objs,int i;for (i =0 ; i < objs->nents; i++) -dma_resv_add_excl_fence (objs->objs[i]->resv, fence); +dma_resv_add_fence (objs->objs[i]->resv, fence, +DMA_RESV_USAGE_WRITE ); }void virtio_gpu_array_put_free (struct virtio_gpu_object_array *objs)
drivers/gpu/drm/vmwgfx/vmwgfx_bo.c +3 -2
@@ -1062,9 +1062,10 @@ void vmw_bo_fence_single(struct ttm_buffer_object *bo,else dma_fence_get (&fence->base); - ret =dma_resv_reserve_shared (bo->base.resv,1 ); + ret =dma_resv_reserve_fences (bo->base.resv,1 );if (!ret) -dma_resv_add_excl_fence (bo->base.resv, &fence->base); +dma_resv_add_fence (bo->base.resv, &fence->base, +DMA_RESV_USAGE_KERNEL );else /* Last resort fallback when we are OOM */ dma_fence_wait (&fence->base, false);
include/linux/dma-resv.h +28 -60
149 lines not on this page · raw message
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 24/26 dma-buf: wait for map to complete for static attachments
In reply to Christian KönigWe have previously done that in the individual drivers but it is more defensive to move that into the common code.
Dynamic attachments should wait for map operations to complete by themselves.
--- drivers/dma-buf/dma-buf.c | 18 +++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 14 +------------- drivers/gpu/drm/nouveau/nouveau_prime.c | 17 +---------------- drivers/gpu/drm/radeon/radeon_prime.c | 16 +++------------- 4 files changed, 20 insertions(+), 45 deletions(-)
drivers/dma-buf/dma-buf.c +15 -3
@@ -660,12 +660,24 @@ static struct sg_table * __map_dma_buf(struct dma_buf_attachment *attach, enum dma_data_direction direction) {struct sg_table *sg_table; +signed long ret; sg_table = attach->dmabuf->ops->map_dma_buf (attach, direction); +if (IS_ERR_OR_NULL (sg_table)) +return sg_table; + +if (!dma_buf_attachment_is_dynamic (attach)) { + ret =dma_resv_wait_timeout (attach->dmabuf->resv, +DMA_RESV_USAGE_KERNEL , true, +MAX_SCHEDULE_TIMEOUT ); +if (ret <0 ) { + attach->dmabuf->ops->unmap_dma_buf (attach, sg_table, + direction); +return ERR_PTR (ret); + } + } -if (!IS_ERR_OR_NULL (sg_table)) -mangle_sg_table (sg_table); - +mangle_sg_table (sg_table);return sg_table; }
drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +1 -13
@@ -105,21 +105,9 @@ static int amdgpu_dma_buf_pin(struct dma_buf_attachment *attach) {struct drm_gem_object *obj = attach->dmabuf->priv;struct amdgpu_bo *bo =gem_to_amdgpu_bo (obj); -int r;/* pin buffer into GTT */ - r =amdgpu_bo_pin (bo,AMDGPU_GEM_DOMAIN_GTT ); -if (r) -return r; - -if (bo->tbo.moving) { - r =dma_fence_wait (bo->tbo.moving, true); -if (r) { -amdgpu_bo_unpin (bo); -return r; - } - } -return 0 ; +return amdgpu_bo_pin (bo,AMDGPU_GEM_DOMAIN_GTT ); }/**
drivers/gpu/drm/nouveau/nouveau_prime.c +1 -16
@@ -93,22 +93,7 @@ int nouveau_gem_prime_pin(struct drm_gem_object *obj)if (ret)return -EINVAL ; - ret =ttm_bo_reserve (&nvbo->bo, false, false,NULL ); -if (ret) - goto error; - -if (nvbo->bo.moving) - ret =dma_fence_wait (nvbo->bo.moving, true); - -ttm_bo_unreserve (&nvbo->bo); -if (ret) - goto error; - -return ret; - -error: -nouveau_bo_unpin (nvbo); -return ret; +return 0 ; }void nouveau_gem_prime_unpin (struct drm_gem_object *obj)
drivers/gpu/drm/radeon/radeon_prime.c +3 -13
@@ -77,19 +77,9 @@ int radeon_gem_prime_pin(struct drm_gem_object *obj)/* pin buffer into GTT */ ret =radeon_bo_pin (bo,RADEON_GEM_DOMAIN_GTT ,NULL ); -if (unlikely (ret)) - goto error; - -if (bo->tbo.moving) { - ret =dma_fence_wait (bo->tbo.moving, false); -if (unlikely (ret)) { -radeon_bo_unpin (bo); - goto error; - } - } - - bo->prime_shared_count++; -error: +if (likely (ret ==0 )) + bo->prime_shared_count++; +radeon_bo_unreserve (bo);return ret; }
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 25/26 amdgpu: remove DMA-buf fence workaround
In reply to Christian KönigNot needed any more now we have that inside the framework.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h | 1 - drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 52 +++------------------ 2 files changed, 6 insertions(+), 47 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h +0 -1
@@ -34,7 +34,6 @@ struct amdgpu_fpriv;struct amdgpu_bo_list_entry {struct ttm_validate_buffer tv;struct amdgpu_bo_va *bo_va; -struct dma_fence_chain *chain;uint32_t priority;struct page **user_pages;bool user_invalidated;
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +6 -46
79 lines not on this page · raw message
Signature · 1 line
2.25.1
Christian König
reviewerPATCH 26/26 drm/ttm: remove bo->moving
In reply to Christian KönigThis is now handled by the DMA-buf framework in the dma_resv obj.
--- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 13 ++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 7 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 11 +++--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 11 ++++-- drivers/gpu/drm/ttm/ttm_bo.c | 10 ++---- drivers/gpu/drm/ttm/ttm_bo_util.c | 7 ---- drivers/gpu/drm/ttm/ttm_bo_vm.c | 34 +++++++------------ drivers/gpu/drm/vmwgfx/vmwgfx_resource.c | 6 ---- include/drm/ttm/ttm_bo_api.h | 2 -- 9 files changed, 40 insertions(+), 61 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +9 -4
@@ -2330,6 +2330,8 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef)struct amdgpu_bo *bo = mem->bo;uint32_t domain = mem->domain;struct kfd_mem_attachment *attachment; +struct dma_resv_iter cursor; +struct dma_fence *fence; total_size +=amdgpu_bo_size (bo);
@@ -2344,10 +2346,13 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) goto validate_map_fail; } } - ret =amdgpu_sync_fence (&sync_obj, bo->tbo.moving); -if (ret) { -pr_debug ("Memory eviction: Sync BO fence failed. Try again\n" ); - goto validate_map_fail; +dma_resv_for_each_fence (&cursor, bo->tbo.base.resv, +DMA_RESV_USAGE_KERNEL , fence) { + ret =amdgpu_sync_fence (&sync_obj, fence); +if (ret) { +pr_debug ("Memory eviction: Sync BO fence failed. Try again\n" ); + goto validate_map_fail; + } }list_for_each_entry (attachment, &mem->attachments, list) {if (!attachment->is_mapped)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -4
@@ -608,9 +608,8 @@ int amdgpu_bo_create(struct amdgpu_device *adev,if (unlikely (r)) goto fail_unreserve; -amdgpu_bo_fence (bo, fence, false); -dma_fence_put (bo->tbo.moving); - bo->tbo.moving =dma_fence_get (fence); +dma_resv_add_fence (bo->tbo.base.resv, fence, +DMA_RESV_USAGE_KERNEL );dma_fence_put (fence); }if (!bp->resv)
@@ -1290,7 +1289,7 @@ void amdgpu_bo_release_notify(struct ttm_buffer_object *bo) r =amdgpu_fill_buffer (abo,AMDGPU_POISON , bo->base.resv, &fence);if (!WARN_ON (r)) { -amdgpu_bo_fence (abo, fence, false); +dma_resv_add_fence (bo->base.resv, fence,DMA_RESV_USAGE_KERNEL );dma_fence_put (fence); }
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c +5 -6
@@ -74,13 +74,12 @@ static int amdgpu_vm_cpu_update(struct amdgpu_vm_update_params *p, {unsigned int i;uint64_t value; -int r; +long r; -if (vmbo->bo.tbo.moving) { - r =dma_fence_wait (vmbo->bo.tbo.moving, true); -if (r) -return r; - } + r =dma_resv_wait_timeout (vmbo->bo.tbo.base.resv,DMA_RESV_USAGE_KERNEL , + true,MAX_SCHEDULE_TIMEOUT ); +if (r <0 ) +return r; pe += (unsigned long )amdgpu_bo_kptr (&vmbo->bo);
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +8 -3
@@ -204,14 +204,19 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,struct amdgpu_bo *bo = &vmbo->bo;enum amdgpu_ib_pool_type pool = p->immediate ?AMDGPU_IB_POOL_IMMEDIATE :AMDGPU_IB_POOL_DELAYED ; +struct dma_resv_iter cursor;unsigned int i, ndw, nptes; +struct dma_fence *fence;uint64_t *pte;int r;/* Wait for PD/PT moves to be completed */ - r =amdgpu_sync_fence (&p->job->sync, bo->tbo.moving); -if (r) -return r; +dma_resv_for_each_fence (&cursor, bo->tbo.base.resv, +DMA_RESV_USAGE_KERNEL , fence) { + r =amdgpu_sync_fence (&p->job->sync, fence); +if (r) +return r; + } do { ndw = p->num_dw_left;
drivers/gpu/drm/ttm/ttm_bo.c +2 -8
@@ -468,7 +468,6 @@ static void ttm_bo_release(struct kref *kref)dma_resv_unlock (bo->base.resv);atomic_dec (&ttm_glob.bo_count); -dma_fence_put (bo->moving); bo->destroy (bo); }
@@ -737,9 +736,8 @@ int ttm_mem_evict_first(struct ttm_device *bdev, }/* -* Add the last move fence to the BO and reserve a new shared slot. We only use -* a shared slot to avoid unecessary sync and rely on the subsequent bo move to -* either stall or use an exclusive fence respectively set bo->moving. +* Add the last move fence to the BO as kernel dependency and reserve a new +* fence slot. */ static int ttm_bo_add_move_fence(struct ttm_buffer_object *bo, struct ttm_resource_manager *man,
@@ -769,9 +767,6 @@ static int ttm_bo_add_move_fence(struct ttm_buffer_object *bo,dma_fence_put (fence);return ret; } - -dma_fence_put (bo->moving); - bo->moving = fence;return 0 ; }
@@ -978,7 +973,6 @@ int ttm_bo_init_reserved(struct ttm_device *bdev, bo->bdev = bdev; bo->type = type; bo->page_alignment = page_alignment; - bo->moving =NULL ; bo->pin_count =0 ; bo->sg = sg;if (resv) {
drivers/gpu/drm/ttm/ttm_bo_util.c +0 -7
@@ -229,7 +229,6 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo,atomic_inc (&ttm_glob.bo_count);INIT_LIST_HEAD (&fbo->base.ddestroy);INIT_LIST_HEAD (&fbo->base.lru); - fbo->base.moving =NULL ;drm_vma_node_reset (&fbo->base.base.vma_node);kref_init (&fbo->base.kref);
@@ -496,9 +495,6 @@ static int ttm_bo_move_to_ghost(struct ttm_buffer_object *bo,* operation has completed. */ -dma_fence_put (bo->moving); - bo->moving =dma_fence_get (fence); - ret =ttm_buffer_object_transfer (bo, &ghost_obj);if (ret)return ret;
@@ -543,9 +539,6 @@ static void ttm_bo_move_pipeline_evict(struct ttm_buffer_object *bo,spin_unlock (&from->move_lock);ttm_resource_free (bo, &bo->resource); - -dma_fence_put (bo->moving); - bo->moving =dma_fence_get (fence); } int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo,
drivers/gpu/drm/ttm/ttm_bo_vm.c +13 -21
@@ -46,17 +46,13 @@static vm_fault_t ttm_bo_vm_fault_idle (struct ttm_buffer_object *bo,struct vm_fault *vmf) { -vm_fault_t ret =0 ; -int err =0 ; - -if (likely (!bo->moving)) - goto out_unlock; +long err =0 ;/* * Quick non-stalling check for idle. */ -if (dma_fence_is_signaled (bo->moving)) - goto out_clear; +if (dma_resv_test_signaled (bo->base.resv,DMA_RESV_USAGE_KERNEL )) +return 0 ;/* * If possible, avoid waiting for GPU with mmap_lock
@@ -64,34 +60,30 @@ static vm_fault_t ttm_bo_vm_fault_idle(struct ttm_buffer_object *bo,* is the first attempt. */ if (fault_flag_allow_retry_first (vmf->flags)) { - ret =VM_FAULT_RETRY ;if (vmf->flags &FAULT_FLAG_RETRY_NOWAIT ) - goto out_unlock; +return VM_FAULT_RETRY ;ttm_bo_get (bo);mmap_read_unlock (vmf->vma->vm_mm); - (void )dma_fence_wait (bo->moving, true); + (void )dma_resv_wait_timeout (bo->base.resv, +DMA_RESV_USAGE_KERNEL , true, +MAX_SCHEDULE_TIMEOUT );dma_resv_unlock (bo->base.resv);ttm_bo_put (bo); - goto out_unlock; +return VM_FAULT_RETRY ; }/* * Ordinary wait. */ - err =dma_fence_wait (bo->moving, true); -if (unlikely (err !=0 )) { - ret = (err != -ERESTARTSYS ) ?VM_FAULT_SIGBUS : + err =dma_resv_wait_timeout (bo->base.resv,DMA_RESV_USAGE_KERNEL , true, +MAX_SCHEDULE_TIMEOUT ); +if (unlikely (err <0 )) { +return (err != -ERESTARTSYS ) ?VM_FAULT_SIGBUS :VM_FAULT_NOPAGE ; - goto out_unlock; } -out_clear: -dma_fence_put (bo->moving); - bo->moving =NULL ; - -out_unlock: -return ret; +return 0 ; } static unsigned long ttm_bo_io_mem_pfn(struct ttm_buffer_object *bo,
drivers/gpu/drm/vmwgfx/vmwgfx_resource.c +0 -6
@@ -1166,12 +1166,6 @@ int vmw_resources_clean(struct vmw_buffer_object *vbo, pgoff_t start, *num_prefault =__KERNEL_DIV_ROUND_UP (last_cleaned - res_start,PAGE_SIZE );vmw_bo_fence_single (bo,NULL ); -if (bo->moving) -dma_fence_put (bo->moving); - -return dma_resv_get_singleton (bo->base.resv, -DMA_RESV_USAGE_KERNEL , - &bo->moving); }return 0 ;
include/drm/ttm/ttm_bo_api.h +0 -2
@@ -98,7 +98,6 @@ struct ttm_tt;* @lru: List head for the lru list. * @ddestroy: List head for the delayed destroy list. * @swap: List head for swap LRU list. -* @moving: Fence set when BO is moving * @offset: The current GPU offset, which can have different meanings * depending on the memory type. For SYSTEM type memory, it should be 0. * @cur_placement: Hint of current placement.
@@ -151,7 +150,6 @@ struct ttm_buffer_object {* Members protected by a bo reservation. */ -struct dma_fence *moving;unsigned priority;unsigned pin_count;
Signature · 1 line
2.25.1
Quoted 31 lines · On Tue, 23 Nov 2021 15:20:45 +0100 "Christian König"…
Hi guys,
as discussed before this set of patches completely rework the
dma_resvsemantic and spreads the new handling over all the existing drivers and users.First of all this drops the DAG approach because it requires that every single driver implements those relatively complicated rules correctly and any violation of that immediately leads to either corruption of freed memory or even more severe security problems.
Instead we just keep all fences around all the time until they are signaled. Only fences with the same context are assumed to be signaled in the correct order since this is exercised elsewhere as well. Replacing fences is now only supported for hardware mechanism like VM page table updates where the hardware can guarantee that the resource can't be accessed any more.
Then the concept of a single exclusive fence and multiple shared fences is dropped as well.
Instead the
dma_resvobject is now just a container fordma_fenceobjects where each fence has associated usage flags. Those use flags describe how the operation represented by thedma_fenceobject is using the resource protected by thedma_resvobject. This allows us to add multiple fences for each usage type.Additionally to the existing WRITE/READ usages this patch set also adds the new KERNEL and OTHER usages. The KERNEL usages is used in cases where the kernel needs to do some operation with the resource protected by the
dma_resvobject, like copies or clears. Those are mandatory to wait for when dynamic memory management is used.
Quoted 4 lines
The OTHER usage is for cases where we don't want that the operation represented by the
dma_fenceobject participate in any implicit sync but needs to be respected by the kernel memory management. Examples for those are VM page table updates and preemption fences.
Hi,
reading just the cover letter I wonder if KERNEL and OTHER could have better names based on what you describe how they are used. WRITE and READ immediately give an idea of semantics, KERNEL and OTHER not so much.
Some suggestions coming to my mind:
KERNEL -> PREPARE or INITIALIZE or SANITIZE
OTHER -> BOOKKEEP
Thanks,
pq
Quoted 16 lines
While doing this the new implementation cleans up existing workarounds all over the place, but especially amdgpu and TTM. Surprisingly I also found two use cases for the KERNEL/OTHER usage in i915 and Nouveau, those might need more thoughts.
In general the existing functionality should been preserved, the only downside is that we now always need to reserve a slot before adding a fence. The newly added call to the reservation function can probably use some more cleanup.
TODOs: Testing, testing, testing, doublechecking the newly added kerneldoc for any typos.
Please review and/or comment,
Christian.
Christian König
reviewerAm 24.11.21 um 09:31 schrieb Pekka Paalanen:
Quoted 45 lines
Quoted 31 lines · On Tue, 23 Nov 2021 15:20:45 +0100 "Christian König"…
Hi guys,
as discussed before this set of patches completely rework the
dma_resvsemantic and spreads the new handling over all the existing drivers and users.First of all this drops the DAG approach because it requires that every single driver implements those relatively complicated rules correctly and any violation of that immediately leads to either corruption of freed memory or even more severe security problems.
Instead we just keep all fences around all the time until they are signaled. Only fences with the same context are assumed to be signaled in the correct order since this is exercised elsewhere as well. Replacing fences is now only supported for hardware mechanism like VM page table updates where the hardware can guarantee that the resource can't be accessed any more.
Then the concept of a single exclusive fence and multiple shared fences is dropped as well.
Instead the
dma_resvobject is now just a container fordma_fenceobjects where each fence has associated usage flags. Those use flags describe how the operation represented by thedma_fenceobject is using the resource protected by thedma_resvobject. This allows us to add multiple fences for each usage type.Additionally to the existing WRITE/READ usages this patch set also adds the new KERNEL and OTHER usages. The KERNEL usages is used in cases where the kernel needs to do some operation with the resource protected by the
dma_resvobject, like copies or clears. Those are mandatory to wait for when dynamic memory management is used.Quoted 4 lines
The OTHER usage is for cases where we don't want that the operation represented by the
dma_fenceobject participate in any implicit sync but needs to be respected by the kernel memory management. Examples for those are VM page table updates and preemption fences.Hi,
reading just the cover letter I wonder if KERNEL and OTHER could have better names based on what you describe how they are used. WRITE and READ immediately give an idea of semantics, KERNEL and OTHER not so much.
Quoted 4 lines
Some suggestions coming to my mind:
KERNEL -> PREPARE or INITIALIZE or SANITIZE
OTHER -> BOOKKEEP
Yes, I was entertaining similar thoughts for quite a while as well.
I think KERNEL fits better than the suggestions because that is really something only the kernel should use and we should not encourage anybody to use that for userspace submissions.
But using BOOKKEEP instead of OTHER sounds like a really good idea to me. Going to keep that in mind and if nobody has any better idea making the change for the next revision.
Thanks,
Christian.
Quoted 21 lines
Thanks,
pqQuoted 16 lines
While doing this the new implementation cleans up existing workarounds all over the place, but especially amdgpu and TTM. Surprisingly I also found two use cases for the KERNEL/OTHER usage in i915 and Nouveau, those might need more thoughts.
In general the existing functionality should been preserved, the only downside is that we now always need to reserve a slot before adding a fence. The newly added call to the reservation function can probably use some more cleanup.
TODOs: Testing, testing, testing, doublechecking the newly added kerneldoc for any typos.
Please review and/or comment,
Christian.
Quoted 41 lines · Christian König
Hi guys,
as discussed before this set of patches completely rework the
dma_resvsemantic and spreads the new handling over all the existing drivers and users.First of all this drops the DAG approach because it requires that every single driver implements those relatively complicated rules correctly and any violation of that immediately leads to either corruption of freed memory or even more severe security problems.
Instead we just keep all fences around all the time until they are signaled. Only fences with the same context are assumed to be signaled in the correct order since this is exercised elsewhere as well. Replacing fences is now only supported for hardware mechanism like VM page table updates where the hardware can guarantee that the resource can't be accessed any more.
Then the concept of a single exclusive fence and multiple shared fences is dropped as well.
Instead the
dma_resvobject is now just a container fordma_fenceobjects where each fence has associated usage flags. Those use flags describe how the operation represented by thedma_fenceobject is using the resource protected by thedma_resvobject. This allows us to add multiple fences for each usage type.Additionally to the existing WRITE/READ usages this patch set also adds the new KERNEL and OTHER usages. The KERNEL usages is used in cases where the kernel needs to do some operation with the resource protected by the
dma_resvobject, like copies or clears. Those are mandatory to wait for when dynamic memory management is used.The OTHER usage is for cases where we don't want that the operation represented by the
dma_fenceobject participate in any implicit sync but needs to be respected by the kernel memory management. Examples for those are VM page table updates and preemption fences.While doing this the new implementation cleans up existing workarounds all over the place, but especially amdgpu and TTM. Surprisingly I also found two use cases for the KERNEL/OTHER usage in i915 and Nouveau, those might need more thoughts.
Quoted 8 lines
In general the existing functionality should been preserved, the only downside is that we now always need to reserve a slot before adding a fence. The newly added call to the reservation function can probably use some more cleanup.
TODOs: Testing, testing, testing, doublechecking the newly added kerneldoc for any typos.
Please review and/or comment,
I like.
Unfortunately also massively burried, but I really like. I think the past few months (years?) of discussions and bikeshed have been worth it, this looks tidy and clear in semantics and in how drivers use it all.
Ofc this will take some time to review/test in detail and land, but I think next steps would be to resurrect Jason's explicit dma-buf fence import/export series (should also clean up nicely I think), and then roll out the new fence semantics to a few vk/compute stacks? I think especially for vk what we want is that normal CS only ever uses OTHER, and any implicit sync that needs to happen for winsys buffers is done through the import/export ioctls. GL might need something slightly different, but normally there's not many shared buffers, so doing a pile of ioctl calls for implicit synced buffers seems fine. But perhaps GL does want a new CS ioctl flag.
Cheers, Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch