Skip to contentSkip to conversation

Waitboost drm syncobj waits

RFC Superseded Tvrtko Ursulin 3 participants 8 messages Feb 9, 2023

Subsystems (5)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover Waitboost drm syncobj waits

From: Tvrtko Ursulin <[email protected]>

In i915 we have this concept of "wait boosting" where we give a priority boost for instance to fences which are actively waited upon from userspace. This has it's pros and cons and can certainly be discussed at lenght. However fact is some workloads really like it.

Problem is that with the arrival of drm syncobj and a new userspace waiting entry point it added, the waitboost mechanism was bypassed. Hence I cooked up this mini series really (really) quickly to see if some discussion can be had.

It adds a concept of "wait count" to dma fence, which is incremented for every explicit dma_fence_enable_sw_signaling and dma_fence_add_wait_callback (like dma_fence_add_callback but from explicit/userspace wait paths).

Individual drivers can then inspect this via dma_fence_wait_count() and decide to wait boost the waits on such fences.

Again, quickly put together and smoke tested only - no guarantees whatsoever and I will rely on interested parties to test and report if it even works or how well.

Tvrtko Ursulin (3):
dma-fence: Track explicit waiters drm/syncobj: Mark syncobj waits as external waiters drm/i915: Waitboost external waits

 drivers/dma-buf/dma-fence.c         | 102 ++++++++++++++++++++--------
 drivers/gpu/drm/drm_syncobj.c       |   6 +-
 drivers/gpu/drm/i915/i915_request.c |  13 +++-
 include/linux/dma-fence.h           |  14 ++++
 4 files changed, 101 insertions(+), 34 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

RFC 1/3 dma-fence: Track explicit waiters

From: Tvrtko Ursulin <[email protected]>

Track how many callers are explicity waiting on a fence to signal and allow querying that via new dma_fence_wait_count() API.

This provides infrastructure on top of which generic "waitboost" concepts can be implemented by individual drivers. Wait-boosting is any reactive activity, such as raising the GPU clocks, which happens while there are active external waiters.

---
 drivers/dma-buf/dma-fence.c | 102 ++++++++++++++++++++++++++----------
 include/linux/dma-fence.h   |  14 +++++
 2 files changed, 88 insertions(+), 28 deletions(-)
drivers/dma-buf/dma-fence.c +74 -28
index 0de0482cd36e..d463c06ebd6d 100644
@@ -344,6 +344,24 @@ void __dma_fence_might_wait(void)
 }
 #endif
 
+static void incr_wait_count(struct dma_fence *fence, struct dma_fence_cb *cb)
+{
+	lockdep_assert_held(fence->lock);
+
+	__set_bit(DMA_FENCE_CB_FLAG_WAITCOUNT_BIT, &cb->flags);
+	fence->waitcount++;
+	WARN_ON_ONCE(!fence->waitcount);
+}
+
+static void decr_wait_count(struct dma_fence *fence, struct dma_fence_cb *cb)
+{
+	lockdep_assert_held(fence->lock);
+
+	if (__test_and_clear_bit(DMA_FENCE_CB_FLAG_WAITCOUNT_BIT, &cb->flags)) {
+		WARN_ON_ONCE(!fence->waitcount);
+		fence->waitcount--;
+	}
+}
 
 /**
  * dma_fence_signal_timestamp_locked - signal completion of a fence
@@ -384,6 +402,7 @@ int dma_fence_signal_timestamp_locked(struct dma_fence *fence,
 
 	list_for_each_entry_safe(cur, tmp, &cb_list, node) {
 		INIT_LIST_HEAD(&cur->node);
+		decr_wait_count(fence, cur);
 		cur->func(fence, cur);
 	}
 
@@ -612,35 +631,15 @@ void dma_fence_enable_sw_signaling(struct dma_fence *fence)
 	unsigned long flags;
 
 	spin_lock_irqsave(fence->lock, flags);
+	fence->waitcount++;
+	WARN_ON_ONCE(!fence->waitcount);
 	__dma_fence_enable_signaling(fence);
 	spin_unlock_irqrestore(fence->lock, flags);
 }
 EXPORT_SYMBOL(dma_fence_enable_sw_signaling);
 
-/**
- * dma_fence_add_callback - add a callback to be called when the fence
- * is signaled
- * @fence: the fence to wait on
- * @cb: the callback to register
- * @func: the function to call
- *
- * Add a software callback to the fence. The caller should keep a reference to
- * the fence.
- *
- * @cb will be initialized by dma_fence_add_callback(), no initialization
- * by the caller is required. Any number of callbacks can be registered
- * to a fence, but a callback can only be registered to one fence at a time.
- *
- * If fence is already signaled, this function will return -ENOENT (and
- * *not* call the callback).
- *
- * Note that the callback can be called from an atomic context or irq context.
- *
- * Returns 0 in case of success, -ENOENT if the fence is already signaled
- * and -EINVAL in case of error.
- */
-int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
-			   dma_fence_func_t func)
+static int add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
+			dma_fence_func_t func, bool wait)
 {
 	unsigned long flags;
 	int ret = 0;
@@ -655,10 +654,15 @@ int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
 
 	spin_lock_irqsave(fence->lock, flags);
 
+	if (wait)
+		incr_wait_count(fence, cb);
+
 	if (__dma_fence_enable_signaling(fence)) {
 		cb->func = func;
 		list_add_tail(&cb->node, &fence->cb_list);
 	} else {
+		if (wait)
+			decr_wait_count(fence, cb);
 		INIT_LIST_HEAD(&cb->node);
 		ret = -ENOENT;
 	}
@@ -667,8 +671,44 @@ int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
 
 	return ret;
 }
+
+/**
+ * dma_fence_add_callback - add a callback to be called when the fence
+ * is signaled
+ * @fence: the fence to wait on
+ * @cb: the callback to register
+ * @func: the function to call
+ *
+ * Add a software callback to the fence. The caller should keep a reference to
+ * the fence.
+ *
+ * @cb will be initialized by dma_fence_add_callback(), no initialization
+ * by the caller is required. Any number of callbacks can be registered
+ * to a fence, but a callback can only be registered to one fence at a time.
+ *
+ * If fence is already signaled, this function will return -ENOENT (and
+ * *not* call the callback).
+ *
+ * Note that the callback can be called from an atomic context or irq context.
+ *
+ * Returns 0 in case of success, -ENOENT if the fence is already signaled
+ * and -EINVAL in case of error.
+ */
+int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
+			   dma_fence_func_t func)
+{
+	return add_callback(fence, cb, func, false);
+}
 EXPORT_SYMBOL(dma_fence_add_callback);
 
+int dma_fence_add_wait_callback(struct dma_fence *fence,
+				struct dma_fence_cb *cb,
+				dma_fence_func_t func)
+{
+	return add_callback(fence, cb, func, true);
+}
+EXPORT_SYMBOL(dma_fence_add_wait_callback);
+
 /**
  * dma_fence_get_status - returns the status upon completion
  * @fence: the dma_fence to query
@@ -721,8 +761,10 @@ dma_fence_remove_callback(struct dma_fence *fence, struct dma_fence_cb *cb)
 	spin_lock_irqsave(fence->lock, flags);
 
 	ret = !list_empty(&cb->node);
-	if (ret)
+	if (ret) {
+		decr_wait_count(fence, cb);
 		list_del_init(&cb->node);
+	}
 
 	spin_unlock_irqrestore(fence->lock, flags);
 
@@ -780,6 +822,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout)
 
 	cb.base.func = dma_fence_default_wait_cb;
 	cb.task = current;
+	incr_wait_count(fence, &cb.base);
 	list_add(&cb.base.node, &fence->cb_list);
 
 	while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) {
@@ -796,8 +839,10 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout)
 			ret = -ERESTARTSYS;
 	}
 
-	if (!list_empty(&cb.base.node))
+	if (!list_empty(&cb.base.node)) {
+		decr_wait_count(fence, &cb.base);
 		list_del(&cb.base.node);
+	}
 	__set_current_state(TASK_RUNNING);
 
 out:
@@ -875,8 +920,8 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count,
 		struct dma_fence *fence = fences[i];
 
 		cb[i].task = current;
-		if (dma_fence_add_callback(fence, &cb[i].base,
-					   dma_fence_default_wait_cb)) {
+		if (dma_fence_add_wait_callback(fence, &cb[i].base,
+						dma_fence_default_wait_cb)) {
 			/* This fence is already signaled */
 			if (idx)
 				*idx = i;
@@ -957,6 +1002,7 @@ dma_fence_init(struct dma_fence *fence, const struct dma_fence_ops *ops,
 	fence->context = context;
 	fence->seqno = seqno;
 	fence->flags = 0UL;
+	fence->waitcount = 0;
 	fence->error = 0;
 
 	trace_dma_fence_init(fence);
include/linux/dma-fence.h +14 -0
index 775cdc0b4f24..d0ed923e4545 100644
@@ -92,6 +92,7 @@ struct dma_fence {
 	u64 seqno;
 	unsigned long flags;
 	struct kref refcount;
+	unsigned int waitcount;
 	int error;
 };
 
@@ -116,6 +117,11 @@ typedef void (*dma_fence_func_t)(struct dma_fence *fence,
 struct dma_fence_cb {
 	struct list_head node;
 	dma_fence_func_t func;
+	unsigned long flags;
+};
+
+enum dma_fence_cb_flag_bits {
+	DMA_FENCE_CB_FLAG_WAITCOUNT_BIT,
 };
 
 /**
@@ -377,6 +383,9 @@ signed long dma_fence_default_wait(struct dma_fence *fence,
 int dma_fence_add_callback(struct dma_fence *fence,
 			   struct dma_fence_cb *cb,
 			   dma_fence_func_t func);
+int dma_fence_add_wait_callback(struct dma_fence *fence,
+				struct dma_fence_cb *cb,
+				dma_fence_func_t func);
 bool dma_fence_remove_callback(struct dma_fence *fence,
 			       struct dma_fence_cb *cb);
 void dma_fence_enable_sw_signaling(struct dma_fence *fence);
@@ -528,6 +537,11 @@ static inline int dma_fence_get_status_locked(struct dma_fence *fence)
 
 int dma_fence_get_status(struct dma_fence *fence);
 
+static inline unsigned int dma_fence_wait_count(struct dma_fence *fence)
+{
+	return fence->waitcount;
+}
+
 /**
  * dma_fence_set_error - flag an error condition on the fence
  * @fence: the dma_fence
Signature · 1 line
2.34.1
#2 lore raw

Re: [RFC 1/3] dma-fence: Track explicit waiters

reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross #…

Hi Tvrtko,

[FYI, it's a private test report for your RFC patch.] [auto build test ERROR on drm-tip/drm-tip]

url:    https://github.com/intel-lab-lkp/linux/commits/Tvrtko-Ursulin/dma-fence-Track-explicit-waiters/20230209-234523
base:   git://anongit.freedesktop.org/drm/drm-tip drm-tip
patch link:    https://lore.kernel.org/r/20230209154315.468682-2-tvrtko.ursulin%40linux.intel.com
patch subject: [RFC 1/3] dma-fence: Track explicit waiters
config: i386-randconfig-a004 (https://download.01.org/0day-ci/archive/20230210/[email protected]/config)
compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/871de6ff345b69a34f0cfe807395b6074b5a29ea
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Tvrtko-Ursulin/dma-fence-Track-explicit-waiters/20230209-234523 git checkout 871de6ff345b69a34f0cfe807395b6074b5a29ea
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/gpu/drm/i915/gt/intel_engine_pm.c:221:3: error: call to __compiletime_assert_548 declared with 'error' attribute: BUILD_BUG_ON failed: sizeof(rq->duration) > sizeof(rq->submitq)
                   BUILD_BUG_ON(sizeof(rq->duration) > sizeof(rq->submitq));
                   ^
   include/linux/build_bug.h:50:2: note: expanded from macro 'BUILD_BUG_ON'
           BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
           ^
   include/linux/build_bug.h:39:37: note: expanded from macro 'BUILD_BUG_ON_MSG'
   #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
                                       ^
   include/linux/compiler_types.h:358:2: note: expanded from macro 'compiletime_assert'
           _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
           ^
   include/linux/compiler_types.h:346:2: note: expanded from macro '_compiletime_assert'
           __compiletime_assert(condition, msg, prefix, suffix)
           ^
   include/linux/compiler_types.h:339:4: note: expanded from macro '__compiletime_assert'
                           prefix ## suffix();                             \
                           ^

<scratch space>:134:1: note: expanded from here __compiletime_assert_548
^
1 error generated.

vim +/error +221 drivers/gpu/drm/i915/gt/intel_engine_pm.c

5cba288466e9b2 Chris Wilson              2019-11-20  140  
79ffac8599c4d8 Chris Wilson              2019-04-24  141  static bool switch_to_kernel_context(struct intel_engine_cs *engine)
79ffac8599c4d8 Chris Wilson              2019-04-24  142  {
5cba288466e9b2 Chris Wilson              2019-11-20  143  	struct intel_context *ce = engine->kernel_context;
79ffac8599c4d8 Chris Wilson              2019-04-24  144  	struct i915_request *rq;
6dcb85a0ad9904 Chris Wilson              2019-08-23  145  	bool result = true;
79ffac8599c4d8 Chris Wilson              2019-04-24  146  
3633242927101b Matthew Brost             2021-10-14  147  	/*
3633242927101b Matthew Brost             2021-10-14  148  	 * This is execlist specific behaviour intended to ensure the GPU is
3633242927101b Matthew Brost             2021-10-14  149  	 * idle by switching to a known 'safe' context. With GuC submission, the
3633242927101b Matthew Brost             2021-10-14  150  	 * same idle guarantee is achieved by other means (disabling
3633242927101b Matthew Brost             2021-10-14  151  	 * scheduling). Further, switching to a 'safe' context has no effect
3633242927101b Matthew Brost             2021-10-14  152  	 * with GuC submission as the scheduler can just switch back again.
3633242927101b Matthew Brost             2021-10-14  153  	 *
3633242927101b Matthew Brost             2021-10-14  154  	 * FIXME: Move this backend scheduler specific behaviour into the
3633242927101b Matthew Brost             2021-10-14  155  	 * scheduler backend.
3633242927101b Matthew Brost             2021-10-14  156  	 */
3633242927101b Matthew Brost             2021-10-14  157  	if (intel_engine_uses_guc(engine))
3633242927101b Matthew Brost             2021-10-14  158  		return true;
3633242927101b Matthew Brost             2021-10-14  159  
45b152f752ea81 Chris Wilson              2019-12-29  160  	/* GPU is pointing to the void, as good as in the kernel context. */
45b152f752ea81 Chris Wilson              2019-12-29  161  	if (intel_gt_is_wedged(engine->gt))
45b152f752ea81 Chris Wilson              2019-12-29  162  		return true;
45b152f752ea81 Chris Wilson              2019-12-29  163  
e6ba76480299a0 Chris Wilson              2019-12-21  164  	GEM_BUG_ON(!intel_context_is_barrier(ce));
2a19abb006292d Chris Wilson              2020-07-14  165  	GEM_BUG_ON(ce->timeline->hwsp_ggtt != engine->status_page.vma);
e6ba76480299a0 Chris Wilson              2019-12-21  166  
79ffac8599c4d8 Chris Wilson              2019-04-24  167  	/* Already inside the kernel context, safe to power down. */
79ffac8599c4d8 Chris Wilson              2019-04-24  168  	if (engine->wakeref_serial == engine->serial)
79ffac8599c4d8 Chris Wilson              2019-04-24  169  		return true;
79ffac8599c4d8 Chris Wilson              2019-04-24  170  
79ffac8599c4d8 Chris Wilson              2019-04-24  171  	/*
79ffac8599c4d8 Chris Wilson              2019-04-24  172  	 * Note, we do this without taking the timeline->mutex. We cannot
79ffac8599c4d8 Chris Wilson              2019-04-24  173  	 * as we may be called while retiring the kernel context and so
79ffac8599c4d8 Chris Wilson              2019-04-24  174  	 * already underneath the timeline->mutex. Instead we rely on the
c34c5bca331adc Chris Wilson              2019-05-03  175  	 * exclusive property of the __engine_park that prevents anyone
79ffac8599c4d8 Chris Wilson              2019-04-24  176  	 * else from creating a request on this engine. This also requires
79ffac8599c4d8 Chris Wilson              2019-04-24  177  	 * that the ring is empty and we avoid any waits while constructing
79ffac8599c4d8 Chris Wilson              2019-04-24  178  	 * the context, as they assume protection by the timeline->mutex.
79ffac8599c4d8 Chris Wilson              2019-04-24  179  	 * This should hold true as we can only park the engine after
79ffac8599c4d8 Chris Wilson              2019-04-24  180  	 * retiring the last request, thus all rings should be empty and
79ffac8599c4d8 Chris Wilson              2019-04-24  181  	 * all timelines idle.
5cba288466e9b2 Chris Wilson              2019-11-20  182  	 *
5cba288466e9b2 Chris Wilson              2019-11-20  183  	 * For unlocking, there are 2 other parties and the GPU who have a
5cba288466e9b2 Chris Wilson              2019-11-20  184  	 * stake here.
5cba288466e9b2 Chris Wilson              2019-11-20  185  	 *
5cba288466e9b2 Chris Wilson              2019-11-20  186  	 * A new gpu user will be waiting on the engine-pm to start their
5cba288466e9b2 Chris Wilson              2019-11-20  187  	 * engine_unpark. New waiters are predicated on engine->wakeref.count
5cba288466e9b2 Chris Wilson              2019-11-20  188  	 * and so intel_wakeref_defer_park() acts like a mutex_unlock of the
5cba288466e9b2 Chris Wilson              2019-11-20  189  	 * engine->wakeref.
5cba288466e9b2 Chris Wilson              2019-11-20  190  	 *
5cba288466e9b2 Chris Wilson              2019-11-20  191  	 * The other party is intel_gt_retire_requests(), which is walking the
5cba288466e9b2 Chris Wilson              2019-11-20  192  	 * list of active timelines looking for completions. Meanwhile as soon
5cba288466e9b2 Chris Wilson              2019-11-20  193  	 * as we call __i915_request_queue(), the GPU may complete our request.
5cba288466e9b2 Chris Wilson              2019-11-20  194  	 * Ergo, if we put ourselves on the timelines.active_list
5cba288466e9b2 Chris Wilson              2019-11-20  195  	 * (se intel_timeline_enter()) before we increment the
5cba288466e9b2 Chris Wilson              2019-11-20  196  	 * engine->wakeref.count, we may see the request completion and retire
0d961c4610ff24 Chen Zhou                 2020-04-01  197  	 * it causing an underflow of the engine->wakeref.
79ffac8599c4d8 Chris Wilson              2019-04-24  198  	 */
bce45c2620e214 Sebastian Andrzej Siewior 2021-12-10  199  	set_bit(CONTEXT_IS_PARKING, &ce->flags);
5cba288466e9b2 Chris Wilson              2019-11-20  200  	GEM_BUG_ON(atomic_read(&ce->timeline->active_count) < 0);
6c69a45445af92 Chris Wilson              2019-08-16  201  
5cba288466e9b2 Chris Wilson              2019-11-20  202  	rq = __i915_request_create(ce, GFP_NOWAIT);
79ffac8599c4d8 Chris Wilson              2019-04-24  203  	if (IS_ERR(rq))
79ffac8599c4d8 Chris Wilson              2019-04-24  204  		/* Context switch failed, hope for the best! Maybe reset? */
6dcb85a0ad9904 Chris Wilson              2019-08-23  205  		goto out_unlock;
79ffac8599c4d8 Chris Wilson              2019-04-24  206  
79ffac8599c4d8 Chris Wilson              2019-04-24  207  	/* Check again on the next retirement. */
79ffac8599c4d8 Chris Wilson              2019-04-24  208  	engine->wakeref_serial = engine->serial + 1;
d8af05ff38ae7a Chris Wilson              2019-08-02  209  	i915_request_add_active_barriers(rq);
a79ca656b648e6 Chris Wilson              2019-08-13  210  
a79ca656b648e6 Chris Wilson              2019-08-13  211  	/* Install ourselves as a preemption barrier */
b5e8e954eb672e Chris Wilson              2019-10-21  212  	rq->sched.attr.priority = I915_PRIORITY_BARRIER;
b81e4d9b594188 Chris Wilson              2019-12-19  213  	if (likely(!__i915_request_commit(rq))) { /* engine should be idle! */
b81e4d9b594188 Chris Wilson              2019-12-19  214  		/*
b81e4d9b594188 Chris Wilson              2019-12-19  215  		 * Use an interrupt for precise measurement of duration,
b81e4d9b594188 Chris Wilson              2019-12-19  216  		 * otherwise we rely on someone else retiring all the requests
b81e4d9b594188 Chris Wilson              2019-12-19  217  		 * which may delay the signaling (i.e. we will likely wait
b81e4d9b594188 Chris Wilson              2019-12-19  218  		 * until the background request retirement running every
b81e4d9b594188 Chris Wilson              2019-12-19  219  		 * second or two).
b81e4d9b594188 Chris Wilson              2019-12-19  220  		 */
b81e4d9b594188 Chris Wilson              2019-12-19 @221  		BUILD_BUG_ON(sizeof(rq->duration) > sizeof(rq->submitq));
b81e4d9b594188 Chris Wilson              2019-12-19  222  		dma_fence_add_callback(&rq->fence, &rq->duration.cb, duration);
b81e4d9b594188 Chris Wilson              2019-12-19  223  		rq->duration.emitted = ktime_get();
b81e4d9b594188 Chris Wilson              2019-12-19  224  	}
79ffac8599c4d8 Chris Wilson              2019-04-24  225  
88a4655e75ac8f Chris Wilson              2019-11-25  226  	/* Expose ourselves to the world */
88a4655e75ac8f Chris Wilson              2019-11-25  227  	__queue_and_release_pm(rq, ce->timeline, engine);
5cba288466e9b2 Chris Wilson              2019-11-20  228  
6dcb85a0ad9904 Chris Wilson              2019-08-23  229  	result = false;
6dcb85a0ad9904 Chris Wilson              2019-08-23  230  out_unlock:
bce45c2620e214 Sebastian Andrzej Siewior 2021-12-10  231  	clear_bit(CONTEXT_IS_PARKING, &ce->flags);
6dcb85a0ad9904 Chris Wilson              2019-08-23  232  	return result;
79ffac8599c4d8 Chris Wilson              2019-04-24  233  }
79ffac8599c4d8 Chris Wilson              2019-04-24  234  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
#3 lore raw

Re: [RFC 1/3] dma-fence: Track explicit waiters

In reply to Tvrtko Ursulin
reproduce (this is a W=1 build): # https://github.com/intel-lab-lkp/linux/commit/871de6ff345b69a34f0cfe807395b6074b5a29ea git remote add linux-review https://github.com/intel-lab-lkp/linux git fetch…

Hi Tvrtko,

[FYI, it's a private test report for your RFC patch.] [auto build test ERROR on drm-tip/drm-tip]

url:    https://github.com/intel-lab-lkp/linux/commits/Tvrtko-Ursulin/dma-fence-Track-explicit-waiters/20230209-234523
base:   git://anongit.freedesktop.org/drm/drm-tip drm-tip
patch link:    https://lore.kernel.org/r/20230209154315.468682-2-tvrtko.ursulin%40linux.intel.com
patch subject: [RFC 1/3] dma-fence: Track explicit waiters
config: i386-defconfig (https://download.01.org/0day-ci/archive/20230210/[email protected]/config)
compiler: gcc-11 (Debian 11.3.0-8) 11.3.0

reproduce (this is a W=1 build):
# https://github.com/intel-lab-lkp/linux/commit/871de6ff345b69a34f0cfe807395b6074b5a29ea
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Tvrtko-Ursulin/dma-fence-Track-explicit-waiters/20230209-234523 git checkout 871de6ff345b69a34f0cfe807395b6074b5a29ea
# save the config file
mkdir build_dir && cp config build_dir/.config
make W=1 O=build_dir ARCH=i386 olddefconfig
make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from <command-line>:
In function 'switch_to_kernel_context',

       inlined from '__engine_park' at drivers/gpu/drm/i915/gt/intel_engine_pm.c:262:7:
>> include/linux/compiler_types.h:358:45: error: call to '__compiletime_assert_547' declared with attribute error: BUILD_BUG_ON failed: sizeof(rq->duration) > sizeof(rq->submitq)
     358 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |                                             ^
   include/linux/compiler_types.h:339:25: note: in definition of macro '__compiletime_assert'
     339 |                         prefix ## suffix();                             \
         |                         ^~~~~~
   include/linux/compiler_types.h:358:9: note: in expansion of macro '_compiletime_assert'
     358 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
      39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
         |                                     ^~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:50:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
      50 |         BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
         |         ^~~~~~~~~~~~~~~~
   drivers/gpu/drm/i915/gt/intel_engine_pm.c:221:17: note: in expansion of macro 'BUILD_BUG_ON'
     221 |                 BUILD_BUG_ON(sizeof(rq->duration) > sizeof(rq->submitq));
         |                 ^~~~~~~~~~~~

vim +/__compiletime_assert_547 +358 include/linux/compiler_types.h

eb5c2d4b45e3d2 Will Deacon 2020-07-21  344  
eb5c2d4b45e3d2 Will Deacon 2020-07-21  345  #define _compiletime_assert(condition, msg, prefix, suffix) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21  346  	__compiletime_assert(condition, msg, prefix, suffix)
eb5c2d4b45e3d2 Will Deacon 2020-07-21  347  
eb5c2d4b45e3d2 Will Deacon 2020-07-21  348  /**
eb5c2d4b45e3d2 Will Deacon 2020-07-21  349   * compiletime_assert - break build and emit msg if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21  350   * @condition: a compile-time constant condition to check
eb5c2d4b45e3d2 Will Deacon 2020-07-21  351   * @msg:       a message to emit if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21  352   *
eb5c2d4b45e3d2 Will Deacon 2020-07-21  353   * In tradition of POSIX assert, this macro will break the build if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21  354   * supplied condition is *false*, emitting the supplied error message if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21  355   * compiler has support to do so.
eb5c2d4b45e3d2 Will Deacon 2020-07-21  356   */
eb5c2d4b45e3d2 Will Deacon 2020-07-21  357  #define compiletime_assert(condition, msg) \

eb5c2d4b45e3d2 Will Deacon 2020-07-21 @358 _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
eb5c2d4b45e3d2 Will Deacon 2020-07-21 359

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
#4 lore raw

RFC 2/3 drm/syncobj: Mark syncobj waits as external waiters

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Use the previously added dma-fence tracking of explicit waiters.

---
 drivers/gpu/drm/drm_syncobj.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/drm_syncobj.c +3 -3
index 0c2be8360525..776b90774a64 100644
@@ -1065,9 +1065,9 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs,
 			if ((flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_AVAILABLE) ||
 			    dma_fence_is_signaled(fence) ||
 			    (!entries[i].fence_cb.func &&
-			     dma_fence_add_callback(fence,
-						    &entries[i].fence_cb,
-						    syncobj_wait_fence_func))) {
+			     dma_fence_add_wait_callback(fence,
+							 &entries[i].fence_cb,
+							 syncobj_wait_fence_func))) {
 				/* The fence has been signaled */
 				if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_ALL) {
 					signaled_count++;
Signature · 1 line
2.34.1
#5 lore raw

RFC 3/3 drm/i915: Waitboost external waits

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Userspace waits coming via the drm_syncobj route have so far been bypassing the waitboost mechanism.

Use the previously added dma-fence wait tracking API and apply the same waitboosting logic which applies to other entry points.

This should fix the perfomance regressions experience by clvk and similar userspace which relies on drm_syncobj.

At the same time, but for documentation purposes only, use the new dma-fence API from i915_request_wait too.

---
 drivers/gpu/drm/i915/i915_request.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/i915_request.c +10 -3
index 7503dcb9043b..e24fac5c1567 100644
@@ -94,7 +94,12 @@ static bool i915_fence_signaled(struct dma_fence *fence)
 
 static bool i915_fence_enable_signaling(struct dma_fence *fence)
 {
-	return i915_request_enable_breadcrumb(to_request(fence));
+	struct i915_request *rq = to_request(fence);
+
+	if (dma_fence_wait_count(&rq->fence) && !i915_request_started(rq))
+		intel_rps_boost(rq);
+
+	return i915_request_enable_breadcrumb(rq);
 }
 
 static signed long i915_fence_wait(struct dma_fence *fence,
@@ -2037,11 +2042,13 @@ long i915_request_wait_timeout(struct i915_request *rq,
 	 * but at a cost of spending more power processing the workload
 	 * (bad for battery).
 	 */
-	if (flags & I915_WAIT_PRIORITY && !i915_request_started(rq))
+	if (((flags & I915_WAIT_PRIORITY) || dma_fence_wait_count(&rq->fence))
+	    && !i915_request_started(rq))
 		intel_rps_boost(rq);
 
 	wait.tsk = current;
-	if (dma_fence_add_callback(&rq->fence, &wait.cb, request_wait_wake))
+	if (dma_fence_add_wait_callback(&rq->fence, &wait.cb,
+					request_wait_wake))
 		goto out;
 
 	/*
Signature · 1 line
2.34.1
#7 lore raw

[Intel-gfx] ✗ Fi.CI.BAT: failure for Waitboost drm syncobj waits

In reply to Tvrtko Ursulin
== Series Details ==

== Series Details ==

Series: Waitboost drm syncobj waits
URL : https://patchwork.freedesktop.org/series/113846/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_12721 -> Patchwork_113846v1
====================================================

Summary
-------

**FAILURE**

Serious unknown changes coming with Patchwork_113846v1 absolutely need to be verified manually.

If you think the reported changes have nothing to do with the changes introduced in Patchwork_113846v1, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/index.html

Participating hosts (37 -> 37)
------------------------------

Additional (2): bat-atsm-1 bat-dg1-6
Missing (2): fi-kbl-soraka fi-snb-2520m

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in Patchwork_113846v1:

### CI changes ###

#### Suppressed ####

The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.

* boot:
- {bat-kbl-2}: [PASS][1] -> [FAIL][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-kbl-2/boot.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-kbl-2/boot.html

### IGT changes ###

#### Possible regressions ####

* igt@i915_module_load@load:
- fi-ilk-650: [PASS][3] -> [ABORT][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-ilk-650/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-ilk-650/igt@[email protected]
- fi-blb-e6850: [PASS][5] -> [ABORT][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-blb-e6850/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-blb-e6850/igt@[email protected]
- fi-bsw-n3050: [PASS][7] -> [ABORT][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-bsw-n3050/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-bsw-n3050/igt@[email protected]
- fi-rkl-11600: [PASS][9] -> [ABORT][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-rkl-11600/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-rkl-11600/igt@[email protected]
- fi-skl-6600u: [PASS][11] -> [ABORT][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-skl-6600u/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-skl-6600u/igt@[email protected]
- fi-apl-guc: [PASS][13] -> [ABORT][14]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-apl-guc/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-apl-guc/igt@[email protected]
- bat-dg1-5: [PASS][15] -> [ABORT][16]
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-dg1-5/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-dg1-5/igt@[email protected]
- fi-pnv-d510: [PASS][17] -> [ABORT][18]
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-pnv-d510/igt@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-pnv-d510/igt@[email protected]
- fi-glk-j4005: [PASS][19] -> [ABORT][20]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-glk-j4005/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-glk-j4005/igt@[email protected]
- fi-skl-guc: [PASS][21] -> [ABORT][22]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-skl-guc/igt@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-skl-guc/igt@[email protected]
- bat-dg1-6: NOTRUN -> [ABORT][23]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-dg1-6/igt@[email protected]
- fi-kbl-7567u: [PASS][24] -> [ABORT][25]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-kbl-7567u/igt@[email protected]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-kbl-7567u/igt@[email protected]
- fi-cfl-8700k: [PASS][26] -> [ABORT][27]
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-cfl-8700k/igt@[email protected]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-cfl-8700k/igt@[email protected]
- fi-elk-e7500: [PASS][28] -> [ABORT][29]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-elk-e7500/igt@[email protected]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-elk-e7500/igt@[email protected]
- fi-bsw-nick: [PASS][30] -> [ABORT][31]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-bsw-nick/igt@[email protected]
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-bsw-nick/igt@[email protected]
- fi-cfl-guc: [PASS][32] -> [ABORT][33]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-cfl-guc/igt@[email protected]
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-cfl-guc/igt@[email protected]
- fi-kbl-x1275: [PASS][34] -> [ABORT][35]
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-kbl-x1275/igt@[email protected]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-kbl-x1275/igt@[email protected]
- fi-hsw-4770: [PASS][36] -> [ABORT][37]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-hsw-4770/igt@[email protected]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-hsw-4770/igt@[email protected]
- fi-cfl-8109u: [PASS][38] -> [ABORT][39]
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-cfl-8109u/igt@[email protected]
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-cfl-8109u/igt@[email protected]
- fi-ivb-3770: [PASS][40] -> [ABORT][41]
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-ivb-3770/igt@[email protected]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-ivb-3770/igt@[email protected]
- fi-kbl-guc: [PASS][42] -> [ABORT][43]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/fi-kbl-guc/igt@[email protected]
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/fi-kbl-guc/igt@[email protected]

#### Suppressed ####

The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.

* igt@core_auth@basic-auth:
- {bat-rpls-2}: [PASS][44] -> [ABORT][45]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-rpls-2/igt@[email protected]
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-rpls-2/igt@[email protected]

* igt@i915_module_load@load:
- {bat-jsl-1}: [PASS][46] -> [ABORT][47]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-jsl-1/igt@[email protected]
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-jsl-1/igt@[email protected]
- {bat-rpls-1}: [PASS][48] -> [ABORT][49]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-rpls-1/igt@[email protected]
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-rpls-1/igt@[email protected]
- {bat-adlp-6}: [PASS][50] -> [ABORT][51]
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-adlp-6/igt@[email protected]
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-adlp-6/igt@[email protected]
- {bat-adls-5}: [PASS][52] -> [ABORT][53]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-adls-5/igt@[email protected]
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-adls-5/igt@[email protected]
- {bat-dg1-7}: [PASS][54] -> [ABORT][55]
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-dg1-7/igt@[email protected]
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-dg1-7/igt@[email protected]
- {bat-jsl-3}: [PASS][56] -> [ABORT][57]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-jsl-3/igt@[email protected]
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-jsl-3/igt@[email protected]
- {bat-adlp-9}: [PASS][58] -> [ABORT][59]
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-adlp-9/igt@[email protected]
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-adlp-9/igt@[email protected]
- {bat-dg2-11}: [PASS][60] -> [ABORT][61]
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-dg2-11/igt@[email protected]
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-dg2-11/igt@[email protected]
- {bat-adln-1}: [PASS][62] -> [ABORT][63]
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-adln-1/igt@[email protected]
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-adln-1/igt@[email protected]
- {bat-adlm-1}: [PASS][64] -> [ABORT][65]
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-adlm-1/igt@[email protected]
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-adlm-1/igt@[email protected]
- {bat-rplp-1}: [PASS][66] -> [ABORT][67]
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-rplp-1/igt@[email protected]
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-rplp-1/igt@[email protected]
- {bat-atsm-1}: NOTRUN -> [ABORT][68]
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-atsm-1/igt@[email protected]
- {bat-dg2-9}: [PASS][69] -> [ABORT][70]
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-dg2-9/igt@[email protected]
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-dg2-9/igt@[email protected]
- {bat-dg2-8}: [PASS][71] -> [ABORT][72]
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12721/bat-dg2-8/igt@[email protected]
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/bat-dg2-8/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

Build changes
-------------

* Linux: CI_DRM_12721 -> Patchwork_113846v1

CI-20190529: 20190529
CI_DRM_12721: e1f9df7a92c1e3da1009afa74dc1d08ef95c3d7c @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_7155: 75c508d4e19c65683d4060cb3a772df600aaf23e @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_113846v1: e1f9df7a92c1e3da1009afa74dc1d08ef95c3d7c @ git://anongit.freedesktop.org/gfx-ci/linux

### Linux commits

e9af776738e5 drm/i915: Waitboost external waits b09ea7b88835 drm/syncobj: Mark syncobj waits as external waiters fe94251ffb8b dma-fence: Track explicit waiters

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_113846v1/index.html