Skip to contentSkip to conversation

GPU scheduler for i915 driver

PATCH Superseded [email protected] 7 participants 82 messages Feb 18, 2016 – Mar 15, 2016 Page 1 of 2

Subsystems (4)

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v5 cover GPU scheduler for i915 driver

From: John Harrison <[email protected]>

Implemented a batch buffer submission scheduler for the i915 DRM driver.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code assigns a unique seqno value and then packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

As each batch buffer completes, it raises an interrupt which wakes up the scheduler. Note that it is possible for multiple buffers to complete before the IRQ handler gets to run. Further, the seqno values of the individual buffers are not necessary incrementing as the scheduler may have re-ordered their submission. However, the scheduler keeps the list of executing buffers in order of hardware submission. Thus it can scan through the list until a matching seqno is found and then mark all in flight nodes from that point on as completed.

A deferred work queue is also poked by the interrupt handler. When this wakes up it can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.

The scheduler also allows high priority batch buffers (e.g. from a desktop compositor) to jump ahead of whatever is already running if the underlying hardware supports pre-emption. In this situation, any work that was pre-empted is returned to the queued list ready to be resubmitted when no more high priority work is outstanding.

Various IGT tests are in progress to test the scheduler's operation and will follow.

v2: Updated for changes in struct fence patch series and other changes to underlying tree (e.g. removal of cliprects). Also changed priority levels to be signed +/-1023 range and reduced mutex lock usage.

v3: More reuse of cached pointers rather than repeated dereferencing (David Gordon).

Moved the dependency generation code out to a seperate function for easier readability. Also added in support for the read-read optimisation.

Major simplification of the DRM file close handler.

Fixed up an overzealous WARN.

Removed unnecessary flushing of the scheduler queue when waiting for a request.

v4: Removed user land fence/sync integration as this is dependent upon the de-staging of the Android sync code. That de-staging is now being done by someone else. The sync support will be added back in to the scheduler in a separate patch series which must wait for the de-staging to be landed first.

Added support for killing batches from contexts that were banned after the batches were submitted to the scheduler.

Changed various comments to fix typos, update to reflect changes to the code, correct formatting and line wrapping, etc. Also wrapped various long lines and twiddled white space to keep the style checker happy.

Changed a bunch of BUG_ONs to WARN_ONs as apparently the latter are preferred.

Used the correct array memory allocator function (kmalloc_array instead of kmalloc).

Fixed a variable type (review comment by Joonas).

Fixed a WARN_ON firing incorrectly when removing killed nodes from the scheduler's queue.

Dropped register definition update patch from this series. The changes are all for pre-emption so it makes more sense for it to be part of that series instead.

v5: Reverted power management changes as they apparently conflict with mutex acquisition. Converted the override mask module parameter to a set of boolean enable flags (just one in this patch set, but others are added later for controlling pre-emption). [Chris Wilson]

Removed lots of whitespace from i915_scheduler.c and re-ordered it to remove all forward declarations. Squashed down the i915_scheduler.c sections of various patches into the initial 'start of scheduler' patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]

Dropped the 'immediate submission' override option. This was a half way house between full scheduler and direct submission and was only really useful during early debug.

Added a re-install of the scheduler's interrupt hook around GPU reset. [Zhipeng Gong]

Used lighter weight spinlocks.

[Patches against drm-intel-nightly tree fetched 19/01/2016 with struct fence conversion patches applied]

Dave Gordon (2):
drm/i915: Cache request pointer in *_submission_final() drm/i915: Add scheduling priority to per-context parameters

John Harrison (33):
drm/i915: Add total count to context status debugfs output drm/i915: Prelude to splitting i915_gem_do_execbuffer in two drm/i915: Split i915_dem_do_execbuffer() in half
drm/i915: Re-instate request->uniq because it is extremely useful drm/i915: Start of GPU scheduler
drm/i915: Prepare retire_requests to handle out-of-order seqnos drm/i915: Disable hardware semaphores when GPU scheduler is enabled drm/i915: Force MMIO flips when scheduler enabled
drm/i915: Added scheduler hook when closing DRM file handles drm/i915: Added scheduler hook into i915_gem_request_notify() drm/i915: Added deferred work handler for scheduler
drm/i915: Redirect execbuffer_final() via scheduler
drm/i915: Keep the reserved space mechanism happy
drm/i915: Added tracking/locking of batch buffer objects drm/i915: Hook scheduler node clean up into retire requests drm/i915: Added scheduler support to __wait_request() calls drm/i915: Added scheduler support to page fault handler drm/i915: Added scheduler flush calls to ring throttle and idle functions drm/i915: Add scheduler hook to GPU reset
drm/i915: Added a module parameter to allow the scheduler to be disabled drm/i915: Support for 'unflushed' ring idle
drm/i915: Defer seqno allocation until actual hardware submission time drm/i915: Added trace points to scheduler
drm/i915: Added scheduler queue throttling by DRM file handle drm/i915: Added debugfs interface to scheduler tuning parameters drm/i915: Added debug state dump facilities to scheduler drm/i915: Add early exit to execbuff_final() if insufficient ring space drm/i915: Added scheduler statistic reporting to debugfs drm/i915: Add scheduler support functions for TDR
drm/i915: Scheduler state dump via debugfs
drm/i915: Enable GPU scheduler by default
drm/i915: Add support for retro-actively banning batch buffers drm/i915: Allow scheduler to manage inter-ring object synchronisation

 drivers/gpu/drm/i915/Makefile              |    1 +
 drivers/gpu/drm/i915/i915_debugfs.c        |  280 ++++-
 drivers/gpu/drm/i915/i915_dma.c            |    6 +
 drivers/gpu/drm/i915/i915_drv.c            |    9 +
 drivers/gpu/drm/i915/i915_drv.h            |   54 +-
 drivers/gpu/drm/i915/i915_gem.c            |  179 ++-
 drivers/gpu/drm/i915/i915_gem_context.c    |   24 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  286 ++++-
 drivers/gpu/drm/i915/i915_params.c         |    4 +
 drivers/gpu/drm/i915/i915_params.h         |    1 +
 drivers/gpu/drm/i915/i915_scheduler.c      | 1663 ++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  159 +++
 drivers/gpu/drm/i915/i915_trace.h          |  228 +++-
 drivers/gpu/drm/i915/intel_display.c       |   10 +-
 drivers/gpu/drm/i915/intel_lrc.c           |  161 ++-
 drivers/gpu/drm/i915/intel_lrc.h           |    1 +
 drivers/gpu/drm/i915/intel_ringbuffer.c    |   49 +-
 drivers/gpu/drm/i915/intel_ringbuffer.h    |    3 +-
 include/uapi/drm/i915_drm.h                |    1 +
 19 files changed, 2976 insertions(+), 143 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH v5 1/35 drm/i915: Add total count to context status debugfs output

From: John Harrison <[email protected]>

When there are lots and lots and even more lots of contexts (e.g. when running with execlists) it is useful to be able to immediately see what the total context count is.

v4: Re-typed a variable (review feedback from Joonas)

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_debugfs.c +4 -1
index b90d6ea..b923949 100644
@@ -1971,7 +1971,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
 	struct drm_i915_private *dev_priv = dev->dev_private;
 	struct intel_engine_cs *ring;
 	struct intel_context *ctx;
-	int ret, i;
+	int ret, i, count = 0;
 
 	ret = mutex_lock_interruptible(&dev->struct_mutex);
 	if (ret)
@@ -1984,6 +1984,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
 
 		seq_puts(m, "HW context ");
 		describe_ctx(m, ctx);
+		count++;
 		for_each_ring(ring, dev_priv, i) {
 			if (ring->default_context == ctx)
 				seq_printf(m, "(default context %s) ",
@@ -2012,6 +2013,8 @@ static int i915_context_status(struct seq_file *m, void *unused)
 		seq_putc(m, '\n');
 	}
 
+	seq_printf(m, "Total: %d contexts\n", count);
+
 	mutex_unlock(&dev->struct_mutex);
 
 	return 0;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

PATCH v5 2/35 drm/i915: Prelude to splitting i915_gem_do_execbuffer in two

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with their submission to the hardware. This basically means splitting the execbuffer() function in half. This change rearranges some code ready for the split to occur.

v5: Dropped runtime PM calls as they conflict with the mutex lock. Instead of being done at the lowest submission level, they are now left right at the top driver entry level. [feedback from Chris Wilson]

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 41 ++++++++++++++++++------------
 drivers/gpu/drm/i915/intel_lrc.c           | 18 +++++++++----
 2 files changed, 38 insertions(+), 21 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +25 -16
index e3e3666..d09130f 100644
@@ -966,10 +966,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return intel_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 static bool
@@ -1237,17 +1234,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	u32 instp_mask;
 	int ret;
 
-	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
-	if (ret)
-		return ret;
-
-	ret = i915_switch_context(params->request);
-	if (ret)
-		return ret;
-
-	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
-	     "%s didn't clear reload\n", ring->name);
-
 	instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
 	instp_mask = I915_EXEC_CONSTANTS_MASK;
 	switch (instp_mode) {
@@ -1281,6 +1267,30 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 		return -EINVAL;
 	}
 
+	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
+	if (ret)
+		return ret;
+
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = intel_ring_invalidate_all_caches(params->request);
+	if (ret)
+		return ret;
+
+	/* Switch to the correct context for the batch */
+	ret = i915_switch_context(params->request);
+	if (ret)
+		return ret;
+
+	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
+	     "%s didn't clear reload\n", ring->name);
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_ring_begin(params->request, 4);
@@ -1317,7 +1327,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;
drivers/gpu/drm/i915/intel_lrc.c +13 -5
index 5657924..53aeb55 100644
@@ -678,10 +678,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return logical_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -957,6 +954,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 	if (ret)
 		return ret;
 
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = logical_ring_invalidate_all_caches(params->request);
+	if (ret)
+		return ret;
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_logical_ring_begin(params->request, 4);
@@ -981,7 +990,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH v5 3/35 drm/i915: Split i915_dem_do_execbuffer() in half

In reply to [email protected]

From: John Harrison <[email protected]>

Split the execbuffer() function in half. The first half collects and validates all the information required to process the batch buffer. It also does all the object pinning, relocations, active list management, etc - basically anything that must be done upfront before the IOCTL returns and allows the user land side to start changing/freeing things. The second half does the actual ring submission.

This change implements the split but leaves the back half being called directly from the end of the front half.

v2: Updated due to changes in underlying tree - addition of sync fence support and removal of cliprects.

v3: Moved local 'ringbuf' variable to make later patches in the series a bit neater.

v4: Corrected a typo in the commit message and downgraded a BUG_ON to a WARN_ON as the latter is preferred. Also removed all the external sync/fence support as that will now be a separate patch series.

v5: Updated for runtime PM changes.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  11 +++
 drivers/gpu/drm/i915/i915_gem.c            |   2 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 103 ++++++++++++++++++++++-------
 drivers/gpu/drm/i915/intel_lrc.c           |  57 +++++++++++-----
 drivers/gpu/drm/i915/intel_lrc.h           |   1 +
 5 files changed, 132 insertions(+), 42 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
index 0584846..8dd811e 100644
@@ -1687,10 +1687,18 @@ struct i915_execbuffer_params {
 	struct drm_device               *dev;
 	struct drm_file                 *file;
 	uint32_t                        dispatch_flags;
+	uint32_t                        args_flags;
 	uint32_t                        args_batch_start_offset;
+	uint32_t                        args_batch_len;
+	uint32_t                        args_num_cliprects;
+	uint32_t                        args_DR1;
+	uint32_t                        args_DR4;
 	uint64_t                        batch_obj_vm_offset;
 	struct intel_engine_cs          *ring;
 	struct drm_i915_gem_object      *batch_obj;
+	struct drm_clip_rect            *cliprects;
+	uint32_t                        instp_mask;
+	int                             instp_mode;
 	struct intel_context            *ctx;
 	struct drm_i915_gem_request     *request;
 };
@@ -1965,6 +1973,7 @@ struct drm_i915_private {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
 				      struct drm_i915_gem_execbuffer2 *args,
 				      struct list_head *vmas);
+		int (*execbuf_final)(struct i915_execbuffer_params *params);
 		int (*init_rings)(struct drm_device *dev);
 		void (*cleanup_ring)(struct intel_engine_cs *ring);
 		void (*stop_ring)(struct intel_engine_cs *ring);
@@ -2835,9 +2844,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
 void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
 					struct drm_i915_gem_request *req);
 void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj);
 int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 				   struct drm_i915_gem_execbuffer2 *args,
 				   struct list_head *vmas);
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params);
 int i915_gem_execbuffer(struct drm_device *dev, void *data,
 			struct drm_file *file_priv);
 int i915_gem_execbuffer2(struct drm_device *dev, void *data,
drivers/gpu/drm/i915/i915_gem.c +2 -0
index 72a37d6..bf39ca4 100644
@@ -5293,11 +5293,13 @@ int i915_gem_init(struct drm_device *dev)
 
 	if (!i915.enable_execlists) {
 		dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
+		dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final;
 		dev_priv->gt.init_rings = i915_gem_init_rings;
 		dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer;
 		dev_priv->gt.stop_ring = intel_stop_ring_buffer;
 	} else {
 		dev_priv->gt.execbuf_submit = intel_execlists_submission;
+		dev_priv->gt.execbuf_final = intel_execlists_submission_final;
 		dev_priv->gt.init_rings = intel_logical_rings_init;
 		dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup;
 		dev_priv->gt.stop_ring = intel_logical_ring_stop;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +78 -25

181 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.c +40 -17

103 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.h +1 -0
index 49af638..d2e387f 100644
@@ -116,6 +116,7 @@ struct i915_execbuffer_params;
 int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas);
+int intel_execlists_submission_final(struct i915_execbuffer_params *params);
 
 void intel_lrc_irq_handler(struct intel_engine_cs *ring);
 void intel_execlists_retire_requests(struct intel_engine_cs *ring);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

PATCH v5 4/35 drm/i915: Cache request pointer in *_submission_final()

In reply to [email protected]

From: Dave Gordon <[email protected]>

Keep a local copy of the request pointer in the _final() functions rather than dereferencing the params block repeatedly.

v3: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++------
 drivers/gpu/drm/i915/intel_lrc.c           | 11 ++++++-----
 2 files changed, 13 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -6
index 3071a2b..7978dae 100644
@@ -1291,6 +1291,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
+	struct drm_i915_gem_request *req = params->request;
 	struct intel_engine_cs  *ring = params->ring;
 	u64 exec_start, exec_len;
 	int ret;
@@ -1302,12 +1303,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = intel_ring_invalidate_all_caches(params->request);
+	ret = intel_ring_invalidate_all_caches(req);
 	if (ret)
 		return ret;
 
 	/* Switch to the correct context for the batch */
-	ret = i915_switch_context(params->request);
+	ret = i915_switch_context(req);
 	if (ret)
 		return ret;
 
@@ -1316,7 +1317,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_ring_begin(params->request, 4);
+		ret = intel_ring_begin(req, 4);
 		if (ret)
 			return ret;
 
@@ -1330,7 +1331,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	}
 
 	if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
-		ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
+		ret = i915_reset_gen7_sol_offsets(params->dev, req);
 		if (ret)
 			return ret;
 	}
@@ -1342,13 +1343,13 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	if (exec_len == 0)
 		exec_len = params->batch_obj->base.size;
 
-	ret = ring->dispatch_execbuffer(params->request,
+	ret = ring->dispatch_execbuffer(req,
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
 		return ret;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
drivers/gpu/drm/i915/intel_lrc.c +6 -5
index 2246a16..12e8949 100644
@@ -973,7 +973,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
-	struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+	struct drm_i915_gem_request *req = params->request;
+	struct intel_ringbuffer *ringbuf = req->ringbuf;
 	struct intel_engine_cs *ring = params->ring;
 	u64 exec_start;
 	int ret;
@@ -985,13 +986,13 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = logical_ring_invalidate_all_caches(params->request);
+	ret = logical_ring_invalidate_all_caches(req);
 	if (ret)
 		return ret;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_logical_ring_begin(params->request, 4);
+		ret = intel_logical_ring_begin(req, 4);
 		if (ret)
 			return ret;
 
@@ -1007,11 +1008,11 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 
-	ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+	ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
 	if (ret)
 		return ret;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

PATCH v5 5/35 drm/i915: Re-instate request->uniq because it is extremely useful

In reply to [email protected]

From: John Harrison <[email protected]>

The seqno value cannot always be used when debugging issues via trace points. This is because it can be reset back to start, especially during TDR type tests. Also, when the scheduler arrives the seqno is only valid while a given request is executing on the hardware. While the request is simply queued waiting for submission, it's seqno value will be zero (meaning invalid).

v4: Wrapped a long line to keep the style checker happy.

v5: Added uniq to the dispatch trace point [Svetlana Kukanova]

For: VIZ-5115

---
 drivers/gpu/drm/i915/i915_drv.h   |  5 +++++
 drivers/gpu/drm/i915/i915_gem.c   |  4 +++-
 drivers/gpu/drm/i915/i915_trace.h | 32 ++++++++++++++++++++++----------
 3 files changed, 30 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +5 -0
index 8dd811e..f4487b9 100644
@@ -1986,6 +1986,8 @@ struct drm_i915_private {
 
 	struct intel_encoder *dig_port_map[I915_MAX_PORTS];
 
+	uint32_t request_uniq;
+
 	/*
 	 * NOTE: This is the dri1/ums dungeon, don't add stuff here. Your patch
 	 * will be rejected. Instead look for a better place.
@@ -2242,6 +2244,9 @@ struct drm_i915_gem_request {
 	  */
 	u32 seqno;
 
+	/* Unique identifier which can be used for trace points & debug */
+	uint32_t uniq;
+
 	/** Position in the ringbuffer of the start of the request */
 	u32 head;
 
drivers/gpu/drm/i915/i915_gem.c +3 -1
index bf39ca4..dfe43ea 100644
@@ -2960,7 +2960,8 @@ static void i915_gem_request_fence_value_str(struct fence *req_fence,
 
 	req = container_of(req_fence, typeof(*req), fence);
 
-	snprintf(str, size, "%d [%d]", req->fence.seqno, req->seqno);
+	snprintf(str, size, "%d [%d:%d]", req->fence.seqno, req->uniq,
+		 req->seqno);
 }
 
 static const struct fence_ops i915_gem_request_fops = {
@@ -3036,6 +3037,7 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
 
 	req->i915 = dev_priv;
 	req->ring = ring;
+	req->uniq = dev_priv->request_uniq++;
 	req->ctx  = ctx;
 	i915_gem_context_reference(req->ctx);
 
drivers/gpu/drm/i915/i915_trace.h +22 -10
index cfe4f03..455c215 100644
@@ -469,6 +469,7 @@ TRACE_EVENT(i915_gem_ring_sync_to,
 			     __field(u32, dev)
 			     __field(u32, sync_from)
 			     __field(u32, sync_to)
+			     __field(u32, uniq_to)
 			     __field(u32, seqno)
 			     ),
 
@@ -476,13 +477,14 @@ TRACE_EVENT(i915_gem_ring_sync_to,
 			   __entry->dev = from->dev->primary->index;
 			   __entry->sync_from = from->id;
 			   __entry->sync_to = to_req->ring->id;
+			   __entry->uniq_to = to_req->uniq;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   ),
 
-	    TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u",
+	    TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u, to_uniq=%u",
 		      __entry->dev,
 		      __entry->sync_from, __entry->sync_to,
-		      __entry->seqno)
+		      __entry->seqno, __entry->uniq_to)
 );
 
 TRACE_EVENT(i915_gem_ring_dispatch,
@@ -492,6 +494,7 @@ TRACE_EVENT(i915_gem_ring_dispatch,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, seqno)
 			     __field(u32, flags)
 			     ),
@@ -501,13 +504,15 @@ TRACE_EVENT(i915_gem_ring_dispatch,
 						i915_gem_request_get_ring(req);
 			   __entry->dev = ring->dev->primary->index;
 			   __entry->ring = ring->id;
+			   __entry->uniq = req->uniq;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   __entry->flags = flags;
 			   i915_trace_irq_get(ring, req);
 			   ),
 
-	    TP_printk("dev=%u, ring=%u, seqno=%u, flags=%x",
-		      __entry->dev, __entry->ring, __entry->seqno, __entry->flags)
+	    TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, flags=%x",
+		      __entry->dev, __entry->ring, __entry->uniq,
+		      __entry->seqno, __entry->flags)
 );
 
 TRACE_EVENT(i915_gem_ring_flush,
@@ -517,6 +522,7 @@ TRACE_EVENT(i915_gem_ring_flush,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, invalidate)
 			     __field(u32, flush)
 			     ),
@@ -524,12 +530,13 @@ TRACE_EVENT(i915_gem_ring_flush,
 	    TP_fast_assign(
 			   __entry->dev = req->ring->dev->primary->index;
 			   __entry->ring = req->ring->id;
+			   __entry->uniq = req->uniq;
 			   __entry->invalidate = invalidate;
 			   __entry->flush = flush;
 			   ),
 
-	    TP_printk("dev=%u, ring=%x, invalidate=%04x, flush=%04x",
-		      __entry->dev, __entry->ring,
+	    TP_printk("dev=%u, ring=%x, request=%u, invalidate=%04x, flush=%04x",
+		      __entry->dev, __entry->ring, __entry->uniq,
 		      __entry->invalidate, __entry->flush)
 );
 
@@ -540,6 +547,7 @@ DECLARE_EVENT_CLASS(i915_gem_request,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, seqno)
 			     ),
 
@@ -548,11 +556,13 @@ DECLARE_EVENT_CLASS(i915_gem_request,
 						i915_gem_request_get_ring(req);
 			   __entry->dev = ring->dev->primary->index;
 			   __entry->ring = ring->id;
+			   __entry->uniq = req ? req->uniq : 0;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   ),
 
-	    TP_printk("dev=%u, ring=%u, seqno=%u",
-		      __entry->dev, __entry->ring, __entry->seqno)
+	    TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u",
+		      __entry->dev, __entry->ring, __entry->uniq,
+		      __entry->seqno)
 );
 
 DEFINE_EVENT(i915_gem_request, i915_gem_request_add,
@@ -601,6 +611,7 @@ TRACE_EVENT(i915_gem_request_wait_begin,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, seqno)
 			     __field(bool, blocking)
 			     ),
@@ -616,13 +627,14 @@ TRACE_EVENT(i915_gem_request_wait_begin,
 						i915_gem_request_get_ring(req);
 			   __entry->dev = ring->dev->primary->index;
 			   __entry->ring = ring->id;
+			   __entry->uniq = req ? req->uniq : 0;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   __entry->blocking =
 				     mutex_is_locked(&ring->dev->struct_mutex);
 			   ),
 
-	    TP_printk("dev=%u, ring=%u, seqno=%u, blocking=%s",
-		      __entry->dev, __entry->ring,
+	    TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, blocking=%s",
+		      __entry->dev, __entry->ring, __entry->uniq,
 		      __entry->seqno, __entry->blocking ?  "yes (NB)" : "no")
 );
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#6 lore raw

PATCH v5 6/35 drm/i915: Start of GPU scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

The scheduler is notified when each batch buffer completes and updates its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed, the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate. All nodes that are not submitted have their priority bumped. This ensures that low priority tasks do not get starved out by busy higher priority ones - everything will eventually get its turn to run.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

v4: Updated the commit message to better reflect the current state of the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.

Fixed a WARN_ON when killing nodes. The dependency removal code checks that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.

Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.

v5: Shuffled various functions around to remove forward declarations as apparently these are frowned upon. Removed lots of white space as apparently having easy to read code is also frowned upon. Split the direct submission scheduler bypass code out into a separate function. Squashed down the i915_scheduler.c sections of various patches into this patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]

Added runtime PM calls as these must be done at the top level before acquiring the driver mutex lock. [Chris Wilson]

Removed some obsolete debug code that had been forgotten about.

Moved more clean up code into the 'i915_gem_scheduler_clean_node()' function rather than replicating it in mutliple places.

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/Makefile         |   1 +
 drivers/gpu/drm/i915/i915_drv.h       |   6 +
 drivers/gpu/drm/i915/i915_gem.c       |   5 +
 drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  95 ++++
 5 files changed, 981 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
 i915-y := i915_drv.o \
 	  i915_irq.o \
 	  i915_params.o \
+	  i915_scheduler.o \
           i915_suspend.o \
 	  i915_sysfs.o \
 	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +6 -0
index f4487b9..03add1a 100644
@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
 	struct drm_i915_gem_request     *request;
 };
 
+struct i915_scheduler;
+
 /* used in computing the new watermarks state */
 struct intel_wm_config {
 	unsigned int num_pipes_active;
@@ -1968,6 +1970,8 @@ struct drm_i915_private {
 
 	struct i915_runtime_pm pm;
 
+	struct i915_scheduler *scheduler;
+
 	/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
 	struct {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
 	/** process identifier submitting this request */
 	struct pid *pid;
 
+	struct i915_scheduler_queue_entry	*scheduler_qe;
+
 	/**
 	 * The ELSP only accepts two elements at a time, so we queue
 	 * context/tail pairs on a given queue (ring->execlist_queue) until the
drivers/gpu/drm/i915/i915_gem.c +5 -0
index dfe43ea..7d9aa24 100644
@@ -32,6 +32,7 @@
 #include "i915_vgpu.h"
 #include "i915_trace.h"
 #include "intel_drv.h"
+#include "i915_scheduler.h"
 #include <linux/shmem_fs.h>
 #include <linux/slab.h>
 #include <linux/swap.h>
@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
 	 */
 	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
 
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+
 	ret = i915_gem_init_userptr(dev);
 	if (ret)
 		goto out_unlock;
drivers/gpu/drm/i915/i915_scheduler.c new file +874 -0

874 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h new file +95 -0

95 lines not on this page · raw message

Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#7 lore raw

Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler

Hi,

Now the code is in reviewable chunks, excellent!

I've added my comments below. A few repeats from last round, but now with more questions about the logic itself.

Quoted 164 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

The scheduler is notified when each batch buffer completes and updates its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed, the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate. All nodes that are not submitted have their priority bumped. This ensures that low priority tasks do not get starved out by busy higher priority ones - everything will eventually get its turn to run.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

v4: Updated the commit message to better reflect the current state of the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.

Fixed a WARN_ON when killing nodes. The dependency removal code checks that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.

Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.

v5: Shuffled various functions around to remove forward declarations as apparently these are frowned upon. Removed lots of white space as apparently having easy to read code is also frowned upon. Split the direct submission scheduler bypass code out into a separate function. Squashed down the i915_scheduler.c sections of various patches into this patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]

Added runtime PM calls as these must be done at the top level before acquiring the driver mutex lock. [Chris Wilson]

Removed some obsolete debug code that had been forgotten about.

Moved more clean up code into the 'i915_gem_scheduler_clean_node()' function rather than replicating it in mutliple places.

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>

---
 drivers/gpu/drm/i915/Makefile         |   1 + drivers/gpu/drm/i915/i915_drv.h       |   6 + drivers/gpu/drm/i915/i915_gem.c       |   5 + drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h |  95 ++++

 5 files changed, 981 insertions(+)

 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h

drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
  i915-y := i915_drv.o \
  	  i915_irq.o \
  	  i915_params.o \
+	  i915_scheduler.o \
            i915_suspend.o \
  	  i915_sysfs.o \
  	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +0 -0
index f4487b9..03add1a 100644
@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
  	struct drm_i915_gem_request     *request;
  };

drivers/gpu/drm/i915/i915_drv.h +2 -0
+struct i915_scheduler;
+

 /* used in computing the new watermarks state */
 struct intel_wm_config {
  unsigned int num_pipes_active;

drivers/gpu/drm/i915/i915_drv.h +0 -0
@@ -1968,6 +1970,8 @@ struct drm_i915_private {


  struct i915_runtime_pm pm;

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1974

Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
+	struct i915_scheduler *scheduler;
+

I think we should have i915.enable_scheduler parameter (just like i915.enable_execlists) and make this a data member, not pointer.

Then we can go forward and have i915.enable_scheduler_preempt and so on to control things at boot time.

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_gem_request { · line 2298

Quoted 9 lines

  /* Abstract the submission mechanism (legacy ringbuffer or execlists) away */ struct {
  int (*execbuf_submit)(struct i915_execbuffer_params *params,

drivers/gpu/drm/i915/i915_drv.h +0 -0
@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
  	/** process identifier submitting this request */
  	struct pid *pid;

drivers/gpu/drm/i915/i915_drv.h +2 -0
+	struct i915_scheduler_queue_entry	*scheduler_qe;
+

Funny whitespace.

Quoted 60 lines

  /**
   * The ELSP only accepts two elements at a time, so we queue
   * context/tail pairs on a given queue (ring->execlist_queue) until the

drivers/gpu/drm/i915/i915_gem.c +1 -0
index dfe43ea..7d9aa24 100644
@@ -32,6 +32,7 @@
  #include "i915_vgpu.h"
  #include "i915_trace.h"
  #include "intel_drv.h"
+#include "i915_scheduler.h"
  #include 
  #include 
  #include 
@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
  	 */
  	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);

drivers/gpu/drm/i915/i915_gem.c +4 -0
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+

  ret = i915_gem_init_userptr(dev);
  if (ret)
  goto out_unlock;

drivers/gpu/drm/i915/i915_scheduler.c new file +28 -0
new file mode 100644
index 0000000..fc23ee7
@@ -0,0 +1,874 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+#include "intel_drv.h"
+#include "i915_scheduler.h"
+

drivers/gpu/drm/i915/i915_scheduler.c line 38

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+/**
+ * i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
+ * @dev: DRM device
+ */
+bool i915_scheduler_is_enabled(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+
+	return dev_priv->scheduler != NULL;
+}

i915.enable_scheduler would be used instead.

drivers/gpu/drm/i915/i915_scheduler.c line 50

Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+
+/**
+ * i915_scheduler_init - Initialise the scheduler.
+ * @dev: DRM device
+ * Returns zero on success or -ENOMEM if memory allocations fail.
+ */
+int i915_scheduler_init(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	int r;
+

This protection -->

drivers/gpu/drm/i915/i915_scheduler.c line 57

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	if (scheduler)
+		return 0;
+
+	scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
+	if (!scheduler)
+		return -ENOMEM;
+

Is not needed if we have data member and i915.enable_scheduler.

Quoted 15 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +15 -0
+	spin_lock_init(&scheduler->lock);
+
+	for (r = 0; r < I915_NUM_RINGS; r++)
+		INIT_LIST_HEAD(&scheduler->node_queue[r]);
+
+	/* Default tuning values: */
+	scheduler->priority_level_min     = -1023;
+	scheduler->priority_level_max     = 1023;
+	scheduler->priority_level_bump    = 50;
+	scheduler->priority_level_preempt = 900;
+	scheduler->min_flying             = 2;
+
+	dev_priv->scheduler = scheduler;
+
+	return 0;

drivers/gpu/drm/i915/i915_scheduler.c line 82

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+}
+
+/*
+ * Add a popped node back in to the queue. For example, because the ring was
+ * hung when execfinal() was called and thus the ring submission needs to be
+ * retried later.
+ */
+static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);

Rather remove this completely as it is an internal function, or use BUG_ON because next line will straight lead to OOPS after warning. I'll pass mentioning rest of the obvious WARN_ON vs BUG_ON's.

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Seqno will be reassigned on relaunch */
+	node->params.request->seqno = 0;
+	node->status = i915_sqs_queued;
+}
+
+/*
+ * Give up on a node completely. For example, because it is causing the
+ * ring to hang or is using some resource that no longer exists.

drivers/gpu/drm/i915/i915_scheduler.c line 102

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+ */
+static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);
+	WARN_ON(I915_SQS_IS_COMPLETE(node));
+
+	node->status = i915_sqs_dead;
+}
+
+/* Mark a node as in flight on the hardware. */

As for documentation, from below, I only assume scheduler lock must be held and node's can be only manipulated when scheduler lock is held? It's good to add a comment describing the expected locking. Or maybe add assert_scheduler_lock_held() helper and sprinkle it around the functions similar to assert_rpm_wakelock_held(), which is self- documenting?

drivers/gpu/drm/i915/i915_scheduler.c line 105

Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
+{
+	struct drm_i915_private *dev_priv = node->params.dev->dev_private;

If node is NULL, this would already OOPS, I do not think it's unreasonable to expect node not to be NULL. The below WARN_ON check is never reached.

drivers/gpu/drm/i915/i915_scheduler.c line 114

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring;
+
+	WARN_ON(!scheduler);
+	WARN_ON(!node);
+	WARN_ON(node->status != i915_sqs_popped);
+
+	ring = node->params.ring;
+

This and similar assignments can be squashed to declaration line, as this is basically an alias.

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	/*
+	 * Add the node (which should currently be in state popped) to the
+	 * front of the queue. This ensure that flying nodes are always held
+	 * in hardware submission order.
+	 */
+	list_add(&node->link, &scheduler->node_queue[ring->id]);
+

drivers/gpu/drm/i915/i915_scheduler.c line 131

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	node->status = i915_sqs_flying;
+
+	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+		bool success = true;
+
+		success = ring->irq_get(ring);
+		if (success)
+			scheduler->flags[ring->id] |= i915_sf_interrupts_enabled;
+		else
+			return -EINVAL;

Shouldn't the node be cleaned up from node_queue in case of an error?

Also, I think above could be written much more compact form (because it looks like something where more logic will be added later). It makes it easier to write and visualize the error paths when reading if there are no nested if's.

I won't mention about the error paths of functions below, I expect the following guidline to be adhered;

	if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED)
		goto out;

	if (!ring->irq_get(ring))
		goto err_irq;

	if (!...)
		goto err_foo;

	scheduler->flags[ring->id] |= I915_SF_INT_ENABLED;
out:
	return 0;
err_foo:
	foobar();
err_irq:
	list_remove()
	return -EINVAL;

drivers/gpu/drm/i915/i915_scheduler.c line 143

Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+	}
+
+	return 0;
+}
+
+static uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+					    struct intel_engine_cs *ring)
+{
+	struct i915_scheduler_queue_entry *node;
+	uint32_t flying = 0;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link)

+1 for the for_each_scheduler_node(...)

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +13 -0
+		if (I915_SQS_IS_FLYING(node))
+			flying++;
+
+	return flying;
+}
+
+static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
+{
+	struct i915_scheduler_queue_entry *node;
+	int i;
+
+	/*
+	 * Ensure circular dependencies don't cause problems and that a bump

drivers/gpu/drm/i915/i915_scheduler.c line 166

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	 * by object usage only bumps each using buffer once:
+	 */
+	for (i = 0; i < I915_NUM_RINGS; i++) {
+		list_for_each_entry(node, &scheduler->node_queue[i], link)
+			node->bumped = false;
+	}
+}
+
+static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *target,

Is there specific reason why struct i915_scheduler_queue_entry are not referred to just as "node" but "qe" and here something else, do "node" and "qe" have a special semantic meaning?

drivers/gpu/drm/i915/i915_scheduler.c line 173

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+				uint32_t bump)
+{
+	uint32_t new_priority;
+	int i, count;
+
+	if (target->priority >= scheduler->priority_level_max)
+		return 1;

So if one node reaches maximum priority the dependencies are expected to be maxed out already?

Quoted 110 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +110 -0
+
+	if (target->bumped)
+		return 0;
+
+	new_priority = target->priority + bump;
+	if ((new_priority <= target->priority) ||
+	    (new_priority > scheduler->priority_level_max))
+		target->priority = scheduler->priority_level_max;
+	else
+		target->priority = new_priority;
+
+	count = 1;
+	target->bumped = true;
+
+	for (i = 0; i < target->num_deps; i++) {
+		if (!target->dep_list[i])
+			continue;
+
+		if (target->dep_list[i]->bumped)
+			continue;
+
+		count += i915_scheduler_priority_bump(scheduler,
+						      target->dep_list[i],
+						      bump);
+	}
+
+	return count;
+}
+
+/*
+ * Nodes are considered valid dependencies if they are queued on any ring or
+ * if they are in flight on a different ring. In flight on the same ring is no
+ * longer interesting for non-premptive nodes as the ring serialises execution.
+ * For pre-empting nodes, all in flight dependencies are valid as they must not
+ * be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */
+static inline bool i915_scheduler_is_dependency_valid(
+			struct i915_scheduler_queue_entry *node, uint32_t idx)
+{
+	struct i915_scheduler_queue_entry *dep;
+
+	dep = node->dep_list[idx];
+	if (!dep)
+		return false;
+
+	if (I915_SQS_IS_QUEUED(dep))
+		return true;
+
+	if (I915_SQS_IS_FLYING(dep)) {
+		if (node->params.ring != dep->params.ring)
+			return true;
+	}
+
+	return false;
+}
+
+static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
+				struct i915_scheduler_queue_entry **pop_node)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *best = NULL;
+	struct i915_scheduler_queue_entry *node;
+	int ret;
+	int i;
+	bool any_queued = false;
+	bool has_local, has_remote, only_remote = false;
+
+	*pop_node = NULL;
+	ret = -ENODATA;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+		any_queued = true;
+
+		has_local  = false;
+		has_remote = false;
+		for (i = 0; i < node->num_deps; i++) {
+			if (!i915_scheduler_is_dependency_valid(node, i))
+				continue;
+
+			if (node->dep_list[i]->params.ring == node->params.ring)
+				has_local = true;
+			else
+				has_remote = true;
+		}
+
+		if (has_remote && !has_local)
+			only_remote = true;
+
+		if (!has_local && !has_remote) {
+			if (!best ||
+			    (node->priority > best->priority))
+				best = node;
+		}
+	}
+
+	if (best) {
+		list_del(&best->link);
+
+		INIT_LIST_HEAD(&best->link);
+		best->status  = i915_sqs_popped;
+
+		ret = 0;
+	} else {
+		/* Can only get here if:
+		 * (a) there are no buffers in the queue

drivers/gpu/drm/i915/i915_scheduler.c line 293

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		 * (b) all queued buffers are dependent on other buffers
+		 *     e.g. on a buffer that is in flight on a different ring
+		 */
+		if (only_remote) {
+			/* The only dependent buffers are on another ring. */
+			ret = -EAGAIN;
+		} else if (any_queued) {
+			/* It seems that something has gone horribly wrong! */
+			DRM_ERROR("Broken dependency tracking on ring %d!\n",
+				  (int) ring->id);

Would this condition be worthy WARN_ONCE(), because it will keep repeating as the situation will persist?

Quoted 61 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +61 -0
+		}
+	}
+
+	*pop_node = best;
+	return ret;
+}
+
+/*
+ * NB: The driver mutex lock must be held before calling this function. It is
+ * only really required during the actual back end submission call. However,
+ * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
+ * And releasing the one before acquiring the other leads to other code
+ * being run and interfering.
+ */
+static int i915_scheduler_submit(struct intel_engine_cs *ring)
+{
+	struct drm_device *dev = ring->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	int ret, count = 0, flying;
+
+	WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+
+	spin_lock_irq(&scheduler->lock);
+
+	WARN_ON(scheduler->flags[ring->id] & i915_sf_submitting);
+	scheduler->flags[ring->id] |= i915_sf_submitting;
+
+	/* First time around, complain if anything unexpected occurs: */
+	ret = i915_scheduler_pop_from_queue_locked(ring, &node);
+	if (ret)
+		goto error;
+
+	do {
+		WARN_ON(!node);
+		WARN_ON(node->params.ring != ring);
+		WARN_ON(node->status != i915_sqs_popped);
+		count++;
+
+		/*
+		 * The call to pop above will have removed the node from the
+		 * list. So add it back in and mark it as in flight.
+		 */
+		i915_scheduler_node_fly(node);
+
+		spin_unlock_irq(&scheduler->lock);
+		ret = dev_priv->gt.execbuf_final(&node->params);
+		spin_lock_irq(&scheduler->lock);
+
+		/*
+		 * Handle failed submission but first check that the
+		 * watchdog/reset code has not nuked the node while we
+		 * weren't looking:
+		 */
+		if (ret && (node->status != i915_sqs_dead)) {
+			bool requeue = true;
+
+			/*
+			 * Oh dear! Either the node is broken or the ring is
+			 * busy. So need to kill the node or requeue it and try

drivers/gpu/drm/i915/i915_scheduler.c line 364

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+			 * again later as appropriate.
+			 */
+
+			switch (-ret) {
+			case ENODEV:
+			case ENOENT:
+				/* Fatal errors. Kill the node. */
+				requeue = false;
+				i915_scheduler_node_kill(node);
+			break;

These break indents still.

Quoted 37 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +37 -0
+
+			case EAGAIN:
+			case EBUSY:
+			case EIO:
+			case ENOMEM:
+			case ERESTARTSYS:
+			case EINTR:
+				/* Supposedly recoverable errors. */
+			break;
+
+			default:
+				/*
+				 * Assume the error is recoverable and hope
+				 * for the best.
+				 */
+				MISSING_CASE(-ret);
+			break;
+			}
+
+			if (requeue) {
+				i915_scheduler_node_requeue(node);
+				/*
+				 * No point spinning if the ring is currently
+				 * unavailable so just give up and come back
+				 * later.
+				 */
+				break;
+			}
+		}
+
+		/* Keep launching until the sky is sufficiently full. */
+		flying = i915_scheduler_count_flying(scheduler, ring);
+		if (flying >= scheduler->min_flying)
+			break;
+
+		/* Grab another node and go round again... */
+		ret = i915_scheduler_pop_from_queue_locked(ring, &node);

drivers/gpu/drm/i915/i915_scheduler.c line 411

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	} while (ret == 0);
+
+	/* Don't complain about not being able to submit extra entries */
+	if (ret == -ENODATA)
+		ret = 0;
+
+	/*
+	 * Bump the priority of everything that was not submitted to prevent
+	 * starvation of low priority tasks by a spamming high priority task.
+	 */

This, this calls for an I-G-T test to make sure we're effective.

Quoted 35 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +35 -0
+	i915_scheduler_priority_bump_clear(scheduler);
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+
+		i915_scheduler_priority_bump(scheduler, node,
+					     scheduler->priority_level_bump);
+	}
+
+	/* On success, return the number of buffers submitted. */
+	if (ret == 0)
+		ret = count;
+
+error:
+	scheduler->flags[ring->id] &= ~i915_sf_submitting;
+	spin_unlock_irq(&scheduler->lock);
+	return ret;
+}
+
+static void i915_generate_dependencies(struct i915_scheduler *scheduler,
+				       struct i915_scheduler_queue_entry *node,
+				       uint32_t ring)
+{
+	struct i915_scheduler_obj_entry *this, *that;
+	struct i915_scheduler_queue_entry *test;
+	int i, j;
+	bool found;
+
+	list_for_each_entry(test, &scheduler->node_queue[ring], link) {
+		if (I915_SQS_IS_COMPLETE(test))
+			continue;
+
+		/*
+		 * Batches on the same ring for the same
+		 * context must be kept in order.

drivers/gpu/drm/i915/i915_scheduler.c line 456

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		 */
+		found = (node->params.ctx == test->params.ctx) &&
+			(node->params.ring == test->params.ring);
+
+		/*
+		 * Batches working on the same objects must
+		 * be kept in order.
+		 */
+		for (i = 0; (i < node->num_objs) && !found; i++) {
+			this = node->saved_objects + i;

node->objs and node->num_objs would be a better naming match. num_objs and saved_objects makes me think we have a bug here, indexing wrong array.

Quoted 23 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +23 -0
+
+			for (j = 0; j < test->num_objs; j++) {
+				that = test->saved_objects + j;
+
+				if (this->obj != that->obj)
+					continue;
+
+				/* Only need to worry about writes */
+				if (this->read_only && that->read_only)
+					continue;
+
+				found = true;
+				break;
+			}
+		}
+
+		if (found) {
+			node->dep_list[node->num_deps] = test;
+			node->num_deps++;
+		}
+	}
+}
+

drivers/gpu/drm/i915/i915_scheduler.c line 489

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_entry *qe)
+{
+	struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	int ret;
+
+	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
+	ret = dev_priv->gt.execbuf_final(&qe->params);
+	scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
+

The above code makes me think of locking again, maybe document at top of this function.

Quoted 32 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +32 -0
+	/*
+	 * Don't do any clean up on failure because the caller will
+	 * do it all anyway.
+	 */
+	if (ret)
+		return ret;
+
+	/* Need to release any resources held by the node: */
+	i915_scheduler_clean_node(qe);
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
+ * scheduler.
+ * @qe: The batch buffer request to be queued.
+ * The expectation is the qe passed in is a local stack variable. This
+ * function will copy its contents into a freshly allocated list node. The
+ * new node takes ownership of said contents so the original qe should simply
+ * be discarded and not cleaned up (i.e. don't free memory it points to or
+ * dereference objects it holds). The node is added to the scheduler's queue
+ * and the batch buffer will be submitted to the hardware at some future
+ * point in time (which may be immediately, before returning or may be quite
+ * a lot later).
+ */
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
+{
+	struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring = qe->params.ring;
+	struct i915_scheduler_queue_entry *node;

drivers/gpu/drm/i915/i915_scheduler.c line 531

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	struct i915_scheduler_queue_entry *test;
+	bool not_flying;
+	int i, r;
+	int incomplete = 0;
+
+	WARN_ON(!scheduler);
+
+	if (1/*!i915.enable_scheduler*/)
+		return i915_scheduler_queue_execbuffer_bypass(qe);
+

I'm thinking, should we branch already before calling a function named i915_scheduler_queue_execbuffer if scheduler is disabled?

drivers/gpu/drm/i915/i915_scheduler.c line 536

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	node = kmalloc(sizeof(*node), GFP_KERNEL);
+	if (!node)
+		return -ENOMEM;
+
+	*node = *qe;

Now I read the added documentation for the function, maybe we should at least zero out qe to avoid future problems?

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+	INIT_LIST_HEAD(&node->link);
+	node->status = i915_sqs_queued;
+	node->stamp  = jiffies;
+	i915_gem_request_reference(node->params.request);
+
+	WARN_ON(node->params.request->scheduler_qe);
+	node->params.request->scheduler_qe = node;
+

drivers/gpu/drm/i915/i915_scheduler.c line 554

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	/* Need to determine the number of incomplete entries in the list as
+	 * that will be the maximum size of the dependency list.
+	 *
+	 * Note that the allocation must not be made with the spinlock acquired
+	 * as kmalloc can sleep. However, the unlock/relock is safe because no
+	 * new entries can be queued up during the unlock as the i915 driver
+	 * mutex is still held. Entries could be removed from the list but that
+	 * just means the dep_list will be over-allocated which is fine.
+	 */
+	spin_lock_irq(&scheduler->lock);

-->

drivers/gpu/drm/i915/i915_scheduler.c line 563

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(test, &scheduler->node_queue[r], link) {
+			if (I915_SQS_IS_COMPLETE(test))
+				continue;
+
+			incomplete++;
+		}
+	}
+

<-- This counting structure, I still think it would be good idea to make it a static helper.

Quoted 23 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +23 -0
+	/* Temporarily unlock to allocate memory: */
+	spin_unlock_irq(&scheduler->lock);
+	if (incomplete) {
+		node->dep_list = kmalloc_array(incomplete,
+					       sizeof(*node->dep_list),
+					       GFP_KERNEL);
+		if (!node->dep_list) {
+			kfree(node);
+			return -ENOMEM;
+		}
+	} else
+		node->dep_list = NULL;
+
+	spin_lock_irq(&scheduler->lock);
+	node->num_deps = 0;
+
+	if (node->dep_list) {
+		for (r = 0; r < I915_NUM_RINGS; r++)
+			i915_generate_dependencies(scheduler, node, r);
+
+		WARN_ON(node->num_deps > incomplete);
+	}
+

drivers/gpu/drm/i915/i915_scheduler.c line 596

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	node->priority = clamp(node->priority,
+			       scheduler->priority_level_min,
+			       scheduler->priority_level_max);
+
+	if ((node->priority > 0) && node->num_deps) {
+		i915_scheduler_priority_bump_clear(scheduler);
+
+		for (i = 0; i < node->num_deps; i++)
+			i915_scheduler_priority_bump(scheduler,
+					node->dep_list[i], node->priority);

If node is posted with maximum priority to begin with, wouldn't the priority propagation stop after first level dependencies due to the check in the beginning of priority_bump function?

Quoted 122 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +122 -0
+
+	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
+
+	not_flying = i915_scheduler_count_flying(scheduler, ring) <
+						 scheduler->min_flying;
+
+	spin_unlock_irq(&scheduler->lock);
+
+	if (not_flying)
+		i915_scheduler_submit(ring);
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_notify_request - Notify the scheduler that the given
+ * request has completed on the hardware.
+ * @req: Request structure which has completed
+ * @preempt: Did it complete pre-emptively?
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not. The return
+ * value is combined for all freshly completed requests and if any were true
+ * then i915_scheduler_wakeup() is called so the scheduler can do further
+ * processing (submit more work) at the end.
+ */
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
+{
+	struct drm_i915_private *dev_priv = to_i915(req->ring->dev);
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
+	unsigned long flags;
+
+	if (!node)
+		return false;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Node was in flight so mark it as complete. */
+	if (req->cancelled)
+		node->status = i915_sqs_dead;
+	else
+		node->status = i915_sqs_complete;
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return true;
+}
+
+static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *remove)
+{
+	struct i915_scheduler_queue_entry *node;
+	int i, r;
+	int count = 0;
+
+	/*
+	 * Ensure that a node is not being removed which is still dependent
+	 * upon other (not completed) work. If that happens, it implies
+	 * something has gone very wrong with the dependency tracking! Note
+	 * that there is no need to worry if this node has been explicitly
+	 * killed for some reason - it might be being killed before it got
+	 * sent to the hardware.
+	 */
+	if (remove->status != i915_sqs_dead) {
+		for (i = 0; i < remove->num_deps; i++)
+			if ((remove->dep_list[i]) &&
+			    (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
+				count++;
+		WARN_ON(count);
+	}
+
+	/*
+	 * Remove this node from the dependency lists of any other node which
+	 * might be waiting on it.
+	 */
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(node, &scheduler->node_queue[r], link) {
+			for (i = 0; i < node->num_deps; i++) {
+				if (node->dep_list[i] != remove)
+					continue;
+
+				node->dep_list[i] = NULL;
+			}
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_wakeup - wake the scheduler's worker thread
+ * @dev: DRM device
+ * Called at the end of seqno interrupt processing if any request has
+ * completed that corresponds to a scheduler node.
+ */
+void i915_scheduler_wakeup(struct drm_device *dev)
+{
+	/* XXX: Need to call i915_scheduler_remove() via work handler. */
+}
+
+/**
+ * i915_scheduler_clean_node - free up any allocations/references
+ * associated with the given scheduler queue entry.
+ * @node: Queue entry structure which is complete
+ * After a give batch buffer completes on the hardware, all the information
+ * required to resubmit it is no longer required. However, the node entry
+ * itself might still be required for tracking purposes for a while longer.
+ * This function should be called as soon as the node is known to be complete
+ * so that these resources may be freed even though the node itself might
+ * hang around.
+ */
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
+{
+	if (!I915_SQS_IS_COMPLETE(node)) {
+		WARN(!node->params.request->cancelled,
+		     "Cleaning active node: %d!\n", node->status);
+		return;

drivers/gpu/drm/i915/i915_scheduler.c line 729

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	}
+
+	if (node->params.batch_obj) {
+		/*
+		 * The batch buffer must be unpinned before it is unreferenced
+		 * otherwise the unpin fails with a missing vma!?
+		 */
+		if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
+			i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
+

This code seems little bit out of place.

drivers/gpu/drm/i915/i915_scheduler.c line 739

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		node->params.batch_obj = NULL;
+	}
+
+	/* And anything else owned by the node: */
+	if (node->params.cliprects) {
+		kfree(node->params.cliprects);
+		node->params.cliprects = NULL;
+	}
+}
+

The below function is quite hacky, I understood this will be mitigated by per-ctx sequence numbering in the future?

Quoted 70 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +70 -0
+static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
+				  struct intel_engine_cs *ring,
+				  struct list_head *remove)
+{
+	struct i915_scheduler_queue_entry *node, *node_next;
+	int flying = 0, queued = 0;
+	bool do_submit;
+	uint32_t min_seqno;
+
+	spin_lock_irq(&scheduler->lock);
+
+	/*
+	 * In the case where the system is idle, starting 'min_seqno' from a big
+	 * number will cause all nodes to be removed as they are now back to
+	 * being in-order. However, this will be a problem if the last one to
+	 * complete was actually out-of-order as the ring seqno value will be
+	 * lower than one or more completed buffers. Thus code looking for the
+	 * completion of said buffers will wait forever.
+	 * Instead, use the hardware seqno as the starting point. This means
+	 * that some buffers might be kept around even in a completely idle
+	 * system but it should guarantee that no-one ever gets confused when
+	 * waiting for buffer completion.
+	 */
+	min_seqno = ring->get_seqno(ring, true);
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->params.request->seqno == 0)
+			continue;
+
+		if (!i915_seqno_passed(node->params.request->seqno, min_seqno))
+			min_seqno = node->params.request->seqno;
+	}
+
+	INIT_LIST_HEAD(remove);
+	list_for_each_entry_safe(node, node_next, &scheduler->node_queue[ring->id], link) {
+		/*
+		 * Only remove completed nodes which have a lower seqno than
+		 * all pending nodes. While there is the possibility of the
+		 * ring's seqno counting backwards, all higher buffers must
+		 * be remembered so that the 'i915_seqno_passed()' test can
+		 * report that they have in fact passed.
+		 *
+		 * NB: This is not true for 'dead' nodes. The GPU reset causes
+		 * the software seqno to restart from its initial value. Thus
+		 * the dead nodes must be removed even though their seqno values
+		 * are potentially vastly greater than the current ring seqno.
+		 */
+		if (!I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->status != i915_sqs_dead) {
+			if (i915_seqno_passed(node->params.request->seqno, min_seqno) &&
+			    (node->params.request->seqno != min_seqno))
+				continue;
+		}
+
+		list_del(&node->link);
+		list_add(&node->link, remove);
+
+		/* Strip the dependency info while the mutex is still locked */
+		i915_scheduler_remove_dependent(scheduler, node);
+
+		continue;

drivers/gpu/drm/i915/i915_scheduler.c line 819

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	}
+
+	/*
+	 * Release the interrupt reference count if there are no longer any
+	 * nodes to worry about.
+	 */
+	if (!flying && !queued &&
+	    (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+		ring->irq_put(ring);
+		scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;

Maybe have this piece of code as a helper? To enable interrupts and disable them, maybe even a reference count?

Quoted 38 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +38 -0
+	}
+
+	/* Launch more packets now? */
+	do_submit = (queued > 0) && (flying < scheduler->min_flying);
+
+	spin_unlock_irq(&scheduler->lock);
+
+	return do_submit;
+}
+
+void i915_scheduler_process_work(struct intel_engine_cs *ring)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	bool do_submit;
+	struct list_head remove;
+
+	if (list_empty(&scheduler->node_queue[ring->id]))
+		return;
+
+	/* Remove completed nodes. */
+	do_submit = i915_scheduler_remove(scheduler, ring, &remove);
+
+	if (!do_submit && list_empty(&remove))
+		return;
+
+	/* Need to grab the pm lock outside of the mutex lock */
+	if (do_submit)
+		intel_runtime_pm_get(dev_priv);
+
+	mutex_lock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		i915_scheduler_submit(ring);
+
+	while (!list_empty(&remove)) {
+		node = list_first_entry(&remove, typeof(*node), link);

drivers/gpu/drm/i915/i915_scheduler.c line 867

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		list_del(&node->link);
+
+		/* Free up all the DRM references */
+		i915_scheduler_clean_node(node);
+
+		/* And anything else owned by the node: */
+		node->params.request->scheduler_qe = NULL;
+		i915_gem_request_unreference(node->params.request);
+		kfree(node->dep_list);
+		kfree(node);

Maybe have a separate helper for freeing a node, would make this much cleaner.

Quoted 47 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	}
+
+	mutex_unlock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		intel_runtime_pm_put(dev_priv);
+}
drivers/gpu/drm/i915/i915_scheduler.h new file +34 -0
new file mode 100644
index 0000000..415fec8
@@ -0,0 +1,95 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _I915_SCHEDULER_H_
+#define _I915_SCHEDULER_H_
+
+enum i915_scheduler_queue_status {
+	/* Limbo: */
+	i915_sqs_none = 0,
+	/* Not yet submitted to hardware: */
+	i915_sqs_queued,
+	/* Popped from queue, ready to fly: */
+	i915_sqs_popped,

drivers/gpu/drm/i915/i915_scheduler.h line 44

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	/* Sent to hardware for processing: */
+	i915_sqs_flying,
+	/* Finished processing on the hardware: */
+	i915_sqs_complete,
+	/* Killed by watchdog or catastrophic submission failure: */
+	i915_sqs_dead,
+	/* Limit value for use with arrays/loops */
+	i915_sqs_MAX
+};
+

To uppercase.

Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +12 -0
+#define I915_SQS_IS_QUEUED(node)	(((node)->status == i915_sqs_queued))
+#define I915_SQS_IS_FLYING(node)	(((node)->status == i915_sqs_flying))
+#define I915_SQS_IS_COMPLETE(node)	(((node)->status == i915_sqs_complete) || \
+					 ((node)->status == i915_sqs_dead))
+
+struct i915_scheduler_obj_entry {
+	struct drm_i915_gem_object          *obj;
+	bool                                read_only;
+};
+
+struct i915_scheduler_queue_entry {
+	struct i915_execbuffer_params       params;

drivers/gpu/drm/i915/i915_scheduler.h line 66

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	/* -1023 = lowest priority, 0 = default, 1023 = highest */
+	int32_t                             priority;
+	struct i915_scheduler_obj_entry     *saved_objects;
+	int                                 num_objs;
+	bool                                bumped;
+	struct i915_scheduler_queue_entry   **dep_list;
+	int                                 num_deps;
+	enum i915_scheduler_queue_status    status;
+	unsigned long                       stamp;
+	struct list_head                    link;

Above whitespace is still incorrect and I could really use comments.

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+};
+
+struct i915_scheduler {
+	struct list_head    node_queue[I915_NUM_RINGS];
+	uint32_t            flags[I915_NUM_RINGS];
+	spinlock_t          lock;
+
+	/* Tuning parameters: */
+	int32_t             priority_level_min;
+	int32_t             priority_level_max;

drivers/gpu/drm/i915/i915_scheduler.h line 86

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	int32_t             priority_level_bump;
+	int32_t             priority_level_preempt;
+	uint32_t            min_flying;
+};
+
+/* Flag bits for i915_scheduler::flags */
+enum {
+	i915_sf_interrupts_enabled  = (1 << 0),
+	i915_sf_submitting          = (1 << 1),
+};

These must be I915_UPPERCASE_SOMETHING by convention.

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +9 -0
+
+bool i915_scheduler_is_enabled(struct drm_device *dev);
+int i915_scheduler_init(struct drm_device *dev);
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
+void i915_scheduler_wakeup(struct drm_device *dev);
+
+#endif  /* _I915_SCHEDULER_H_ */
Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#8 lore raw

Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler

Quoted 172 lines · Joonas Lahtinen

Hi,

Now the code is in reviewable chunks, excellent!

I've added my comments below. A few repeats from last round, but now with more questions about the logic itself.

Quoted 164 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

The scheduler is notified when each batch buffer completes and updates its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed, the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate. All nodes that are not submitted have their priority bumped. This ensures that low priority tasks do not get starved out by busy higher priority ones - everything will eventually get its turn to run.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

v4: Updated the commit message to better reflect the current state of the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.

Fixed a WARN_ON when killing nodes. The dependency removal code checks that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.

Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.

v5: Shuffled various functions around to remove forward declarations as apparently these are frowned upon. Removed lots of white space as apparently having easy to read code is also frowned upon. Split the direct submission scheduler bypass code out into a separate function. Squashed down the i915_scheduler.c sections of various patches into this patch. Thus the later patches simply hook in existing code into various parts of the driver rather than adding the code as well. Added documentation to various functions. Re-worked the submit function in terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp' macro. [Joonas Lahtinen]

Added runtime PM calls as these must be done at the top level before acquiring the driver mutex lock. [Chris Wilson]

Removed some obsolete debug code that had been forgotten about.

Moved more clean up code into the 'i915_gem_scheduler_clean_node()' function rather than replicating it in mutliple places.

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
  drivers/gpu/drm/i915/Makefile         |   1 +
  drivers/gpu/drm/i915/i915_drv.h       |   6 +
  drivers/gpu/drm/i915/i915_gem.c       |   5 +
  drivers/gpu/drm/i915/i915_scheduler.c | 874 ++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/i915_scheduler.h |  95 ++++
  5 files changed, 981 insertions(+)

create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h

drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
  i915-y := i915_drv.o \
  	  i915_irq.o \
  	  i915_params.o \
+	  i915_scheduler.o \
            i915_suspend.o \
  	  i915_sysfs.o \
  	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +2 -0
index f4487b9..03add1a 100644
@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
  	struct drm_i915_gem_request     *request;
  };
 
+struct i915_scheduler;
+
  /* used in computing the new watermarks state */
  struct intel_wm_config {
  	unsigned int num_pipes_active;
@@ -1968,6 +1970,8 @@ struct drm_i915_private {
 
  	struct i915_runtime_pm pm;
 
Quoted 7 lines

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1974

Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
+	struct i915_scheduler *scheduler;
+

I think we should have i915.enable_scheduler parameter (just like i915.enable_execlists) and make this a data member, not pointer.

Then we can go forward and have i915.enable_scheduler_preempt and so on to control things at boot time.

There are. The enable_scheduler comes in as patch #21. Including it from the start means modifying i915_params.[ch] in this patch as well. And then you are getting into one big patch that contains everything and is unreviewable due to its sheer size. The enable_preemption parameter comes in with the pre-emption code. No point in adding that until there is something for it to control.

Quoted 75 lines

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_gem_request { · line 2298

Quoted 10 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
  	/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
  	struct {
  		int (*execbuf_submit)(struct i915_execbuffer_params *params,
@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
  	/** process identifier submitting this request */
  	struct pid *pid;
 
+	struct i915_scheduler_queue_entry	*scheduler_qe;
+

Funny whitespace.

Quoted 63 lines
drivers/gpu/drm/i915/i915_drv.h +0 -0
  	/**
  	 * The ELSP only accepts two elements at a time, so we queue
  	 * context/tail pairs on a given queue (ring->execlist_queue) until the
drivers/gpu/drm/i915/i915_gem.c +5 -0
index dfe43ea..7d9aa24 100644
@@ -32,6 +32,7 @@
  #include "i915_vgpu.h"
  #include "i915_trace.h"
  #include "intel_drv.h"
+#include "i915_scheduler.h"
  #include
  #include
  #include
@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
  	 */
  	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
 
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+
  	ret = i915_gem_init_userptr(dev);
  	if (ret)
  		goto out_unlock;
drivers/gpu/drm/i915/i915_scheduler.c new file +31 -0
new file mode 100644
index 0000000..fc23ee7
@@ -0,0 +1,874 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+#include "intel_drv.h"
+#include "i915_scheduler.h"
+
+/**
+ * i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
+ * @dev: DRM device
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 38

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+ */
+bool i915_scheduler_is_enabled(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+
+	return dev_priv->scheduler != NULL;
+}

i915.enable_scheduler would be used instead.

It is when it arrives.

Quoted 15 lines

drivers/gpu/drm/i915/i915_scheduler.c line 50

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+
+/**
+ * i915_scheduler_init - Initialise the scheduler.
+ * @dev: DRM device
+ * Returns zero on success or -ENOMEM if memory allocations fail.
+ */
+int i915_scheduler_init(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	int r;
+

This protection -->

Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 57

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	if (scheduler)
+		return 0;
+
+	scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
+	if (!scheduler)
+		return -ENOMEM;
+

Is not needed if we have data member and i915.enable_scheduler.

Making the scheduler structure a object rather than a pointer basically means moving the whole of i915_scheduler.h into i915_drv.h. There is too much interconnectedness between the various structures. I would say that keeping the code nicely partitioned and readable is worth the one dynamic allocation at driver load time.

Quoted 21 lines
Quoted 21 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +20 -0
+	spin_lock_init(&scheduler->lock);
+
+	for (r = 0; r < I915_NUM_RINGS; r++)
+		INIT_LIST_HEAD(&scheduler->node_queue[r]);
+
+	/* Default tuning values: */
+	scheduler->priority_level_min     = -1023;
+	scheduler->priority_level_max     = 1023;
+	scheduler->priority_level_bump    = 50;
+	scheduler->priority_level_preempt = 900;
+	scheduler->min_flying             = 2;
+
+	dev_priv->scheduler = scheduler;
+
+	return 0;
+}
+
+/*
+ * Add a popped node back in to the queue. For example, because the ring was
+ * hung when execfinal() was called and thus the ring submission needs to be
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 82

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+ * retried later.
+ */
+static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);

Rather remove this completely as it is an internal function, or use BUG_ON because next line will straight lead to OOPS after warning. I'll pass mentioning rest of the obvious WARN_ON vs BUG_ON's.

Yeah, most of the WARN/BUG_ONs are probably unnecessary now. They were scattered liberally around during initial development when strange and unexpected things could happen. Now the the code paths are pretty well established they aren't really much use. I'll reduce them down to just the ones that still make sense.

Quoted 57 lines
Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Seqno will be reassigned on relaunch */
+	node->params.request->seqno = 0;
+	node->status = i915_sqs_queued;
+}
+
+/*
+ * Give up on a node completely. For example, because it is causing the
+ * ring to hang or is using some resource that no longer exists.

drivers/gpu/drm/i915/i915_scheduler.c line 102

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+ */
+static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);
+	WARN_ON(I915_SQS_IS_COMPLETE(node));
+
+	node->status = i915_sqs_dead;
+}
+
+/* Mark a node as in flight on the hardware. */

As for documentation, from below, I only assume scheduler lock must be held and node's can be only manipulated when scheduler lock is held? It's good to add a comment describing the expected locking. Or maybe add assert_scheduler_lock_held() helper and sprinkle it around the functions similar to assert_rpm_wakelock_held(), which is self- documenting?

drivers/gpu/drm/i915/i915_scheduler.c line 105

Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
+{
+	struct drm_i915_private *dev_priv = node->params.dev->dev_private;

If node is NULL, this would already OOPS, I do not think it's unreasonable to expect node not to be NULL. The below WARN_ON check is never reached.

drivers/gpu/drm/i915/i915_scheduler.c line 114

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring;
+
+	WARN_ON(!scheduler);
+	WARN_ON(!node);
+	WARN_ON(node->status != i915_sqs_popped);
+
+	ring = node->params.ring;
+

This and similar assignments can be squashed to declaration line, as this is basically an alias.

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	/*
+	 * Add the node (which should currently be in state popped) to the
+	 * front of the queue. This ensure that flying nodes are always held
+	 * in hardware submission order.
+	 */
+	list_add(&node->link, &scheduler->node_queue[ring->id]);
+
+	node->status = i915_sqs_flying;
+
+	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 131

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+		bool success = true;
+
+		success = ring->irq_get(ring);
+		if (success)
+			scheduler->flags[ring->id] |= i915_sf_interrupts_enabled;
+		else
+			return -EINVAL;

Shouldn't the node be cleaned up from node_queue in case of an error?

Not really. The batch buffer will still be executed regardless of whether interrupts could be enabled or not. The return code is actually ignored so this should really be a void function, I guess. Without interrupts then completion might take a while to be noticed but stuff should still basically work.

Quoted 59 lines

Also, I think above could be written much more compact form (because it looks like something where more logic will be added later). It makes it easier to write and visualize the error paths when reading if there are no nested if's.

I won't mention about the error paths of functions below, I expect the following guidline to be adhered;

	if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED)
		goto out;

	if (!ring->irq_get(ring))
		goto err_irq;

	if (!...)
		goto err_foo;

	scheduler->flags[ring->id] |= I915_SF_INT_ENABLED;
out:
	return 0;
err_foo:
	foobar();
err_irq:
	list_remove()
	return -EINVAL;

drivers/gpu/drm/i915/i915_scheduler.c line 143

Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+	}
+
+	return 0;
+}
+
+static uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+					    struct intel_engine_cs *ring)
+{
+	struct i915_scheduler_queue_entry *node;
+	uint32_t flying = 0;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link)

+1 for the for_each_scheduler_node(...)

Quoted 18 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +18 -0
+		if (I915_SQS_IS_FLYING(node))
+			flying++;
+
+	return flying;
+}
+
+static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
+{
+	struct i915_scheduler_queue_entry *node;
+	int i;
+
+	/*
+	 * Ensure circular dependencies don't cause problems and that a bump
+	 * by object usage only bumps each using buffer once:
+	 */
+	for (i = 0; i < I915_NUM_RINGS; i++) {
+		list_for_each_entry(node, &scheduler->node_queue[i], link)
+			node->bumped = false;
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 166

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	}
+}
+
+static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *target,

Is there specific reason why struct i915_scheduler_queue_entry are not referred to just as "node" but "qe" and here something else, do "node" and "qe" have a special semantic meaning?

Not sure what you mean. The use of 'node' is because almost everything an i915_scheduler_queue_entry object is being used it is as a node in a list that is being iterated over. The only 'qe' variable is in 'i915_scheduler_queue_execbuffer[_bypass]' and there it is because the qe is being passed in from outside and is copied to a local 'node' that is then added to the list. I think the use of 'target' here is largely historical. The subsequent array search was originally a list search and hence had a 'node' that was used as the iterator. Thus the qe passed in was called something else to avoid conflicts and target seemed appropriate.

Quoted 9 lines

drivers/gpu/drm/i915/i915_scheduler.c line 173

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+				uint32_t bump)
+{
+	uint32_t new_priority;
+	int i, count;
+
+	if (target->priority >= scheduler->priority_level_max)
+		return 1;

So if one node reaches maximum priority the dependencies are expected to be maxed out already?

Yes. Dependencies should always be of equal or higher priority to the dependee. When a new dependency is created, the priority of the dependee is added on to ensure that something of high priority can never be stuck waiting on something of low priority. Also, it means that a batch buffer that lots of other buffers are waiting on will be bumped to higher and higher priority the more waiters there are and thus is more likely to be executed sooner and free them all up.

Quoted 236 lines
Quoted 110 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +110 -0
+
+	if (target->bumped)
+		return 0;
+
+	new_priority = target->priority + bump;
+	if ((new_priority <= target->priority) ||
+	    (new_priority > scheduler->priority_level_max))
+		target->priority = scheduler->priority_level_max;
+	else
+		target->priority = new_priority;
+
+	count = 1;
+	target->bumped = true;
+
+	for (i = 0; i < target->num_deps; i++) {
+		if (!target->dep_list[i])
+			continue;
+
+		if (target->dep_list[i]->bumped)
+			continue;
+
+		count += i915_scheduler_priority_bump(scheduler,
+						      target->dep_list[i],
+						      bump);
+	}
+
+	return count;
+}
+
+/*
+ * Nodes are considered valid dependencies if they are queued on any ring or
+ * if they are in flight on a different ring. In flight on the same ring is no
+ * longer interesting for non-premptive nodes as the ring serialises execution.
+ * For pre-empting nodes, all in flight dependencies are valid as they must not
+ * be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */
+static inline bool i915_scheduler_is_dependency_valid(
+			struct i915_scheduler_queue_entry *node, uint32_t idx)
+{
+	struct i915_scheduler_queue_entry *dep;
+
+	dep = node->dep_list[idx];
+	if (!dep)
+		return false;
+
+	if (I915_SQS_IS_QUEUED(dep))
+		return true;
+
+	if (I915_SQS_IS_FLYING(dep)) {
+		if (node->params.ring != dep->params.ring)
+			return true;
+	}
+
+	return false;
+}
+
+static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
+				struct i915_scheduler_queue_entry **pop_node)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *best = NULL;
+	struct i915_scheduler_queue_entry *node;
+	int ret;
+	int i;
+	bool any_queued = false;
+	bool has_local, has_remote, only_remote = false;
+
+	*pop_node = NULL;
+	ret = -ENODATA;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+		any_queued = true;
+
+		has_local  = false;
+		has_remote = false;
+		for (i = 0; i < node->num_deps; i++) {
+			if (!i915_scheduler_is_dependency_valid(node, i))
+				continue;
+
+			if (node->dep_list[i]->params.ring == node->params.ring)
+				has_local = true;
+			else
+				has_remote = true;
+		}
+
+		if (has_remote && !has_local)
+			only_remote = true;
+
+		if (!has_local && !has_remote) {
+			if (!best ||
+			    (node->priority > best->priority))
+				best = node;
+		}
+	}
+
+	if (best) {
+		list_del(&best->link);
+
+		INIT_LIST_HEAD(&best->link);
+		best->status  = i915_sqs_popped;
+
+		ret = 0;
+	} else {
+		/* Can only get here if:
+		 * (a) there are no buffers in the queue

drivers/gpu/drm/i915/i915_scheduler.c line 293

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		 * (b) all queued buffers are dependent on other buffers
+		 *     e.g. on a buffer that is in flight on a different ring
+		 */
+		if (only_remote) {
+			/* The only dependent buffers are on another ring. */
+			ret = -EAGAIN;
+		} else if (any_queued) {
+			/* It seems that something has gone horribly wrong! */
+			DRM_ERROR("Broken dependency tracking on ring %d!\n",
+				  (int) ring->id);

Would this condition be worthy WARN_ONCE(), because it will keep repeating as the situation will persist?

Quoted 61 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +61 -0
+		}
+	}
+
+	*pop_node = best;
+	return ret;
+}
+
+/*
+ * NB: The driver mutex lock must be held before calling this function. It is
+ * only really required during the actual back end submission call. However,
+ * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
+ * And releasing the one before acquiring the other leads to other code
+ * being run and interfering.
+ */
+static int i915_scheduler_submit(struct intel_engine_cs *ring)
+{
+	struct drm_device *dev = ring->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	int ret, count = 0, flying;
+
+	WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+
+	spin_lock_irq(&scheduler->lock);
+
+	WARN_ON(scheduler->flags[ring->id] & i915_sf_submitting);
+	scheduler->flags[ring->id] |= i915_sf_submitting;
+
+	/* First time around, complain if anything unexpected occurs: */
+	ret = i915_scheduler_pop_from_queue_locked(ring, &node);
+	if (ret)
+		goto error;
+
+	do {
+		WARN_ON(!node);
+		WARN_ON(node->params.ring != ring);
+		WARN_ON(node->status != i915_sqs_popped);
+		count++;
+
+		/*
+		 * The call to pop above will have removed the node from the
+		 * list. So add it back in and mark it as in flight.
+		 */
+		i915_scheduler_node_fly(node);
+
+		spin_unlock_irq(&scheduler->lock);
+		ret = dev_priv->gt.execbuf_final(&node->params);
+		spin_lock_irq(&scheduler->lock);
+
+		/*
+		 * Handle failed submission but first check that the
+		 * watchdog/reset code has not nuked the node while we
+		 * weren't looking:
+		 */
+		if (ret && (node->status != i915_sqs_dead)) {
+			bool requeue = true;
+
+			/*
+			 * Oh dear! Either the node is broken or the ring is
+			 * busy. So need to kill the node or requeue it and try

drivers/gpu/drm/i915/i915_scheduler.c line 364

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+			 * again later as appropriate.
+			 */
+
+			switch (-ret) {
+			case ENODEV:
+			case ENOENT:
+				/* Fatal errors. Kill the node. */
+				requeue = false;
+				i915_scheduler_node_kill(node);
+			break;

These break indents still.

Quoted 40 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +40 -0
+
+			case EAGAIN:
+			case EBUSY:
+			case EIO:
+			case ENOMEM:
+			case ERESTARTSYS:
+			case EINTR:
+				/* Supposedly recoverable errors. */
+			break;
+
+			default:
+				/*
+				 * Assume the error is recoverable and hope
+				 * for the best.
+				 */
+				MISSING_CASE(-ret);
+			break;
+			}
+
+			if (requeue) {
+				i915_scheduler_node_requeue(node);
+				/*
+				 * No point spinning if the ring is currently
+				 * unavailable so just give up and come back
+				 * later.
+				 */
+				break;
+			}
+		}
+
+		/* Keep launching until the sky is sufficiently full. */
+		flying = i915_scheduler_count_flying(scheduler, ring);
+		if (flying >= scheduler->min_flying)
+			break;
+
+		/* Grab another node and go round again... */
+		ret = i915_scheduler_pop_from_queue_locked(ring, &node);
+	} while (ret == 0);
+
+	/* Don't complain about not being able to submit extra entries */
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 411

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	if (ret == -ENODATA)
+		ret = 0;
+
+	/*
+	 * Bump the priority of everything that was not submitted to prevent
+	 * starvation of low priority tasks by a spamming high priority task.
+	 */

This, this calls for an I-G-T test to make sure we're effective.

Yeah, IGTs are under development.

Quoted 76 lines
Quoted 35 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +35 -0
+	i915_scheduler_priority_bump_clear(scheduler);
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+
+		i915_scheduler_priority_bump(scheduler, node,
+					     scheduler->priority_level_bump);
+	}
+
+	/* On success, return the number of buffers submitted. */
+	if (ret == 0)
+		ret = count;
+
+error:
+	scheduler->flags[ring->id] &= ~i915_sf_submitting;
+	spin_unlock_irq(&scheduler->lock);
+	return ret;
+}
+
+static void i915_generate_dependencies(struct i915_scheduler *scheduler,
+				       struct i915_scheduler_queue_entry *node,
+				       uint32_t ring)
+{
+	struct i915_scheduler_obj_entry *this, *that;
+	struct i915_scheduler_queue_entry *test;
+	int i, j;
+	bool found;
+
+	list_for_each_entry(test, &scheduler->node_queue[ring], link) {
+		if (I915_SQS_IS_COMPLETE(test))
+			continue;
+
+		/*
+		 * Batches on the same ring for the same
+		 * context must be kept in order.

drivers/gpu/drm/i915/i915_scheduler.c line 456

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		 */
+		found = (node->params.ctx == test->params.ctx) &&
+			(node->params.ring == test->params.ring);
+
+		/*
+		 * Batches working on the same objects must
+		 * be kept in order.
+		 */
+		for (i = 0; (i < node->num_objs) && !found; i++) {
+			this = node->saved_objects + i;

node->objs and node->num_objs would be a better naming match. num_objs and saved_objects makes me think we have a bug here, indexing wrong array.

Quoted 27 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +27 -0
+
+			for (j = 0; j < test->num_objs; j++) {
+				that = test->saved_objects + j;
+
+				if (this->obj != that->obj)
+					continue;
+
+				/* Only need to worry about writes */
+				if (this->read_only && that->read_only)
+					continue;
+
+				found = true;
+				break;
+			}
+		}
+
+		if (found) {
+			node->dep_list[node->num_deps] = test;
+			node->num_deps++;
+		}
+	}
+}
+
+static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_entry *qe)
+{
+	struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 489

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+	int ret;
+
+	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
+	ret = dev_priv->gt.execbuf_final(&qe->params);
+	scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
+

The above code makes me think of locking again, maybe document at top of this function.

No need to acquire the spinlock at this point. If the scheduler is in bypass mode then there is no internal state to protect. The driver mutex lock will be held because there is only one code path to get here which is the execbuff IOCTL. The whole point of bypass mode is that it is a single contiguous execution path from IOCTL entry all the way through to hardware submission. A mutex check could be added but it seems unnecessary given the limited calling scope.

Quoted 37 lines
Quoted 37 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +36 -0
+	/*
+	 * Don't do any clean up on failure because the caller will
+	 * do it all anyway.
+	 */
+	if (ret)
+		return ret;
+
+	/* Need to release any resources held by the node: */
+	i915_scheduler_clean_node(qe);
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
+ * scheduler.
+ * @qe: The batch buffer request to be queued.
+ * The expectation is the qe passed in is a local stack variable. This
+ * function will copy its contents into a freshly allocated list node. The
+ * new node takes ownership of said contents so the original qe should simply
+ * be discarded and not cleaned up (i.e. don't free memory it points to or
+ * dereference objects it holds). The node is added to the scheduler's queue
+ * and the batch buffer will be submitted to the hardware at some future
+ * point in time (which may be immediately, before returning or may be quite
+ * a lot later).
+ */
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
+{
+	struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring = qe->params.ring;
+	struct i915_scheduler_queue_entry *node;
+	struct i915_scheduler_queue_entry *test;
+	bool not_flying;
+	int i, r;
+	int incomplete = 0;
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 531

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+
+	WARN_ON(!scheduler);
+
+	if (1/*!i915.enable_scheduler*/)
+		return i915_scheduler_queue_execbuffer_bypass(qe);
+

I'm thinking, should we branch already before calling a function named i915_scheduler_queue_execbuffer if scheduler is disabled?

That would require putting if(scheduler) logic in random points throughout the driver. Doing it this way, the driver as a whole has a single execution path which is via the scheduler. It is then up to the scheduler itself to decide whether to run in bypass mode, simple re-ordering mode, full pre-emption mode or any other mode we feel a need to add.

Quoted 7 lines

drivers/gpu/drm/i915/i915_scheduler.c line 536

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	node = kmalloc(sizeof(*node), GFP_KERNEL);
+	if (!node)
+		return -ENOMEM;
+
+	*node = *qe;

Now I read the added documentation for the function, maybe we should at least zero out qe to avoid future problems?

The execbuff success path hands the qe contents over and pretty much immediately drops all the way back out of the IOCTL. The failure path does the full cleanup of de-allocating all the qe stuff it had previously allocated. IMO there doesn't seem to be much need to zero out the structure at this point.

Quoted 61 lines
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+	INIT_LIST_HEAD(&node->link);
+	node->status = i915_sqs_queued;
+	node->stamp  = jiffies;
+	i915_gem_request_reference(node->params.request);
+
+	WARN_ON(node->params.request->scheduler_qe);
+	node->params.request->scheduler_qe = node;
+

drivers/gpu/drm/i915/i915_scheduler.c line 554

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	/* Need to determine the number of incomplete entries in the list as
+	 * that will be the maximum size of the dependency list.
+	 *
+	 * Note that the allocation must not be made with the spinlock acquired
+	 * as kmalloc can sleep. However, the unlock/relock is safe because no
+	 * new entries can be queued up during the unlock as the i915 driver
+	 * mutex is still held. Entries could be removed from the list but that
+	 * just means the dep_list will be over-allocated which is fine.
+	 */
+	spin_lock_irq(&scheduler->lock);

-->

drivers/gpu/drm/i915/i915_scheduler.c line 563

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(test, &scheduler->node_queue[r], link) {
+			if (I915_SQS_IS_COMPLETE(test))
+				continue;
+
+			incomplete++;
+		}
+	}
+

<-- This counting structure, I still think it would be good idea to make it a static helper.

Quoted 28 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +28 -0
+	/* Temporarily unlock to allocate memory: */
+	spin_unlock_irq(&scheduler->lock);
+	if (incomplete) {
+		node->dep_list = kmalloc_array(incomplete,
+					       sizeof(*node->dep_list),
+					       GFP_KERNEL);
+		if (!node->dep_list) {
+			kfree(node);
+			return -ENOMEM;
+		}
+	} else
+		node->dep_list = NULL;
+
+	spin_lock_irq(&scheduler->lock);
+	node->num_deps = 0;
+
+	if (node->dep_list) {
+		for (r = 0; r < I915_NUM_RINGS; r++)
+			i915_generate_dependencies(scheduler, node, r);
+
+		WARN_ON(node->num_deps > incomplete);
+	}
+
+	node->priority = clamp(node->priority,
+			       scheduler->priority_level_min,
+			       scheduler->priority_level_max);
+
+	if ((node->priority > 0) && node->num_deps) {
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 596

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+		i915_scheduler_priority_bump_clear(scheduler);
+
+		for (i = 0; i < node->num_deps; i++)
+			i915_scheduler_priority_bump(scheduler,
+					node->dep_list[i], node->priority);

If node is posted with maximum priority to begin with, wouldn't the priority propagation stop after first level dependencies due to the check in the beginning of priority_bump function?

The recursion will stop whenever it encounters a node already at maximum priority. However, it is not starting from the newly submitted node but from that node's dependencies. If they are already at max priority then their dependencies must also be and so on down the chain and no further processing is required. On the other hand, if they are not then they will be bumped and their dependencies checked and so on down the chain.

Quoted 125 lines
Quoted 125 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +125 -0
+
+	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
+
+	not_flying = i915_scheduler_count_flying(scheduler, ring) <
+						 scheduler->min_flying;
+
+	spin_unlock_irq(&scheduler->lock);
+
+	if (not_flying)
+		i915_scheduler_submit(ring);
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_notify_request - Notify the scheduler that the given
+ * request has completed on the hardware.
+ * @req: Request structure which has completed
+ * @preempt: Did it complete pre-emptively?
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not. The return
+ * value is combined for all freshly completed requests and if any were true
+ * then i915_scheduler_wakeup() is called so the scheduler can do further
+ * processing (submit more work) at the end.
+ */
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
+{
+	struct drm_i915_private *dev_priv = to_i915(req->ring->dev);
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
+	unsigned long flags;
+
+	if (!node)
+		return false;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Node was in flight so mark it as complete. */
+	if (req->cancelled)
+		node->status = i915_sqs_dead;
+	else
+		node->status = i915_sqs_complete;
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return true;
+}
+
+static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *remove)
+{
+	struct i915_scheduler_queue_entry *node;
+	int i, r;
+	int count = 0;
+
+	/*
+	 * Ensure that a node is not being removed which is still dependent
+	 * upon other (not completed) work. If that happens, it implies
+	 * something has gone very wrong with the dependency tracking! Note
+	 * that there is no need to worry if this node has been explicitly
+	 * killed for some reason - it might be being killed before it got
+	 * sent to the hardware.
+	 */
+	if (remove->status != i915_sqs_dead) {
+		for (i = 0; i < remove->num_deps; i++)
+			if ((remove->dep_list[i]) &&
+			    (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
+				count++;
+		WARN_ON(count);
+	}
+
+	/*
+	 * Remove this node from the dependency lists of any other node which
+	 * might be waiting on it.
+	 */
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(node, &scheduler->node_queue[r], link) {
+			for (i = 0; i < node->num_deps; i++) {
+				if (node->dep_list[i] != remove)
+					continue;
+
+				node->dep_list[i] = NULL;
+			}
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_wakeup - wake the scheduler's worker thread
+ * @dev: DRM device
+ * Called at the end of seqno interrupt processing if any request has
+ * completed that corresponds to a scheduler node.
+ */
+void i915_scheduler_wakeup(struct drm_device *dev)
+{
+	/* XXX: Need to call i915_scheduler_remove() via work handler. */
+}
+
+/**
+ * i915_scheduler_clean_node - free up any allocations/references
+ * associated with the given scheduler queue entry.
+ * @node: Queue entry structure which is complete
+ * After a give batch buffer completes on the hardware, all the information
+ * required to resubmit it is no longer required. However, the node entry
+ * itself might still be required for tracking purposes for a while longer.
+ * This function should be called as soon as the node is known to be complete
+ * so that these resources may be freed even though the node itself might
+ * hang around.
+ */
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
+{
+	if (!I915_SQS_IS_COMPLETE(node)) {
+		WARN(!node->params.request->cancelled,
+		     "Cleaning active node: %d!\n", node->status);
+		return;
+	}
+
+	if (node->params.batch_obj) {
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 729

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+		/*
+		 * The batch buffer must be unpinned before it is unreferenced
+		 * otherwise the unpin fails with a missing vma!?
+		 */
+		if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
+			i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
+

This code seems little bit out of place.

Not sure where else you could put it. The code itself is just a copy of the existing clean up code that was already in the execbuff code path. The cleanup can no longer be done within the execbuff call due to the batch buffer execution being arbitrarily delayed by the scheduler. Thus it is down to the scheduler to do all the necessary cleanup when it knows that the batch buffer is no longer required.

Quoted 13 lines

drivers/gpu/drm/i915/i915_scheduler.c line 739

Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		node->params.batch_obj = NULL;
+	}
+
+	/* And anything else owned by the node: */
+	if (node->params.cliprects) {
+		kfree(node->params.cliprects);
+		node->params.cliprects = NULL;
+	}
+}
+

The below function is quite hacky, I understood this will be mitigated by per-ctx sequence numbering in the future?

To be honest, I think it is already obsolete due to the previous work of converting from seqno tests everywhere to testing request structures. The point of the code below was to cope with seqno values popping out of the hardware in random order due to the re-ordering of batch buffers prior to execution. Whereas, the struct request conversion allowed the seqno to be late allocated and thus kept in order.

Unfortunately, pre-emption confuses matters again. Although the intention is that seqno values will still be kept in order. It's just that a given request might have multiple different seqno values assigned to it throughout its life. The pre-emption code hasn't quite settled down yet though, especially with interesting new hardware planned for future chips. Thus I don't really want to rip out all of this code quite yet just in case we do still need it.

As you say, per-ctx seqnos would definitely have an effect here as well. So yes, long term the intention is to remove all this nastyness. Short term, it seems best to keep it around until proven unnecessary.

Quoted 75 lines
Quoted 75 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +74 -0
+static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
+				  struct intel_engine_cs *ring,
+				  struct list_head *remove)
+{
+	struct i915_scheduler_queue_entry *node, *node_next;
+	int flying = 0, queued = 0;
+	bool do_submit;
+	uint32_t min_seqno;
+
+	spin_lock_irq(&scheduler->lock);
+
+	/*
+	 * In the case where the system is idle, starting 'min_seqno' from a big
+	 * number will cause all nodes to be removed as they are now back to
+	 * being in-order. However, this will be a problem if the last one to
+	 * complete was actually out-of-order as the ring seqno value will be
+	 * lower than one or more completed buffers. Thus code looking for the
+	 * completion of said buffers will wait forever.
+	 * Instead, use the hardware seqno as the starting point. This means
+	 * that some buffers might be kept around even in a completely idle
+	 * system but it should guarantee that no-one ever gets confused when
+	 * waiting for buffer completion.
+	 */
+	min_seqno = ring->get_seqno(ring, true);
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->params.request->seqno == 0)
+			continue;
+
+		if (!i915_seqno_passed(node->params.request->seqno, min_seqno))
+			min_seqno = node->params.request->seqno;
+	}
+
+	INIT_LIST_HEAD(remove);
+	list_for_each_entry_safe(node, node_next, &scheduler->node_queue[ring->id], link) {
+		/*
+		 * Only remove completed nodes which have a lower seqno than
+		 * all pending nodes. While there is the possibility of the
+		 * ring's seqno counting backwards, all higher buffers must
+		 * be remembered so that the 'i915_seqno_passed()' test can
+		 * report that they have in fact passed.
+		 *
+		 * NB: This is not true for 'dead' nodes. The GPU reset causes
+		 * the software seqno to restart from its initial value. Thus
+		 * the dead nodes must be removed even though their seqno values
+		 * are potentially vastly greater than the current ring seqno.
+		 */
+		if (!I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->status != i915_sqs_dead) {
+			if (i915_seqno_passed(node->params.request->seqno, min_seqno) &&
+			    (node->params.request->seqno != min_seqno))
+				continue;
+		}
+
+		list_del(&node->link);
+		list_add(&node->link, remove);
+
+		/* Strip the dependency info while the mutex is still locked */
+		i915_scheduler_remove_dependent(scheduler, node);
+
+		continue;
+	}
+
+	/*
+	 * Release the interrupt reference count if there are no longer any
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 819

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+	 * nodes to worry about.
+	 */
+	if (!flying && !queued &&
+	    (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+		ring->irq_put(ring);
+		scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;

Maybe have this piece of code as a helper? To enable interrupts and disable them, maybe even a reference count?

Not sure how much you could abstract out into a helper other than the two lines of irq_get|put + flag set|clear. The flying|queued test is particular to this one instance. Also, there is no need to reference count as the irq code does that already.

Quoted 43 lines
Quoted 43 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +42 -0
+	}
+
+	/* Launch more packets now? */
+	do_submit = (queued > 0) && (flying < scheduler->min_flying);
+
+	spin_unlock_irq(&scheduler->lock);
+
+	return do_submit;
+}
+
+void i915_scheduler_process_work(struct intel_engine_cs *ring)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	bool do_submit;
+	struct list_head remove;
+
+	if (list_empty(&scheduler->node_queue[ring->id]))
+		return;
+
+	/* Remove completed nodes. */
+	do_submit = i915_scheduler_remove(scheduler, ring, &remove);
+
+	if (!do_submit && list_empty(&remove))
+		return;
+
+	/* Need to grab the pm lock outside of the mutex lock */
+	if (do_submit)
+		intel_runtime_pm_get(dev_priv);
+
+	mutex_lock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		i915_scheduler_submit(ring);
+
+	while (!list_empty(&remove)) {
+		node = list_first_entry(&remove, typeof(*node), link);
+		list_del(&node->link);
+
+		/* Free up all the DRM references */
+		i915_scheduler_clean_node(node);
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 867

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+
+		/* And anything else owned by the node: */
+		node->params.request->scheduler_qe = NULL;
+		i915_gem_request_unreference(node->params.request);
+		kfree(node->dep_list);
+		kfree(node);

Maybe have a separate helper for freeing a node, would make this much cleaner.

There is a helper for freeing up the contents of the node - see i915_scheduler_clean_node() above. However, that function is called from multiple places and this is the only instance where the node itself is actually freed. The other callers either need to keep the node around for a while longer (i.e. until this piece of code can run) or don't own the node because it is stack allocated (in the bypass case). So the 'anything else' section cannot be moved into the existing clean function and any secondary helper would be single usage and only four lines long. It doesn't seem worth it.

Quoted 51 lines
Quoted 51 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	}
+
+	mutex_unlock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		intel_runtime_pm_put(dev_priv);
+}
drivers/gpu/drm/i915/i915_scheduler.h new file +37 -0
new file mode 100644
index 0000000..415fec8
@@ -0,0 +1,95 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _I915_SCHEDULER_H_
+#define _I915_SCHEDULER_H_
+
+enum i915_scheduler_queue_status {
+	/* Limbo: */
+	i915_sqs_none = 0,
+	/* Not yet submitted to hardware: */
+	i915_sqs_queued,
+	/* Popped from queue, ready to fly: */
+	i915_sqs_popped,
+	/* Sent to hardware for processing: */
+	i915_sqs_flying,
+	/* Finished processing on the hardware: */
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.h line 44

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +7 -0
+	i915_sqs_complete,
+	/* Killed by watchdog or catastrophic submission failure: */
+	i915_sqs_dead,
+	/* Limit value for use with arrays/loops */
+	i915_sqs_MAX
+};
+

To uppercase.

Grrr. Linux kernel style guide is daft.

Quoted 47 lines
Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +12 -0
+#define I915_SQS_IS_QUEUED(node)	(((node)->status == i915_sqs_queued))
+#define I915_SQS_IS_FLYING(node)	(((node)->status == i915_sqs_flying))
+#define I915_SQS_IS_COMPLETE(node)	(((node)->status == i915_sqs_complete) || \
+					 ((node)->status == i915_sqs_dead))
+
+struct i915_scheduler_obj_entry {
+	struct drm_i915_gem_object          *obj;
+	bool                                read_only;
+};
+
+struct i915_scheduler_queue_entry {
+	struct i915_execbuffer_params       params;

drivers/gpu/drm/i915/i915_scheduler.h line 66

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	/* -1023 = lowest priority, 0 = default, 1023 = highest */
+	int32_t                             priority;
+	struct i915_scheduler_obj_entry     *saved_objects;
+	int                                 num_objs;
+	bool                                bumped;
+	struct i915_scheduler_queue_entry   **dep_list;
+	int                                 num_deps;
+	enum i915_scheduler_queue_status    status;
+	unsigned long                       stamp;
+	struct list_head                    link;

Above whitespace is still incorrect and I could really use comments.

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+};
+
+struct i915_scheduler {
+	struct list_head    node_queue[I915_NUM_RINGS];
+	uint32_t            flags[I915_NUM_RINGS];
+	spinlock_t          lock;
+
+	/* Tuning parameters: */
+	int32_t             priority_level_min;
+	int32_t             priority_level_max;

drivers/gpu/drm/i915/i915_scheduler.h line 86

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	int32_t             priority_level_bump;
+	int32_t             priority_level_preempt;
+	uint32_t            min_flying;
+};
+
+/* Flag bits for i915_scheduler::flags */
+enum {
+	i915_sf_interrupts_enabled  = (1 << 0),
+	i915_sf_submitting          = (1 << 1),
+};

These must be I915_UPPERCASE_SOMETHING by convention.

Quoted 9 lines

drivers/gpu/drm/i915/i915_scheduler.h line 95

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +9 -0
+
+bool i915_scheduler_is_enabled(struct drm_device *dev);
+int i915_scheduler_init(struct drm_device *dev);
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
+void i915_scheduler_wakeup(struct drm_device *dev);
+
+#endif  /* _I915_SCHEDULER_H_ */

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

maintainer
#9 lore raw

Re: [PATCH v5 06/35] drm/i915: Start of GPU scheduler

Hi,

The below answers are reasonable. So v6 should be the version.

Regards, Joonas

Quoted 1,834 lines · John Harrison
Quoted 243 lines · Joonas Lahtinen

Hi,

Now the code is in reviewable chunks, excellent!

I've added my comments below. A few repeats from last round, but now
with more questions about the logic itself.

Quoted 228 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in
to the driver yet. It also leaves the scheduler code in 'pass through'
mode so that even when it is hooked in, it will not actually do very
much. This allows the hooks to be added one at a time in bite size
chunks and only when the scheduler is finally enabled at the end does
anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the
information required to execute the batch buffer at a later time. This
package is given over to the scheduler which adds it to an internal
node list. The scheduler also scans the list of objects associated
with the batch buffer and compares them against the objects already in
use by other buffers in the node list. If matches are found then the
new batch buffer node is marked as being dependent upon the matching
node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the
more dependencies a given batch buffer has the more important it is
likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers
in flight on the hardware at any given time. If fewer than this are
currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added
to the queue and the driver returns back to user land.

The scheduler is notified when each batch buffer completes and updates
its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed,
the scheduler's deferred worker thread is woken up. This can do more
involved processing such as actually removing completed nodes from the
queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.).
The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node
list for the highest priority node that has no unmet dependencies.
Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note
also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or
the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for
submission to the hardware. The in flight count is then re- checked and
a new node popped from the list if appropriate. All nodes that are not
submitted have their priority bumped. This ensures that low priority
tasks do not get starved out by busy higher priority ones - everything
will eventually get its turn to run.

Note that this patch does not implement pre-emptive scheduling. Only
basic scheduling by re-ordering batch buffer submission is currently
implemented. Pre-emption of actively executing batch buffers comes in
the next patch series.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915
code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler
but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be
told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable.
Also added in read-read optimisation support - i.e., don't treat a
shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

v4: Updated the commit message to better reflect the current state of
the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array
memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently
to keep the style checker happy.

Fixed a WARN_ON when killing nodes. The dependency removal code checks
that nodes being destroyed do not have any oustanding dependencies
(which would imply they should not have been executed yet). In the
case of nodes being destroyed, e.g. due to context banning, then this
might well be the case - they have not been executed and do indeed
have outstanding dependencies.

Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have
been fixed now.

v5: Shuffled various functions around to remove forward declarations
as apparently these are frowned upon. Removed lots of white space as
apparently having easy to read code is also frowned upon. Split the
direct submission scheduler bypass code out into a separate function.
Squashed down the i915_scheduler.c sections of various patches into
this patch. Thus the later patches simply hook in existing code into
various parts of the driver rather than adding the code as well. Added
documentation to various functions. Re-worked the submit function in
terms of mutex locking, error handling and exit paths. Split the delayed work handler function in half. Made use of the kernel 'clamp'
macro. [Joonas Lahtinen]

Added runtime PM calls as these must be done at the top level before
acquiring the driver mutex lock. [Chris Wilson]

Removed some obsolete debug code that had been forgotten about.

Moved more clean up code into the 'i915_gem_scheduler_clean_node()'
function rather than replicating it in mutliple places.

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>

---
  drivers/gpu/drm/i915/Makefile         |   1 + drivers/gpu/drm/i915/i915_drv.h       |   6 + drivers/gpu/drm/i915/i915_gem.c       |   5 + drivers/gpu/drm/i915/i915_scheduler.c | 874
++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/i915_scheduler.h |  95 ++++

  5 files changed, 981 insertions(+)

  create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h

drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
   i915-y := i915_drv.o \
   	  i915_irq.o \
   	  i915_params.o \
+	  i915_scheduler.o \
             i915_suspend.o \
   	  i915_sysfs.o \
   	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +0 -0
index f4487b9..03add1a 100644
@@ -1703,6 +1703,8 @@ struct i915_execbuffer_params {
   	struct drm_i915_gem_request     *request;
   };

drivers/gpu/drm/i915/i915_drv.h +2 -0
+struct i915_scheduler;
+

  /* used in computing the new watermarks state */ struct intel_wm_config {
   unsigned int num_pipes_active;

drivers/gpu/drm/i915/i915_drv.h +0 -0
@@ -1968,6 +1970,8 @@ struct drm_i915_private {


   struct i915_runtime_pm pm;

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1974

Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
+	struct i915_scheduler *scheduler;
+

I think we should have i915.enable_scheduler parameter (just like i915.enable_execlists) and make this a data member, not pointer.

Quoted 3 lines

Then we can go forward and have i915.enable_scheduler_preempt and so on
to control things at boot time.

There are. The enable_scheduler comes in as patch #21. Including it from
the start means modifying i915_params.[ch] in this patch as well. And
then you are getting into one big patch that contains everything and is
unreviewable due to its sheer size. The enable_preemption parameter comes in with the pre-emption code. No point in adding that until there
is something for it to control.

Quoted 99 lines
Quoted 12 lines

   /* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
   struct {
   int (*execbuf_submit)(struct i915_execbuffer_params *params,

drivers/gpu/drm/i915/i915_drv.h +0 -0
@@ -2290,6 +2294,8 @@ struct drm_i915_gem_request {
   	/** process identifier submitting this request */
   	struct pid *pid;

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_gem_request { · line 2298

Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
+	struct i915_scheduler_queue_entry	*scheduler_qe;
+

Funny whitespace.

Quoted 83 lines

   /**
    * The ELSP only accepts two elements at a time, so we queue
    * context/tail pairs on a given queue (ring-

Quoted 1 line

execlist_queue) until the

drivers/gpu/drm/i915/i915_gem.c +1 -0
index dfe43ea..7d9aa24 100644
@@ -32,6 +32,7 @@
   #include "i915_vgpu.h"
   #include "i915_trace.h"
   #include "intel_drv.h"
+#include "i915_scheduler.h"
   #include
   #include
   #include
@@ -5315,6 +5316,10 @@ int i915_gem_init(struct drm_device *dev)
   	 */
   	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);

drivers/gpu/drm/i915/i915_gem.c +4 -0
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+

   ret = i915_gem_init_userptr(dev);
   if (ret)
   goto out_unlock;

drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
new file mode 100644
index 0000000..fc23ee7
@@ -0,0 +1,874 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+
+#include "i915_drv.h"
+#include "intel_drv.h"
+#include "i915_scheduler.h"
+
+/**
+ * i915_scheduler_is_enabled - Returns true if the scheduler is

enabled.
+ * @dev: DRM device

Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 38

Quoted 7 lines

+ */

drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+bool i915_scheduler_is_enabled(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+
+	return dev_priv->scheduler != NULL;
+}

i915.enable_scheduler would be used instead.

It is when it arrives.

Quoted 19 lines
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+
+/**
+ * i915_scheduler_init - Initialise the scheduler.
+ * @dev: DRM device
+ * Returns zero on success or -ENOMEM if memory allocations

fail.
+ */

drivers/gpu/drm/i915/i915_scheduler.c line 50

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+int i915_scheduler_init(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	int r;
+

This protection -->

Quoted 1 line
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 57

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	if (scheduler)
+		return 0;
+
+	scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
+	if (!scheduler)
+		return -ENOMEM;
+

Is not needed if we have data member and i915.enable_scheduler.

Making the scheduler structure a object rather than a pointer basically
means moving the whole of i915_scheduler.h into i915_drv.h. There is too
much interconnectedness between the various structures. I would say that
keeping the code nicely partitioned and readable is worth the one dynamic allocation at driver load time.

Quoted 27 lines
Quoted 27 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +19 -0
+	spin_lock_init(&scheduler->lock);
+
+	for (r = 0; r < I915_NUM_RINGS; r++)
+		INIT_LIST_HEAD(&scheduler->node_queue[r]);
+
+	/* Default tuning values: */
+	scheduler->priority_level_min     = -1023;
+	scheduler->priority_level_max     = 1023;
+	scheduler->priority_level_bump    = 50;
+	scheduler->priority_level_preempt = 900;
+	scheduler->min_flying             = 2;
+
+	dev_priv->scheduler = scheduler;
+
+	return 0;
+}
+
+/*
+ * Add a popped node back in to the queue. For example, because

the ring was
+ * hung when execfinal() was called and thus the ring submission needs to be
+ * retried later.
+ */

Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c

Quoted 4 lines

+static void i915_scheduler_node_requeue(struct
i915_scheduler_queue_entry *node)

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+{
+	WARN_ON(!node);

Rather remove this completely as it is an internal function, or use BUG_ON because next line will straight lead to OOPS after warning. I'll
pass mentioning rest of the obvious WARN_ON vs BUG_ON's.

Yeah, most of the WARN/BUG_ONs are probably unnecessary now. They were
scattered liberally around during initial development when strange and
unexpected things could happen. Now the the code paths are pretty well
established they aren't really much use. I'll reduce them down to just
the ones that still make sense.

Quoted 76 lines
Quoted 17 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Seqno will be reassigned on relaunch */
+	node->params.request->seqno = 0;
+	node->status = i915_sqs_queued;
+}
+
+/*
+ * Give up on a node completely. For example, because it is

causing the
+ * ring to hang or is using some resource that no longer exists.
+ */
+static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)

drivers/gpu/drm/i915/i915_scheduler.c line 102

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+{
+	WARN_ON(!node);
+	WARN_ON(I915_SQS_IS_COMPLETE(node));
+
+	node->status = i915_sqs_dead;
+}
+
+/* Mark a node as in flight on the hardware. */

As for documentation, from below, I only assume scheduler lock must be
held and node's can be only manipulated when scheduler lock is held?
It's good to add a comment describing the expected locking. Or maybe
add assert_scheduler_lock_held() helper and sprinkle it around the functions similar to assert_rpm_wakelock_held(), which is self- documenting?

Quoted 6 lines

+static int i915_scheduler_node_fly(struct
i915_scheduler_queue_entry *node)

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+{
+	struct drm_i915_private *dev_priv = node->params.dev-
Quoted 1 line

dev_private;

If node is NULL, this would already OOPS, I do not think it's unreasonable to expect node not to be NULL. The below WARN_ON check is
never reached.

drivers/gpu/drm/i915/i915_scheduler.c line 114

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring;
+
+	WARN_ON(!scheduler);
+	WARN_ON(!node);
+	WARN_ON(node->status != i915_sqs_popped);
+
+	ring = node->params.ring;
+

This and similar assignments can be squashed to declaration line, as
this is basically an alias.

Quoted 16 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	/*
+	 * Add the node (which should currently be in state

popped) to the

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * front of the queue. This ensure that flying nodes are

always held

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	 * in hardware submission order.
+	 */
+	list_add(&node->link, &scheduler->node_queue[ring->id]);
+
+	node->status = i915_sqs_flying;
+
+	if (!(scheduler->flags[ring->id] &

i915_sf_interrupts_enabled)) {

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+		bool success = true;
+
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 131

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+		success = ring->irq_get(ring);
+		if (success)
+			scheduler->flags[ring->id] |=

i915_sf_interrupts_enabled;

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+		else
+			return -EINVAL;

Shouldn't the node be cleaned up from node_queue in case of an error?

Not really. The batch buffer will still be executed regardless of whether interrupts could be enabled or not. The return code is actually
ignored so this should really be a void function, I guess. Without interrupts then completion might take a while to be noticed but stuff
should still basically work.

Quoted 71 lines

Also, I think above could be written much more compact form (because it
looks like something where more logic will be added later). It makes it
easier to write and visualize the error paths when reading if there are
no nested if's.

I won't mention about the error paths of functions below, I expect the
following guidline to be adhered;

	if (scheduler->flags[ring->id] & I915_SF_INT_ENABLED)
		goto out;

	if (!ring->irq_get(ring))
		goto err_irq;

	if (!...)
		goto err_foo;

	scheduler->flags[ring->id] |= I915_SF_INT_ENABLED;
out:
	return 0;
err_foo:
	foobar();
err_irq:
	list_remove()
	return -EINVAL;
Quoted 15 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+	}
+
+	return 0;
+}
+
+static uint32_t i915_scheduler_count_flying(struct

i915_scheduler *scheduler,

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+					    struct

intel_engine_cs *ring)

drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+{
+	struct i915_scheduler_queue_entry *node;
+	uint32_t flying = 0;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring-
Quoted 1 line
Quoted 1 line

id], link)

+1 for the for_each_scheduler_node(...)

Quoted 21 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+		if (I915_SQS_IS_FLYING(node))
+			flying++;
+
+	return flying;
+}
+
+static void i915_scheduler_priority_bump_clear(struct

i915_scheduler *scheduler)

drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+{
+	struct i915_scheduler_queue_entry *node;
+	int i;
+
+	/*
+	 * Ensure circular dependencies don't cause problems and

that a bump

drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+	 * by object usage only bumps each using buffer once:
+	 */
+	for (i = 0; i < I915_NUM_RINGS; i++) {
+		list_for_each_entry(node, &scheduler-
Quoted 1 line

node_queue[i], link)

Quoted 13 lines
Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+			node->bumped = false;
+	}
+}
+
+static int i915_scheduler_priority_bump(struct i915_scheduler

*scheduler,

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+				struct

i915_scheduler_queue_entry *target,

Is there specific reason why struct i915_scheduler_queue_entry are not
referred to just as "node" but "qe" and here something else, do "node"
and "qe" have a special semantic meaning?

Not sure what you mean. The use of 'node' is because almost everything
an i915_scheduler_queue_entry object is being used it is as a node in a
list that is being iterated over. The only 'qe' variable is in 'i915_scheduler_queue_execbuffer[_bypass]' and there it is because the
qe is being passed in from outside and is copied to a local 'node' that
is then added to the list. I think the use of 'target' here is largely
historical. The subsequent array search was originally a list search and
hence had a 'node' that was used as the iterator. Thus the qe passed in
was called something else to avoid conflicts and target seemed appropriate.

Quoted 12 lines

drivers/gpu/drm/i915/i915_scheduler.c line 173

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+				uint32_t bump)
+{
+	uint32_t new_priority;
+	int i, count;
+
+	if (target->priority >= scheduler->priority_level_max)
+		return 1;

So if one node reaches maximum priority the dependencies are expected
to be maxed out already?

Yes. Dependencies should always be of equal or higher priority to the
dependee. When a new dependency is created, the priority of the dependee
is added on to ensure that something of high priority can never be stuck
waiting on something of low priority. Also, it means that a batch buffer
that lots of other buffers are waiting on will be bumped to higher and
higher priority the more waiters there are and thus is more likely to be
executed sooner and free them all up.

Quoted 282 lines
Quoted 139 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+
+	if (target->bumped)
+		return 0;
+
+	new_priority = target->priority + bump;
+	if ((new_priority <= target->priority) ||
+	    (new_priority > scheduler->priority_level_max))
+		target->priority = scheduler-
Quoted 1 line

priority_level_max;

drivers/gpu/drm/i915/i915_scheduler.c new file +15 -0
+	else
+		target->priority = new_priority;
+
+	count = 1;
+	target->bumped = true;
+
+	for (i = 0; i < target->num_deps; i++) {
+		if (!target->dep_list[i])
+			continue;
+
+		if (target->dep_list[i]->bumped)
+			continue;
+
+		count += i915_scheduler_priority_bump(scheduler,
+						      target-
Quoted 1 line

dep_list[i],

drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+						      bump);
+	}
+
+	return count;
+}
+
+/*
+ * Nodes are considered valid dependencies if they are queued on

any ring or
+ * if they are in flight on a different ring. In flight on the same ring is no
+ * longer interesting for non-premptive nodes as the ring serialises execution.
+ * For pre-empting nodes, all in flight dependencies are valid as they must not
+ * be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+static inline bool i915_scheduler_is_dependency_valid(
+			struct i915_scheduler_queue_entry *node,

uint32_t idx)

drivers/gpu/drm/i915/i915_scheduler.c new file +19 -0
+{
+	struct i915_scheduler_queue_entry *dep;
+
+	dep = node->dep_list[idx];
+	if (!dep)
+		return false;
+
+	if (I915_SQS_IS_QUEUED(dep))
+		return true;
+
+	if (I915_SQS_IS_FLYING(dep)) {
+		if (node->params.ring != dep->params.ring)
+			return true;
+	}
+
+	return false;
+}
+
+static int i915_scheduler_pop_from_queue_locked(struct

intel_engine_cs *ring,

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+				struct

i915_scheduler_queue_entry **pop_node)

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+{
+	struct drm_i915_private *dev_priv = ring->dev-
Quoted 1 line

dev_private;

drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *best = NULL;
+	struct i915_scheduler_queue_entry *node;
+	int ret;
+	int i;
+	bool any_queued = false;
+	bool has_local, has_remote, only_remote = false;
+
+	*pop_node = NULL;
+	ret = -ENODATA;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring-
Quoted 1 line

id], link) {

drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+		any_queued = true;
+
+		has_local  = false;
+		has_remote = false;
+		for (i = 0; i < node->num_deps; i++) {
+			if

(!i915_scheduler_is_dependency_valid(node, i))

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+				continue;
+
+			if (node->dep_list[i]->params.ring ==

node->params.ring)

drivers/gpu/drm/i915/i915_scheduler.c new file +26 -0
+				has_local = true;
+			else
+				has_remote = true;
+		}
+
+		if (has_remote && !has_local)
+			only_remote = true;
+
+		if (!has_local && !has_remote) {
+			if (!best ||
+			    (node->priority > best->priority))
+				best = node;
+		}
+	}
+
+	if (best) {
+		list_del(&best->link);
+
+		INIT_LIST_HEAD(&best->link);
+		best->status  = i915_sqs_popped;
+
+		ret = 0;
+	} else {
+		/* Can only get here if:
+		 * (a) there are no buffers in the queue
+		 * (b) all queued buffers are dependent on other

buffers

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 *     e.g. on a buffer that is in flight on a

different ring

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+		 */
+		if (only_remote) {
+			/* The only dependent buffers are on

another ring. */

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+			ret = -EAGAIN;
+		} else if (any_queued) {
+			/* It seems that something has gone

horribly wrong! */

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+			DRM_ERROR("Broken dependency tracking on

ring %d!\n",

drivers/gpu/drm/i915/i915_scheduler.c line 293

Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+				  (int) ring->id);

Would this condition be worthy WARN_ONCE(), because it will keep repeating as the situation will persist?

Quoted 82 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+		}
+	}
+
+	*pop_node = best;
+	return ret;
+}
+
+/*
+ * NB: The driver mutex lock must be held before calling this

function. It is
+ * only really required during the actual back end submission call. However,
+ * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
+ * And releasing the one before acquiring the other leads to other code
+ * being run and interfering.
+ */

drivers/gpu/drm/i915/i915_scheduler.c new file +13 -0
+static int i915_scheduler_submit(struct intel_engine_cs *ring)
+{
+	struct drm_device *dev = ring->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	int ret, count = 0, flying;
+
+	WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+
+	spin_lock_irq(&scheduler->lock);
+
+	WARN_ON(scheduler->flags[ring->id] &

i915_sf_submitting);

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+	scheduler->flags[ring->id] |= i915_sf_submitting;
+
+	/* First time around, complain if anything unexpected

occurs: */

drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+	ret = i915_scheduler_pop_from_queue_locked(ring, &node);
+	if (ret)
+		goto error;
+
+	do {
+		WARN_ON(!node);
+		WARN_ON(node->params.ring != ring);
+		WARN_ON(node->status != i915_sqs_popped);
+		count++;
+
+		/*
+		 * The call to pop above will have removed the

node from the

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * list. So add it back in and mark it as in

flight.

drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+		 */
+		i915_scheduler_node_fly(node);
+
+		spin_unlock_irq(&scheduler->lock);
+		ret = dev_priv->gt.execbuf_final(&node->params);
+		spin_lock_irq(&scheduler->lock);
+
+		/*
+		 * Handle failed submission but first check that

the

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * watchdog/reset code has not nuked the node

while we

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+		 * weren't looking:
+		 */
+		if (ret && (node->status != i915_sqs_dead)) {
+			bool requeue = true;
+
+			/*
+			 * Oh dear! Either the node is broken or

the ring is

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+			 * busy. So need to kill the node or

requeue it and try

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+			 * again later as appropriate.
+			 */
+
+			switch (-ret) {
+			case ENODEV:
+			case ENOENT:
+				/* Fatal errors. Kill the node.

*/

drivers/gpu/drm/i915/i915_scheduler.c line 364

Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+				requeue = false;
+				i915_scheduler_node_kill(node);
+			break;

These break indents still.

Quoted 52 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+
+			case EAGAIN:
+			case EBUSY:
+			case EIO:
+			case ENOMEM:
+			case ERESTARTSYS:
+			case EINTR:
+				/* Supposedly recoverable

errors. */

drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+			break;
+
+			default:
+				/*
+				 * Assume the error is

recoverable and hope

drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+				 * for the best.
+				 */
+				MISSING_CASE(-ret);
+			break;
+			}
+
+			if (requeue) {
+				i915_scheduler_node_requeue(node

);

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+				/*
+				 * No point spinning if the ring

is currently

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+				 * unavailable so just give up

and come back

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+				 * later.
+				 */
+				break;
+			}
+		}
+
+		/* Keep launching until the sky is sufficiently

full. */

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		flying = i915_scheduler_count_flying(scheduler,

ring);

drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+		if (flying >= scheduler->min_flying)
+			break;
+
+		/* Grab another node and go round again... */
+		ret = i915_scheduler_pop_from_queue_locked(ring,

&node);

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+	} while (ret == 0);
+
+	/* Don't complain about not being able to submit extra

entries */

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	if (ret == -ENODATA)
+		ret = 0;
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+
+	/*
+	 * Bump the priority of everything that was not

submitted to prevent

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * starvation of low priority tasks by a spamming high

priority task.

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 */

This, this calls for an I-G-T test to make sure we're effective.

Yeah, IGTs are under development.

Quoted 92 lines
Quoted 55 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	i915_scheduler_priority_bump_clear(scheduler);
+	list_for_each_entry(node, &scheduler->node_queue[ring-
Quoted 1 line

id], link) {

drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+
+		i915_scheduler_priority_bump(scheduler, node,
+					     scheduler-
Quoted 1 line

priority_level_bump);

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+	}
+
+	/* On success, return the number of buffers submitted.

*/

drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	if (ret == 0)
+		ret = count;
+
+error:
+	scheduler->flags[ring->id] &= ~i915_sf_submitting;
+	spin_unlock_irq(&scheduler->lock);
+	return ret;
+}
+
+static void i915_generate_dependencies(struct i915_scheduler

*scheduler,

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+				       struct

i915_scheduler_queue_entry *node,

drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+				       uint32_t ring)
+{
+	struct i915_scheduler_obj_entry *this, *that;
+	struct i915_scheduler_queue_entry *test;
+	int i, j;
+	bool found;
+
+	list_for_each_entry(test, &scheduler->node_queue[ring],

link) {

drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+		if (I915_SQS_IS_COMPLETE(test))
+			continue;
+
+		/*
+		 * Batches on the same ring for the same
+		 * context must be kept in order.
+		 */
+		found = (node->params.ctx == test->params.ctx)

&&

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+			(node->params.ring == test-
Quoted 1 line

params.ring);

drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+
+		/*
+		 * Batches working on the same objects must
+		 * be kept in order.
+		 */
+		for (i = 0; (i < node->num_objs) && !found; i++)

{

drivers/gpu/drm/i915/i915_scheduler.c line 456

Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+			this = node->saved_objects + i;

node->objs and node->num_objs would be a better naming match. num_objs
and saved_objects makes me think we have a bug here, indexing wrong array.

Quoted 31 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+
+			for (j = 0; j < test->num_objs; j++) {
+				that = test->saved_objects + j;
+
+				if (this->obj != that->obj)
+					continue;
+
+				/* Only need to worry about

writes */

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+				if (this->read_only && that-
Quoted 1 line

read_only)

drivers/gpu/drm/i915/i915_scheduler.c new file +15 -0
+					continue;
+
+				found = true;
+				break;
+			}
+		}
+
+		if (found) {
+			node->dep_list[node->num_deps] = test;
+			node->num_deps++;
+		}
+	}
+}
+
+static int i915_scheduler_queue_execbuffer_bypass(struct

i915_scheduler_queue_entry *qe)

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+{
+	struct drm_i915_private *dev_priv = qe->params.dev-
Quoted 1 line

dev_private;

Quoted 12 lines
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	int ret;
+
+	scheduler->flags[qe->params.ring->id] |=

i915_sf_submitting;

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	ret = dev_priv->gt.execbuf_final(&qe->params);
+	scheduler->flags[qe->params.ring->id] &=

~i915_sf_submitting;
+

The above code makes me think of locking again, maybe document at top
of this function.

No need to acquire the spinlock at this point. If the scheduler is in
bypass mode then there is no internal state to protect. The driver mutex
lock will be held because there is only one code path to get here which
is the execbuff IOCTL. The whole point of bypass mode is that it is a
single contiguous execution path from IOCTL entry all the way through to
hardware submission. A mutex check could be added but it seems unnecessary given the limited calling scope.

Quoted 52 lines
Quoted 52 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	/*
+	 * Don't do any clean up on failure because the caller

will

drivers/gpu/drm/i915/i915_scheduler.c new file +13 -0
+	 * do it all anyway.
+	 */
+	if (ret)
+		return ret;
+
+	/* Need to release any resources held by the node: */
+	i915_scheduler_clean_node(qe);
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_queue_execbuffer - Submit a batch buffer

request to the
+ * scheduler.
+ * @qe: The batch buffer request to be queued.
+ * The expectation is the qe passed in is a local stack variable. This
+ * function will copy its contents into a freshly allocated list node. The
+ * new node takes ownership of said contents so the original qe should simply
+ * be discarded and not cleaned up (i.e. don't free memory it points to or
+ * dereference objects it holds). The node is added to the scheduler's queue
+ * and the batch buffer will be submitted to the hardware at some future
+ * point in time (which may be immediately, before returning or may be quite
+ * a lot later).
+ */
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+{
+	struct drm_i915_private *dev_priv = qe->params.dev-
Quoted 1 line

dev_private;

drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring = qe->params.ring;
+	struct i915_scheduler_queue_entry *node;
+	struct i915_scheduler_queue_entry *test;
+	bool not_flying;
+	int i, r;
+	int incomplete = 0;
+
+	WARN_ON(!scheduler);
Quoted 8 lines
Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+
+	if (1/*!i915.enable_scheduler*/)
+		return

i915_scheduler_queue_execbuffer_bypass(qe);
+

I'm thinking, should we branch already before calling a function named
i915_scheduler_queue_execbuffer if scheduler is disabled?

That would require putting if(scheduler) logic in random points throughout the driver. Doing it this way, the driver as a whole has a
single execution path which is via the scheduler. It is then up to the
scheduler itself to decide whether to run in bypass mode, simple re-ordering mode, full pre-emption mode or any other mode we feel a need
to add.

Quoted 10 lines

drivers/gpu/drm/i915/i915_scheduler.c line 536

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	node = kmalloc(sizeof(*node), GFP_KERNEL);
+	if (!node)
+		return -ENOMEM;
+
+	*node = *qe;

Now I read the added documentation for the function, maybe we should at
least zero out qe to avoid future problems?

The execbuff success path hands the qe contents over and pretty much immediately drops all the way back out of the IOCTL. The failure path
does the full cleanup of de-allocating all the qe stuff it had previously allocated. IMO there doesn't seem to be much need to zero out
the structure at this point.

Quoted 79 lines
Quoted 25 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	INIT_LIST_HEAD(&node->link);
+	node->status = i915_sqs_queued;
+	node->stamp  = jiffies;
+	i915_gem_request_reference(node->params.request);
+
+	WARN_ON(node->params.request->scheduler_qe);
+	node->params.request->scheduler_qe = node;
+
+	/* Need to determine the number of incomplete entries in

the list as

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+	 * that will be the maximum size of the dependency list.
+	 *
+	 * Note that the allocation must not be made with the

spinlock acquired

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * as kmalloc can sleep. However, the unlock/relock is

safe because no

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * new entries can be queued up during the unlock as the

i915 driver

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * mutex is still held. Entries could be removed from

the list but that

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * just means the dep_list will be over-allocated which

is fine.

drivers/gpu/drm/i915/i915_scheduler.c

Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	 */
+	spin_lock_irq(&scheduler->lock);

-->

drivers/gpu/drm/i915/i915_scheduler.c line 563

Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(test, &scheduler-
Quoted 1 line

node_queue[r], link) {

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+			if (I915_SQS_IS_COMPLETE(test))
+				continue;
+
+			incomplete++;
+		}
+	}
+

<-- This counting structure, I still think it would be good idea to make it a static helper.

Quoted 36 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	/* Temporarily unlock to allocate memory: */
+	spin_unlock_irq(&scheduler->lock);
+	if (incomplete) {
+		node->dep_list = kmalloc_array(incomplete,
+					       sizeof(*node-
Quoted 1 line

dep_list),

drivers/gpu/drm/i915/i915_scheduler.c new file +14 -0
+					       GFP_KERNEL);
+		if (!node->dep_list) {
+			kfree(node);
+			return -ENOMEM;
+		}
+	} else
+		node->dep_list = NULL;
+
+	spin_lock_irq(&scheduler->lock);
+	node->num_deps = 0;
+
+	if (node->dep_list) {
+		for (r = 0; r < I915_NUM_RINGS; r++)
+			i915_generate_dependencies(scheduler,

node, r);

drivers/gpu/drm/i915/i915_scheduler.c new file +14 -0
+
+		WARN_ON(node->num_deps > incomplete);
+	}
+
+	node->priority = clamp(node->priority,
+			       scheduler->priority_level_min,
+			       scheduler->priority_level_max);
+
+	if ((node->priority > 0) && node->num_deps) {
+		i915_scheduler_priority_bump_clear(scheduler);
+
+		for (i = 0; i < node->num_deps; i++)
+			i915_scheduler_priority_bump(scheduler,
+					node->dep_list[i], node-
Quoted 4 lines
Quoted 1 line
Quoted 1 line

priority);

If node is posted with maximum priority to begin with, wouldn't the priority propagation stop after first level dependencies due to the check in the beginning of priority_bump function?

The recursion will stop whenever it encounters a node already at maximum
priority. However, it is not starting from the newly submitted node but
from that node's dependencies. If they are already at max priority then
their dependencies must also be and so on down the chain and no further
processing is required. On the other hand, if they are not then they will be bumped and their dependencies checked and so on down the chain.

Quoted 156 lines
Quoted 156 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+
+	list_add_tail(&node->link, &scheduler->node_queue[ring-
Quoted 1 line

id]);

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+
+	not_flying = i915_scheduler_count_flying(scheduler,

ring) <

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+						 scheduler-
Quoted 1 line

min_flying;

drivers/gpu/drm/i915/i915_scheduler.c new file +11 -0
+
+	spin_unlock_irq(&scheduler->lock);
+
+	if (not_flying)
+		i915_scheduler_submit(ring);
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_notify_request - Notify the scheduler that the

given
+ * request has completed on the hardware.
+ * @req: Request structure which has completed
+ * @preempt: Did it complete pre-emptively?
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not. The return
+ * value is combined for all freshly completed requests and if any were true
+ * then i915_scheduler_wakeup() is called so the scheduler can do further
+ * processing (submit more work) at the end.
+ */
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+{
+	struct drm_i915_private *dev_priv = to_i915(req->ring-
Quoted 1 line

dev);

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node = req-
Quoted 1 line

scheduler_qe;

drivers/gpu/drm/i915/i915_scheduler.c new file +21 -0
+	unsigned long flags;
+
+	if (!node)
+		return false;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Node was in flight so mark it as complete. */
+	if (req->cancelled)
+		node->status = i915_sqs_dead;
+	else
+		node->status = i915_sqs_complete;
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return true;
+}
+
+static int i915_scheduler_remove_dependent(struct i915_scheduler

*scheduler,

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+				struct

i915_scheduler_queue_entry *remove)

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+{
+	struct i915_scheduler_queue_entry *node;
+	int i, r;
+	int count = 0;
+
+	/*
+	 * Ensure that a node is not being removed which is

still dependent

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * upon other (not completed) work. If that happens, it

implies

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * something has gone very wrong with the dependency

tracking! Note

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * that there is no need to worry if this node has been

explicitly

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * killed for some reason - it might be being killed

before it got

drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+	 * sent to the hardware.
+	 */
+	if (remove->status != i915_sqs_dead) {
+		for (i = 0; i < remove->num_deps; i++)
+			if ((remove->dep_list[i]) &&
+			    (!I915_SQS_IS_COMPLETE(remove-
Quoted 1 line

dep_list[i])))

drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+				count++;
+		WARN_ON(count);
+	}
+
+	/*
+	 * Remove this node from the dependency lists of any

other node which

drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+	 * might be waiting on it.
+	 */
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(node, &scheduler-
Quoted 1 line

node_queue[r], link) {

drivers/gpu/drm/i915/i915_scheduler.c new file +16 -0
+			for (i = 0; i < node->num_deps; i++) {
+				if (node->dep_list[i] != remove)
+					continue;
+
+				node->dep_list[i] = NULL;
+			}
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_wakeup - wake the scheduler's worker thread
+ * @dev: DRM device
+ * Called at the end of seqno interrupt processing if any

request has
+ * completed that corresponds to a scheduler node.
+ */

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+void i915_scheduler_wakeup(struct drm_device *dev)
+{
+	/* XXX: Need to call i915_scheduler_remove() via work

handler. */

drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+}
+
+/**
+ * i915_scheduler_clean_node - free up any

allocations/references
+ * associated with the given scheduler queue entry.
+ * @node: Queue entry structure which is complete
+ * After a give batch buffer completes on the hardware, all the information
+ * required to resubmit it is no longer required. However, the node entry
+ * itself might still be required for tracking purposes for a while longer.
+ * This function should be called as soon as the node is known to be complete
+ * so that these resources may be freed even though the node itself might
+ * hang around.
+ */
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)

drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+{
+	if (!I915_SQS_IS_COMPLETE(node)) {
+		WARN(!node->params.request->cancelled,
+		     "Cleaning active node: %d!\n", node-
Quoted 1 line

status);

Quoted 16 lines
Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+		return;
+	}
+
+	if (node->params.batch_obj) {
+		/*
+		 * The batch buffer must be unpinned before it

is unreferenced

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * otherwise the unpin fails with a missing

vma!?

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+		 */
+		if (node->params.dispatch_flags &

I915_DISPATCH_SECURE)

drivers/gpu/drm/i915/i915_scheduler.c

Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -1
+			i915_gem_execbuff_release_batch_obj(node
->params.batch_obj);
+

This code seems little bit out of place.

Not sure where else you could put it. The code itself is just a copy of
the existing clean up code that was already in the execbuff code path.
The cleanup can no longer be done within the execbuff call due to the
batch buffer execution being arbitrarily delayed by the scheduler. Thus
it is down to the scheduler to do all the necessary cleanup when it knows that the batch buffer is no longer required.

Quoted 16 lines

drivers/gpu/drm/i915/i915_scheduler.c line 739

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		node->params.batch_obj = NULL;
+	}
+
+	/* And anything else owned by the node: */
+	if (node->params.cliprects) {
+		kfree(node->params.cliprects);
+		node->params.cliprects = NULL;
+	}
+}
+

The below function is quite hacky, I understood this will be mitigated
by per-ctx sequence numbering in the future?

To be honest, I think it is already obsolete due to the previous work of
converting from seqno tests everywhere to testing request structures.
The point of the code below was to cope with seqno values popping out of
the hardware in random order due to the re-ordering of batch buffers prior to execution. Whereas, the struct request conversion allowed the
seqno to be late allocated and thus kept in order.

Unfortunately, pre-emption confuses matters again. Although the intention is that seqno values will still be kept in order. It's just
that a given request might have multiple different seqno values assigned
to it throughout its life. The pre-emption code hasn't quite settled down yet though, especially with interesting new hardware planned for
future chips. Thus I don't really want to rip out all of this code quite
yet just in case we do still need it.

As you say, per-ctx seqnos would definitely have an effect here as well.
So yes, long term the intention is to remove all this nastyness. Short
term, it seems best to keep it around until proven unnecessary.

Quoted 104 lines
Quoted 104 lines

+static bool i915_scheduler_remove(struct i915_scheduler *scheduler,

drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+				  struct intel_engine_cs *ring,
+				  struct list_head *remove)
+{
+	struct i915_scheduler_queue_entry *node, *node_next;
+	int flying = 0, queued = 0;
+	bool do_submit;
+	uint32_t min_seqno;
+
+	spin_lock_irq(&scheduler->lock);
+
+	/*
+	 * In the case where the system is idle, starting

'min_seqno' from a big

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * number will cause all nodes to be removed as they are

now back to

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * being in-order. However, this will be a problem if

the last one to

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * complete was actually out-of-order as the ring seqno

value will be

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * lower than one or more completed buffers. Thus code

looking for the

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	 * completion of said buffers will wait forever.
+	 * Instead, use the hardware seqno as the starting

point. This means

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * that some buffers might be kept around even in a

completely idle

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	 * system but it should guarantee that no-one ever gets

confused when

drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	 * waiting for buffer completion.
+	 */
+	min_seqno = ring->get_seqno(ring, true);
+
+	list_for_each_entry(node, &scheduler->node_queue[ring-
Quoted 1 line

id], link) {

drivers/gpu/drm/i915/i915_scheduler.c new file +11 -0
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->params.request->seqno == 0)
+			continue;
+
+		if (!i915_seqno_passed(node->params.request-
Quoted 1 line

seqno, min_seqno))

drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+			min_seqno = node->params.request->seqno;
+	}
+
+	INIT_LIST_HEAD(remove);
+	list_for_each_entry_safe(node, node_next, &scheduler-
Quoted 1 line

node_queue[ring->id], link) {

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+		/*
+		 * Only remove completed nodes which have a

lower seqno than

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * all pending nodes. While there is the

possibility of the

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * ring's seqno counting backwards, all higher

buffers must

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * be remembered so that the

'i915_seqno_passed()' test can

drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+		 * report that they have in fact passed.
+		 *
+		 * NB: This is not true for 'dead' nodes. The

GPU reset causes

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * the software seqno to restart from its

initial value. Thus

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * the dead nodes must be removed even though

their seqno values

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		 * are potentially vastly greater than the

current ring seqno.

drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+		 */
+		if (!I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->status != i915_sqs_dead) {
+			if (i915_seqno_passed(node-
Quoted 1 line

params.request->seqno, min_seqno) &&

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+			    (node->params.request->seqno !=

min_seqno))

drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+				continue;
+		}
+
+		list_del(&node->link);
+		list_add(&node->link, remove);
+
+		/* Strip the dependency info while the mutex is

still locked */

drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+		i915_scheduler_remove_dependent(scheduler,

node);

drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+
+		continue;
+	}
+
+	/*
+	 * Release the interrupt reference count if there are no

longer any

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	 * nodes to worry about.
+	 */
Quoted 8 lines
Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+	if (!flying && !queued &&
+	    (scheduler->flags[ring->id] &

i915_sf_interrupts_enabled)) {

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+		ring->irq_put(ring);
+		scheduler->flags[ring->id] &=

~i915_sf_interrupts_enabled;

Maybe have this piece of code as a helper? To enable interrupts and disable them, maybe even a reference count?

Not sure how much you could abstract out into a helper other than the
two lines of irq_get|put + flag set|clear. The flying|queued test is particular to this one instance. Also, there is no need to reference count as the irq code does that already.

Quoted 53 lines
Quoted 53 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+	}
+
+	/* Launch more packets now? */
+	do_submit = (queued > 0) && (flying < scheduler-
Quoted 1 line

min_flying);

drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+
+	spin_unlock_irq(&scheduler->lock);
+
+	return do_submit;
+}
+
+void i915_scheduler_process_work(struct intel_engine_cs *ring)
+{
+	struct drm_i915_private *dev_priv = ring->dev-
Quoted 1 line

dev_private;

drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	bool do_submit;
+	struct list_head remove;
+
+	if (list_empty(&scheduler->node_queue[ring->id]))
+		return;
+
+	/* Remove completed nodes. */
+	do_submit = i915_scheduler_remove(scheduler, ring,

&remove);

drivers/gpu/drm/i915/i915_scheduler.c new file +15 -0
+
+	if (!do_submit && list_empty(&remove))
+		return;
+
+	/* Need to grab the pm lock outside of the mutex lock */
+	if (do_submit)
+		intel_runtime_pm_get(dev_priv);
+
+	mutex_lock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		i915_scheduler_submit(ring);
+
+	while (!list_empty(&remove)) {
+		node = list_first_entry(&remove, typeof(*node),

link);

drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+		list_del(&node->link);
+
+		/* Free up all the DRM references */
+		i915_scheduler_clean_node(node);
+
+		/* And anything else owned by the node: */
+		node->params.request->scheduler_qe = NULL;
+		i915_gem_request_unreference(node-
Quoted 6 lines

drivers/gpu/drm/i915/i915_scheduler.c line 867

Quoted 3 lines
Quoted 1 line

params.request);

drivers/gpu/drm/i915/i915_scheduler.c new file +2 -0
+		kfree(node->dep_list);
+		kfree(node);

Maybe have a separate helper for freeing a node, would make this much
cleaner.

There is a helper for freeing up the contents of the node - see i915_scheduler_clean_node() above. However, that function is called from
multiple places and this is the only instance where the node itself is
actually freed. The other callers either need to keep the node around
for a while longer (i.e. until this piece of code can run) or don't own
the node because it is stack allocated (in the bypass case). So the 'anything else' section cannot be moved into the existing clean function
and any secondary helper would be single usage and only four lines long.
It doesn't seem worth it.

Quoted 69 lines
Quoted 69 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	}
+
+	mutex_unlock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		intel_runtime_pm_put(dev_priv);
+}
drivers/gpu/drm/i915/i915_scheduler.h new file +4 -0
new file mode 100644
index 0000000..415fec8
@@ -0,0 +1,95 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person

obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */

drivers/gpu/drm/i915/i915_scheduler.h new file +15 -0
+
+#ifndef _I915_SCHEDULER_H_
+#define _I915_SCHEDULER_H_
+
+enum i915_scheduler_queue_status {
+	/* Limbo: */
+	i915_sqs_none = 0,
+	/* Not yet submitted to hardware: */
+	i915_sqs_queued,
+	/* Popped from queue, ready to fly: */
+	i915_sqs_popped,
+	/* Sent to hardware for processing: */
+	i915_sqs_flying,
+	/* Finished processing on the hardware: */
+	i915_sqs_complete,
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.h line 44

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +1 -0
+	/* Killed by watchdog or catastrophic submission

failure: */

drivers/gpu/drm/i915/i915_scheduler.h new file +5 -0
+	i915_sqs_dead,
+	/* Limit value for use with arrays/loops */
+	i915_sqs_MAX
+};
+

To uppercase.

Grrr. Linux kernel style guide is daft.

Quoted 69 lines
Quoted 21 lines

+#define I915_SQS_IS_QUEUED(node) (((node)->status ==
i915_sqs_queued))
+#define I915_SQS_IS_FLYING(node) (((node)->status ==
i915_sqs_flying))
+#define I915_SQS_IS_COMPLETE(node) (((node)->status ==
i915_sqs_complete) || \

drivers/gpu/drm/i915/i915_scheduler.h new file +1 -0
+					 ((node)->status ==

i915_sqs_dead))

drivers/gpu/drm/i915/i915_scheduler.h new file +9 -0
+
+struct i915_scheduler_obj_entry {
+	struct drm_i915_gem_object          *obj;
+	bool                                read_only;
+};
+
+struct i915_scheduler_queue_entry {
+	struct i915_execbuffer_params       params;
+	/* -1023 = lowest priority, 0 = default, 1023 = highest

*/

drivers/gpu/drm/i915/i915_scheduler.h line 66

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +9 -0
+	int32_t                             priority;
+	struct i915_scheduler_obj_entry     *saved_objects;
+	int                                 num_objs;
+	bool                                bumped;
+	struct i915_scheduler_queue_entry   **dep_list;
+	int                                 num_deps;
+	enum i915_scheduler_queue_status    status;
+	unsigned long                       stamp;
+	struct list_head                    link;

Above whitespace is still incorrect and I could really use comments.

Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+};
+
+struct i915_scheduler {
+	struct list_head    node_queue[I915_NUM_RINGS];
+	uint32_t            flags[I915_NUM_RINGS];
+	spinlock_t          lock;
+
+	/* Tuning parameters: */
+	int32_t             priority_level_min;
+	int32_t             priority_level_max;

drivers/gpu/drm/i915/i915_scheduler.h line 86

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	int32_t             priority_level_bump;
+	int32_t             priority_level_preempt;
+	uint32_t            min_flying;
+};
+
+/* Flag bits for i915_scheduler::flags */
+enum {
+	i915_sf_interrupts_enabled  = (1 << 0),
+	i915_sf_submitting          = (1 << 1),
+};

These must be I915_UPPERCASE_SOMETHING by convention.

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +4 -0
+
+bool i915_scheduler_is_enabled(struct drm_device *dev);
+int i915_scheduler_init(struct drm_device *dev);
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry

*node);
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);

drivers/gpu/drm/i915/i915_scheduler.h new file +3 -0
+void i915_scheduler_wakeup(struct drm_device *dev);
+
+#endif  /* _I915_SCHEDULER_H_ */
Signature · 8 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#11 lore raw

PATCH v5 7/35 drm/i915: Prepare retire_requests to handle out-of-order seqnos

In reply to [email protected]

From: John Harrison <[email protected]>

A major point of the GPU scheduler is that it re-orders batch buffers after they have been submitted to the driver. This leads to requests completing out of order. In turn, this means that the retire processing can no longer assume that all completed entries are at the front of the list. Rather than attempting to re-order the request list on a regular basis, it is better to simply scan the entire list.

v2: Removed deferred free code as no longer necessary due to request handling updates.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +13 -18
index 7d9aa24..0003cfc 100644
@@ -3233,6 +3233,7 @@ void i915_gem_reset(struct drm_device *dev)
 void
 i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
 {
+	struct drm_i915_gem_object *obj, *obj_next;
 	struct drm_i915_gem_request *req, *req_next;
 	LIST_HEAD(list_head);
 
@@ -3245,37 +3246,31 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
 	 */
 	i915_gem_request_notify(ring, false);
 
+	/*
+	 * Note that request entries might be out of order due to rescheduling
+	 * and pre-emption. Thus both lists must be processed in their entirety
+	 * rather than stopping at the first non-complete entry.
+	 */
+
 	/* Retire requests first as we use it above for the early return.
 	 * If we retire requests last, we may use a later seqno and so clear
 	 * the requests lists without clearing the active list, leading to
 	 * confusion.
 	 */
-	while (!list_empty(&ring->request_list)) {
-		struct drm_i915_gem_request *request;
-
-		request = list_first_entry(&ring->request_list,
-					   struct drm_i915_gem_request,
-					   list);
-
-		if (!i915_gem_request_completed(request))
-			break;
+	list_for_each_entry_safe(req, req_next, &ring->request_list, list) {
+		if (!i915_gem_request_completed(req))
+			continue;
 
-		i915_gem_request_retire(request);
+		i915_gem_request_retire(req);
 	}
 
 	/* Move any buffers on the active list that are no longer referenced
 	 * by the ringbuffer to the flushing/inactive lists as appropriate,
 	 * before we free the context associated with the requests.
 	 */
-	while (!list_empty(&ring->active_list)) {
-		struct drm_i915_gem_object *obj;
-
-		obj = list_first_entry(&ring->active_list,
-				      struct drm_i915_gem_object,
-				      ring_list[ring->id]);
-
+	list_for_each_entry_safe(obj, obj_next, &ring->active_list, ring_list[ring->id]) {
 		if (!list_empty(&obj->last_read_req[ring->id]->list))
-			break;
+			continue;
 
 		i915_gem_object_retire__read(obj, ring->id);
 	}
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 7/35
#12 lore raw

Re: [PATCH v5 07/35] drm/i915: Prepare retire_requests to handle out-of-order seqnos

Quoted 71 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

A major point of the GPU scheduler is that it re-orders batch buffers after they have been submitted to the driver. This leads to requests completing out of order. In turn, this means that the retire processing can no longer assume that all completed entries are at the front of the list. Rather than attempting to re-order the request list on a regular basis, it is better to simply scan the entire list.

v2: Removed deferred free code as no longer necessary due to request handling updates.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +11 -14
index 7d9aa24..0003cfc 100644
@@ -3233,6 +3233,7 @@ void i915_gem_reset(struct drm_device *dev)
 void
 i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
 {
+	struct drm_i915_gem_object *obj, *obj_next;
 	struct drm_i915_gem_request *req, *req_next;
 	LIST_HEAD(list_head);
 
@@ -3245,37 +3246,31 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
 	 */
 	i915_gem_request_notify(ring, false);
 
+	/*
+	 * Note that request entries might be out of order due to rescheduling
+	 * and pre-emption. Thus both lists must be processed in their entirety
+	 * rather than stopping at the first non-complete entry.
+	 */
+
 	/* Retire requests first as we use it above for the early return.
 	 * If we retire requests last, we may use a later seqno and so clear
 	 * the requests lists without clearing the active list, leading to
 	 * confusion.
 	 */
-	while (!list_empty(&ring->request_list)) {
-		struct drm_i915_gem_request *request;
-
-		request = list_first_entry(&ring->request_list,
-					   struct drm_i915_gem_request,
-					   list);
-
-		if (!i915_gem_request_completed(request))
-			break;
+	list_for_each_entry_safe(req, req_next, &ring->request_list, list) {
+		if (!i915_gem_request_completed(req))
+			continue;
 
-		i915_gem_request_retire(request);
+		i915_gem_request_retire(req);
 	}
 
 	/* Move any buffers on the active list that are no longer referenced
 	 * by the ringbuffer to the flushing/inactive lists as appropriate,
 	 * before we free the context associated with the requests.
 	 */
-	while (!list_empty(&ring->active_list)) {
-		struct drm_i915_gem_object *obj;
-
-		obj = list_first_entry(&ring->active_list,

drivers/gpu/drm/i915/i915_gem.c i915_gem_retire_requests_ring(struct intel_engine_cs *ring) · line 3276

Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +2 -4
-				      struct drm_i915_gem_object,
-				      ring_list[ring->id]);
-
+	list_for_each_entry_safe(obj, obj_next, &ring->active_list, ring_list[ring->id]) {
 		if (!list_empty(&obj->last_read_req[ring->id]->list))
-			break;
+			continue;
 
 		i915_gem_object_retire__read(obj, ring->id);
 	}

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#13 lore raw

PATCH v5 8/35 drm/i915: Disable hardware semaphores when GPU scheduler is enabled

In reply to [email protected]

From: John Harrison <[email protected]>

Hardware sempahores require seqno values to be continuously incrementing. However, the scheduler's reordering of batch buffers means that the seqno values going through the hardware could be out of order. Thus semaphores can not be used.

On the other hand, the scheduler superceeds the need for hardware semaphores anyway. Having one ring stall waiting for something to complete on another ring is inefficient if that ring could be working on some other, independent task. This is what the scheduler is meant to do - keep the hardware as busy as possible by reordering batch buffers to avoid dependency stalls.

v4: Downgraded a BUG_ON to WARN_ON as the latter is preferred.

v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_drv.c         | 9 +++++++++
 drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++
 2 files changed, 13 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +9 -0
index 975af35..5760a17 100644
@@ -34,6 +34,7 @@
 #include "i915_drv.h"
 #include "i915_trace.h"
 #include "intel_drv.h"
+#include "i915_scheduler.h"
 
 #include <linux/console.h>
 #include <linux/module.h>
@@ -517,6 +518,14 @@ void intel_detect_pch(struct drm_device *dev)
 
 bool i915_semaphore_is_enabled(struct drm_device *dev)
 {
+	/* Hardware semaphores are not compatible with the scheduler due to the
+	 * seqno values being potentially out of order. However, semaphores are
+	 * also not required as the scheduler will handle interring dependencies
+	 * and try do so in a way that does not cause dead time on the hardware.
+	 */
+	if (i915_scheduler_is_enabled(dev))
+		return false;
+
 	if (INTEL_INFO(dev)->gen < 6)
 		return false;
 
drivers/gpu/drm/i915/intel_ringbuffer.c +4 -0
index 9d4f19d..ca7b8af 100644
@@ -33,6 +33,7 @@
 #include <drm/i915_drm.h>
 #include "i915_trace.h"
 #include "intel_drv.h"
+#include "i915_scheduler.h"
 
 int __intel_ring_space(int head, int tail, int size)
 {
@@ -1400,6 +1401,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
 	u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];
 	int ret;
 
+	/* Arithmetic on sequence numbers is unreliable with a scheduler. */
+	WARN_ON(i915_scheduler_is_enabled(signaller->dev));
+
 	/* Throughout all of the GEM code, seqno passed implies our current
 	 * seqno is >= the last seqno executed. However for hardware the
 	 * comparison is strictly greater than.
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 8/35
#14 lore raw

Re: [PATCH v5 08/35] drm/i915: Disable hardware semaphores when GPU scheduler is enabled

Quoted 68 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Hardware sempahores require seqno values to be continuously incrementing. However, the scheduler's reordering of batch buffers means that the seqno values going through the hardware could be out of order. Thus semaphores can not be used.

On the other hand, the scheduler superceeds the need for hardware semaphores anyway. Having one ring stall waiting for something to complete on another ring is inefficient if that ring could be working on some other, independent task. This is what the scheduler is meant to do - keep the hardware as busy as possible by reordering batch buffers to avoid dependency stalls.

v4: Downgraded a BUG_ON to WARN_ON as the latter is preferred.

v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_drv.c         | 9 +++++++++
 drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++
 2 files changed, 13 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +9 -0
index 975af35..5760a17 100644
@@ -34,6 +34,7 @@
 #include "i915_drv.h"
 #include "i915_trace.h"
 #include "intel_drv.h"
+#include "i915_scheduler.h"
 
 #include <linux/console.h>
 #include <linux/module.h>
@@ -517,6 +518,14 @@ void intel_detect_pch(struct drm_device *dev)
 
 bool i915_semaphore_is_enabled(struct drm_device *dev)
 {
+	/* Hardware semaphores are not compatible with the scheduler due to the
+	 * seqno values being potentially out of order. However, semaphores are
+	 * also not required as the scheduler will handle interring dependencies
+	 * and try do so in a way that does not cause dead time on the hardware.
+	 */
+	if (i915_scheduler_is_enabled(dev))
+		return false;
+
 	if (INTEL_INFO(dev)->gen < 6)
 		return false;
 
drivers/gpu/drm/i915/intel_ringbuffer.c +1 -0
index 9d4f19d..ca7b8af 100644
@@ -33,6 +33,7 @@
 #include <drm/i915_drm.h>
 #include "i915_trace.h"
 #include "intel_drv.h"
+#include "i915_scheduler.h"
 
 int __intel_ring_space(int head, int tail, int size)
 {

drivers/gpu/drm/i915/intel_ringbuffer.c gen6_ring_sync(struct drm_i915_gem_request *waiter_req, · line 1409

Quoted 9 lines
drivers/gpu/drm/i915/intel_ringbuffer.c +3 -0
@@ -1400,6 +1401,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
 	u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];
 	int ret;
 
+	/* Arithmetic on sequence numbers is unreliable with a scheduler. */
+	WARN_ON(i915_scheduler_is_enabled(signaller->dev));
+
 	/* Throughout all of the GEM code, seqno passed implies our current
 	 * seqno is >= the last seqno executed. However for hardware the
 	 * comparison is strictly greater than.

I'd rather get rid of this altogether, but I guess we'll need it for the older gens. Another option would be to make the sync_to hook NULL in the scheduler case, though I guess the failure mode is less desirable there.

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#15 lore raw

PATCH v5 9/35 drm/i915: Force MMIO flips when scheduler enabled

In reply to [email protected]

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

For: VIZ-1587

---
 drivers/gpu/drm/i915/intel_display.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +3 -0
index 6e12ed7..731d20a 100644
@@ -46,6 +46,7 @@
 #include <linux/dma_remapping.h>
 #include <linux/reservation.h>
 #include <linux/dma-buf.h>
+#include "i915_scheduler.h"
 
 /* Primary plane formats for gen <= 3 */
 static const uint32_t i8xx_primary_formats[] = {
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
 		return true;
 	else if (i915.enable_execlists)
 		return true;
+	else if (i915_scheduler_is_enabled(ring->dev))
+		return true;
 	else if (obj->base.dma_buf &&
 		 !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
 						       false))
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 9/35
#16 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

Quoted 28 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

For: VIZ-1587

---
 drivers/gpu/drm/i915/intel_display.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +1 -0
index 6e12ed7..731d20a 100644
@@ -46,6 +46,7 @@
 #include <linux/dma_remapping.h>
 #include <linux/reservation.h>
 #include <linux/dma-buf.h>
+#include "i915_scheduler.h"
 
 /* Primary plane formats for gen <= 3 */
 static const uint32_t i8xx_primary_formats[] = {

drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338

Quoted 8 lines
drivers/gpu/drm/i915/intel_display.c +2 -0
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
 		return true;
 	else if (i915.enable_execlists)
 		return true;
+	else if (i915_scheduler_is_enabled(ring->dev))
+		return true;
 	else if (obj->base.dma_buf &&
 		 !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
 						       false))

Why can't we use mmio flips unconditionally? Maarten or Ville?

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#17 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

Quoted 25 lines · Jesse Barnes
Quoted 24 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

For: VIZ-1587

---
 drivers/gpu/drm/i915/intel_display.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +1 -0
index 6e12ed7..731d20a 100644
@@ -46,6 +46,7 @@
 #include <linux/dma_remapping.h>
 #include <linux/reservation.h>
 #include <linux/dma-buf.h>
+#include "i915_scheduler.h"
 
Quoted 14 lines

drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338

Quoted 12 lines
drivers/gpu/drm/i915/intel_display.c +2 -0
 /* Primary plane formats for gen <= 3 */
 static const uint32_t i8xx_primary_formats[] = {
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
 		return true;
 	else if (i915.enable_execlists)
 		return true;
+	else if (i915_scheduler_is_enabled(ring->dev))
+		return true;
 	else if (obj->base.dma_buf &&
 		 !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
 						       false))

Why can't we use mmio flips unconditionally? Maarten or Ville?

We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.

For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.

I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.

Signature · 6 lines
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

Quoted 41 lines · Ville Syrjälä
Quoted 25 lines · Jesse Barnes
Quoted 24 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

For: VIZ-1587

---
 drivers/gpu/drm/i915/intel_display.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +1 -0
index 6e12ed7..731d20a 100644
@@ -46,6 +46,7 @@
 #include <linux/dma_remapping.h>
 #include <linux/reservation.h>
 #include <linux/dma-buf.h>
+#include "i915_scheduler.h"
 
Quoted 14 lines

drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338

Quoted 12 lines
drivers/gpu/drm/i915/intel_display.c +2 -0
 /* Primary plane formats for gen <= 3 */
 static const uint32_t i8xx_primary_formats[] = {
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
 		return true;
 	else if (i915.enable_execlists)
 		return true;
+	else if (i915_scheduler_is_enabled(ring->dev))
+		return true;
 	else if (obj->base.dma_buf &&
 		 !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
 						       false))

Why can't we use mmio flips unconditionally? Maarten or Ville?

Quoted 2 lines

We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.

didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).

Quoted 11 lines

For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.

I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.

Yeah that's probably the way to go; I haven't checked the behavior on SKL either.

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#19 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

Hey,

Jesse Barnes schreef op vr 19-02-2016 om 12:01 [-0800]:

Quoted 42 lines · Ville Syrjälä
Quoted 41 lines · Jesse Barnes
Quoted 39 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
Quoted 28 lines

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

For: VIZ-1587

---
 drivers/gpu/drm/i915/intel_display.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +1 -0
index 6e12ed7..731d20a 100644
@@ -46,6 +46,7 @@
 #include <linux/dma_remapping.h>
 #include <linux/reservation.h>
 #include <linux/dma-buf.h>
+#include "i915_scheduler.h"
 
 /* Primary plane formats for gen <= 3 */
 static const uint32_t i8xx_primary_formats[] = {

drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338

Quoted 8 lines
drivers/gpu/drm/i915/intel_display.c +2 -0
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
 		return true;
 	else if (i915.enable_execlists)
 		return true;
+	else if (i915_scheduler_is_enabled(ring->dev))
+		return true;
 	else if (obj->base.dma_buf &&
 		 !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
 						       false))

Why can't we use mmio flips unconditionally? Maarten or Ville?

Quoted 4 lines
Quoted 2 lines

We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.

didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).

I have a patch that would enable support for mmio flips on all platforms, but because of Chris Wilson's objections I still didn't force mmio flips by default.

Quoted 6 lines
Quoted 6 lines

For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.

It was easy to fix in a way similar to that.

Quoted 6 lines
Quoted 4 lines

I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.

Yeah that's probably the way to go; I haven't checked the behavior on SKL either.

---------------------------------------------------------------------
Intel International B.V.
Registered in The Netherlands under number 34098535 Statutory seat: Haarlemmermeer
Registered address: Capronilaan 37, 1119NG Schiphol-Rijk

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#20 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

So is the summary that currently MMIO flips do not work on some platforms?

That could be a problem because the scheduler is intended to be enabled for everything and thus will be forcing MMIO flips on everything. So should I reverse the logic here - only enable the scheduler if MMIO flips are already enabled?

Thanks,
John.

Quoted 51 lines · Maarten

Hey,

Jesse Barnes schreef op vr 19-02-2016 om 12:01 [-0800]:

Quoted 40 lines · Ville Syrjälä
Quoted 39 lines · Jesse Barnes
Quoted 38 lines · On 02/18/2016 06:26 AM, [email protected] wrote:
Quoted 28 lines

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now but more importantly, pipe based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

For: VIZ-1587

---
  drivers/gpu/drm/i915/intel_display.c | 3 +++
  1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +1 -0
index 6e12ed7..731d20a 100644
@@ -46,6 +46,7 @@
  #include <linux/dma_remapping.h>
  #include <linux/reservation.h>
  #include <linux/dma-buf.h>
+#include "i915_scheduler.h"
 
  /* Primary plane formats for gen <= 3 */
  static const uint32_t i8xx_primary_formats[] = {

drivers/gpu/drm/i915/intel_display.c static bool use_mmio_flip(struct intel_engine_cs *ring, · line 11338

Quoted 8 lines
drivers/gpu/drm/i915/intel_display.c +2 -0
@@ -11330,6 +11331,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
  		return true;
  	else if (i915.enable_execlists)
  		return true;
+	else if (i915_scheduler_is_enabled(ring->dev))
+		return true;
  	else if (obj->base.dma_buf &&
  		 !reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
  						       false))

Why can't we use mmio flips unconditionally? Maarten or Ville?

Quoted 3 lines
Quoted 2 lines

We do when execlists are used, which is always on gen9+. So I guess I'm missing the point of this patch. For gen5+ we could also do it trivially.

didn't check if the scheduler is also enabled for gen8 (I guess it would be nice, that would cover BDW and BSW).

I have a patch that would enable support for mmio flips on all platforms, but because of Chris Wilson's objections I still didn't force mmio flips by default.

Quoted 12 lines
Quoted 5 lines
Quoted 5 lines

For older platforms it'd require a bit of work since we'd need to complete the flips from the vblank interrupt. Well, we actually do that already even with CS flips on those platforms, but we do look at the flip pending interrupt to figure out if CS already issued the flip or not. So that part would need changing.

It was easy to fix in a way similar to that.

Quoted 5 lines
Quoted 4 lines

I also think we should switch to using the vblank interrupt for this stuff on all platforms, mainly since the flip done interrupt is somewhat broken on at least BDW (no idea if it got fixed in SKL or later), and doing things in more than one way certainly won't decrease our bug count.

Yeah that's probably the way to go; I haven't checked the behavior on SKL either.

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#21 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

In reply to Jesse Barnes
Quoted 4 lines · Jesse Barnes
Quoted 3 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now

Because introducing variable latency in waking up a big core is a good idea?

Quoted 5 lines

but more importantly, pipe

Quoted 4 lines

based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

No. That is just incredibily bad design.

Quoted 1 line

Why can't we use mmio flips unconditionally? Maarten or Ville?

Why would we want to? CS flips work just fine in execlists and no reason was ever given as to why they were not enabled, just laziness. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#22 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

Quoted 8 lines · Chris Wilson
Quoted 4 lines · Jesse Barnes
Quoted 3 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now

Because introducing variable latency in waking up a big core is a good idea?

Is the latency variability really that bad? I'm not too concerned about waking up the core either, I think it's going to be hit up quite a bit in these situations anyway, and vblanks should be driving the rendering as well.

Quoted 6 lines
Quoted 5 lines

but more importantly, pipe

Quoted 4 lines

based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

Quoted 6 lines

No. That is just incredibily bad design.

Quoted 1 line

Why can't we use mmio flips unconditionally? Maarten or Ville?

Why would we want to? CS flips work just fine in execlists and no reason was ever given as to why they were not enabled, just laziness.

I'm not saying it can't be done, but I thought with atomic we decided to go with mmio because it makes things a lot simpler (iirc Ville's design did that way back?).

I'm fine with either, but it seems like we should settle on one rather than trying to maintain two different ways of flipping. We'll have work to do either way.

Jesse

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#23 lore raw

Re: [PATCH v5 09/35] drm/i915: Force MMIO flips when scheduler enabled

Quoted 11 lines · Jesse Barnes
Quoted 8 lines · Chris Wilson
Quoted 4 lines · Jesse Barnes
Quoted 3 lines · On 02/18/2016 06:26 AM, [email protected] wrote:

From: John Harrison <[email protected]>

MMIO flips are the preferred mechanism now

Because introducing variable latency in waking up a big core is a good idea?

Is the latency variability really that bad? I'm not too concerned about waking up the core either, I think it's going to be hit up quite a bit in these situations anyway, and vblanks should be driving the rendering as well.

The workqueue we use is subject to being preempted by the user, so the latency can be made arbitrarily large. Also, we don't control what tasks are ahead of the flip.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

but more importantly, pipe

Quoted 4 lines

based flips cause issues for the scheduler. Specifically, submitting work to the rings around the side of the scheduler could cause that work to be lost if the scheduler generates a pre-emption event on that ring.

Quoted 8 lines
Quoted 6 lines

No. That is just incredibily bad design.

Quoted 1 line

Why can't we use mmio flips unconditionally? Maarten or Ville?

Why would we want to? CS flips work just fine in execlists and no reason was ever given as to why they were not enabled, just laziness.

I'm not saying it can't be done, but I thought with atomic we decided to go with mmio because it makes things a lot simpler (iirc Ville's design did that way back?).

My point is that the interface to the scheduler excludes even the possibility of doing so because it is not cleanly segrated into a tool for determining the next request to execute but instead subsumes half of GEM and half of the engine backend becoming another unwanted midlayer incorporating its own policy. The scheduler doesn't even deal in requests, which is what prevents us from constructing a request here with the known dependency and passing it to the scheduler for execution.

As I saw it, CS flips would be one of the best demonstrations for a scheduler - because to execute the flip within the deadline should cause a priority boost for all the prerequisites, and then ideally reduce the number of missed target msc under load. (Though we can equally promote the last_write_request through another scheduler interface.)
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#24 lore raw

PATCH v5 10/35 drm/i915: Added scheduler hook when closing DRM file handles

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to close its DRM file handle while there is still work outstanding. That means the scheduler needs to know about file close events so it can remove the file pointer from such orphaned batch buffers and not attempt to dereference it later.

v3: Updated to not wait for outstanding work to complete but merely remove the file handle reference. The wait was getting excessively complicated with inter-ring dependencies, pre-emption, and other such issues.

v4: Changed some white space to keep the style checker happy.

v5: Added function documentation and removed apparently objectionable white space. [Joonas Lahtinen]

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_dma.c       |  3 +++
 drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  2 ++
 3 files changed, 53 insertions(+)
drivers/gpu/drm/i915/i915_dma.c +3 -0
index a0f5659..678adc7 100644
@@ -46,6 +46,7 @@
 #include <linux/vga_switcheroo.h>
 #include <linux/slab.h>
 #include <acpi/video.h>
+#include "i915_scheduler.h"
 #include <linux/pm.h>
 #include <linux/pm_runtime.h>
 #include <linux/oom.h>
@@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)
 
 void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
 {
+	i915_scheduler_closefile(dev, file);
+
 	mutex_lock(&dev->struct_mutex);
 	i915_gem_context_close(dev, file);
 	i915_gem_release(dev, file);
drivers/gpu/drm/i915/i915_scheduler.c +48 -0

51 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 415fec8..0e8b6a9 100644
@@ -87,6 +87,8 @@ enum {
 
 bool i915_scheduler_is_enabled(struct drm_device *dev);
 int i915_scheduler_init(struct drm_device *dev);
+int i915_scheduler_closefile(struct drm_device *dev,
+			     struct drm_file *file);
 void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
 int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer Reviewed-by on v5 10/35
#25 lore raw

Re: [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles

Quoted 46 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to close its DRM file handle while there is still work outstanding. That means the scheduler needs to know about file close events so it can remove the file pointer from such orphaned batch buffers and not attempt to dereference it later.

v3: Updated to not wait for outstanding work to complete but merely remove the file handle reference. The wait was getting excessively complicated with inter-ring dependencies, pre-emption, and other such issues.

v4: Changed some white space to keep the style checker happy.

v5: Added function documentation and removed apparently objectionable white space. [Joonas Lahtinen]

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>

---
 drivers/gpu/drm/i915/i915_dma.c       |  3 +++ drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h |  2 ++

 3 files changed, 53 insertions(+)
drivers/gpu/drm/i915/i915_dma.c +1 -0
index a0f5659..678adc7 100644
@@ -46,6 +46,7 @@
  #include 
  #include 
  #include 
+#include "i915_scheduler.h"
  #include 
  #include 
  #include 
@@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)


 void i915_driver_preclose(struct drm_device *dev, struct drm_file *file) {

drivers/gpu/drm/i915/i915_dma.c void i915_driver_lastclose(struct drm_device *dev) · line 1262

Quoted 1 line
drivers/gpu/drm/i915/i915_dma.c +1 -0
+	i915_scheduler_closefile(dev, file);

Any reason not to put this inside the struct_mutex lock?

Quoted 18 lines

+
  mutex_lock(&dev->struct_mutex);
  i915_gem_context_close(dev, file);
  i915_gem_release(dev, file);

drivers/gpu/drm/i915/i915_scheduler.c +6 -0
index fc23ee7..ab5007a 100644
@@ -872,3 +872,51 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
  	if (do_submit)
  		intel_runtime_pm_put(dev_priv);
  }
+
+/**
+ * i915_scheduler_closefile - notify the scheduler that a DRM file handle
+ * has been closed.
+ * @dev: DRM device
+ * @file: file being closed

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 890

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+ *
+ * Goes through the scheduler's queues and removes all connections to the
+ * disappearing file handle that still exist. There is an argument to say
+ * that this should also flush such outstanding work through the hardware.
+ * However, with pre-emption, TDR and other such complications doing so
+ * becomes a locking nightmare. So instead, just warn with a debug message
+ * if the application is leaking uncompleted work and make sure a null
+ * pointer dereference will not follow.
+ */
+int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)

Return value not used, should be void if this can not fail.

Other than that,

Quoted 45 lines
drivers/gpu/drm/i915/i915_scheduler.c +32 -0
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring;
+	int i;
+
+	if (!scheduler)
+		return 0;
+
+	spin_lock_irq(&scheduler->lock);
+
+	for_each_ring(ring, dev_priv, i) {
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (node->params.file != file)
+				continue;
+
+			if (!I915_SQS_IS_COMPLETE(node))
+				DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+						 node->params.request->uniq,
+						 node->params.request->seqno,
+						 node->status,
+						 ring->name);
+
+			node->params.file = NULL;
+		}
+	}
+
+	spin_unlock_irq(&scheduler->lock);
+
+	return 0;
+}
drivers/gpu/drm/i915/i915_scheduler.h +0 -0
index 415fec8..0e8b6a9 100644
@@ -87,6 +87,8 @@ enum {


 bool i915_scheduler_is_enabled(struct drm_device *dev); int i915_scheduler_init(struct drm_device *dev);

drivers/gpu/drm/i915/i915_scheduler.h +2 -0
+int i915_scheduler_closefile(struct drm_device *dev,
+			     struct drm_file *file);

 void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node); int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe); bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#26 lore raw

Re: [PATCH v5 10/35] drm/i915: Added scheduler hook when closing DRM file handles

Quoted 45 lines · Joonas Lahtinen
Quoted 44 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to close its DRM file handle while there is still work outstanding. That means the scheduler needs to know about file close events so it can remove the file pointer from such orphaned batch buffers and not attempt to dereference it later.

v3: Updated to not wait for outstanding work to complete but merely remove the file handle reference. The wait was getting excessively complicated with inter-ring dependencies, pre-emption, and other such issues.

v4: Changed some white space to keep the style checker happy.

v5: Added function documentation and removed apparently objectionable white space. [Joonas Lahtinen]

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
  drivers/gpu/drm/i915/i915_dma.c       |  3 +++
  drivers/gpu/drm/i915/i915_scheduler.c | 48 +++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/i915_scheduler.h |  2 ++
  3 files changed, 53 insertions(+)
drivers/gpu/drm/i915/i915_dma.c +1 -0
index a0f5659..678adc7 100644
@@ -46,6 +46,7 @@
  #include
  #include
  #include
+#include "i915_scheduler.h"
  #include
  #include
  #include
@@ -1258,6 +1259,8 @@ void i915_driver_lastclose(struct drm_device *dev)
 
Quoted 4 lines

drivers/gpu/drm/i915/i915_dma.c void i915_driver_lastclose(struct drm_device *dev) · line 1262

Quoted 3 lines
drivers/gpu/drm/i915/i915_dma.c +1 -0
  void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
  {
+	i915_scheduler_closefile(dev, file);

Any reason not to put this inside the struct_mutex lock?

Any reason to put it inside? The smaller the processing done while holding any given lock the better. The implementation only works on scheduler internal data structures so while it does require the scheduler's lock, it does not require the driver lock.

Quoted 22 lines
Quoted 22 lines
drivers/gpu/drm/i915/i915_dma.c +1 -0
+
  	mutex_lock(&dev->struct_mutex);
  	i915_gem_context_close(dev, file);
  	i915_gem_release(dev, file);
drivers/gpu/drm/i915/i915_scheduler.c +9 -0
index fc23ee7..ab5007a 100644
@@ -872,3 +872,51 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
  	if (do_submit)
  		intel_runtime_pm_put(dev_priv);
  }
+
+/**
+ * i915_scheduler_closefile - notify the scheduler that a DRM file handle
+ * has been closed.
+ * @dev: DRM device
+ * @file: file being closed
+ *
+ * Goes through the scheduler's queues and removes all connections to the
+ * disappearing file handle that still exist. There is an argument to say
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 890

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c +7 -0
+ * that this should also flush such outstanding work through the hardware.
+ * However, with pre-emption, TDR and other such complications doing so
+ * becomes a locking nightmare. So instead, just warn with a debug message
+ * if the application is leaking uncompleted work and make sure a null
+ * pointer dereference will not follow.
+ */
+int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)

Return value not used, should be void if this can not fail.

Left over from a previous implementation that was much more complicated.

Quoted 42 lines

Other than that,

Quoted 38 lines
drivers/gpu/drm/i915/i915_scheduler.c +32 -0
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring;
+	int i;
+
+	if (!scheduler)
+		return 0;
+
+	spin_lock_irq(&scheduler->lock);
+
+	for_each_ring(ring, dev_priv, i) {
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (node->params.file != file)
+				continue;
+
+			if (!I915_SQS_IS_COMPLETE(node))
+				DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+						 node->params.request->uniq,
+						 node->params.request->seqno,
+						 node->status,
+						 ring->name);
+
+			node->params.file = NULL;
+		}
+	}
+
+	spin_unlock_irq(&scheduler->lock);
+
+	return 0;
+}
drivers/gpu/drm/i915/i915_scheduler.h +0 -0
index 415fec8..0e8b6a9 100644
@@ -87,6 +87,8 @@ enum {
 
Quoted 7 lines

drivers/gpu/drm/i915/i915_scheduler.h enum { · line 94

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
  bool i915_scheduler_is_enabled(struct drm_device *dev);
  int i915_scheduler_init(struct drm_device *dev);
+int i915_scheduler_closefile(struct drm_device *dev,
+			     struct drm_file *file);
  void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
  int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
  bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#27 lore raw

PATCH v5 11/35 drm/i915: Added scheduler hook into i915_gem_request_notify()

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler needs to know when requests have completed so that it can keep its own internal state up to date and can submit new requests to the hardware from its queue.

v2: Updated due to changes in request handling. The operation is now reversed from before. Rather than the scheduler being in control of completion events, it is now the request code itself. The scheduler merely receives a notification event. It can then optionally request it's worker thread be woken up after all completion processing is complete.

v4: Downgraded a BUG_ON to a WARN_ON as the latter is preferred.

v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_gem.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +16 -0
index 0003cfc..c3b7def 100644
@@ -2872,6 +2872,7 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 {
 	struct drm_i915_gem_request *req, *req_next;
 	unsigned long flags;
+	bool wake_sched = false;
 	u32 seqno;
 
 	if (list_empty(&ring->fence_signal_list)) {
@@ -2908,6 +2909,14 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 		 */
 		list_del_init(&req->signal_link);
 
+		/*
+		 * NB: Must notify the scheduler before signalling
+		 * the node. Otherwise the node can get retired first
+		 * and call scheduler_clean() while the scheduler
+		 * thinks it is still active.
+		 */
+		wake_sched |= i915_scheduler_notify_request(req);
+
 		if (!req->cancelled) {
 			fence_signal_locked(&req->fence);
 			trace_i915_gem_request_complete(req);
@@ -2924,6 +2933,13 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 
 	if (!fence_locked)
 		spin_unlock_irqrestore(&ring->fence_lock, flags);
+
+	/* Necessary? Or does the fence_signal() call do an implicit wakeup? */
+	wake_up_all(&ring->irq_queue);
+
+	/* Final scheduler processing after all individual updates are done. */
+	if (wake_sched)
+		i915_scheduler_wakeup(ring->dev);
 }
 
 static const char *i915_gem_request_get_driver_name(struct fence *req_fence)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#28 lore raw

Re: [PATCH v5 11/35] drm/i915: Added scheduler hook into i915_gem_request_notify()

Quoted 41 lines · On to, 2016-02-18 at 14:26 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler needs to know when requests have completed so that it can keep its own internal state up to date and can submit new requests to the hardware from its queue.

v2: Updated due to changes in request handling. The operation is now reversed from before. Rather than the scheduler being in control of completion events, it is now the request code itself. The scheduler merely receives a notification event. It can then optionally request it's worker thread be woken up after all completion processing is complete.

v4: Downgraded a BUG_ON to a WARN_ON as the latter is preferred.

v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>

---
 drivers/gpu/drm/i915/i915_gem.c | 16 ++++++++++++++++

 1 file changed, 16 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +1 -0
index 0003cfc..c3b7def 100644
@@ -2872,6 +2872,7 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
  {
  	struct drm_i915_gem_request *req, *req_next;
  	unsigned long flags;
+	bool wake_sched = false;
  	u32 seqno;


  if (list_empty(&ring->fence_signal_list)) {

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -2908,6 +2909,14 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
  		 */
  		list_del_init(&req->signal_link);

drivers/gpu/drm/i915/i915_gem.c void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked) · line 2918

Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7 -0
+		/*
+		 * NB: Must notify the scheduler before signalling
+		 * the node. Otherwise the node can get retired first
+		 * and call scheduler_clean() while the scheduler
+		 * thinks it is still active.
+		 */
+		wake_sched |= i915_scheduler_notify_request(req);
		if (i915_scheduler_notify_request(req))
			wake_sched = true;

drivers/gpu/drm/i915/i915_gem.c void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked) · line 2938

Quoted 11 lines

+
  if (!req->cancelled) { fence_signal_locked(&req->fence); trace_i915_gem_request_complete(req);

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -2924,6 +2933,13 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)


  if (!fence_locked) spin_unlock_irqrestore(&ring->fence_lock, flags);

drivers/gpu/drm/i915/i915_gem.c +3 -0
+
+	/* Necessary? Or does the fence_signal() call do an implicit wakeup? */
+	wake_up_all(&ring->irq_queue);

This should probably be figured out after the struct fence series is merged.

Regards, Joonas

Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +4 -0
+
+	/* Final scheduler processing after all individual updates are done. */
+	if (wake_sched)
+		i915_scheduler_wakeup(ring->dev);

 }

 static const char *i915_gem_request_get_driver_name(struct fence *req_fence)

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#29 lore raw

PATCH v5 12/35 drm/i915: Added deferred work handler for scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler needs to do interrupt triggered work that is too complex to do in the interrupt handler. Thus it requires a deferred work handler to process such tasks asynchronously.

v2: Updated to reduce mutex lock usage. The lock is now only held for the minimum time within the remove function rather than for the whole of the worker thread's operation.

v5: Removed objectionable white space and added some documentation. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_dma.c       |  3 +++
 drivers/gpu/drm/i915/i915_drv.h       | 10 ++++++++++
 drivers/gpu/drm/i915/i915_gem.c       |  2 ++
 drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 5 files changed, 43 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_dma.c +3 -0
index 678adc7..c3d382d 100644
@@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)
 	WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
 	unregister_shrinker(&dev_priv->mm.shrinker);
 
+	/* Cancel the scheduler work handler, which should be idle now. */
+	cancel_work_sync(&dev_priv->mm.scheduler_work);
+
 	io_mapping_free(dev_priv->gtt.mappable);
 	arch_phys_wc_del(dev_priv->gtt.mtrr);
 
drivers/gpu/drm/i915/i915_drv.h +10 -0
index 03add1a..4d544f1 100644
@@ -1291,6 +1291,16 @@ struct i915_gem_mm {
 	struct delayed_work retire_work;
 
 	/**
+	 * New scheme is to get an interrupt after every work packet
+	 * in order to allow the low latency scheduling of pending
+	 * packets. The idea behind adding new packets to a pending
+	 * queue rather than directly into the hardware ring buffer
+	 * is to allow high priority packets to over take low priority
+	 * ones.
+	 */
+	struct work_struct scheduler_work;
+
+	/**
 	 * When we detect an idle GPU, we want to turn on
 	 * powersaving features. So once we see that there
 	 * are no more requests outstanding and no more
drivers/gpu/drm/i915/i915_gem.c +2 -0
index c3b7def..1ab7256 100644
@@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev)
 			  i915_gem_retire_work_handler);
 	INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
 			  i915_gem_idle_work_handler);
+	INIT_WORK(&dev_priv->mm.scheduler_work,
+				i915_scheduler_work_handler);
 	init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
 
 	dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
drivers/gpu/drm/i915/i915_scheduler.c +27 -2
index ab5007a..3986890 100644
@@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
  */
 void i915_scheduler_wakeup(struct drm_device *dev)
 {
-	/* XXX: Need to call i915_scheduler_remove() via work handler. */
+	struct drm_i915_private *dev_priv = to_i915(dev);
+
+	queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
 }
 
 /**
@@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
 	return do_submit;
 }
 
-void i915_scheduler_process_work(struct intel_engine_cs *ring)
+static void i915_scheduler_process_work(struct intel_engine_cs *ring)
 {
 	struct drm_i915_private *dev_priv = ring->dev->dev_private;
 	struct i915_scheduler *scheduler = dev_priv->scheduler;
@@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
 }
 
 /**
+ * i915_scheduler_work_handler - scheduler's work handler callback.
+ * @work: Work structure
+ * A lot of the scheduler's work must be done asynchronously in response to
+ * an interrupt or other event. However, that work cannot be done at
+ * interrupt time or in the context of the event signaller (which might in
+ * fact be an interrupt). Thus a worker thread is required. This function
+ * will cause the thread to wake up and do its processing.
+ */
+void i915_scheduler_work_handler(struct work_struct *work)
+{
+	struct intel_engine_cs *ring;
+	struct drm_i915_private *dev_priv;
+	struct drm_device *dev;
+	int i;
+
+	dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
+	dev = dev_priv->dev;
+
+	for_each_ring(ring, dev_priv, i)
+		i915_scheduler_process_work(ring);
+}
+
+/**
  * i915_scheduler_closefile - notify the scheduler that a DRM file handle
  * has been closed.
  * @dev: DRM device
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 0e8b6a9..180d75f 100644
@@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
 int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void i915_scheduler_wakeup(struct drm_device *dev);
+void i915_scheduler_work_handler(struct work_struct *work);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer Reviewed-by on v5 12/35
#30 lore raw

Re: [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler

Quoted 91 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler needs to do interrupt triggered work that is too complex to do in the interrupt handler. Thus it requires a deferred work handler to process such tasks asynchronously.

v2: Updated to reduce mutex lock usage. The lock is now only held for the minimum time within the remove function rather than for the whole of the worker thread's operation.

v5: Removed objectionable white space and added some documentation. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>

---

 drivers/gpu/drm/i915/i915_dma.c       |  3 +++
 drivers/gpu/drm/i915/i915_drv.h       | 10 ++++++++++
 drivers/gpu/drm/i915/i915_gem.c       |  2 ++
 drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 5 files changed, 43 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_dma.c +0 -0
index 678adc7..c3d382d 100644
@@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)
  	WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
  	unregister_shrinker(&dev_priv->mm.shrinker);

drivers/gpu/drm/i915/i915_dma.c +3 -0
+	/* Cancel the scheduler work handler, which should be idle now. */
+	cancel_work_sync(&dev_priv->mm.scheduler_work);
+

  io_mapping_free(dev_priv->gtt.mappable);
  arch_phys_wc_del(dev_priv->gtt.mtrr);

drivers/gpu/drm/i915/i915_drv.h +0 -0
index 03add1a..4d544f1 100644
@@ -1291,6 +1291,16 @@ struct i915_gem_mm {
  	struct delayed_work retire_work;


  /**

drivers/gpu/drm/i915/i915_drv.h +10 -0
+	 * New scheme is to get an interrupt after every work packet
+	 * in order to allow the low latency scheduling of pending
+	 * packets. The idea behind adding new packets to a pending
+	 * queue rather than directly into the hardware ring buffer
+	 * is to allow high priority packets to over take low priority
+	 * ones.
+	 */
+	struct work_struct scheduler_work;
+
+	/**

   * When we detect an idle GPU, we want to turn on
   * powersaving features. So once we see that there
   * are no more requests outstanding and no more

drivers/gpu/drm/i915/i915_gem.c +2 -0
index c3b7def..1ab7256 100644
@@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev)
  			  i915_gem_retire_work_handler);
  	INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
  			  i915_gem_idle_work_handler);
+	INIT_WORK(&dev_priv->mm.scheduler_work,
+				i915_scheduler_work_handler);
  	init_waitqueue_head(&dev_priv->gpu_error.reset_queue);


  dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;

drivers/gpu/drm/i915/i915_scheduler.c +3 -1
index ab5007a..3986890 100644
@@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
   */
  void i915_scheduler_wakeup(struct drm_device *dev)
  {
-	/* XXX: Need to call i915_scheduler_remove() via work handler. */
+	struct drm_i915_private *dev_priv = to_i915(dev);
+
+	queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
  }


 /**

drivers/gpu/drm/i915/i915_scheduler.c +0 -0
@@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
  	return do_submit;
  }

drivers/gpu/drm/i915/i915_scheduler.c static bool i915_scheduler_remove(struct i915_scheduler *scheduler, · line 832

Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.c +1 -1
-void i915_scheduler_process_work(struct intel_engine_cs *ring)
+static void i915_scheduler_process_work(struct intel_engine_cs *ring)

This is an odd change, maybe should have been static void to begin with?

Quoted 15 lines

 {
  struct drm_i915_private *dev_priv = ring->dev->dev_private; struct i915_scheduler *scheduler = dev_priv->scheduler;

drivers/gpu/drm/i915/i915_scheduler.c +0 -0
@@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
  }


 /**
+ * i915_scheduler_work_handler - scheduler's work handler callback.
+ * @work: Work structure
+ * A lot of the scheduler's work must be done asynchronously in response to
+ * an interrupt or other event. However, that work cannot be done at
+ * interrupt time or in the context of the event signaller (which might in
+ * fact be an interrupt). Thus a worker thread is required. This function
+ * will cause the thread to wake up and do its processing.
+ */

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 895

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c +9 -0
+void i915_scheduler_work_handler(struct work_struct *work)
+{
+	struct intel_engine_cs *ring;
+	struct drm_i915_private *dev_priv;
+	struct drm_device *dev;
+	int i;
+
+	dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
+	dev = dev_priv->dev;

dev is unused variable?

With those changes,

Quoted 20 lines
drivers/gpu/drm/i915/i915_scheduler.c +6 -0
+
+	for_each_ring(ring, dev_priv, i)
+		i915_scheduler_process_work(ring);
+}
+
+/**

  * i915_scheduler_closefile - notify the scheduler that a DRM file handle
  * has been closed.
  * @dev: DRM device

drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 0e8b6a9..180d75f 100644
@@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
  int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
  bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
  void i915_scheduler_wakeup(struct drm_device *dev);
+void i915_scheduler_work_handler(struct work_struct *work);


 #endif  /* _I915_SCHEDULER_H_ */

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#31 lore raw

Re: [PATCH v5 12/35] drm/i915: Added deferred work handler for scheduler

Quoted 92 lines · Joonas Lahtinen
Quoted 91 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler needs to do interrupt triggered work that is too complex to do in the interrupt handler. Thus it requires a deferred work handler to process such tasks asynchronously.

v2: Updated to reduce mutex lock usage. The lock is now only held for the minimum time within the remove function rather than for the whole of the worker thread's operation.

v5: Removed objectionable white space and added some documentation. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
  drivers/gpu/drm/i915/i915_dma.c       |  3 +++
  drivers/gpu/drm/i915/i915_drv.h       | 10 ++++++++++
  drivers/gpu/drm/i915/i915_gem.c       |  2 ++
  drivers/gpu/drm/i915/i915_scheduler.c | 29 +++++++++++++++++++++++++++--
  drivers/gpu/drm/i915/i915_scheduler.h |  1 +
  5 files changed, 43 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_dma.c +3 -0
index 678adc7..c3d382d 100644
@@ -1158,6 +1158,9 @@ int i915_driver_unload(struct drm_device *dev)
  	WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
  	unregister_shrinker(&dev_priv->mm.shrinker);
 
+	/* Cancel the scheduler work handler, which should be idle now. */
+	cancel_work_sync(&dev_priv->mm.scheduler_work);
+
  	io_mapping_free(dev_priv->gtt.mappable);
  	arch_phys_wc_del(dev_priv->gtt.mtrr);
 
drivers/gpu/drm/i915/i915_drv.h +10 -0
index 03add1a..4d544f1 100644
@@ -1291,6 +1291,16 @@ struct i915_gem_mm {
  	struct delayed_work retire_work;
 
  	/**
+	 * New scheme is to get an interrupt after every work packet
+	 * in order to allow the low latency scheduling of pending
+	 * packets. The idea behind adding new packets to a pending
+	 * queue rather than directly into the hardware ring buffer
+	 * is to allow high priority packets to over take low priority
+	 * ones.
+	 */
+	struct work_struct scheduler_work;
+
+	/**
  	 * When we detect an idle GPU, we want to turn on
  	 * powersaving features. So once we see that there
  	 * are no more requests outstanding and no more
drivers/gpu/drm/i915/i915_gem.c +2 -0
index c3b7def..1ab7256 100644
@@ -5427,6 +5427,8 @@ i915_gem_load(struct drm_device *dev)
  			  i915_gem_retire_work_handler);
  	INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
  			  i915_gem_idle_work_handler);
+	INIT_WORK(&dev_priv->mm.scheduler_work,
+				i915_scheduler_work_handler);
  	init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
 
  	dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
drivers/gpu/drm/i915/i915_scheduler.c +3 -1
index ab5007a..3986890 100644
@@ -697,7 +697,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
   */
  void i915_scheduler_wakeup(struct drm_device *dev)
  {
-	/* XXX: Need to call i915_scheduler_remove() via work handler. */
+	struct drm_i915_private *dev_priv = to_i915(dev);
+
+	queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
  }
 
  /**
@@ -827,7 +829,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
  	return do_submit;
  }
 
Quoted 4 lines

drivers/gpu/drm/i915/i915_scheduler.c static bool i915_scheduler_remove(struct i915_scheduler *scheduler, · line 832

Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.c +1 -1
-void i915_scheduler_process_work(struct intel_engine_cs *ring)
+static void i915_scheduler_process_work(struct intel_engine_cs *ring)

This is an odd change, maybe should have been static void to begin with?

The problems of introducing code in patch A and plumbing it to the driver in patch B. Making it static from the start gives an unused function compiler warning.

Quoted 49 lines
Quoted 14 lines
drivers/gpu/drm/i915/i915_scheduler.c +7 -0
  {
  	struct drm_i915_private *dev_priv = ring->dev->dev_private;
  	struct i915_scheduler *scheduler = dev_priv->scheduler;
@@ -874,6 +876,29 @@ void i915_scheduler_process_work(struct intel_engine_cs *ring)
  }
 
  /**
+ * i915_scheduler_work_handler - scheduler's work handler callback.
+ * @work: Work structure
+ * A lot of the scheduler's work must be done asynchronously in response to
+ * an interrupt or other event. However, that work cannot be done at
+ * interrupt time or in the context of the event signaller (which might in
+ * fact be an interrupt). Thus a worker thread is required. This function
+ * will cause the thread to wake up and do its processing.

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_process_work(struct intel_engine_cs *ring) · line 895

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+ */
+void i915_scheduler_work_handler(struct work_struct *work)
+{
+	struct intel_engine_cs *ring;
+	struct drm_i915_private *dev_priv;
+	struct drm_device *dev;
+	int i;
+
+	dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
+	dev = dev_priv->dev;

dev is unused variable?

With those changes,

Quoted 19 lines
drivers/gpu/drm/i915/i915_scheduler.c +6 -0
+
+	for_each_ring(ring, dev_priv, i)
+		i915_scheduler_process_work(ring);
+}
+
+/**
   * i915_scheduler_closefile - notify the scheduler that a DRM file handle
   * has been closed.
   * @dev: DRM device
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 0e8b6a9..180d75f 100644
@@ -93,5 +93,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
  int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
  bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
  void i915_scheduler_wakeup(struct drm_device *dev);
+void i915_scheduler_work_handler(struct work_struct *work);
 
Quoted 1 line

drivers/gpu/drm/i915/i915_scheduler.h void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node); · line 98

Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.h +0 -0
  #endif  /* _I915_SCHEDULER_H_ */

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#32 lore raw

PATCH v5 13/35 drm/i915: Redirect execbuffer_final() via scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

Updated the execbuffer() code to pass the packaged up batch buffer information to the scheduler rather than calling execbuffer_final() directly. The scheduler queue() code is currently a stub which simply chains on to _final() immediately.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++-----------
 drivers/gpu/drm/i915/intel_lrc.c           | 12 ++++--------
 2 files changed, 11 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -11
index 7978dae..09c5ce9 100644
@@ -33,6 +33,7 @@
 #include "intel_drv.h"
 #include <linux/dma_remapping.h>
 #include <linux/uaccess.h>
+#include "i915_scheduler.h"
 
 #define  __EXEC_OBJECT_HAS_PIN (1<<31)
 #define  __EXEC_OBJECT_HAS_FENCE (1<<30)
@@ -1226,6 +1227,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas)
 {
+	struct i915_scheduler_queue_entry *qe;
 	struct drm_device *dev = params->dev;
 	struct intel_engine_cs *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -1270,17 +1272,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	ret = dev_priv->gt.execbuf_final(params);
+	qe = container_of(params, typeof(*qe), params);
+	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
 		return ret;
 
-	/*
-	 * Free everything that was stored in the QE structure (until the
-	 * scheduler arrives and does it instead):
-	 */
-	if (params->dispatch_flags & I915_DISPATCH_SECURE)
-		i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
 	return 0;
 }
 
@@ -1420,8 +1416,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	struct intel_engine_cs *ring;
 	struct intel_context *ctx;
 	struct i915_address_space *vm;
-	struct i915_execbuffer_params params_master; /* XXX: will be removed later */
-	struct i915_execbuffer_params *params = &params_master;
+	struct i915_scheduler_queue_entry qe;
+	struct i915_execbuffer_params *params = &qe.params;
 	const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
 	u32 dispatch_flags;
 	int ret;
@@ -1529,7 +1525,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	else
 		vm = &dev_priv->gtt.base;
 
-	memset(&params_master, 0x00, sizeof(params_master));
+	memset(&qe, 0x00, sizeof(qe));
 
 	eb = eb_create(args);
 	if (eb == NULL) {
drivers/gpu/drm/i915/intel_lrc.c +4 -8
index 12e8949..ff4565f 100644
@@ -136,6 +136,7 @@
 #include <drm/i915_drm.h>
 #include "i915_drv.h"
 #include "intel_mocs.h"
+#include "i915_scheduler.h"
 
 #define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
 #define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -910,6 +911,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas)
 {
+	struct i915_scheduler_queue_entry *qe;
 	struct drm_device       *dev = params->dev;
 	struct intel_engine_cs  *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -952,17 +954,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	ret = dev_priv->gt.execbuf_final(params);
+	qe = container_of(params, typeof(*qe), params);
+	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
 		return ret;
 
-	/*
-	 * Free everything that was stored in the QE structure (until the
-	 * scheduler arrives and does it instead):
-	 */
-	if (params->dispatch_flags & I915_DISPATCH_SECURE)
-		i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
 	return 0;
 }
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 13/35
#33 lore raw

Re: [PATCH v5 13/35] drm/i915: Redirect execbuffer_final() via scheduler

Quoted 105 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Updated the execbuffer() code to pass the packaged up batch buffer information to the scheduler rather than calling execbuffer_final() directly. The scheduler queue() code is currently a stub which simply chains on to _final() immediately.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++-----------
 drivers/gpu/drm/i915/intel_lrc.c           | 12 ++++--------
 2 files changed, 11 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -11
index 7978dae..09c5ce9 100644
@@ -33,6 +33,7 @@
 #include "intel_drv.h"
 #include <linux/dma_remapping.h>
 #include <linux/uaccess.h>
+#include "i915_scheduler.h"
 
 #define  __EXEC_OBJECT_HAS_PIN (1<<31)
 #define  __EXEC_OBJECT_HAS_FENCE (1<<30)
@@ -1226,6 +1227,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas)
 {
+	struct i915_scheduler_queue_entry *qe;
 	struct drm_device *dev = params->dev;
 	struct intel_engine_cs *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -1270,17 +1272,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	ret = dev_priv->gt.execbuf_final(params);
+	qe = container_of(params, typeof(*qe), params);
+	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
 		return ret;
 
-	/*
-	 * Free everything that was stored in the QE structure (until the
-	 * scheduler arrives and does it instead):
-	 */
-	if (params->dispatch_flags & I915_DISPATCH_SECURE)
-		i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
 	return 0;
 }
 
@@ -1420,8 +1416,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	struct intel_engine_cs *ring;
 	struct intel_context *ctx;
 	struct i915_address_space *vm;
-	struct i915_execbuffer_params params_master; /* XXX: will be removed later */
-	struct i915_execbuffer_params *params = &params_master;
+	struct i915_scheduler_queue_entry qe;
+	struct i915_execbuffer_params *params = &qe.params;
 	const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
 	u32 dispatch_flags;
 	int ret;
@@ -1529,7 +1525,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	else
 		vm = &dev_priv->gtt.base;
 
-	memset(&params_master, 0x00, sizeof(params_master));
+	memset(&qe, 0x00, sizeof(qe));
 
 	eb = eb_create(args);
 	if (eb == NULL) {
drivers/gpu/drm/i915/intel_lrc.c +4 -1
index 12e8949..ff4565f 100644
@@ -136,6 +136,7 @@
 #include <drm/i915_drm.h>
 #include "i915_drv.h"
 #include "intel_mocs.h"
+#include "i915_scheduler.h"
 
 #define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
 #define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -910,6 +911,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas)
 {
+	struct i915_scheduler_queue_entry *qe;
 	struct drm_device       *dev = params->dev;
 	struct intel_engine_cs  *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -952,17 +954,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	ret = dev_priv->gt.execbuf_final(params);
+	qe = container_of(params, typeof(*qe), params);
+	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
 		return ret;
 

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 964

Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +0 -7
-	/*
-	 * Free everything that was stored in the QE structure (until the
-	 * scheduler arrives and does it instead):
-	 */
-	if (params->dispatch_flags & I915_DISPATCH_SECURE)
-		i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
 	return 0;
 }
 

I think this is ok, but might need changes if some of the earlier patches see changes due to Joonas's reviews.

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#34 lore raw

PATCH v5 14/35 drm/i915: Keep the reserved space mechanism happy

In reply to [email protected]

From: John Harrison <[email protected]>

Ring space is reserved when constructing a request to ensure that the subsequent 'add_request()' call cannot fail due to waiting for space on a busy or broken GPU. However, the scheduler jumps in to the middle of the execbuffer process between request creation and request submission. Thus it needs to cancel the reserved space when the request is simply added to the scheduler's queue and not yet submitted. Similarly, it needs to re-reserve the space when it finally does want to send the batch buffer to the hardware.

v3: Updated to use locally cached request pointer.

v5: Updated due to changes to earlier patches in series - for runtime PM calls and splitting bypass mode into a separate function.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 20 ++++++++++++++------
 drivers/gpu/drm/i915/i915_scheduler.c      |  4 ++++
 drivers/gpu/drm/i915/intel_lrc.c           | 13 +++++++++++--
 3 files changed, 29 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +14 -6
index 09c5ce9..11bea8d 100644
@@ -1295,18 +1295,22 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_ring_reserve_space(req);
+	if (ret)
+		goto error;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
 	ret = intel_ring_invalidate_all_caches(req);
 	if (ret)
-		return ret;
+		goto error;
 
 	/* Switch to the correct context for the batch */
 	ret = i915_switch_context(req);
 	if (ret)
-		return ret;
+		goto error;
 
 	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
 	     "%s didn't clear reload\n", ring->name);
@@ -1315,7 +1319,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	    params->instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_ring_begin(req, 4);
 		if (ret)
-			return ret;
+			goto error;
 
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
@@ -1329,7 +1333,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
 		ret = i915_reset_gen7_sol_offsets(params->dev, req);
 		if (ret)
-			return ret;
+			goto error;
 	}
 
 	exec_len   = params->args_batch_len;
@@ -1343,13 +1347,17 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto error;
 
 	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
-	return 0;
+error:
+	if (ret)
+		intel_ring_reserved_space_cancel(req->ringbuf);
+
+	return ret;
 }
 
 /**
drivers/gpu/drm/i915/i915_scheduler.c +4 -0
index 3986890..a3ffd04 100644
@@ -483,6 +483,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
 	struct i915_scheduler *scheduler = dev_priv->scheduler;
 	int ret;
 
+	intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
+
 	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
 	ret = dev_priv->gt.execbuf_final(&qe->params);
 	scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
@@ -539,6 +541,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	node->stamp  = jiffies;
 	i915_gem_request_reference(node->params.request);
 
+	intel_ring_reserved_space_cancel(node->params.request->ringbuf);
+
 	WARN_ON(node->params.request->scheduler_qe);
 	node->params.request->scheduler_qe = node;
 
drivers/gpu/drm/i915/intel_lrc.c +11 -2
index ff4565f..f4bab82 100644
@@ -978,13 +978,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_logical_ring_reserve_space(req);
+	if (ret)
+		goto err;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
 	ret = logical_ring_invalidate_all_caches(req);
 	if (ret)
-		return ret;
+		goto err;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
@@ -1006,13 +1010,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 
 	ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto err;
 
 	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;
+
+err:
+	intel_ring_reserved_space_cancel(params->request->ringbuf);
+
+	return ret;
 }
 
 void intel_execlists_retire_requests(struct intel_engine_cs *ring)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 14/35
#35 lore raw

Re: [PATCH v5 14/35] drm/i915: Keep the reserved space mechanism happy

Quoted 147 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Ring space is reserved when constructing a request to ensure that the subsequent 'add_request()' call cannot fail due to waiting for space on a busy or broken GPU. However, the scheduler jumps in to the middle of the execbuffer process between request creation and request submission. Thus it needs to cancel the reserved space when the request is simply added to the scheduler's queue and not yet submitted. Similarly, it needs to re-reserve the space when it finally does want to send the batch buffer to the hardware.

v3: Updated to use locally cached request pointer.

v5: Updated due to changes to earlier patches in series - for runtime PM calls and splitting bypass mode into a separate function.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 20 ++++++++++++++------
 drivers/gpu/drm/i915/i915_scheduler.c      |  4 ++++
 drivers/gpu/drm/i915/intel_lrc.c           | 13 +++++++++++--
 3 files changed, 29 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +14 -6
index 09c5ce9..11bea8d 100644
@@ -1295,18 +1295,22 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_ring_reserve_space(req);
+	if (ret)
+		goto error;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
 	ret = intel_ring_invalidate_all_caches(req);
 	if (ret)
-		return ret;
+		goto error;
 
 	/* Switch to the correct context for the batch */
 	ret = i915_switch_context(req);
 	if (ret)
-		return ret;
+		goto error;
 
 	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
 	     "%s didn't clear reload\n", ring->name);
@@ -1315,7 +1319,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	    params->instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_ring_begin(req, 4);
 		if (ret)
-			return ret;
+			goto error;
 
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
@@ -1329,7 +1333,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
 		ret = i915_reset_gen7_sol_offsets(params->dev, req);
 		if (ret)
-			return ret;
+			goto error;
 	}
 
 	exec_len   = params->args_batch_len;
@@ -1343,13 +1347,17 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto error;
 
 	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
-	return 0;
+error:
+	if (ret)
+		intel_ring_reserved_space_cancel(req->ringbuf);
+
+	return ret;
 }
 
 /**
drivers/gpu/drm/i915/i915_scheduler.c +4 -0
index 3986890..a3ffd04 100644
@@ -483,6 +483,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
 	struct i915_scheduler *scheduler = dev_priv->scheduler;
 	int ret;
 
+	intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
+
 	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
 	ret = dev_priv->gt.execbuf_final(&qe->params);
 	scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
@@ -539,6 +541,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	node->stamp  = jiffies;
 	i915_gem_request_reference(node->params.request);
 
+	intel_ring_reserved_space_cancel(node->params.request->ringbuf);
+
 	WARN_ON(node->params.request->scheduler_qe);
 	node->params.request->scheduler_qe = node;
 
drivers/gpu/drm/i915/intel_lrc.c +6 -2
index ff4565f..f4bab82 100644
@@ -978,13 +978,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_logical_ring_reserve_space(req);
+	if (ret)
+		goto err;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
 	ret = logical_ring_invalidate_all_caches(req);
 	if (ret)
-		return ret;
+		goto err;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
@@ -1006,13 +1010,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 
 	ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto err;
 
 	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission_final(struct i915_execbuffer_params *params) · line 1027

Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +5 -0
 
 	return 0;
+
+err:
+	intel_ring_reserved_space_cancel(params->request->ringbuf);
+
+	return ret;
 }
 
 void intel_execlists_retire_requests(struct intel_engine_cs *ring)

I had to double check that it was ok to cancel space if the reserve failed (I guess it is), so seems ok.

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#36 lore raw

PATCH v5 15/35 drm/i915: Added tracking/locking of batch buffer objects

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler needs to track interdependencies between batch buffers. These are calculated by analysing the object lists of the buffers and looking for commonality. The scheduler also needs to keep those buffers locked long after the initial IOCTL call has returned to user land.

v3: Updated to support read-read optimisation.

v5: Updated due to changes to earlier patches in series for splitting bypass mode into a separate function and consoliding the clean up code.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 48 ++++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.c      | 15 ++++++++++
 2 files changed, 61 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +46 -2
index 11bea8d..f45f4dc 100644
@@ -1428,7 +1428,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	struct i915_execbuffer_params *params = &qe.params;
 	const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
 	u32 dispatch_flags;
-	int ret;
+	int ret, i;
 	bool need_relocs;
 
 	if (!i915_gem_check_execbuffer(args))
@@ -1543,6 +1543,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 		goto pre_mutex_err;
 	}
 
+	qe.saved_objects = kzalloc(
+			sizeof(*qe.saved_objects) * args->buffer_count,
+			GFP_KERNEL);
+	if (!qe.saved_objects) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
 	/* Look up object handles */
 	ret = eb_lookup_vmas(eb, exec, args, vm, file);
 	if (ret)
@@ -1663,7 +1671,30 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	params->args_DR1                = args->DR1;
 	params->args_DR4                = args->DR4;
 	params->batch_obj               = batch_obj;
-	params->ctx                     = ctx;
+
+	/*
+	 * Save away the list of objects used by this batch buffer for the
+	 * purpose of tracking inter-buffer dependencies.
+	 */
+	for (i = 0; i < args->buffer_count; i++) {
+		struct drm_i915_gem_object *obj;
+
+		/*
+		 * NB: 'drm_gem_object_lookup()' increments the object's
+		 * reference count and so must be matched by a
+		 * 'drm_gem_object_unreference' call.
+		 */
+		obj = to_intel_bo(drm_gem_object_lookup(dev, file,
+							  exec[i].handle));
+		qe.saved_objects[i].obj       = obj;
+		qe.saved_objects[i].read_only = obj->base.pending_write_domain == 0;
+
+	}
+	qe.num_objs = i;
+
+	/* Lock and save the context object as well. */
+	i915_gem_context_reference(ctx);
+	params->ctx = ctx;
 
 	ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
 	if (ret)
@@ -1696,6 +1727,19 @@ err:
 	i915_gem_context_unreference(ctx);
 	eb_destroy(eb);
 
+	/* Need to release the objects: */
+	if (qe.saved_objects) {
+		for (i = 0; i < qe.num_objs; i++)
+			drm_gem_object_unreference(
+					&qe.saved_objects[i].obj->base);
+
+		kfree(qe.saved_objects);
+	}
+
+	/* Context too */
+	if (params->ctx)
+		i915_gem_context_unreference(params->ctx);
+
 	/*
 	 * If the request was created but not successfully submitted then it
 	 * must be freed again. If it was submitted then it is being tracked
drivers/gpu/drm/i915/i915_scheduler.c +15 -0
index a3ffd04..60a59d3 100644
@@ -719,6 +719,8 @@ void i915_scheduler_wakeup(struct drm_device *dev)
  */
 void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
 {
+	int i;
+
 	if (!I915_SQS_IS_COMPLETE(node)) {
 		WARN(!node->params.request->cancelled,
 		     "Cleaning active node: %d!\n", node->status);
@@ -736,6 +738,19 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
 		node->params.batch_obj = NULL;
 	}
 
+	/* Release the locked buffers: */
+	for (i = 0; i < node->num_objs; i++)
+		drm_gem_object_unreference(&node->saved_objects[i].obj->base);
+	kfree(node->saved_objects);
+	node->saved_objects = NULL;
+	node->num_objs = 0;
+
+	/* Context too: */
+	if (node->params.ctx) {
+		i915_gem_context_unreference(node->params.ctx);
+		node->params.ctx = NULL;
+	}
+
 	/* And anything else owned by the node: */
 	if (node->params.cliprects) {
 		kfree(node->params.cliprects);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 15/35
#37 lore raw

Re: [PATCH v5 15/35] drm/i915: Added tracking/locking of batch buffer objects

Quoted 124 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler needs to track interdependencies between batch buffers. These are calculated by analysing the object lists of the buffers and looking for commonality. The scheduler also needs to keep those buffers locked long after the initial IOCTL call has returned to user land.

v3: Updated to support read-read optimisation.

v5: Updated due to changes to earlier patches in series for splitting bypass mode into a separate function and consoliding the clean up code.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 48 ++++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.c      | 15 ++++++++++
 2 files changed, 61 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +46 -2
index 11bea8d..f45f4dc 100644
@@ -1428,7 +1428,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	struct i915_execbuffer_params *params = &qe.params;
 	const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
 	u32 dispatch_flags;
-	int ret;
+	int ret, i;
 	bool need_relocs;
 
 	if (!i915_gem_check_execbuffer(args))
@@ -1543,6 +1543,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 		goto pre_mutex_err;
 	}
 
+	qe.saved_objects = kzalloc(
+			sizeof(*qe.saved_objects) * args->buffer_count,
+			GFP_KERNEL);
+	if (!qe.saved_objects) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
 	/* Look up object handles */
 	ret = eb_lookup_vmas(eb, exec, args, vm, file);
 	if (ret)
@@ -1663,7 +1671,30 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	params->args_DR1                = args->DR1;
 	params->args_DR4                = args->DR4;
 	params->batch_obj               = batch_obj;
-	params->ctx                     = ctx;
+
+	/*
+	 * Save away the list of objects used by this batch buffer for the
+	 * purpose of tracking inter-buffer dependencies.
+	 */
+	for (i = 0; i < args->buffer_count; i++) {
+		struct drm_i915_gem_object *obj;
+
+		/*
+		 * NB: 'drm_gem_object_lookup()' increments the object's
+		 * reference count and so must be matched by a
+		 * 'drm_gem_object_unreference' call.
+		 */
+		obj = to_intel_bo(drm_gem_object_lookup(dev, file,
+							  exec[i].handle));
+		qe.saved_objects[i].obj       = obj;
+		qe.saved_objects[i].read_only = obj->base.pending_write_domain == 0;
+
+	}
+	qe.num_objs = i;
+
+	/* Lock and save the context object as well. */
+	i915_gem_context_reference(ctx);
+	params->ctx = ctx;
 
 	ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
 	if (ret)
@@ -1696,6 +1727,19 @@ err:
 	i915_gem_context_unreference(ctx);
 	eb_destroy(eb);
 
+	/* Need to release the objects: */
+	if (qe.saved_objects) {
+		for (i = 0; i < qe.num_objs; i++)
+			drm_gem_object_unreference(
+					&qe.saved_objects[i].obj->base);
+
+		kfree(qe.saved_objects);
+	}
+
+	/* Context too */
+	if (params->ctx)
+		i915_gem_context_unreference(params->ctx);
+
 	/*
 	 * If the request was created but not successfully submitted then it
 	 * must be freed again. If it was submitted then it is being tracked
drivers/gpu/drm/i915/i915_scheduler.c +8 -0
index a3ffd04..60a59d3 100644
@@ -719,6 +719,8 @@ void i915_scheduler_wakeup(struct drm_device *dev)
  */
 void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
 {
+	int i;
+
 	if (!I915_SQS_IS_COMPLETE(node)) {
 		WARN(!node->params.request->cancelled,
 		     "Cleaning active node: %d!\n", node->status);
@@ -736,6 +738,19 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
 		node->params.batch_obj = NULL;
 	}
 
+	/* Release the locked buffers: */
+	for (i = 0; i < node->num_objs; i++)
+		drm_gem_object_unreference(&node->saved_objects[i].obj->base);
+	kfree(node->saved_objects);
+	node->saved_objects = NULL;
+	node->num_objs = 0;

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) · line 756

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +7 -0
+
+	/* Context too: */
+	if (node->params.ctx) {
+		i915_gem_context_unreference(node->params.ctx);
+		node->params.ctx = NULL;
+	}
+
 	/* And anything else owned by the node: */
 	if (node->params.cliprects) {
 		kfree(node->params.cliprects);

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#38 lore raw

PATCH v5 16/35 drm/i915: Hook scheduler node clean up into retire requests

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler keeps its own lock on various DRM objects in order to guarantee safe access long after the original execbuff IOCTL has completed. This is especially important when pre-emption is enabled as the batch buffer might need to be submitted to the hardware multiple times. This patch hooks the clean up of these locks into the request retire function. The request can only be retired after it has completed on the hardware and thus is no longer eligible for re-submission. Thus there is no point holding on to the locks beyond that time.

v3: Updated to not WARN when cleaning a node that is being cancelled. The clean will happen later so skipping it at the point of cancellation is fine.

v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_gem.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +3 -0
index 1ab7256..2dd9b55 100644
@@ -1489,6 +1489,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
 		fence_signal_locked(&request->fence);
 	}
 
+	if (request->scheduler_qe)
+		i915_scheduler_clean_node(request->scheduler_qe);
+
 	i915_gem_request_unreference(request);
 }
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 16/35
#39 lore raw

Re: [PATCH v5 16/35] drm/i915: Hook scheduler node clean up into retire requests

Quoted 4 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler keeps its own lock on various DRM objects in order to guarantee safe access long after the original execbuff IOCTL has

execbuf is getting bigger, but I'm not sure if it qualifies as "buff" yet. Intentional misspelling? :)

Quoted 28 lines

completed. This is especially important when pre-emption is enabled as the batch buffer might need to be submitted to the hardware multiple times. This patch hooks the clean up of these locks into the request retire function. The request can only be retired after it has completed on the hardware and thus is no longer eligible for re-submission. Thus there is no point holding on to the locks beyond that time.

v3: Updated to not WARN when cleaning a node that is being cancelled. The clean will happen later so skipping it at the point of cancellation is fine.

v5: Squashed the i915_scheduler.c portions down into the 'start of scheduler' patch. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_gem.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +0 -0
index 1ab7256..2dd9b55 100644

drivers/gpu/drm/i915/i915_gem.c static void i915_gem_request_retire(struct drm_i915_gem_request *request) · line 1497

Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +3 -0
@@ -1489,6 +1489,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
 		fence_signal_locked(&request->fence);
 	}
 
+	if (request->scheduler_qe)
+		i915_scheduler_clean_node(request->scheduler_qe);
+
 	i915_gem_request_unreference(request);
 }
 

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#40 lore raw

PATCH v5 17/35 drm/i915: Added scheduler support to __wait_request() calls

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.

This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of __wait_request() release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.

This change hooks the scheduler in to the __wait_request() code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.

v3: Removed the explicit scheduler flush from i915_wait_request(). This is no longer necessary and was causing unintended changes to the scheduler priority level which broke a validation team test.

v4: Corrected the format of some comments to keep the style checker happy.

v5: Added function description. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_drv.h         |  3 ++-
 drivers/gpu/drm/i915/i915_gem.c         | 37 ++++++++++++++++++++++++++-------
 drivers/gpu/drm/i915/i915_scheduler.c   | 31 +++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h   |  2 ++
 drivers/gpu/drm/i915/intel_display.c    |  5 +++--
 drivers/gpu/drm/i915/intel_ringbuffer.c |  2 +-
 6 files changed, 69 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +2 -1
index 4d544f1..5eeeced 100644
@@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			unsigned reset_counter,
 			bool interruptible,
 			s64 *timeout,
-			struct intel_rps_client *rps);
+			struct intel_rps_client *rps,
+			bool is_locked);
 int __must_check i915_wait_request(struct drm_i915_gem_request *req);
 int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
 int __must_check
drivers/gpu/drm/i915/i915_gem.c +30 -7
index 2dd9b55..17b44b3 100644
@@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			unsigned reset_counter,
 			bool interruptible,
 			s64 *timeout,
-			struct intel_rps_client *rps)
+			struct intel_rps_client *rps,
+			bool is_locked)
 {
 	struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
 	struct drm_device *dev = ring->dev;
@@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 	DEFINE_WAIT(wait);
 	unsigned long timeout_expire;
 	s64 before = 0; /* Only to silence a compiler warning. */
-	int ret;
+	int ret = 0;
+	bool    busy;
 
+	might_sleep();
 	WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
 
 	if (i915_gem_request_completed(req))
@@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			break;
 		}
 
+		if (is_locked) {
+			/*
+			 * If this request is being processed by the scheduler
+			 * then it is unsafe to sleep with the mutex lock held
+			 * as the scheduler may require the lock in order to
+			 * progress the request.
+			 */
+			if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
+				if (busy) {
+					ret = -EAGAIN;
+					break;
+				}
+			}
+
+			/*
+			 * If the request is not tracked by the scheduler
+			 * then the regular test can be done.
+			 */
+		}
+
 		if (i915_gem_request_completed(req)) {
 			ret = 0;
 			break;
@@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
 
 	ret = __i915_wait_request(req,
 				  atomic_read(&dev_priv->gpu_error.reset_counter),
-				  interruptible, NULL, NULL);
+				  interruptible, NULL, NULL, true);
 	if (ret)
 		return ret;
 
@@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
 	mutex_unlock(&dev->struct_mutex);
 	for (i = 0; ret == 0 && i < n; i++)
 		ret = __i915_wait_request(requests[i], reset_counter, true,
-					  NULL, rps);
+					  NULL, rps, false);
 	mutex_lock(&dev->struct_mutex);
 
 	for (i = 0; i < n; i++) {
@@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		if (ret == 0)
 			ret = __i915_wait_request(req[i], reset_counter, true,
 						  args->timeout_ns > 0 ? &args->timeout_ns : NULL,
-						  to_rps_client(file));
+						  to_rps_client(file), false);
 		i915_gem_request_unreference(req[i]);
 	}
 	return ret;
@@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
 					  atomic_read(&i915->gpu_error.reset_counter),
 					  i915->mm.interruptible,
 					  NULL,
-					  &i915->rps.semaphores);
+					  &i915->rps.semaphores, true);
 		if (ret)
 			return ret;
 
@@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	if (target == NULL)
 		return 0;
 
-	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);
 	if (ret == 0)
 		queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
 
drivers/gpu/drm/i915/i915_scheduler.c +31 -0
index 60a59d3..edab63d 100644
@@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work)
 }
 
 /**
+ * i915_scheduler_is_request_tracked - return info to say what the scheduler's
+ * connection to this request is (if any).
+ * @req: request to be queried
+ * @compeleted: if non-null, set to completion status
+ * @busy: if non-null set to busy status
+ *
+ * Looks up the given request in the scheduler's internal queue and reports
+ * on whether the request has completed or is still pending.
+ * Returns true if the request was found or false if it was not.
+ */
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+				       bool *completed, bool *busy)
+{
+	struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+	if (!scheduler)
+		return false;
+
+	if (req->scheduler_qe == NULL)
+		return false;
+
+	if (completed)
+		*completed = I915_SQS_IS_COMPLETE(req->scheduler_qe);
+	if (busy)
+		*busy      = I915_SQS_IS_QUEUED(req->scheduler_qe);
+
+	return true;
+}
+
+/**
  * i915_scheduler_closefile - notify the scheduler that a DRM file handle
  * has been closed.
  * @dev: DRM device
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 180d75f..a88adce 100644
@@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void i915_scheduler_wakeup(struct drm_device *dev);
 void i915_scheduler_work_handler(struct work_struct *work);
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+				       bool *completed, bool *busy);
 
 #endif  /* _I915_SCHEDULER_H_ */
drivers/gpu/drm/i915/intel_display.c +3 -2
index 731d20a..5953590 100644
@@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
 		WARN_ON(__i915_wait_request(mmio_flip->req,
 					    mmio_flip->crtc->reset_counter,
 					    false, NULL,
-					    &mmio_flip->i915->rps.mmioflips));
+					    &mmio_flip->i915->rps.mmioflips,
+					    false));
 		i915_gem_request_unreference(mmio_flip->req);
 	}
 
@@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
 
 			ret = __i915_wait_request(intel_plane_state->wait_req,
 						  reset_counter, true,
-						  NULL, NULL);
+						  NULL, NULL, false);
 
 			/* Swallow -EIO errors to allow updates during hw lockup. */
 			if (ret == -EIO)
drivers/gpu/drm/i915/intel_ringbuffer.c +1 -1
index ca7b8af..a2093f5 100644
@@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
 	return __i915_wait_request(req,
 				   atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
 				   to_i915(ring->dev)->mm.interruptible,
-				   NULL, NULL);
+				   NULL, NULL, true);
 }
 
 int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#41 lore raw

Re: [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls

Quoted 51 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.

This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of __wait_request() release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.

This change hooks the scheduler in to the __wait_request() code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.

v3: Removed the explicit scheduler flush from i915_wait_request(). This is no longer necessary and was causing unintended changes to the scheduler priority level which broke a validation team test.

v4: Corrected the format of some comments to keep the style checker happy.

v5: Added function description. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>

---
 drivers/gpu/drm/i915/i915_drv.h         |  3 ++- drivers/gpu/drm/i915/i915_gem.c         | 37 ++++++++++++++++++++++++++------- drivers/gpu/drm/i915/i915_scheduler.c   | 31 +++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h   |  2 ++ drivers/gpu/drm/i915/intel_display.c    |  5 +++-- drivers/gpu/drm/i915/intel_ringbuffer.c |  2 +-

 6 files changed, 69 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +0 -0
index 4d544f1..5eeeced 100644

drivers/gpu/drm/i915/i915_drv.h int __i915_wait_request(struct drm_i915_gem_request *req, · line 3075

Quoted 6 lines
drivers/gpu/drm/i915/i915_drv.h +2 -1
@@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  			unsigned reset_counter,
  			bool interruptible,
  			s64 *timeout,
-			struct intel_rps_client *rps);
+			struct intel_rps_client *rps,
+			bool is_locked);

Multiple bool parameters are better converted to int flags, it makes the callsite code more readable if you have enum I915_WAIT_REQ_* flags.

Quoted 21 lines
drivers/gpu/drm/i915/i915_drv.h +0 -0
  int __must_check i915_wait_request(struct drm_i915_gem_request *req);
  int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);

 int __must_check

drivers/gpu/drm/i915/i915_gem.c +2 -1
index 2dd9b55..17b44b3 100644
@@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  			unsigned reset_counter,
  			bool interruptible,
  			s64 *timeout,
-			struct intel_rps_client *rps)
+			struct intel_rps_client *rps,
+			bool is_locked)
  {
  	struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
  	struct drm_device *dev = ring->dev;
@@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  	DEFINE_WAIT(wait);
  	unsigned long timeout_expire;

  s64 before = 0; /* Only to silence a compiler warning. */

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1273

Quoted 3 lines
drivers/gpu/drm/i915/i915_gem.c +2 -1
-	int ret;
+	int ret = 0;
+	bool    busy;

Strange whitespace.

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1275

Quoted 2 lines

drivers/gpu/drm/i915/i915_gem.c +1 -0
+	might_sleep();

This will splat if is_locked is true? So maybe it should be protected by if (!is_locked)?

Quoted 7 lines

  WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
  if (i915_gem_request_completed(req))

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  			break;
  		}

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1336

Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7 -0
+		if (is_locked) {
+			/*
+			 * If this request is being processed by the scheduler
+			 * then it is unsafe to sleep with the mutex lock held
+			 * as the scheduler may require the lock in order to
+			 * progress the request.
+			 */

This is becoming a monstrous function, I think it could be split down to two functions, __i915_wait_request and __i915_wait_request_locked/nonblocking, because if you take all the timeout/sleeping related stuff away, not much is left.

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1342

Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +6 -0
+			if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
+				if (busy) {
+					ret = -EAGAIN;
+					break;
+				}
+			}

If this is kept in single function, would not it be enough to execute this before the loop?

Quoted 75 lines
drivers/gpu/drm/i915/i915_gem.c +7 -0
+
+			/*
+			 * If the request is not tracked by the scheduler
+			 * then the regular test can be done.
+			 */
+		}
+

  if (i915_gem_request_completed(req)) { ret = 0;
  break;

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req)


  ret = __i915_wait_request(req, atomic_read(&dev_priv->gpu_error.reset_counter),

drivers/gpu/drm/i915/i915_gem.c +1 -1
-				  interruptible, NULL, NULL);
+				  interruptible, NULL, NULL, true);

  if (ret)
  return ret;

drivers/gpu/drm/i915/i915_gem.c +1 -1
@@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
  	mutex_unlock(&dev->struct_mutex);
  	for (i = 0; ret == 0 && i < n; i++)
  		ret = __i915_wait_request(requests[i], reset_counter, true,
-					  NULL, rps);
+					  NULL, rps, false);
  	mutex_lock(&dev->struct_mutex);


  for (i = 0; i < n; i++) {

drivers/gpu/drm/i915/i915_gem.c +2 -2
@@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
  		if (ret == 0)
  			ret = __i915_wait_request(req[i], reset_counter, true,
  						  args->timeout_ns > 0 ? &args->timeout_ns : NULL,
-						  to_rps_client(file));
+						  to_rps_client(file), false);
  		i915_gem_request_unreference(req[i]);
  	}
  	return ret;
@@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
  					  atomic_read(&i915->gpu_error.reset_counter),
  					  i915->mm.interruptible,
  					  NULL,
-					  &i915->rps.semaphores);
+					  &i915->rps.semaphores, true);
  		if (ret)
  			return ret;

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
  	if (target == NULL)
  		return 0;

drivers/gpu/drm/i915/i915_gem.c +1 -1
-	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);

  if (ret == 0)
  queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);

drivers/gpu/drm/i915/i915_scheduler.c +0 -0
index 60a59d3..edab63d 100644
@@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work)
  }


 /**
+ * i915_scheduler_is_request_tracked - return info to say what the scheduler's
+ * connection to this request is (if any).
+ * @req: request to be queried
+ * @compeleted: if non-null, set to completion status
+ * @busy: if non-null set to busy status
+ *
+ * Looks up the given request in the scheduler's internal queue and reports
+ * on whether the request has completed or is still pending.
+ * Returns true if the request was found or false if it was not.
+ */

drivers/gpu/drm/i915/i915_scheduler.c +2 -0
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+				       bool *completed, bool *busy)

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 942

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+{
+	struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+	if (!scheduler)
+		return false;
+
+	if (req->scheduler_qe == NULL)
+		return false;
+

These better be their own functions, making the code more readable.

_is_request_completed()
_is_request_busy()

Quoted 60 lines
drivers/gpu/drm/i915/i915_scheduler.c +9 -0
+	if (completed)
+		*completed = I915_SQS_IS_COMPLETE(req->scheduler_qe);
+	if (busy)
+		*busy      = I915_SQS_IS_QUEUED(req->scheduler_qe);
+
+	return true;
+}
+
+/**

  * i915_scheduler_closefile - notify the scheduler that a DRM file handle
  * has been closed.
  * @dev: DRM device

drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 180d75f..a88adce 100644
@@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
  bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
  void i915_scheduler_wakeup(struct drm_device *dev);
  void i915_scheduler_work_handler(struct work_struct *work);
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+				       bool *completed, bool *busy);


 #endif  /* _I915_SCHEDULER_H_ */

drivers/gpu/drm/i915/intel_display.c +2 -1
index 731d20a..5953590 100644
@@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
  		WARN_ON(__i915_wait_request(mmio_flip->req,
  					    mmio_flip->crtc->reset_counter,
  					    false, NULL,
-					    &mmio_flip->i915->rps.mmioflips));
+					    &mmio_flip->i915->rps.mmioflips,
+					    false));
  		i915_gem_request_unreference(mmio_flip->req);
  	}

drivers/gpu/drm/i915/intel_display.c +0 -0
@@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,


  ret = __i915_wait_request(intel_plane_state->wait_req, reset_counter, true,

drivers/gpu/drm/i915/intel_display.c +1 -1
-						  NULL, NULL);
+						  NULL, NULL, false);


  /* Swallow -EIO errors to allow updates during hw lockup. */ if (ret == -EIO)

drivers/gpu/drm/i915/intel_ringbuffer.c +1 -1
index ca7b8af..a2093f5 100644
@@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
  	return __i915_wait_request(req,
  				   atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
  				   to_i915(ring->dev)->mm.interruptible,
-				   NULL, NULL);
+				   NULL, NULL, true);
  }


 int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#42 lore raw

Re: [PATCH v5 17/35] drm/i915: Added scheduler support to __wait_request() calls

Quoted 88 lines · Joonas Lahtinen
Quoted 51 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.

This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of __wait_request() release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.

This change hooks the scheduler in to the __wait_request() code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.

v3: Removed the explicit scheduler flush from i915_wait_request(). This is no longer necessary and was causing unintended changes to the scheduler priority level which broke a validation team test.

v4: Corrected the format of some comments to keep the style checker happy.

v5: Added function description. [Joonas Lahtinen]

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
  drivers/gpu/drm/i915/i915_drv.h         |  3 ++-
  drivers/gpu/drm/i915/i915_gem.c         | 37 ++++++++++++++++++++++++++-------
  drivers/gpu/drm/i915/i915_scheduler.c   | 31 +++++++++++++++++++++++++++
  drivers/gpu/drm/i915/i915_scheduler.h   |  2 ++
  drivers/gpu/drm/i915/intel_display.c    |  5 +++--
  drivers/gpu/drm/i915/intel_ringbuffer.c |  2 +-
  6 files changed, 69 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +0 -0
index 4d544f1..5eeeced 100644

drivers/gpu/drm/i915/i915_drv.h int __i915_wait_request(struct drm_i915_gem_request *req, · line 3075

Quoted 6 lines
drivers/gpu/drm/i915/i915_drv.h +2 -1
@@ -3071,7 +3071,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  			unsigned reset_counter,
  			bool interruptible,
  			s64 *timeout,
-			struct intel_rps_client *rps);
+			struct intel_rps_client *rps,
+			bool is_locked);

Multiple bool parameters are better converted to int flags, it makes the callsite code more readable if you have enum I915_WAIT_REQ_* flags.

Quoted 18 lines
drivers/gpu/drm/i915/i915_drv.h +0 -0
  int __must_check i915_wait_request(struct drm_i915_gem_request *req);
  int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
  int __must_check
drivers/gpu/drm/i915/i915_gem.c +2 -1
index 2dd9b55..17b44b3 100644
@@ -1258,7 +1258,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  			unsigned reset_counter,
  			bool interruptible,
  			s64 *timeout,
-			struct intel_rps_client *rps)
+			struct intel_rps_client *rps,
+			bool is_locked)
  {
  	struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
  	struct drm_device *dev = ring->dev;

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1273

Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +2 -1
@@ -1268,8 +1269,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  	DEFINE_WAIT(wait);
  	unsigned long timeout_expire;
  	s64 before = 0; /* Only to silence a compiler warning. */
-	int ret;
+	int ret = 0;
+	bool    busy;

Strange whitespace.

Quoted 1 line
Quoted 3 lines

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1275

Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +1 -0
+	might_sleep();

This will splat if is_locked is true? So maybe it should be protected by if (!is_locked)?

The locking here is just the driver mutex lock. Whereas the test will only go bang if in atomic context (spinlock, IRQ, etc.). So it should be safe. If it isn't then the driver is already broken because the whole point of _wait_request() is that it will sleep! Although one could argue it is already broken because it quite likes sleeping with the mutex lock held.

Quoted 7 lines
Quoted 7 lines
  	WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");

if (i915_gem_request_completed(req))

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -1324,6 +1327,26 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
  			break;
  		}
 
Quoted 11 lines

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1336

Quoted 7 lines
drivers/gpu/drm/i915/i915_gem.c +7 -0
+		if (is_locked) {
+			/*
+			 * If this request is being processed by the scheduler
+			 * then it is unsafe to sleep with the mutex lock held
+			 * as the scheduler may require the lock in order to
+			 * progress the request.
+			 */

This is becoming a monstrous function, I think it could be split down to two functions, __i915_wait_request and __i915_wait_request_locked/nonblocking, because if you take all the timeout/sleeping related stuff away, not much is left.

I don't see what you mean. You can't take away all the timeout/sleeping related stuff, that is the whole point of the function! Everything that is inside the loop needs to be inside the loop. It is a hideously complicated wait function but as far as I can tell, it needs to be such. I recall at some point someone did a refactor to move the contents of the loop into a separate function but that doesn't seem to have made it upstream. Or maybe it did and it has since been reverted?

The scheduler addition is now much simpler due to the re-implementing of the _is_tracked() function - see comments below.

Quoted 8 lines

drivers/gpu/drm/i915/i915_gem.c int __i915_wait_request(struct drm_i915_gem_request *req, · line 1342

Quoted 6 lines
drivers/gpu/drm/i915/i915_gem.c +6 -0
+			if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
+				if (busy) {
+					ret = -EAGAIN;
+					break;
+				}
+			}

If this is kept in single function, would not it be enough to execute this before the loop?

It might have changed. The sleep might have been aborted due to something that means the scheduler is now busy and sleeping with the mutex lock held will cause a deadlock.

Quoted 87 lines
Quoted 75 lines
drivers/gpu/drm/i915/i915_gem.c +12 -5
+
+			/*
+			 * If the request is not tracked by the scheduler
+			 * then the regular test can be done.
+			 */
+		}
+
  		if (i915_gem_request_completed(req)) {
  			ret = 0;
  			break;
@@ -1542,7 +1565,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
 
  	ret = __i915_wait_request(req,
  				  atomic_read(&dev_priv->gpu_error.reset_counter),
-				  interruptible, NULL, NULL);
+				  interruptible, NULL, NULL, true);
  	if (ret)
  		return ret;
 
@@ -1655,7 +1678,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
  	mutex_unlock(&dev->struct_mutex);
  	for (i = 0; ret == 0 && i < n; i++)
  		ret = __i915_wait_request(requests[i], reset_counter, true,
-					  NULL, rps);
+					  NULL, rps, false);
  	mutex_lock(&dev->struct_mutex);
 
  	for (i = 0; i < n; i++) {
@@ -3511,7 +3534,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
  		if (ret == 0)
  			ret = __i915_wait_request(req[i], reset_counter, true,
  						  args->timeout_ns > 0 ? &args->timeout_ns : NULL,
-						  to_rps_client(file));
+						  to_rps_client(file), false);
  		i915_gem_request_unreference(req[i]);
  	}
  	return ret;
@@ -3544,7 +3567,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
  					  atomic_read(&i915->gpu_error.reset_counter),
  					  i915->mm.interruptible,
  					  NULL,
-					  &i915->rps.semaphores);
+					  &i915->rps.semaphores, true);
  		if (ret)
  			return ret;
 
@@ -4523,7 +4546,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
  	if (target == NULL)
  		return 0;
 
-	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);
  	if (ret == 0)
  		queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
 
drivers/gpu/drm/i915/i915_scheduler.c +12 -0
index 60a59d3..edab63d 100644
@@ -918,6 +918,37 @@ void i915_scheduler_work_handler(struct work_struct *work)
  }
 
  /**
+ * i915_scheduler_is_request_tracked - return info to say what the scheduler's
+ * connection to this request is (if any).
+ * @req: request to be queried
+ * @compeleted: if non-null, set to completion status
+ * @busy: if non-null set to busy status
+ *
+ * Looks up the given request in the scheduler's internal queue and reports
+ * on whether the request has completed or is still pending.
+ * Returns true if the request was found or false if it was not.
+ */
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+				       bool *completed, bool *busy)

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 942

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+{
+	struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+	if (!scheduler)
+		return false;
+
+	if (req->scheduler_qe == NULL)
+		return false;
+

These better be their own functions, making the code more readable.

Quoted 2 lines

_is_request_completed()
_is_request_busy()

Yeah, the original need has actually changed somewhat since this was written. There is no longer any code which requires all three items together. So I've split it into two simpler functions which are more appropriate to their use cases.

Quoted 59 lines
Quoted 59 lines
drivers/gpu/drm/i915/i915_scheduler.c +9 -0
+	if (completed)
+		*completed = I915_SQS_IS_COMPLETE(req->scheduler_qe);
+	if (busy)
+		*busy      = I915_SQS_IS_QUEUED(req->scheduler_qe);
+
+	return true;
+}
+
+/**
   * i915_scheduler_closefile - notify the scheduler that a DRM file handle
   * has been closed.
   * @dev: DRM device
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 180d75f..a88adce 100644
@@ -94,5 +94,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
  bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
  void i915_scheduler_wakeup(struct drm_device *dev);
  void i915_scheduler_work_handler(struct work_struct *work);
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+				       bool *completed, bool *busy);
 
  #endif  /* _I915_SCHEDULER_H_ */
drivers/gpu/drm/i915/intel_display.c +3 -2
index 731d20a..5953590 100644
@@ -11458,7 +11458,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
  		WARN_ON(__i915_wait_request(mmio_flip->req,
  					    mmio_flip->crtc->reset_counter,
  					    false, NULL,
-					    &mmio_flip->i915->rps.mmioflips));
+					    &mmio_flip->i915->rps.mmioflips,
+					    false));
  		i915_gem_request_unreference(mmio_flip->req);
  	}
 
@@ -13523,7 +13524,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
 
  			ret = __i915_wait_request(intel_plane_state->wait_req,
  						  reset_counter, true,
-						  NULL, NULL);
+						  NULL, NULL, false);
 
  			/* Swallow -EIO errors to allow updates during hw lockup. */
  			if (ret == -EIO)
drivers/gpu/drm/i915/intel_ringbuffer.c +1 -1
index ca7b8af..a2093f5 100644
@@ -2304,7 +2304,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
  	return __i915_wait_request(req,
  				   atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
  				   to_i915(ring->dev)->mm.interruptible,
-				   NULL, NULL);
+				   NULL, NULL, true);
  }
 
Quoted 1 line

drivers/gpu/drm/i915/intel_ringbuffer.c int intel_ring_idle(struct intel_engine_cs *ring) · line 2310

Quoted 1 line
drivers/gpu/drm/i915/intel_ringbuffer.c +0 -0
  int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#43 lore raw

PATCH v5 18/35 drm/i915: Added scheduler support to page fault handler

In reply to [email protected]

From: John Harrison <[email protected]>

GPU page faults can now require scheduler operation in order to complete. For example, in order to free up sufficient memory to handle the fault the handler must wait for a batch buffer to complete that has not even been sent to the hardware yet. Thus EAGAIN no longer means a GPU hang, it can occur under normal operation.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +7 -2
index 17b44b3..a47a495 100644
@@ -2003,10 +2003,15 @@ out:
 		}
 	case -EAGAIN:
 		/*
-		 * EAGAIN means the gpu is hung and we'll wait for the error
-		 * handler to reset everything when re-faulting in
+		 * EAGAIN can mean the gpu is hung and we'll have to wait for
+		 * the error handler to reset everything when re-faulting in
 		 * i915_mutex_lock_interruptible.
+		 *
+		 * It can also indicate various other nonfatal errors for which
+		 * the best response is to give other threads a chance to run,
+		 * and then retry the failing operation in its entirety.
 		 */
+		/*FALLTHRU*/
 	case 0:
 	case -ERESTARTSYS:
 	case -EINTR:
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 18/35
#44 lore raw

Re: [PATCH v5 18/35] drm/i915: Added scheduler support to page fault handler

Quoted 27 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

GPU page faults can now require scheduler operation in order to complete. For example, in order to free up sufficient memory to handle the fault the handler must wait for a batch buffer to complete that has not even been sent to the hardware yet. Thus EAGAIN no longer means a GPU hang, it can occur under normal operation.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +2 -2
index 17b44b3..a47a495 100644
@@ -2003,10 +2003,15 @@ out:
 		}
 	case -EAGAIN:
 		/*
-		 * EAGAIN means the gpu is hung and we'll wait for the error
-		 * handler to reset everything when re-faulting in
+		 * EAGAIN can mean the gpu is hung and we'll have to wait for
+		 * the error handler to reset everything when re-faulting in

drivers/gpu/drm/i915/i915_gem.c out: · line 2017

Quoted 10 lines
drivers/gpu/drm/i915/i915_gem.c +5 -0
 		 * i915_mutex_lock_interruptible.
+		 *
+		 * It can also indicate various other nonfatal errors for which
+		 * the best response is to give other threads a chance to run,
+		 * and then retry the failing operation in its entirety.
 		 */
+		/*FALLTHRU*/
 	case 0:
 	case -ERESTARTSYS:
 	case -EINTR:

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#45 lore raw

PATCH v5 19/35 drm/i915: Added scheduler flush calls to ring throttle and idle functions

In reply to [email protected]

From: John Harrison <[email protected]>

When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.

v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.

v3: Moved submit_max_priority() to this patch from an earlier patch is it is no longer required in the other.

v4: Corrected the format of a comment to keep the style checker happy. Downgraded a BUG_ON to a WARN_ON as the latter is preferred.

v5: Shuffled functions around to remove forward prototypes, removed similarly offensive white space and added documentation. Re-worked the mutex locking around the submit function. [Joonas Lahtinen]

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_gem.c       |  24 ++++-
 drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |   3 +
 3 files changed, 204 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +23 -1
index a47a495..d946f53 100644
@@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)
 
 	/* Flush everything onto the inactive list. */
 	for_each_ring(ring, dev_priv, i) {
+		ret = i915_scheduler_flush(ring, true);
+		if (ret < 0)
+			return ret;
+
 		if (!i915.enable_execlists) {
 			struct drm_i915_gem_request *req;
 
@@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
 	struct drm_i915_gem_request *request, *target = NULL;
 	unsigned reset_counter;
-	int ret;
+	int i, ret;
+	struct intel_engine_cs *ring;
 
 	ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
 	if (ret)
@@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	if (ret)
 		return ret;
 
+	for_each_ring(ring, dev_priv, i) {
+		/*
+		 * Flush out scheduler entries that are getting 'stale'. Note
+		 * that the following recent_enough test will only check
+		 * against the time at which the request was submitted to the
+		 * hardware (i.e. when it left the scheduler) not the time it
+		 * was submitted to the driver.
+		 *
+		 * Also, there is not much point worring about busy return
+		 * codes from the scheduler flush call. Even if more work
+		 * cannot be submitted right now for whatever reason, we
+		 * still want to throttle against stale work that has already
+		 * been submitted.
+		 */
+		i915_scheduler_flush_stamp(ring, recent_enough, false);
+	}
+
 	spin_lock(&file_priv->mm.lock);
 	list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
 		if (time_after_eq(request->emitted_jiffies, recent_enough))
drivers/gpu/drm/i915/i915_scheduler.c +178 -0

196 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +3 -0
index a88adce..839b048 100644
@@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void i915_scheduler_wakeup(struct drm_device *dev);
 void i915_scheduler_work_handler(struct work_struct *work);
+int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
+int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+			       unsigned long stamp, bool is_locked);
 bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 				       bool *completed, bool *busy);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#46 lore raw

Re: [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions

Quoted 104 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.

v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.

v3: Moved submit_max_priority() to this patch from an earlier patch is it is no longer required in the other.

v4: Corrected the format of a comment to keep the style checker happy. Downgraded a BUG_ON to a WARN_ON as the latter is preferred.

v5: Shuffled functions around to remove forward prototypes, removed similarly offensive white space and added documentation. Re-worked the mutex locking around the submit function. [Joonas Lahtinen]

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>

---
 drivers/gpu/drm/i915/i915_gem.c       |  24 ++++- drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++ drivers/gpu/drm/i915/i915_scheduler.h |   3 +

 3 files changed, 204 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +0 -0
index a47a495..d946f53 100644
@@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)


  /* Flush everything onto the inactive list. */ for_each_ring(ring, dev_priv, i) {

drivers/gpu/drm/i915/i915_gem.c +4 -0
+		ret = i915_scheduler_flush(ring, true);
+		if (ret < 0)
+			return ret;
+

  if (!i915.enable_execlists) { struct drm_i915_gem_request *req;

drivers/gpu/drm/i915/i915_gem.c +2 -1
@@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
  	unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
  	struct drm_i915_gem_request *request, *target = NULL;
  	unsigned reset_counter;
-	int ret;
+	int i, ret;
+	struct intel_engine_cs *ring;


  ret = i915_gem_wait_for_error(&dev_priv->gpu_error); if (ret)

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
  	if (ret)
  		return ret;

drivers/gpu/drm/i915/i915_gem.c +17 -0
+	for_each_ring(ring, dev_priv, i) {
+		/*
+		 * Flush out scheduler entries that are getting 'stale'. Note
+		 * that the following recent_enough test will only check
+		 * against the time at which the request was submitted to the
+		 * hardware (i.e. when it left the scheduler) not the time it
+		 * was submitted to the driver.
+		 *
+		 * Also, there is not much point worring about busy return
+		 * codes from the scheduler flush call. Even if more work
+		 * cannot be submitted right now for whatever reason, we
+		 * still want to throttle against stale work that has already
+		 * been submitted.
+		 */
+		i915_scheduler_flush_stamp(ring, recent_enough, false);
+	}
+

  spin_lock(&file_priv->mm.lock); list_for_each_entry(request, &file_priv->mm.request_list, client_list) { if (time_after_eq(request->emitted_jiffies, recent_enough))

drivers/gpu/drm/i915/i915_scheduler.c +4 -0
index edab63d..8130a9c 100644
@@ -304,6 +304,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
   * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
   * And releasing the one before acquiring the other leads to other code
   * being run and interfering.
+ *
+ * Hence any caller that does not already have the mutex lock for other
+ * reasons should call i915_scheduler_submit_unlocked() instead in order to
+ * obtain the lock first.
   */
  static int i915_scheduler_submit(struct intel_engine_cs *ring)
  {
@@ -428,6 +432,22 @@ error:
  	return ret;
  }

drivers/gpu/drm/i915/i915_scheduler.c +6 -0
+static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
+{
+	struct drm_device *dev = ring->dev;
+	int ret;
+
+	ret = i915_mutex_lock_interruptible(dev);

drivers/gpu/drm/i915/i915_scheduler.c error: · line 450

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+	if (ret)
+		return ret;
+
+	ret = i915_scheduler_submit(ring);
+
+	mutex_unlock(&dev->struct_mutex);
+
+	return ret;
+}
+

This kind of code should really be factored to upper level functions.

Quoted 28 lines

 static void i915_generate_dependencies(struct i915_scheduler *scheduler, struct i915_scheduler_queue_entry *node, uint32_t ring)

drivers/gpu/drm/i915/i915_scheduler.c +0 -0
@@ -917,6 +937,164 @@ void i915_scheduler_work_handler(struct work_struct *work)
  		i915_scheduler_process_work(ring);
  }

drivers/gpu/drm/i915/i915_scheduler.c +21 -0
+static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
+					      bool is_locked)
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	int ret, count = 0;
+	bool found;
+
+	do {
+		found = false;
+		spin_lock_irq(&scheduler->lock);
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (!I915_SQS_IS_QUEUED(node))
+				continue;
+
+			if (node->priority < scheduler->priority_level_max)
+				continue;
+
+			found = true;
+			break;

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 970

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+		}
+		spin_unlock_irq(&scheduler->lock);
+
+		if (!found)
+			break;
+
+		if (is_locked)
+			ret = i915_scheduler_submit(ring);
+		else
+			ret = i915_scheduler_submit_unlocked(ring);

The widespread use of 'is_locked' notation is really bothering me, I think the difference in code path should be handled in the upper level, instead of being passed down, for example this will cause repetitive lock/unlock in loop.

Quoted 38 lines
drivers/gpu/drm/i915/i915_scheduler.c +38 -0
+		if (ret < 0)
+			return ret;
+
+		count += ret;
+	} while (found);
+
+	return count;
+}
+
+/**
+ * i915_scheduler_flush_stamp - force requests of a given age through the
+ * scheduler.
+ * @ring: Ring to be flushed
+ * @target: Jiffy based time stamp to flush up to
+ * @is_locked: Is the driver mutex lock held?
+ * DRM has a throttle by age of request facility. This requires waiting for
+ * outstanding work over a given age. This function helps that by forcing
+ * queued batch buffers over said age through the system.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+			       unsigned long target,
+			       bool is_locked)
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv;
+	struct i915_scheduler *scheduler;
+	int flush_count = 0;
+
+	if (!ring)
+		return -EINVAL;
+
+	dev_priv  = ring->dev->dev_private;
+	scheduler = dev_priv->scheduler;
+
+	if (!scheduler)

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 1018

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+		return 0;
+
+	if (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)) {
+		/*
+		 * Scheduler is busy already submitting another batch,
+		 * come back later rather than going recursive...
+		 */
+		return -EAGAIN;
+	}
+

Again, this would be handled better at higher level.

So I would like to see those 'is_locked' and other similar constructs factored out from the series. It should make the code much more robust to modify at later point.

Regards, Joonas

Quoted 96 lines
drivers/gpu/drm/i915/i915_scheduler.c +79 -0
+	spin_lock_irq(&scheduler->lock);
+	i915_scheduler_priority_bump_clear(scheduler);
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+
+		if (node->stamp > target)
+			continue;
+
+		flush_count = i915_scheduler_priority_bump(scheduler,
+					node, scheduler->priority_level_max);
+	}
+	spin_unlock_irq(&scheduler->lock);
+
+	if (flush_count) {
+		DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", ring->name, flush_count);
+		flush_count = i915_scheduler_submit_max_priority(ring, is_locked);
+	}
+
+	return flush_count;
+}
+
+/**
+ * i915_scheduler_flush - force all requests through the scheduler.
+ * @ring: Ring to be flushed
+ * @is_locked: Is the driver mutex lock held?
+ * For various reasons it is sometimes necessary to the scheduler out, e.g.
+ * due to ring reset.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked)
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv;
+	struct i915_scheduler *scheduler;
+	bool found;
+	int ret;
+	uint32_t count = 0;
+
+	if (!ring)
+		return -EINVAL;
+
+	dev_priv  = ring->dev->dev_private;
+	scheduler = dev_priv->scheduler;
+
+	if (!scheduler)
+		return 0;
+
+	WARN_ON(is_locked && (scheduler->flags[ring->id] & i915_sf_submitting));
+
+	do {
+		found = false;
+		spin_lock_irq(&scheduler->lock);
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (!I915_SQS_IS_QUEUED(node))
+				continue;
+
+			found = true;
+			break;
+		}
+		spin_unlock_irq(&scheduler->lock);
+
+		if (found) {
+			if (is_locked)
+				ret = i915_scheduler_submit(ring);
+			else
+				ret = i915_scheduler_submit_unlocked(ring);
+			if (ret < 0)
+				return ret;
+
+			count += ret;
+		}
+	} while (found);
+
+	return count;
+}
+

 /**
  * i915_scheduler_is_request_tracked - return info to say what the scheduler's
  * connection to this request is (if any).

drivers/gpu/drm/i915/i915_scheduler.h +3 -0
index a88adce..839b048 100644
@@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
  bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
  void i915_scheduler_wakeup(struct drm_device *dev);
  void i915_scheduler_work_handler(struct work_struct *work);
+int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
+int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+			       unsigned long stamp, bool is_locked);
  bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
  				       bool *completed, bool *busy);

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#47 lore raw

Re: [PATCH v5 19/35] drm/i915: Added scheduler flush calls to ring throttle and idle functions

Quoted 108 lines · Joonas Lahtinen
Quoted 107 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.

v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.

v3: Moved submit_max_priority() to this patch from an earlier patch is it is no longer required in the other.

v4: Corrected the format of a comment to keep the style checker happy. Downgraded a BUG_ON to a WARN_ON as the latter is preferred.

v5: Shuffled functions around to remove forward prototypes, removed similarly offensive white space and added documentation. Re-worked the mutex locking around the submit function. [Joonas Lahtinen]

Used lighter weight spinlocks.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
  drivers/gpu/drm/i915/i915_gem.c       |  24 ++++-
  drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/i915_scheduler.h |   3 +
  3 files changed, 204 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +23 -1
index a47a495..d946f53 100644
@@ -3786,6 +3786,10 @@ int i915_gpu_idle(struct drm_device *dev)
 
  	/* Flush everything onto the inactive list. */
  	for_each_ring(ring, dev_priv, i) {
+		ret = i915_scheduler_flush(ring, true);
+		if (ret < 0)
+			return ret;
+
  		if (!i915.enable_execlists) {
  			struct drm_i915_gem_request *req;
 
@@ -4519,7 +4523,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
  	unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
  	struct drm_i915_gem_request *request, *target = NULL;
  	unsigned reset_counter;
-	int ret;
+	int i, ret;
+	struct intel_engine_cs *ring;
 
  	ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
  	if (ret)
@@ -4529,6 +4534,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
  	if (ret)
  		return ret;
 
+	for_each_ring(ring, dev_priv, i) {
+		/*
+		 * Flush out scheduler entries that are getting 'stale'. Note
+		 * that the following recent_enough test will only check
+		 * against the time at which the request was submitted to the
+		 * hardware (i.e. when it left the scheduler) not the time it
+		 * was submitted to the driver.
+		 *
+		 * Also, there is not much point worring about busy return
+		 * codes from the scheduler flush call. Even if more work
+		 * cannot be submitted right now for whatever reason, we
+		 * still want to throttle against stale work that has already
+		 * been submitted.
+		 */
+		i915_scheduler_flush_stamp(ring, recent_enough, false);
+	}
+
  	spin_lock(&file_priv->mm.lock);
  	list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
  		if (time_after_eq(request->emitted_jiffies, recent_enough))
drivers/gpu/drm/i915/i915_scheduler.c +13 -0
index edab63d..8130a9c 100644
@@ -304,6 +304,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
   * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
   * And releasing the one before acquiring the other leads to other code
   * being run and interfering.
+ *
+ * Hence any caller that does not already have the mutex lock for other
+ * reasons should call i915_scheduler_submit_unlocked() instead in order to
+ * obtain the lock first.
   */
  static int i915_scheduler_submit(struct intel_engine_cs *ring)
  {
@@ -428,6 +432,22 @@ error:
  	return ret;
  }
 
+static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
+{
+	struct drm_device *dev = ring->dev;
+	int ret;
+
+	ret = i915_mutex_lock_interruptible(dev);
+	if (ret)
+		return ret;
+
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c error: · line 450

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c +7 -0
+	ret = i915_scheduler_submit(ring);
+
+	mutex_unlock(&dev->struct_mutex);
+
+	return ret;
+}
+

This kind of code should really be factored to upper level functions.

See below.

Quoted 34 lines
Quoted 34 lines
drivers/gpu/drm/i915/i915_scheduler.c +27 -0
  static void i915_generate_dependencies(struct i915_scheduler *scheduler,
  				       struct i915_scheduler_queue_entry *node,
  				       uint32_t ring)
@@ -917,6 +937,164 @@ void i915_scheduler_work_handler(struct work_struct *work)
  		i915_scheduler_process_work(ring);
  }
 
+static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
+					      bool is_locked)
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	int ret, count = 0;
+	bool found;
+
+	do {
+		found = false;
+		spin_lock_irq(&scheduler->lock);
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (!I915_SQS_IS_QUEUED(node))
+				continue;
+
+			if (node->priority < scheduler->priority_level_max)
+				continue;
+
+			found = true;
+			break;
+		}
+		spin_unlock_irq(&scheduler->lock);
+
+		if (!found)
+			break;
+
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 970

Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c +4 -0
+		if (is_locked)
+			ret = i915_scheduler_submit(ring);
+		else
+			ret = i915_scheduler_submit_unlocked(ring);

The widespread use of 'is_locked' notation is really bothering me, I think the difference in code path should be handled in the upper level, instead of being passed down, for example this will cause repetitive lock/unlock in loop.

My aim has been to try to keep the amount of code inside any given lock to the absolute minimum. One could just push the mutex lock acquire all the way up to the top level before any of the scheduler code is called. To me, that just seems an excessive and unnecessary hog of the global driver lock.

Quoted 51 lines
Quoted 39 lines
drivers/gpu/drm/i915/i915_scheduler.c +38 -0
+		if (ret < 0)
+			return ret;
+
+		count += ret;
+	} while (found);
+
+	return count;
+}
+
+/**
+ * i915_scheduler_flush_stamp - force requests of a given age through the
+ * scheduler.
+ * @ring: Ring to be flushed
+ * @target: Jiffy based time stamp to flush up to
+ * @is_locked: Is the driver mutex lock held?
+ * DRM has a throttle by age of request facility. This requires waiting for
+ * outstanding work over a given age. This function helps that by forcing
+ * queued batch buffers over said age through the system.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+			       unsigned long target,
+			       bool is_locked)
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv;
+	struct i915_scheduler *scheduler;
+	int flush_count = 0;
+
+	if (!ring)
+		return -EINVAL;
+
+	dev_priv  = ring->dev->dev_private;
+	scheduler = dev_priv->scheduler;
+
+	if (!scheduler)

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_work_handler(struct work_struct *work) · line 1018

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+		return 0;
+
+	if (is_locked && (scheduler->flags[ring->id] & i915_sf_submitting)) {
+		/*
+		 * Scheduler is busy already submitting another batch,
+		 * come back later rather than going recursive...
+		 */
+		return -EAGAIN;
+	}
+

Again, this would be handled better at higher level.

Quoted 3 lines

So I would like to see those 'is_locked' and other similar constructs factored out from the series. It should make the code much more robust to modify at later point.

As above. There is no need to lock the global driver mutex while doing internal scheduler processing. If you think the trade off of unnecessary locking for code simplification is best as just lock and be simple then sure, all the 'is_locked' stuff can disappear. Otherwise, I don't see any easy way to delay the mutex lock until it is actually required.

Quoted 91 lines

Regards, Joonas

Quoted 88 lines
drivers/gpu/drm/i915/i915_scheduler.c +79 -0
+	spin_lock_irq(&scheduler->lock);
+	i915_scheduler_priority_bump_clear(scheduler);
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+
+		if (node->stamp > target)
+			continue;
+
+		flush_count = i915_scheduler_priority_bump(scheduler,
+					node, scheduler->priority_level_max);
+	}
+	spin_unlock_irq(&scheduler->lock);
+
+	if (flush_count) {
+		DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", ring->name, flush_count);
+		flush_count = i915_scheduler_submit_max_priority(ring, is_locked);
+	}
+
+	return flush_count;
+}
+
+/**
+ * i915_scheduler_flush - force all requests through the scheduler.
+ * @ring: Ring to be flushed
+ * @is_locked: Is the driver mutex lock held?
+ * For various reasons it is sometimes necessary to the scheduler out, e.g.
+ * due to ring reset.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked)
+{
+	struct i915_scheduler_queue_entry *node;
+	struct drm_i915_private *dev_priv;
+	struct i915_scheduler *scheduler;
+	bool found;
+	int ret;
+	uint32_t count = 0;
+
+	if (!ring)
+		return -EINVAL;
+
+	dev_priv  = ring->dev->dev_private;
+	scheduler = dev_priv->scheduler;
+
+	if (!scheduler)
+		return 0;
+
+	WARN_ON(is_locked && (scheduler->flags[ring->id] & i915_sf_submitting));
+
+	do {
+		found = false;
+		spin_lock_irq(&scheduler->lock);
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (!I915_SQS_IS_QUEUED(node))
+				continue;
+
+			found = true;
+			break;
+		}
+		spin_unlock_irq(&scheduler->lock);
+
+		if (found) {
+			if (is_locked)
+				ret = i915_scheduler_submit(ring);
+			else
+				ret = i915_scheduler_submit_unlocked(ring);
+			if (ret < 0)
+				return ret;
+
+			count += ret;
+		}
+	} while (found);
+
+	return count;
+}
+
  /**
   * i915_scheduler_is_request_tracked - return info to say what the scheduler's
   * connection to this request is (if any).
drivers/gpu/drm/i915/i915_scheduler.h +0 -0
index a88adce..839b048 100644
@@ -94,6 +94,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
  bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.h int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe); · line 102

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.h +3 -0
  void i915_scheduler_wakeup(struct drm_device *dev);
  void i915_scheduler_work_handler(struct work_struct *work);
+int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
+int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+			       unsigned long stamp, bool is_locked);
  bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
  				       bool *completed, bool *busy);
 

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#48 lore raw

PATCH v5 20/35 drm/i915: Add scheduler hook to GPU reset

In reply to [email protected]

From: John Harrison <[email protected]>

When the watchdog resets the GPU, all interrupts get disabled despite the reference count remaining. As the scheduler probably had interrupts enabled during the reset (it would have been waiting for the bad batch to complete), it must be poked to tell it that the interrupt has been disabled.

v5: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c       |  2 ++
 drivers/gpu/drm/i915/i915_scheduler.c | 11 +++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 3 files changed, 14 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +2 -0
index d946f53..d7f7f7a 100644
@@ -3248,6 +3248,8 @@ static void i915_gem_reset_ring_cleanup(struct drm_i915_private *dev_priv,
 		buffer->last_retired_head = buffer->tail;
 		intel_ring_update_space(buffer);
 	}
+
+	i915_scheduler_reset_cleanup(ring);
 }
 
 void i915_gem_reset(struct drm_device *dev)
drivers/gpu/drm/i915/i915_scheduler.c +11 -0
index 8130a9c..4f25bf2 100644
@@ -778,6 +778,17 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
 	}
 }
 
+void i915_scheduler_reset_cleanup(struct intel_engine_cs *ring)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+	if (scheduler->flags[ring->id] & i915_sf_interrupts_enabled) {
+		ring->irq_put(ring);
+		scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
+	}
+}
+
 static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
 				  struct intel_engine_cs *ring,
 				  struct list_head *remove)
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 839b048..075befb 100644
@@ -89,6 +89,7 @@ bool i915_scheduler_is_enabled(struct drm_device *dev);
 int i915_scheduler_init(struct drm_device *dev);
 int i915_scheduler_closefile(struct drm_device *dev,
 			     struct drm_file *file);
+void i915_scheduler_reset_cleanup(struct intel_engine_cs *ring);
 void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
 int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 20/35
#49 lore raw

Re: [PATCH v5 20/35] drm/i915: Add scheduler hook to GPU reset

Quoted 40 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

When the watchdog resets the GPU, all interrupts get disabled despite the reference count remaining. As the scheduler probably had interrupts enabled during the reset (it would have been waiting for the bad batch to complete), it must be poked to tell it that the interrupt has been disabled.

v5: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c       |  2 ++
 drivers/gpu/drm/i915/i915_scheduler.c | 11 +++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 3 files changed, 14 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +2 -0
index d946f53..d7f7f7a 100644
@@ -3248,6 +3248,8 @@ static void i915_gem_reset_ring_cleanup(struct drm_i915_private *dev_priv,
 		buffer->last_retired_head = buffer->tail;
 		intel_ring_update_space(buffer);
 	}
+
+	i915_scheduler_reset_cleanup(ring);
 }
 
 void i915_gem_reset(struct drm_device *dev)
drivers/gpu/drm/i915/i915_scheduler.c +1 -0
index 8130a9c..4f25bf2 100644
@@ -778,6 +778,17 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
 	}
 }
 
+void i915_scheduler_reset_cleanup(struct intel_engine_cs *ring)

drivers/gpu/drm/i915/i915_scheduler.c void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node) · line 791

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+	if (scheduler->flags[ring->id] & i915_sf_interrupts_enabled) {
+		ring->irq_put(ring);
+		scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
+	}
+}
+

So I guess these flags are also protected by the struct_mutex? If so, I guess it looks ok.

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#50 lore raw

PATCH v5 21/35 drm/i915: Added a module parameter to allow the scheduler to be disabled

In reply to [email protected]

From: John Harrison <[email protected]>

It can be useful to be able to disable the GPU scheduler via a module parameter for debugging purposes.

v5: Converted from a multi-feature 'overrides' mask to a single 'enable' boolean. Further features (e.g. pre-emption) will now be separate 'enable' booleans added later. [Chris Wilson]

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_params.c    | 4 ++++
 drivers/gpu/drm/i915/i915_params.h    | 1 +
 drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++-
 3 files changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_params.c +4 -0
index d0eba58..0ef3159 100644
@@ -57,6 +57,7 @@ struct i915_params i915 __read_mostly = {
 	.edp_vswing = 0,
 	.enable_guc_submission = true,
 	.guc_log_level = -1,
+	.enable_scheduler = 0,
 };
 
 module_param_named(modeset, i915.modeset, int, 0400);
@@ -203,3 +204,6 @@ MODULE_PARM_DESC(enable_guc_submission, "Enable GuC submission (default:false)")
 module_param_named(guc_log_level, i915.guc_log_level, int, 0400);
 MODULE_PARM_DESC(guc_log_level,
 	"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
+
+module_param_named_unsafe(enable_scheduler, i915.enable_scheduler, int, 0600);
+MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable [default], 1 = enable)");
drivers/gpu/drm/i915/i915_params.h +1 -0
index 5299290..f855c86 100644
@@ -60,6 +60,7 @@ struct i915_params {
 	bool enable_guc_submission;
 	bool verbose_state_checks;
 	bool nuclear_pageflip;
+	int enable_scheduler;
 };
 
 extern struct i915_params i915 __read_mostly;
drivers/gpu/drm/i915/i915_scheduler.c +4 -1
index 4f25bf2..47d7de4 100644
@@ -34,6 +34,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
 {
 	struct drm_i915_private *dev_priv = dev->dev_private;
 
+	if (!i915.enable_scheduler)
+		return false;
+
 	return dev_priv->scheduler != NULL;
 }
 
@@ -548,7 +551,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
 	WARN_ON(!scheduler);
 
-	if (1/*!i915.enable_scheduler*/)
+	if (!i915.enable_scheduler)
 		return i915_scheduler_queue_execbuffer_bypass(qe);
 
 	node = kmalloc(sizeof(*node), GFP_KERNEL);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 21/35
#51 lore raw

Re: [PATCH v5 21/35] drm/i915: Added a module parameter to allow the scheduler to be disabled

Quoted 64 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

It can be useful to be able to disable the GPU scheduler via a module parameter for debugging purposes.

v5: Converted from a multi-feature 'overrides' mask to a single 'enable' boolean. Further features (e.g. pre-emption) will now be separate 'enable' booleans added later. [Chris Wilson]

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_params.c    | 4 ++++
 drivers/gpu/drm/i915/i915_params.h    | 1 +
 drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++-
 3 files changed, 9 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_params.c +4 -0
index d0eba58..0ef3159 100644
@@ -57,6 +57,7 @@ struct i915_params i915 __read_mostly = {
 	.edp_vswing = 0,
 	.enable_guc_submission = true,
 	.guc_log_level = -1,
+	.enable_scheduler = 0,
 };
 
 module_param_named(modeset, i915.modeset, int, 0400);
@@ -203,3 +204,6 @@ MODULE_PARM_DESC(enable_guc_submission, "Enable GuC submission (default:false)")
 module_param_named(guc_log_level, i915.guc_log_level, int, 0400);
 MODULE_PARM_DESC(guc_log_level,
 	"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
+
+module_param_named_unsafe(enable_scheduler, i915.enable_scheduler, int, 0600);
+MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable [default], 1 = enable)");
drivers/gpu/drm/i915/i915_params.h +1 -0
index 5299290..f855c86 100644
@@ -60,6 +60,7 @@ struct i915_params {
 	bool enable_guc_submission;
 	bool verbose_state_checks;
 	bool nuclear_pageflip;
+	int enable_scheduler;
 };
 
 extern struct i915_params i915 __read_mostly;
drivers/gpu/drm/i915/i915_scheduler.c +3 -0
index 4f25bf2..47d7de4 100644
@@ -34,6 +34,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
 {
 	struct drm_i915_private *dev_priv = dev->dev_private;
 
+	if (!i915.enable_scheduler)
+		return false;
+
 	return dev_priv->scheduler != NULL;
 }
 

drivers/gpu/drm/i915/i915_scheduler.c int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe) · line 557

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c +1 -1
@@ -548,7 +551,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
 	WARN_ON(!scheduler);
 
-	if (1/*!i915.enable_scheduler*/)
+	if (!i915.enable_scheduler)
 		return i915_scheduler_queue_execbuffer_bypass(qe);
 
 	node = kmalloc(sizeof(*node), GFP_KERNEL);

I did a double take here; maybe a comment along the lines of "if the scheduler is disabled, queue the buffer immediately" would help, and something similar for where the if (1) is added temporarily.

Doesn't matter too much though.

Thanks,
Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#52 lore raw

PATCH v5 22/35 drm/i915: Support for 'unflushed' ring idle

In reply to [email protected]

From: John Harrison <[email protected]>

When the seqno wraps around zero, the entire GPU is forced to be idle for some reason (possibly only to work around issues with hardware semaphores but no-one seems too sure!). This causes a problem if the force idle occurs at an inopportune moment such as in the middle of submitting a batch buffer. Specifically, it would lead to recursive submits - submitting work requires a new seqno, the new seqno requires idling the ring, idling the ring requires submitting work, submitting work requires a new seqno...

This change adds a 'flush' parameter to the idle function call which specifies whether the scheduler queues should be flushed out. I.e. is the call intended to just idle the ring as it is right now (no flush) or is it intended to force all outstanding work out of the system (with flush).

In the seqno wrap case, pending work is not an issue because the next operation will be to submit it. However, in other cases, the intention is to make sure everything that could be done has been done.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c         |  4 ++--
 drivers/gpu/drm/i915/intel_lrc.c        |  2 +-
 drivers/gpu/drm/i915/intel_ringbuffer.c | 17 +++++++++++++++--
 drivers/gpu/drm/i915/intel_ringbuffer.h |  2 +-
 4 files changed, 19 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +2 -2
index d7f7f7a..a249e52 100644
@@ -2564,7 +2564,7 @@ i915_gem_init_seqno(struct drm_device *dev, u32 seqno)
 
 	/* Carefully retire all requests without writing to the rings */
 	for_each_ring(ring, dev_priv, i) {
-		ret = intel_ring_idle(ring);
+		ret = intel_ring_idle(ring, false);
 		if (ret)
 			return ret;
 	}
@@ -3808,7 +3808,7 @@ int i915_gpu_idle(struct drm_device *dev)
 			i915_add_request_no_flush(req);
 		}
 
-		ret = intel_ring_idle(ring);
+		ret = intel_ring_idle(ring, true);
 		if (ret)
 			return ret;
 	}
drivers/gpu/drm/i915/intel_lrc.c +1 -1
index f4bab82..e056875 100644
@@ -1058,7 +1058,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *ring)
 	if (!intel_ring_initialized(ring))
 		return;
 
-	ret = intel_ring_idle(ring);
+	ret = intel_ring_idle(ring, true);
 	if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
 		DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
 			  ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.c +15 -2
index a2093f5..70ef9f0 100644
@@ -2288,9 +2288,22 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)
 	intel_ring_update_space(ringbuf);
 }
 
-int intel_ring_idle(struct intel_engine_cs *ring)
+int intel_ring_idle(struct intel_engine_cs *ring, bool flush)
 {
 	struct drm_i915_gem_request *req;
+	int ret;
+
+	/*
+	 * NB: Must not flush the scheduler if this idle request is from
+	 * within an execbuff submission (i.e. due to 'get_seqno' calling
+	 * 'wrap_seqno' calling 'idle'). As that would lead to recursive
+	 * flushes!
+	 */
+	if (flush) {
+		ret = i915_scheduler_flush(ring, true);
+		if (ret)
+			return ret;
+	}
 
 	/* Wait upon the last request to be completed */
 	if (list_empty(&ring->request_list))
@@ -3095,7 +3108,7 @@ intel_stop_ring_buffer(struct intel_engine_cs *ring)
 	if (!intel_ring_initialized(ring))
 		return;
 
-	ret = intel_ring_idle(ring);
+	ret = intel_ring_idle(ring, true);
 	if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
 		DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
 			  ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.h +1 -1
index ada93a9..cca476f 100644
@@ -478,7 +478,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
 int intel_ring_space(struct intel_ringbuffer *ringbuf);
 bool intel_ring_stopped(struct intel_engine_cs *ring);
 
-int __must_check intel_ring_idle(struct intel_engine_cs *ring);
+int __must_check intel_ring_idle(struct intel_engine_cs *ring, bool flush);
 void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno);
 int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
 int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 22/35
#53 lore raw

Re: [PATCH v5 22/35] drm/i915: Support for 'unflushed' ring idle

Quoted 108 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

When the seqno wraps around zero, the entire GPU is forced to be idle for some reason (possibly only to work around issues with hardware semaphores but no-one seems too sure!). This causes a problem if the force idle occurs at an inopportune moment such as in the middle of submitting a batch buffer. Specifically, it would lead to recursive submits - submitting work requires a new seqno, the new seqno requires idling the ring, idling the ring requires submitting work, submitting work requires a new seqno...

This change adds a 'flush' parameter to the idle function call which specifies whether the scheduler queues should be flushed out. I.e. is the call intended to just idle the ring as it is right now (no flush) or is it intended to force all outstanding work out of the system (with flush).

In the seqno wrap case, pending work is not an issue because the next operation will be to submit it. However, in other cases, the intention is to make sure everything that could be done has been done.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c         |  4 ++--
 drivers/gpu/drm/i915/intel_lrc.c        |  2 +-
 drivers/gpu/drm/i915/intel_ringbuffer.c | 17 +++++++++++++++--
 drivers/gpu/drm/i915/intel_ringbuffer.h |  2 +-
 4 files changed, 19 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +2 -2
index d7f7f7a..a249e52 100644
@@ -2564,7 +2564,7 @@ i915_gem_init_seqno(struct drm_device *dev, u32 seqno)
 
 	/* Carefully retire all requests without writing to the rings */
 	for_each_ring(ring, dev_priv, i) {
-		ret = intel_ring_idle(ring);
+		ret = intel_ring_idle(ring, false);
 		if (ret)
 			return ret;
 	}
@@ -3808,7 +3808,7 @@ int i915_gpu_idle(struct drm_device *dev)
 			i915_add_request_no_flush(req);
 		}
 
-		ret = intel_ring_idle(ring);
+		ret = intel_ring_idle(ring, true);
 		if (ret)
 			return ret;
 	}
drivers/gpu/drm/i915/intel_lrc.c +1 -1
index f4bab82..e056875 100644
@@ -1058,7 +1058,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *ring)
 	if (!intel_ring_initialized(ring))
 		return;
 
-	ret = intel_ring_idle(ring);
+	ret = intel_ring_idle(ring, true);
 	if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
 		DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
 			  ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.c +15 -2
index a2093f5..70ef9f0 100644
@@ -2288,9 +2288,22 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)
 	intel_ring_update_space(ringbuf);
 }
 
-int intel_ring_idle(struct intel_engine_cs *ring)
+int intel_ring_idle(struct intel_engine_cs *ring, bool flush)
 {
 	struct drm_i915_gem_request *req;
+	int ret;
+
+	/*
+	 * NB: Must not flush the scheduler if this idle request is from
+	 * within an execbuff submission (i.e. due to 'get_seqno' calling
+	 * 'wrap_seqno' calling 'idle'). As that would lead to recursive
+	 * flushes!
+	 */
+	if (flush) {
+		ret = i915_scheduler_flush(ring, true);
+		if (ret)
+			return ret;
+	}
 
 	/* Wait upon the last request to be completed */
 	if (list_empty(&ring->request_list))
@@ -3095,7 +3108,7 @@ intel_stop_ring_buffer(struct intel_engine_cs *ring)
 	if (!intel_ring_initialized(ring))
 		return;
 
-	ret = intel_ring_idle(ring);
+	ret = intel_ring_idle(ring, true);
 	if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
 		DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
 			  ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.h +0 -0
index ada93a9..cca476f 100644

drivers/gpu/drm/i915/intel_ringbuffer.h void intel_ring_update_space(struct intel_ringbuffer *ringbuf); · line 484

Quoted 8 lines
drivers/gpu/drm/i915/intel_ringbuffer.h +1 -1
@@ -478,7 +478,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
 int intel_ring_space(struct intel_ringbuffer *ringbuf);
 bool intel_ring_stopped(struct intel_engine_cs *ring);
 
-int __must_check intel_ring_idle(struct intel_engine_cs *ring);
+int __must_check intel_ring_idle(struct intel_engine_cs *ring, bool flush);
 void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno);
 int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
 int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);

Maybe Chris remembers the history here; I think the wraparound idle goes all the way back to Eric's original work with wrapping (something we had a lot of trouble with early on iirc).

My only suggestion here is to add wrappers for a new __intel_ring_idle(ring, bool), one for the existing usage of intel_ring_idle(ring) (which would pass false) and a new intel_ring_flush(ring) that passes true, along with some kdoc explaining the difference. Otherwise everyone will have to look up the param all the time. :)

With that change (because I'm a bool param hater, at least in exposed APIs):

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#54 lore raw

PATCH v5 23/35 drm/i915: Defer seqno allocation until actual hardware submission time

In reply to [email protected]

From: John Harrison <[email protected]>

The seqno value is now only used for the final test for completion of a request. It is no longer used to track the request through the software stack. Thus it is no longer necessary to allocate the seqno immediately with the request. Instead, it can be done lazily and left until the request is actually sent to the hardware. This is particular advantageous with a GPU scheduler as the requests can then be re-ordered between their creation and their hardware submission without having out of order seqnos.

v2: i915_add_request() can't fail!

Combine with 'drm/i915: Assign seqno at start of exec_final()' Various bits of code during the execbuf code path need a seqno value to be assigned to the request. This change makes this assignment explicit at the start of submission_final() rather than relying on an auto-generated seqno to have happened already. This is in preparation for a future patch which changes seqno values to be assigned lazily (during add_request).

v3: Updated to use locally cached request pointer.

v4: Changed some white space and comment formatting to keep the style checker happy.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  1 +
 drivers/gpu/drm/i915/i915_gem.c            | 23 ++++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 14 ++++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           | 14 ++++++++++++++
 4 files changed, 51 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 5eeeced..071a27b 100644
@@ -2257,6 +2257,7 @@ struct drm_i915_gem_request {
 	  * has finished processing this request.
 	  */
 	u32 seqno;
+	u32 reserved_seqno;
 
 	/* Unique identifier which can be used for trace points & debug */
 	uint32_t uniq;
drivers/gpu/drm/i915/i915_gem.c +22 -1
index a249e52..a2c136d 100644
@@ -2616,6 +2616,11 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)
 
 	/* reserve 0 for non-seqno */
 	if (dev_priv->next_seqno == 0) {
+		/*
+		 * Why is the full re-initialisation required? Is it only for
+		 * hardware semaphores? If so, could skip it in the case where
+		 * semaphores are disabled?
+		 */
 		int ret = i915_gem_init_seqno(dev, 0);
 		if (ret)
 			return ret;
@@ -2673,6 +2678,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,
 		WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
 	}
 
+	/* Make the request's seqno 'live': */
+	if (!request->seqno) {
+		request->seqno = request->reserved_seqno;
+		WARN_ON(request->seqno != dev_priv->last_seqno);
+	}
+
 	/* Record the position of the start of the request so that
 	 * should we detect the updated seqno part-way through the
 	 * GPU processing the request, we never over-estimate the
@@ -2930,6 +2941,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 
 	list_for_each_entry_safe(req, req_next, &ring->fence_signal_list, signal_link) {
 		if (!req->cancelled) {
+			/* How can this happen? */
+			WARN_ON(req->seqno == 0);
+
 			if (!i915_seqno_passed(seqno, req->seqno))
 				break;
 		}
@@ -3079,7 +3093,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
 	if (req == NULL)
 		return -ENOMEM;
 
-	ret = i915_gem_get_seqno(ring->dev, &req->seqno);
+	/*
+	 * Assign an identifier to track this request through the hardware
+	 * but don't make it live yet. It could change in the future if this
+	 * request gets overtaken. However, it still needs to be allocated
+	 * in advance because the point of submission must not fail and seqno
+	 * allocation can fail.
+	 */
+	ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
 	if (ret)
 		goto err;
 
drivers/gpu/drm/i915/i915_gem_execbuffer.c +14 -0
index f45f4dc..b9ad0fd 100644
@@ -1295,6 +1295,20 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	/* Make sure the request's seqno is the latest and greatest: */
+	if (req->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+		if (ret)
+			return ret;
+	}
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front.
+	 */
+	WARN_ON(req->seqno);
+	req->seqno = req->reserved_seqno;
+	WARN_ON(req->seqno != dev_priv->last_seqno);
+
 	ret = intel_ring_reserve_space(req);
 	if (ret)
 		goto error;
drivers/gpu/drm/i915/intel_lrc.c +14 -0
index e056875..9c7a79a 100644
@@ -978,6 +978,20 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	/* Make sure the request's seqno is the latest and greatest: */
+	if (req->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+		if (ret)
+			return ret;
+	}
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front.
+	 */
+	WARN_ON(req->seqno);
+	req->seqno = req->reserved_seqno;
+	WARN_ON(req->seqno != dev_priv->last_seqno);
+
 	ret = intel_logical_ring_reserve_space(req);
 	if (ret)
 		goto err;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#55 lore raw

Re: [PATCH v5 23/35] drm/i915: Defer seqno allocation until actual hardware submission time

Quoted 110 lines · On to, 2016-02-18 at 14:27 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The seqno value is now only used for the final test for completion of a request. It is no longer used to track the request through the software stack. Thus it is no longer necessary to allocate the seqno immediately with the request. Instead, it can be done lazily and left until the request is actually sent to the hardware. This is particular advantageous with a GPU scheduler as the requests can then be re-ordered between their creation and their hardware submission without having out of order seqnos.

v2: i915_add_request() can't fail!

Combine with 'drm/i915: Assign seqno at start of exec_final()' Various bits of code during the execbuf code path need a seqno value to be assigned to the request. This change makes this assignment explicit at the start of submission_final() rather than relying on an auto-generated seqno to have happened already. This is in preparation for a future patch which changes seqno values to be assigned lazily (during add_request).

v3: Updated to use locally cached request pointer.

v4: Changed some white space and comment formatting to keep the style checker happy.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  1 + drivers/gpu/drm/i915/i915_gem.c            | 23 ++++++++++++++++++++++- drivers/gpu/drm/i915/i915_gem_execbuffer.c | 14 ++++++++++++++ drivers/gpu/drm/i915/intel_lrc.c           | 14 ++++++++++++++

 4 files changed, 51 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 5eeeced..071a27b 100644
@@ -2257,6 +2257,7 @@ struct drm_i915_gem_request {
  	  * has finished processing this request.
  	  */
  	u32 seqno;
+	u32 reserved_seqno;


  /* Unique identifier which can be used for trace points & debug */ uint32_t uniq;

drivers/gpu/drm/i915/i915_gem.c +0 -0
index a249e52..a2c136d 100644
@@ -2616,6 +2616,11 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)


  /* reserve 0 for non-seqno */
  if (dev_priv->next_seqno == 0) {

drivers/gpu/drm/i915/i915_gem.c +5 -0
+		/*
+		 * Why is the full re-initialisation required? Is it only for
+		 * hardware semaphores? If so, could skip it in the case where
+		 * semaphores are disabled?
+		 */

  int ret = i915_gem_init_seqno(dev, 0); if (ret)
  return ret;

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -2673,6 +2678,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,
  		WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
  	}

drivers/gpu/drm/i915/i915_gem.c +6 -0
+	/* Make the request's seqno 'live': */
+	if (!request->seqno) {
+		request->seqno = request->reserved_seqno;
+		WARN_ON(request->seqno != dev_priv->last_seqno);
+	}
+

  /* Record the position of the start of the request so that
   * should we detect the updated seqno part-way through the
   * GPU processing the request, we never over-estimate the

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -2930,6 +2941,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)


  list_for_each_entry_safe(req, req_next, &ring->fence_signal_list, signal_link) { if (!req->cancelled) {

drivers/gpu/drm/i915/i915_gem.c +3 -0
+			/* How can this happen? */
+			WARN_ON(req->seqno == 0);
+

  if (!i915_seqno_passed(seqno, req->seqno)) break;
  }

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -3079,7 +3093,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
  	if (req == NULL)
  		return -ENOMEM;

drivers/gpu/drm/i915/i915_gem.c +8 -1
-	ret = i915_gem_get_seqno(ring->dev, &req->seqno);
+	/*
+	 * Assign an identifier to track this request through the hardware
+	 * but don't make it live yet. It could change in the future if this
+	 * request gets overtaken. However, it still needs to be allocated
+	 * in advance because the point of submission must not fail and seqno
+	 * allocation can fail.
+	 */
+	ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);

  if (ret)
  goto err;

drivers/gpu/drm/i915/i915_gem_execbuffer.c +0 -0
index f45f4dc..b9ad0fd 100644
@@ -1295,6 +1295,20 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
  	/* The mutex must be acquired before calling this function */
  	WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));

drivers/gpu/drm/i915/i915_gem_execbuffer.c int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params) · line 1303

Quoted 6 lines
drivers/gpu/drm/i915/i915_gem_execbuffer.c +6 -0
+	/* Make sure the request's seqno is the latest and greatest: */
+	if (req->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+		if (ret)
+			return ret;
+	}

Would this situation be caused by reordering? Kinda strange to pregenerate and then re-generate if some request got prioritized over us. I'd assume that to happen constantly with compositor running.

What exactly stops from generating the seqno always at submission time, if we're able to re-generate it anyway?

Regards, Joonas

Quoted 36 lines
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8 -0
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front.
+	 */
+	WARN_ON(req->seqno);
+	req->seqno = req->reserved_seqno;
+	WARN_ON(req->seqno != dev_priv->last_seqno);
+

  ret = intel_ring_reserve_space(req);
  if (ret)
  goto error;

drivers/gpu/drm/i915/intel_lrc.c +0 -0
index e056875..9c7a79a 100644
@@ -978,6 +978,20 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
  	/* The mutex must be acquired before calling this function */
  	WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));

drivers/gpu/drm/i915/intel_lrc.c +14 -0
+	/* Make sure the request's seqno is the latest and greatest: */
+	if (req->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+		if (ret)
+			return ret;
+	}
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front.
+	 */
+	WARN_ON(req->seqno);
+	req->seqno = req->reserved_seqno;
+	WARN_ON(req->seqno != dev_priv->last_seqno);
+

  ret = intel_logical_ring_reserve_space(req); if (ret)
  goto err;

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#56 lore raw

PATCH v5 24/35 drm/i915: Added trace points to scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.

v2: Updated for new request completion tracking implementation.

v3: Updated for changes to node kill code.

v4: Wrapped some long lines to keep the style checker happy.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |   2 +
 drivers/gpu/drm/i915/i915_scheduler.c      |  26 ++++
 drivers/gpu/drm/i915/i915_trace.h          | 196 +++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           |   2 +
 4 files changed, 226 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +2 -0
index b9ad0fd..d4de8c7 100644
@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +26 -0
index 47d7de4..e56ce08 100644
@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
 	/* Seqno will be reassigned on relaunch */
 	node->params.request->seqno = 0;
 	node->status = i915_sqs_queued;
+	trace_i915_scheduler_unfly(node->params.ring, node);
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /*
@@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
 	WARN_ON(!node);
 	WARN_ON(I915_SQS_IS_COMPLETE(node));
 
+	if (I915_SQS_IS_FLYING(node))
+		trace_i915_scheduler_unfly(node->params.ring, node);
+
 	node->status = i915_sqs_dead;
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /* Mark a node as in flight on the hardware. */
@@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
 
 	node->status = i915_sqs_flying;
 
+	trace_i915_scheduler_fly(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
 		bool success = true;
 
@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		INIT_LIST_HEAD(&best->link);
 		best->status  = i915_sqs_popped;
 
+		trace_i915_scheduler_node_state_change(ring, best);
+
 		ret = 0;
 	} else {
 		/* Can only get here if:
@@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		}
 	}
 
+	trace_i915_scheduler_pop_from_queue(ring, best);
+
 	*pop_node = best;
 	return ret;
 }
@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
 	struct i915_scheduler *scheduler = dev_priv->scheduler;
 	int ret;
 
+	trace_i915_scheduler_queue(qe->params.ring, qe);
+
 	intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
 
 	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	not_flying = i915_scheduler_count_flying(scheduler, ring) <
 						 scheduler->min_flying;
 
+	trace_i915_scheduler_queue(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	spin_unlock_irq(&scheduler->lock);
 
 	if (not_flying)
@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
 	unsigned long flags;
 
+	trace_i915_scheduler_landing(req);
+
 	if (!node)
 		return false;
 
@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	else
 		node->status = i915_sqs_complete;
 
+	trace_i915_scheduler_node_state_change(req->ring, node);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	return true;
@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
 	/* Launch more packets now? */
 	do_submit = (queued > 0) && (flying < scheduler->min_flying);
 
+	trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
 	spin_unlock_irq(&scheduler->lock);
 
 	return do_submit;
@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
 		node = list_first_entry(&remove, typeof(*node), link);
 		list_del(&node->link);
 
+		trace_i915_scheduler_destroy(ring, node);
+
 		/* Free up all the DRM references */
 		i915_scheduler_clean_node(node);
 
drivers/gpu/drm/i915/i915_trace.h +196 -0

208 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.c +2 -0
index 9c7a79a..2b9f49c 100644
@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 24/35
#57 lore raw

Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler

Quoted 361 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.

v2: Updated for new request completion tracking implementation.

v3: Updated for changes to node kill code.

v4: Wrapped some long lines to keep the style checker happy.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |   2 +
 drivers/gpu/drm/i915/i915_scheduler.c      |  26 ++++
 drivers/gpu/drm/i915/i915_trace.h          | 196 +++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           |   2 +
 4 files changed, 226 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +2 -0
index b9ad0fd..d4de8c7 100644
@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +26 -0
index 47d7de4..e56ce08 100644
@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
 	/* Seqno will be reassigned on relaunch */
 	node->params.request->seqno = 0;
 	node->status = i915_sqs_queued;
+	trace_i915_scheduler_unfly(node->params.ring, node);
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /*
@@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
 	WARN_ON(!node);
 	WARN_ON(I915_SQS_IS_COMPLETE(node));
 
+	if (I915_SQS_IS_FLYING(node))
+		trace_i915_scheduler_unfly(node->params.ring, node);
+
 	node->status = i915_sqs_dead;
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /* Mark a node as in flight on the hardware. */
@@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
 
 	node->status = i915_sqs_flying;
 
+	trace_i915_scheduler_fly(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
 		bool success = true;
 
@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		INIT_LIST_HEAD(&best->link);
 		best->status  = i915_sqs_popped;
 
+		trace_i915_scheduler_node_state_change(ring, best);
+
 		ret = 0;
 	} else {
 		/* Can only get here if:
@@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		}
 	}
 
+	trace_i915_scheduler_pop_from_queue(ring, best);
+
 	*pop_node = best;
 	return ret;
 }
@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
 	struct i915_scheduler *scheduler = dev_priv->scheduler;
 	int ret;
 
+	trace_i915_scheduler_queue(qe->params.ring, qe);
+
 	intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
 
 	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	not_flying = i915_scheduler_count_flying(scheduler, ring) <
 						 scheduler->min_flying;
 
+	trace_i915_scheduler_queue(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	spin_unlock_irq(&scheduler->lock);
 
 	if (not_flying)
@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
 	unsigned long flags;
 
+	trace_i915_scheduler_landing(req);
+
 	if (!node)
 		return false;
 
@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	else
 		node->status = i915_sqs_complete;
 
+	trace_i915_scheduler_node_state_change(req->ring, node);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	return true;
@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
 	/* Launch more packets now? */
 	do_submit = (queued > 0) && (flying < scheduler->min_flying);
 
+	trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
 	spin_unlock_irq(&scheduler->lock);
 
 	return do_submit;
@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
 		node = list_first_entry(&remove, typeof(*node), link);
 		list_del(&node->link);
 
+		trace_i915_scheduler_destroy(ring, node);
+
 		/* Free up all the DRM references */
 		i915_scheduler_clean_node(node);
 
drivers/gpu/drm/i915/i915_trace.h +196 -0
index 455c215..c3c4e58 100644
@@ -9,6 +9,7 @@
 #include "i915_drv.h"
 #include "intel_drv.h"
 #include "intel_ringbuffer.h"
+#include "i915_scheduler.h"
 
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM i915
@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
 		  __entry->dev, __entry->ring, __entry->to, __entry->vm)
 );
 
+TRACE_EVENT(i915_scheduler_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_fly,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_unfly,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_landing,
+	    TP_PROTO(struct drm_i915_gem_request *req),
+	    TP_ARGS(req),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     __field(u32, status)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring   = req->ring->id;
+			   __entry->uniq   = req->uniq;
+			   __entry->seqno  = req->seqno;
+			   __entry->status = req->scheduler_qe ?
+						req->scheduler_qe->status : ~0U;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno,
+		      __entry->status)
+);
+
+TRACE_EVENT(i915_scheduler_remove,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     u32 min_seqno, bool do_submit),
+	    TP_ARGS(ring, min_seqno, do_submit),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, min_seqno)
+			     __field(bool, do_submit)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring      = ring->id;
+			   __entry->min_seqno = min_seqno;
+			   __entry->do_submit = do_submit;
+			   ),
+
+	    TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
+		      __entry->ring, __entry->min_seqno, __entry->do_submit)
+);
+
+TRACE_EVENT(i915_scheduler_destroy,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_pop_from_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_node_state_change,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     __field(u32, status)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring   = ring->id;
+			   __entry->uniq   = node ? node->params.request->uniq  : 0;
+			   __entry->seqno  = node->params.request->seqno;
+			   __entry->status = node->status;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno,
+		      __entry->status)
+);
+
+TRACE_EVENT(i915_gem_ring_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_execbuffer_params *params),
+	    TP_ARGS(ring, params),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = params->request->uniq;
+			   __entry->seqno = params->request->seqno;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
+		      __entry->uniq, __entry->seqno)
+);
+
 #endif /* _I915_TRACE_H_ */
 
 /* This part must be outside protection */
drivers/gpu/drm/i915/intel_lrc.c +0 -0
index 9c7a79a..2b9f49c 100644

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 961

Quoted 8 lines
drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)

Looks fine.

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#58 lore raw

Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler

In reply to [email protected]
Quoted 361 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.

v2: Updated for new request completion tracking implementation.

v3: Updated for changes to node kill code.

v4: Wrapped some long lines to keep the style checker happy.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |   2 +
 drivers/gpu/drm/i915/i915_scheduler.c      |  26 ++++
 drivers/gpu/drm/i915/i915_trace.h          | 196 +++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           |   2 +
 4 files changed, 226 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +2 -0
index b9ad0fd..d4de8c7 100644
@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +26 -0
index 47d7de4..e56ce08 100644
@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
 	/* Seqno will be reassigned on relaunch */
 	node->params.request->seqno = 0;
 	node->status = i915_sqs_queued;
+	trace_i915_scheduler_unfly(node->params.ring, node);
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /*
@@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
 	WARN_ON(!node);
 	WARN_ON(I915_SQS_IS_COMPLETE(node));
 
+	if (I915_SQS_IS_FLYING(node))
+		trace_i915_scheduler_unfly(node->params.ring, node);
+
 	node->status = i915_sqs_dead;
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /* Mark a node as in flight on the hardware. */
@@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
 
 	node->status = i915_sqs_flying;
 
+	trace_i915_scheduler_fly(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
 		bool success = true;
 
@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		INIT_LIST_HEAD(&best->link);
 		best->status  = i915_sqs_popped;
 
+		trace_i915_scheduler_node_state_change(ring, best);
+
 		ret = 0;
 	} else {
 		/* Can only get here if:
@@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		}
 	}
 
+	trace_i915_scheduler_pop_from_queue(ring, best);
+
 	*pop_node = best;
 	return ret;
 }
@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
 	struct i915_scheduler *scheduler = dev_priv->scheduler;
 	int ret;
 
+	trace_i915_scheduler_queue(qe->params.ring, qe);
+
 	intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
 
 	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	not_flying = i915_scheduler_count_flying(scheduler, ring) <
 						 scheduler->min_flying;
 
+	trace_i915_scheduler_queue(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	spin_unlock_irq(&scheduler->lock);
 
 	if (not_flying)
@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
 	unsigned long flags;
 
+	trace_i915_scheduler_landing(req);
+
 	if (!node)
 		return false;
 
@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	else
 		node->status = i915_sqs_complete;
 
+	trace_i915_scheduler_node_state_change(req->ring, node);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	return true;
@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
 	/* Launch more packets now? */
 	do_submit = (queued > 0) && (flying < scheduler->min_flying);
 
+	trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
 	spin_unlock_irq(&scheduler->lock);
 
 	return do_submit;
@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
 		node = list_first_entry(&remove, typeof(*node), link);
 		list_del(&node->link);
 
+		trace_i915_scheduler_destroy(ring, node);
+
 		/* Free up all the DRM references */
 		i915_scheduler_clean_node(node);
 
drivers/gpu/drm/i915/i915_trace.h +196 -0
index 455c215..c3c4e58 100644
@@ -9,6 +9,7 @@
 #include "i915_drv.h"
 #include "intel_drv.h"
 #include "intel_ringbuffer.h"
+#include "i915_scheduler.h"
 
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM i915
@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
 		  __entry->dev, __entry->ring, __entry->to, __entry->vm)
 );
 
+TRACE_EVENT(i915_scheduler_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_fly,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_unfly,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_landing,
+	    TP_PROTO(struct drm_i915_gem_request *req),
+	    TP_ARGS(req),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     __field(u32, status)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring   = req->ring->id;
+			   __entry->uniq   = req->uniq;
+			   __entry->seqno  = req->seqno;
+			   __entry->status = req->scheduler_qe ?
+						req->scheduler_qe->status : ~0U;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno,
+		      __entry->status)
+);
+
+TRACE_EVENT(i915_scheduler_remove,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     u32 min_seqno, bool do_submit),
+	    TP_ARGS(ring, min_seqno, do_submit),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, min_seqno)
+			     __field(bool, do_submit)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring      = ring->id;
+			   __entry->min_seqno = min_seqno;
+			   __entry->do_submit = do_submit;
+			   ),
+
+	    TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
+		      __entry->ring, __entry->min_seqno, __entry->do_submit)
+);
+
+TRACE_EVENT(i915_scheduler_destroy,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_pop_from_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_node_state_change,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     __field(u32, status)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring   = ring->id;
+			   __entry->uniq   = node ? node->params.request->uniq  : 0;
+			   __entry->seqno  = node->params.request->seqno;
+			   __entry->status = node->status;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno,
+		      __entry->status)
+);
+
+TRACE_EVENT(i915_gem_ring_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_execbuffer_params *params),
+	    TP_ARGS(ring, params),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = params->request->uniq;
+			   __entry->seqno = params->request->seqno;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
+		      __entry->uniq, __entry->seqno)
+);
+
 #endif /* _I915_TRACE_H_ */
 
 /* This part must be outside protection */
drivers/gpu/drm/i915/intel_lrc.c +0 -0
index 9c7a79a..2b9f49c 100644

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 961

Quoted 8 lines
drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)

Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.

Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#59 lore raw

Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler

Quoted 364 lines · Jesse Barnes
Quoted 363 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.

v2: Updated for new request completion tracking implementation.

v3: Updated for changes to node kill code.

v4: Wrapped some long lines to keep the style checker happy.

For: VIZ-1587

---
  drivers/gpu/drm/i915/i915_gem_execbuffer.c |   2 +
  drivers/gpu/drm/i915/i915_scheduler.c      |  26 ++++
  drivers/gpu/drm/i915/i915_trace.h          | 196 +++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/intel_lrc.c           |   2 +
  4 files changed, 226 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +2 -0
index b9ad0fd..d4de8c7 100644
@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
  	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
  	qe = container_of(params, typeof(*qe), params);
  	ret = i915_scheduler_queue_execbuffer(qe);
  	if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +26 -0
index 47d7de4..e56ce08 100644
@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
  	/* Seqno will be reassigned on relaunch */
  	node->params.request->seqno = 0;
  	node->status = i915_sqs_queued;
+	trace_i915_scheduler_unfly(node->params.ring, node);
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
  }
 
  /*
@@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
  	WARN_ON(!node);
  	WARN_ON(I915_SQS_IS_COMPLETE(node));
 
+	if (I915_SQS_IS_FLYING(node))
+		trace_i915_scheduler_unfly(node->params.ring, node);
+
  	node->status = i915_sqs_dead;
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
  }
 
  /* Mark a node as in flight on the hardware. */
@@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
 
  	node->status = i915_sqs_flying;
 
+	trace_i915_scheduler_fly(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
  	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
  		bool success = true;
 
@@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
  		INIT_LIST_HEAD(&best->link);
  		best->status  = i915_sqs_popped;
 
+		trace_i915_scheduler_node_state_change(ring, best);
+
  		ret = 0;
  	} else {
  		/* Can only get here if:
@@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
  		}
  	}
 
+	trace_i915_scheduler_pop_from_queue(ring, best);
+
  	*pop_node = best;
  	return ret;
  }
@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
  	struct i915_scheduler *scheduler = dev_priv->scheduler;
  	int ret;
 
+	trace_i915_scheduler_queue(qe->params.ring, qe);
+
  	intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
 
  	scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
  	not_flying = i915_scheduler_count_flying(scheduler, ring) <
  						 scheduler->min_flying;
 
+	trace_i915_scheduler_queue(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
  	spin_unlock_irq(&scheduler->lock);
 
  	if (not_flying)
@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
  	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
  	unsigned long flags;
 
+	trace_i915_scheduler_landing(req);
+
  	if (!node)
  		return false;
 
@@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
  	else
  		node->status = i915_sqs_complete;
 
+	trace_i915_scheduler_node_state_change(req->ring, node);
+
  	spin_unlock_irqrestore(&scheduler->lock, flags);
 
  	return true;
@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
  	/* Launch more packets now? */
  	do_submit = (queued > 0) && (flying < scheduler->min_flying);
 
+	trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
  	spin_unlock_irq(&scheduler->lock);
 
  	return do_submit;
@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
  		node = list_first_entry(&remove, typeof(*node), link);
  		list_del(&node->link);
 
+		trace_i915_scheduler_destroy(ring, node);
+
  		/* Free up all the DRM references */
  		i915_scheduler_clean_node(node);
 
drivers/gpu/drm/i915/i915_trace.h +196 -0
index 455c215..c3c4e58 100644
@@ -9,6 +9,7 @@
  #include "i915_drv.h"
  #include "intel_drv.h"
  #include "intel_ringbuffer.h"
+#include "i915_scheduler.h"
 
  #undef TRACE_SYSTEM
  #define TRACE_SYSTEM i915
@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
  		  __entry->dev, __entry->ring, __entry->to, __entry->vm)
  );
 
+TRACE_EVENT(i915_scheduler_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_fly,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_unfly,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_landing,
+	    TP_PROTO(struct drm_i915_gem_request *req),
+	    TP_ARGS(req),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     __field(u32, status)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring   = req->ring->id;
+			   __entry->uniq   = req->uniq;
+			   __entry->seqno  = req->seqno;
+			   __entry->status = req->scheduler_qe ?
+						req->scheduler_qe->status : ~0U;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno,
+		      __entry->status)
+);
+
+TRACE_EVENT(i915_scheduler_remove,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     u32 min_seqno, bool do_submit),
+	    TP_ARGS(ring, min_seqno, do_submit),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, min_seqno)
+			     __field(bool, do_submit)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring      = ring->id;
+			   __entry->min_seqno = min_seqno;
+			   __entry->do_submit = do_submit;
+			   ),
+
+	    TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
+		      __entry->ring, __entry->min_seqno, __entry->do_submit)
+);
+
+TRACE_EVENT(i915_scheduler_destroy,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_pop_from_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = node ? node->params.request->uniq  : 0;
+			   __entry->seqno = node ? node->params.request->seqno : 0;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_node_state_change,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_scheduler_queue_entry *node),
+	    TP_ARGS(ring, node),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     __field(u32, status)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring   = ring->id;
+			   __entry->uniq   = node ? node->params.request->uniq  : 0;
+			   __entry->seqno  = node->params.request->seqno;
+			   __entry->status = node->status;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+		      __entry->ring, __entry->uniq, __entry->seqno,
+		      __entry->status)
+);
+
+TRACE_EVENT(i915_gem_ring_queue,
+	    TP_PROTO(struct intel_engine_cs *ring,
+		     struct i915_execbuffer_params *params),
+	    TP_ARGS(ring, params),
+
+	    TP_STRUCT__entry(
+			     __field(u32, ring)
+			     __field(u32, uniq)
+			     __field(u32, seqno)
+			     ),
+
+	    TP_fast_assign(
+			   __entry->ring  = ring->id;
+			   __entry->uniq  = params->request->uniq;
+			   __entry->seqno = params->request->seqno;
+			   ),
+
+	    TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
+		      __entry->uniq, __entry->seqno)
+);
+
  #endif /* _I915_TRACE_H_ */
 
  /* This part must be outside protection */
drivers/gpu/drm/i915/intel_lrc.c +0 -0
index 9c7a79a..2b9f49c 100644
@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
  	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
Quoted 7 lines

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 961

Quoted 6 lines
drivers/gpu/drm/i915/intel_lrc.c +2 -0
+	trace_i915_gem_ring_queue(ring, params);
+
  	qe = container_of(params, typeof(*qe), params);
  	ret = i915_scheduler_queue_execbuffer(qe);
  	if (ret)

Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.

Nothing at the moment. They have been used to debug issues by looking through trace logs. Also the validation team have tests that hook in to the various trace points to verify the internal operation of the scheduler and other parts of the i915 driver.

Quoted 2 lines

Jesse

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#60 lore raw

Re: [PATCH v5 24/35] drm/i915: Added trace points to scheduler

Quoted 345 lines · John Harrison
Quoted 336 lines · Jesse Barnes
Quoted 335 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.

v2: Updated for new request completion tracking implementation.

v3: Updated for changes to node kill code.

v4: Wrapped some long lines to keep the style checker happy.

For: VIZ-1587

---
  drivers/gpu/drm/i915/i915_gem_execbuffer.c |   2 +
  drivers/gpu/drm/i915/i915_scheduler.c      |  26 ++++
  drivers/gpu/drm/i915/i915_trace.h          | 196 +++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/intel_lrc.c           |   2 +
  4 files changed, 226 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +1 -0
index b9ad0fd..d4de8c7 100644
@@ -1272,6 +1272,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
        i915_gem_execbuffer_move_to_active(vmas, params->request);
  +    trace_i915_gem_ring_queue(ring, params);
+
      qe = container_of(params, typeof(*qe), params);
      ret = i915_scheduler_queue_execbuffer(qe);
      if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +16 -0
index 47d7de4..e56ce08 100644
@@ -88,6 +88,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
      /* Seqno will be reassigned on relaunch */
      node->params.request->seqno = 0;
      node->status = i915_sqs_queued;
+    trace_i915_scheduler_unfly(node->params.ring, node);
+    trace_i915_scheduler_node_state_change(node->params.ring, node);
  }
    /*
@@ -99,7 +101,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
      WARN_ON(!node);
      WARN_ON(I915_SQS_IS_COMPLETE(node));
  +    if (I915_SQS_IS_FLYING(node))
+        trace_i915_scheduler_unfly(node->params.ring, node);
+
      node->status = i915_sqs_dead;
+    trace_i915_scheduler_node_state_change(node->params.ring, node);
  }
    /* Mark a node as in flight on the hardware. */
@@ -124,6 +130,9 @@ static int i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
        node->status = i915_sqs_flying;
  +    trace_i915_scheduler_fly(ring, node);
+    trace_i915_scheduler_node_state_change(ring, node);
+
      if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
          bool success = true;
  @@ -280,6 +289,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
          INIT_LIST_HEAD(&best->link);
          best->status  = i915_sqs_popped;
  +        trace_i915_scheduler_node_state_change(ring, best);
+
          ret = 0;
      } else {
          /* Can only get here if:
@@ -297,6 +308,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
          }
      }
  +    trace_i915_scheduler_pop_from_queue(ring, best);
+
      *pop_node = best;
      return ret;
  }
@@ -506,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
      struct i915_scheduler *scheduler = dev_priv->scheduler;
      int ret;
  +    trace_i915_scheduler_queue(qe->params.ring, qe);
+
      intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
        scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
@@ -628,6 +643,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
      not_flying = i915_scheduler_count_flying(scheduler, ring) <
                           scheduler->min_flying;
  +    trace_i915_scheduler_queue(ring, node);
+    trace_i915_scheduler_node_state_change(ring, node);
+
      spin_unlock_irq(&scheduler->lock);
        if (not_flying)
@@ -657,6 +675,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
      struct i915_scheduler_queue_entry *node = req->scheduler_qe;
      unsigned long flags;
  +    trace_i915_scheduler_landing(req);
+
      if (!node)
          return false;
  @@ -670,6 +690,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
      else
          node->status = i915_sqs_complete;
  +    trace_i915_scheduler_node_state_change(req->ring, node);
+
      spin_unlock_irqrestore(&scheduler->lock, flags);
        return true;
@@ -877,6 +899,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
      /* Launch more packets now? */
      do_submit = (queued > 0) && (flying < scheduler->min_flying);
  +    trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
      spin_unlock_irq(&scheduler->lock);
        return do_submit;
@@ -912,6 +936,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *ring)
          node = list_first_entry(&remove, typeof(*node), link);
          list_del(&node->link);
  +        trace_i915_scheduler_destroy(ring, node);
+
          /* Free up all the DRM references */
          i915_scheduler_clean_node(node);
  diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h

index 455c215..c3c4e58 100644

drivers/gpu/drm/i915/i915_trace.h +195 -0
@@ -9,6 +9,7 @@
  #include "i915_drv.h"
  #include "intel_drv.h"
  #include "intel_ringbuffer.h"
+#include "i915_scheduler.h"
    #undef TRACE_SYSTEM
  #define TRACE_SYSTEM i915
@@ -826,6 +827,201 @@ TRACE_EVENT(switch_mm,
            __entry->dev, __entry->ring, __entry->to, __entry->vm)
  );
  +TRACE_EVENT(i915_scheduler_queue,
+        TP_PROTO(struct intel_engine_cs *ring,
+             struct i915_scheduler_queue_entry *node),
+        TP_ARGS(ring, node),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring  = ring->id;
+               __entry->uniq  = node ? node->params.request->uniq  : 0;
+               __entry->seqno = node ? node->params.request->seqno : 0;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d",
+              __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_fly,
+        TP_PROTO(struct intel_engine_cs *ring,
+             struct i915_scheduler_queue_entry *node),
+        TP_ARGS(ring, node),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring  = ring->id;
+               __entry->uniq  = node ? node->params.request->uniq  : 0;
+               __entry->seqno = node ? node->params.request->seqno : 0;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d",
+              __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_unfly,
+        TP_PROTO(struct intel_engine_cs *ring,
+             struct i915_scheduler_queue_entry *node),
+        TP_ARGS(ring, node),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring  = ring->id;
+               __entry->uniq  = node ? node->params.request->uniq  : 0;
+               __entry->seqno = node ? node->params.request->seqno : 0;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d",
+              __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_landing,
+        TP_PROTO(struct drm_i915_gem_request *req),
+        TP_ARGS(req),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 __field(u32, status)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring   = req->ring->id;
+               __entry->uniq   = req->uniq;
+               __entry->seqno  = req->seqno;
+               __entry->status = req->scheduler_qe ?
+                        req->scheduler_qe->status : ~0U;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+              __entry->ring, __entry->uniq, __entry->seqno,
+              __entry->status)
+);
+
+TRACE_EVENT(i915_scheduler_remove,
+        TP_PROTO(struct intel_engine_cs *ring,
+             u32 min_seqno, bool do_submit),
+        TP_ARGS(ring, min_seqno, do_submit),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, min_seqno)
+                 __field(bool, do_submit)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring      = ring->id;
+               __entry->min_seqno = min_seqno;
+               __entry->do_submit = do_submit;
+               ),
+
+        TP_printk("ring=%d, min_seqno = %d, do_submit=%d",
+              __entry->ring, __entry->min_seqno, __entry->do_submit)
+);
+
+TRACE_EVENT(i915_scheduler_destroy,
+        TP_PROTO(struct intel_engine_cs *ring,
+             struct i915_scheduler_queue_entry *node),
+        TP_ARGS(ring, node),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring  = ring->id;
+               __entry->uniq  = node ? node->params.request->uniq  : 0;
+               __entry->seqno = node ? node->params.request->seqno : 0;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d",
+              __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_pop_from_queue,
+        TP_PROTO(struct intel_engine_cs *ring,
+             struct i915_scheduler_queue_entry *node),
+        TP_ARGS(ring, node),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring  = ring->id;
+               __entry->uniq  = node ? node->params.request->uniq  : 0;
+               __entry->seqno = node ? node->params.request->seqno : 0;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d",
+              __entry->ring, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_node_state_change,
+        TP_PROTO(struct intel_engine_cs *ring,
+             struct i915_scheduler_queue_entry *node),
+        TP_ARGS(ring, node),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 __field(u32, status)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring   = ring->id;
+               __entry->uniq   = node ? node->params.request->uniq  : 0;
+               __entry->seqno  = node->params.request->seqno;
+               __entry->status = node->status;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d, status=%d",
+              __entry->ring, __entry->uniq, __entry->seqno,
+              __entry->status)
+);
+
+TRACE_EVENT(i915_gem_ring_queue,
+        TP_PROTO(struct intel_engine_cs *ring,
+             struct i915_execbuffer_params *params),
+        TP_ARGS(ring, params),
+
+        TP_STRUCT__entry(
+                 __field(u32, ring)
+                 __field(u32, uniq)
+                 __field(u32, seqno)
+                 ),
+
+        TP_fast_assign(
+               __entry->ring  = ring->id;
+               __entry->uniq  = params->request->uniq;
+               __entry->seqno = params->request->seqno;
+               ),
+
+        TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
+              __entry->uniq, __entry->seqno)
+);
+
  #endif /* _I915_TRACE_H_ */
    /* This part must be outside protection */
drivers/gpu/drm/i915/intel_lrc.c +0 -0
index 9c7a79a..2b9f49c 100644
@@ -954,6 +954,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
Quoted 8 lines

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 960

Quoted 7 lines
drivers/gpu/drm/i915/intel_lrc.c +1 -0
        i915_gem_execbuffer_move_to_active(vmas, params->request);
  +    trace_i915_gem_ring_queue(ring, params);
+
      qe = container_of(params, typeof(*qe), params);
      ret = i915_scheduler_queue_execbuffer(qe);
      if (ret)
 

Oops, forgot to ask if there are any tools for igt that make use of these, e.g. some kind of scheduler top util that looks for bubbles or starved batches or something.

Quoted 1 line

Nothing at the moment. They have been used to debug issues by looking through trace logs. Also the validation team have tests that hook in to the various trace points to verify the internal operation of the scheduler and other parts of the i915 driver.

Nice. I wasn't asking because it's a blocker or anything, just curious.

Thanks,
Jesse

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#61 lore raw

PATCH v5 25/35 drm/i915: Added scheduler queue throttling by DRM file handle

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.

v3: Added a missing decrement of the file queue counter.

v4: Updated a comment.

v5: Updated due to changes to earlier patches in series - removing forward declarations and white space. Also added some documentation. [Joonas Lahtinen]

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  2 ++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  8 +++++
 drivers/gpu/drm/i915/i915_scheduler.c      | 48 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  2 ++
 4 files changed, 60 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index 071a27b..3f4c4f0 100644
@@ -336,6 +336,8 @@ struct drm_i915_file_private {
 	} rps;
 
 	struct intel_engine_cs *bsd_ring;
+
+	u32 scheduler_queue_length;
 };
 
 enum intel_dpll_id {
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8 -0
index d4de8c7..dff120c 100644
@@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	/* Copy in the exec list from userland */
 	exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
 	exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
@@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
 			     GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
 	if (exec2_list == NULL)
drivers/gpu/drm/i915/i915_scheduler.c +48 -0

78 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 075befb..b78de12 100644
@@ -77,6 +77,7 @@ struct i915_scheduler {
 	int32_t             priority_level_bump;
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
+	uint32_t            file_queue_max;
 };
 
 /* Flag bits for i915_scheduler::flags */
@@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
 			       unsigned long stamp, bool is_locked);
 bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 				       bool *completed, bool *busy);
+bool i915_scheduler_file_queue_is_full(struct drm_file *file);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 25/35
#62 lore raw

Re: [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle

Quoted 169 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.

v3: Added a missing decrement of the file queue counter.

v4: Updated a comment.

v5: Updated due to changes to earlier patches in series - removing forward declarations and white space. Also added some documentation. [Joonas Lahtinen]

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  2 ++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  8 +++++
 drivers/gpu/drm/i915/i915_scheduler.c      | 48 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  2 ++
 4 files changed, 60 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index 071a27b..3f4c4f0 100644
@@ -336,6 +336,8 @@ struct drm_i915_file_private {
 	} rps;
 
 	struct intel_engine_cs *bsd_ring;
+
+	u32 scheduler_queue_length;
 };
 
 enum intel_dpll_id {
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8 -0
index d4de8c7..dff120c 100644
@@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	/* Copy in the exec list from userland */
 	exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
 	exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
@@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
 			     GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
 	if (exec2_list == NULL)
drivers/gpu/drm/i915/i915_scheduler.c +48 -0
index e56ce08..f7f29d5 100644
@@ -69,6 +69,7 @@ int i915_scheduler_init(struct drm_device *dev)
 	scheduler->priority_level_bump    = 50;
 	scheduler->priority_level_preempt = 900;
 	scheduler->min_flying             = 2;
+	scheduler->file_queue_max         = 64;
 
 	dev_priv->scheduler = scheduler;
 
@@ -464,6 +465,44 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
 	return ret;
 }
 
+/**
+ * i915_scheduler_file_queue_is_full - Returns true if the queue is full.
+ * @file: File object to query.
+ * This allows throttling of applications by limiting the total number of
+ * outstanding requests to a specified level. Once that limit is reached,
+ * this call will return true and no more requests should be accepted.
+ */
+bool i915_scheduler_file_queue_is_full(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+	struct drm_i915_private *dev_priv  = file_priv->dev_priv;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+	return file_priv->scheduler_queue_length >= scheduler->file_queue_max;
+}
+
+/**
+ * i915_scheduler_file_queue_inc - Increment the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_inc(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length++;
+}
+
+/**
+ * i915_scheduler_file_queue_dec - Decrement the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_dec(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length--;
+}
+
 static void i915_generate_dependencies(struct i915_scheduler *scheduler,
 				       struct i915_scheduler_queue_entry *node,
 				       uint32_t ring)
@@ -640,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
 	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
 
+	i915_scheduler_file_queue_inc(node->params.file);
+
 	not_flying = i915_scheduler_count_flying(scheduler, ring) <
 						 scheduler->min_flying;
 
@@ -883,6 +924,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
 		/* Strip the dependency info while the mutex is still locked */
 		i915_scheduler_remove_dependent(scheduler, node);
 
+		/* Likewise clean up the file pointer. */
+		if (node->params.file) {
+			i915_scheduler_file_queue_dec(node->params.file);
+			node->params.file = NULL;
+		}
+
 		continue;
 	}
 
@@ -1205,6 +1252,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 						 node->status,
 						 ring->name);
 
+			i915_scheduler_file_queue_dec(node->params.file);
 			node->params.file = NULL;
 		}
 	}
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 075befb..b78de12 100644
@@ -77,6 +77,7 @@ struct i915_scheduler {
 	int32_t             priority_level_bump;
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
+	uint32_t            file_queue_max;
 };
 
 /* Flag bits for i915_scheduler::flags */

drivers/gpu/drm/i915/i915_scheduler.h int i915_scheduler_flush_stamp(struct intel_engine_cs *ring, · line 106

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
@@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
 			       unsigned long stamp, bool is_locked);
 bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 				       bool *completed, bool *busy);
+bool i915_scheduler_file_queue_is_full(struct drm_file *file);
 
 #endif  /* _I915_SCHEDULER_H_ */

Just to clarify and make sure I understood the previous stuff: a queued execbuf that has not yet been dispatched does not reserve and pin pages right? That occurs at actual dispatch time? If so, I guess clients will hit this 64 queued item limit pretty regularly... How much metadata overhead does that involve? Has it been derived from some performance work with a bunch of workloads? It's fine if not, I can imagine that different mixes of workloads would be affected by lower or higher queue depths (e.g. small batch tests).

If this is tunable, I guess it should be clamped like a nice or rlimit value, with values outside that range requiring CAP_SYS_ADMIN.

Jesse
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#63 lore raw

Re: [PATCH v5 25/35] drm/i915: Added scheduler queue throttling by DRM file handle

Quoted 174 lines · Jesse Barnes
Quoted 173 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.

v3: Added a missing decrement of the file queue counter.

v4: Updated a comment.

v5: Updated due to changes to earlier patches in series - removing forward declarations and white space. Also added some documentation. [Joonas Lahtinen]

For: VIZ-1587

---
  drivers/gpu/drm/i915/i915_drv.h            |  2 ++
  drivers/gpu/drm/i915/i915_gem_execbuffer.c |  8 +++++
  drivers/gpu/drm/i915/i915_scheduler.c      | 48 ++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/i915_scheduler.h      |  2 ++
  4 files changed, 60 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index 071a27b..3f4c4f0 100644
@@ -336,6 +336,8 @@ struct drm_i915_file_private {
  	} rps;
 
  	struct intel_engine_cs *bsd_ring;
+
+	u32 scheduler_queue_length;
  };
 
  enum intel_dpll_id {
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8 -0
index d4de8c7..dff120c 100644
@@ -1803,6 +1803,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
  		return -EINVAL;
  	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
  	/* Copy in the exec list from userland */
  	exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
  	exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
@@ -1893,6 +1897,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
  		return -EINVAL;
  	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
  	exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
  			     GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
  	if (exec2_list == NULL)
drivers/gpu/drm/i915/i915_scheduler.c +48 -0
index e56ce08..f7f29d5 100644
@@ -69,6 +69,7 @@ int i915_scheduler_init(struct drm_device *dev)
  	scheduler->priority_level_bump    = 50;
  	scheduler->priority_level_preempt = 900;
  	scheduler->min_flying             = 2;
+	scheduler->file_queue_max         = 64;
 
  	dev_priv->scheduler = scheduler;
 
@@ -464,6 +465,44 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *ring)
  	return ret;
  }
 
+/**
+ * i915_scheduler_file_queue_is_full - Returns true if the queue is full.
+ * @file: File object to query.
+ * This allows throttling of applications by limiting the total number of
+ * outstanding requests to a specified level. Once that limit is reached,
+ * this call will return true and no more requests should be accepted.
+ */
+bool i915_scheduler_file_queue_is_full(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+	struct drm_i915_private *dev_priv  = file_priv->dev_priv;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+	return file_priv->scheduler_queue_length >= scheduler->file_queue_max;
+}
+
+/**
+ * i915_scheduler_file_queue_inc - Increment the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_inc(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length++;
+}
+
+/**
+ * i915_scheduler_file_queue_dec - Decrement the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_dec(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length--;
+}
+
  static void i915_generate_dependencies(struct i915_scheduler *scheduler,
  				       struct i915_scheduler_queue_entry *node,
  				       uint32_t ring)
@@ -640,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
  	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
 
+	i915_scheduler_file_queue_inc(node->params.file);
+
  	not_flying = i915_scheduler_count_flying(scheduler, ring) <
  						 scheduler->min_flying;
 
@@ -883,6 +924,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
  		/* Strip the dependency info while the mutex is still locked */
  		i915_scheduler_remove_dependent(scheduler, node);
 
+		/* Likewise clean up the file pointer. */
+		if (node->params.file) {
+			i915_scheduler_file_queue_dec(node->params.file);
+			node->params.file = NULL;
+		}
+
  		continue;
  	}
 
@@ -1205,6 +1252,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
  						 node->status,
  						 ring->name);
 
+			i915_scheduler_file_queue_dec(node->params.file);
  			node->params.file = NULL;
  		}
  	}
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 075befb..b78de12 100644
@@ -77,6 +77,7 @@ struct i915_scheduler {
  	int32_t             priority_level_bump;
  	int32_t             priority_level_preempt;
  	uint32_t            min_flying;
+	uint32_t            file_queue_max;
  };
 
  /* Flag bits for i915_scheduler::flags */
@@ -100,5 +101,6 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
  			       unsigned long stamp, bool is_locked);
  bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
  				       bool *completed, bool *busy);
+bool i915_scheduler_file_queue_is_full(struct drm_file *file);
 
Quoted 3 lines

drivers/gpu/drm/i915/i915_scheduler.h int i915_scheduler_flush_stamp(struct intel_engine_cs *ring, · line 106

Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.h +0 -0
  #endif  /* _I915_SCHEDULER_H_ */

Just to clarify and make sure I understood the previous stuff: a queued execbuf that has not yet been dispatched does not reserve and pin pages right? That occurs at actual dispatch time? If so, I guess clients will hit this 64 queued item limit pretty regularly... How much metadata overhead does that involve? Has it been derived from some performance work with a bunch of workloads? It's fine if not, I can imagine that different mixes of workloads would be affected by lower or higher queue depths (e.g. small batch tests).

Whether a client will hit the limit is entirely down to how that client is written. A GL program that renders double buffered with one batch per frame will only ever have two batch buffer requests outstanding. Whereas, the gem_exec_nop IGT test submits a thousand batches before waiting for anything to complete.

Quoted 1 line

If this is tunable, I guess it should be clamped like a nice or rlimit value, with values outside that range requiring CAP_SYS_ADMIN.

What is considered a valid range? Previously the limit would have been the size of the ring buffer. In times past, they were large and you could have thousands of batches in flight. With the GuC we are down to 4K words. At, for example, 32 words per batch buffer request that makes for 128 outstanding batches. So set the maximum permitted to 256? Larger? Smaller?

Quoted 4 lines

Jesse

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#64 lore raw

PATCH v5 26/35 drm/i915: Added debugfs interface to scheduler tuning parameters

In reply to [email protected]

From: John Harrison <[email protected]>

There are various parameters within the scheduler which can be tuned to improve performance, reduce memory footprint, etc. This change adds support for altering these via debugfs.

v2: Updated for priorities now being signed values.

v5: Squashed priority bumping entries into this patch rather than a separate patch all of their own.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++
 1 file changed, 169 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +169 -0
index b923949..7d01c07 100644
@@ -39,6 +39,7 @@
 #include "intel_ringbuffer.h"
 #include <drm/i915_drm.h>
 #include "i915_drv.h"
+#include "i915_scheduler.h"
 
 enum {
 	ACTIVE_LIST,
@@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
 			i915_next_seqno_get, i915_next_seqno_set,
 			"0x%llx\n");
 
+static int
+i915_scheduler_priority_min_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_min;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_min_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_min = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
+			i915_scheduler_priority_min_get,
+			i915_scheduler_priority_min_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_max;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_max = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
+			i915_scheduler_priority_max_get,
+			i915_scheduler_priority_max_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_bump_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_bump;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_bump_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_bump = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
+			i915_scheduler_priority_bump_get,
+			i915_scheduler_priority_bump_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_preempt_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_preempt;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_preempt_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_preempt = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
+			i915_scheduler_priority_preempt_get,
+			i915_scheduler_priority_preempt_set,
+			"%lld\n");
+
+static int
+i915_scheduler_min_flying_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->min_flying;
+	return 0;
+}
+
+static int
+i915_scheduler_min_flying_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->min_flying = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
+			i915_scheduler_min_flying_get,
+			i915_scheduler_min_flying_set,
+			"%llu\n");
+
+static int
+i915_scheduler_file_queue_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->file_queue_max;
+	return 0;
+}
+
+static int
+i915_scheduler_file_queue_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->file_queue_max = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
+			i915_scheduler_file_queue_max_get,
+			i915_scheduler_file_queue_max_set,
+			"%llu\n");
+
 static int i915_frequency_info(struct seq_file *m, void *unused)
 {
 	struct drm_info_node *node = m->private;
@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
 	{"i915_gem_drop_caches", &i915_drop_caches_fops},
 	{"i915_error_state", &i915_error_state_fops},
 	{"i915_next_seqno", &i915_next_seqno_fops},
+	{"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
+	{"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+	{"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
+	{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
+	{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
+	{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
 	{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
 	{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
 	{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v5 26/35
#65 lore raw

Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters

Quoted 202 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

There are various parameters within the scheduler which can be tuned to improve performance, reduce memory footprint, etc. This change adds support for altering these via debugfs.

v2: Updated for priorities now being signed values.

v5: Squashed priority bumping entries into this patch rather than a separate patch all of their own.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++
 1 file changed, 169 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +163 -0
index b923949..7d01c07 100644
@@ -39,6 +39,7 @@
 #include "intel_ringbuffer.h"
 #include <drm/i915_drm.h>
 #include "i915_drv.h"
+#include "i915_scheduler.h"
 
 enum {
 	ACTIVE_LIST,
@@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
 			i915_next_seqno_get, i915_next_seqno_set,
 			"0x%llx\n");
 
+static int
+i915_scheduler_priority_min_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_min;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_min_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_min = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
+			i915_scheduler_priority_min_get,
+			i915_scheduler_priority_min_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_max;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_max = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
+			i915_scheduler_priority_max_get,
+			i915_scheduler_priority_max_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_bump_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_bump;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_bump_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_bump = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
+			i915_scheduler_priority_bump_get,
+			i915_scheduler_priority_bump_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_preempt_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_preempt;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_preempt_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_preempt = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
+			i915_scheduler_priority_preempt_get,
+			i915_scheduler_priority_preempt_set,
+			"%lld\n");
+
+static int
+i915_scheduler_min_flying_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->min_flying;
+	return 0;
+}
+
+static int
+i915_scheduler_min_flying_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->min_flying = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
+			i915_scheduler_min_flying_get,
+			i915_scheduler_min_flying_set,
+			"%llu\n");
+
+static int
+i915_scheduler_file_queue_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->file_queue_max;
+	return 0;
+}
+
+static int
+i915_scheduler_file_queue_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->file_queue_max = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
+			i915_scheduler_file_queue_max_get,
+			i915_scheduler_file_queue_max_set,
+			"%llu\n");
+
 static int i915_frequency_info(struct seq_file *m, void *unused)
 {
 	struct drm_info_node *node = m->private;
@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
 	{"i915_gem_drop_caches", &i915_drop_caches_fops},
 	{"i915_error_state", &i915_error_state_fops},

drivers/gpu/drm/i915/i915_debugfs.c static const struct i915_debugfs_files { · line 5598

Quoted 10 lines
drivers/gpu/drm/i915/i915_debugfs.c +6 -0
 	{"i915_next_seqno", &i915_next_seqno_fops},
+	{"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
+	{"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+	{"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
+	{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
+	{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
+	{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
 	{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
 	{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
 	{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},

Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.

If not,

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#66 lore raw

Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters

Quoted 206 lines · Jesse Barnes
Quoted 205 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

There are various parameters within the scheduler which can be tuned to improve performance, reduce memory footprint, etc. This change adds support for altering these via debugfs.

v2: Updated for priorities now being signed values.

v5: Squashed priority bumping entries into this patch rather than a separate patch all of their own.

For: VIZ-1587

---
  drivers/gpu/drm/i915/i915_debugfs.c | 169 ++++++++++++++++++++++++++++++++++++
  1 file changed, 169 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +166 -0
index b923949..7d01c07 100644
@@ -39,6 +39,7 @@
  #include "intel_ringbuffer.h"
  #include <drm/i915_drm.h>
  #include "i915_drv.h"
+#include "i915_scheduler.h"
 
  enum {
  	ACTIVE_LIST,
@@ -1122,6 +1123,168 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
  			i915_next_seqno_get, i915_next_seqno_set,
  			"0x%llx\n");
 
+static int
+i915_scheduler_priority_min_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_min;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_min_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_min = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
+			i915_scheduler_priority_min_get,
+			i915_scheduler_priority_min_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_max;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_max = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
+			i915_scheduler_priority_max_get,
+			i915_scheduler_priority_max_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_bump_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_bump;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_bump_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_bump = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
+			i915_scheduler_priority_bump_get,
+			i915_scheduler_priority_bump_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_preempt_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_preempt;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_preempt_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_preempt = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
+			i915_scheduler_priority_preempt_get,
+			i915_scheduler_priority_preempt_set,
+			"%lld\n");
+
+static int
+i915_scheduler_min_flying_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->min_flying;
+	return 0;
+}
+
+static int
+i915_scheduler_min_flying_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->min_flying = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
+			i915_scheduler_min_flying_get,
+			i915_scheduler_min_flying_set,
+			"%llu\n");
+
+static int
+i915_scheduler_file_queue_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->file_queue_max;
+	return 0;
+}
+
+static int
+i915_scheduler_file_queue_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->file_queue_max = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
+			i915_scheduler_file_queue_max_get,
+			i915_scheduler_file_queue_max_set,
+			"%llu\n");
+
  static int i915_frequency_info(struct seq_file *m, void *unused)
  {
  	struct drm_info_node *node = m->private;
@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
  	{"i915_gem_drop_caches", &i915_drop_caches_fops},
  	{"i915_error_state", &i915_error_state_fops},
  	{"i915_next_seqno", &i915_next_seqno_fops},
+	{"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
+	{"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+	{"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
Quoted 8 lines

drivers/gpu/drm/i915/i915_debugfs.c static const struct i915_debugfs_files { · line 5598

Quoted 7 lines
drivers/gpu/drm/i915/i915_debugfs.c +3 -0
+	{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
+	{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
+	{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
  	{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
  	{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
  	{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},

Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.

Serialised with what? Each other or the scheduler operation? Neither should be necessary. The scheduler will read the current values whenever it tests against one of these limits. If multiple are being changed while the system is busy, it doesn't really matter. They are just tuning values and best guesses type of numbers not array indices or other things that would cause kernel panics if you got them wrong. E.g. if you set the max file queue depth to smaller than the current queue contents that just means you won't be able to submit more stuff until the queue has drained - which is presumably the intended result of lowering the max queue value anyway. The queue won't leak the extra entries or get into an inconsistent state.

Quoted 3 lines

If not,

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#67 lore raw

Re: [PATCH v5 26/35] drm/i915: Added debugfs interface to scheduler tuning parameters

Quoted 24 lines · John Harrison
Quoted 15 lines · Jesse Barnes
Quoted 14 lines · On 02/18/2016 06:27 AM, [email protected] wrote:

From: John Harrison <[email protected]>

      struct drm_info_node *node = m->private;
drivers/gpu/drm/i915/i915_debugfs.c +6 -0
@@ -5424,6 +5587,12 @@ static const struct i915_debugfs_files {
      {"i915_gem_drop_caches", &i915_drop_caches_fops},
      {"i915_error_state", &i915_error_state_fops},
      {"i915_next_seqno", &i915_next_seqno_fops},
+    {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
+    {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+    {"i915_scheduler_priority_bump",
 &i915_scheduler_priority_bump_fops},
+    {"i915_scheduler_priority_preempt",
 &i915_scheduler_priority_preempt_fops},
+    {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
+    {"i915_scheduler_file_queue_max",
Quoted 8 lines
Quoted 5 lines
(quoted diff) +0 -0
 &i915_scheduler_file_queue_max_fops},
      {"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
      {"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
      {"i915_spr_wm_latency", &i915_spr_wm_latency_fops},

Do these need to be serialized at all? I guess some raciness doesn't hurt too much for these guys, unless somehow an inconsistent set of values would cause a livelock in the scheduler or something.

Quoted 11 lines

Serialised with what? Each other or the scheduler operation? Neither should be necessary. The scheduler will read the current values whenever it tests against one of these limits. If multiple are being changed while the system is busy, it doesn't really matter. They are just tuning values and best guesses type of numbers not array indices or other things that would cause kernel panics if you got them wrong. E.g. if you set the max file queue depth to smaller than the current queue contents that just means you won't be able to submit more stuff until the queue has drained - which is presumably the intended result of lowering the max queue value anyway. The queue won't leak the extra entries or get into an inconsistent state.

Yeah I meant both serialized against scheduler accesses/usage and atomicity with respect to one another. Sounds like it doesn't matter too much though, so it's fine with me.

Jesse

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#68 lore raw

PATCH v5 27/35 drm/i915: Added debug state dump facilities to scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

When debugging batch buffer submission issues, it is useful to be able to see what the current state of the scheduler is. This change adds functions for decoding the internal scheduler state and reporting it.

v3: Updated a debug message with the new state_str() function.

v4: Wrapped some long lines to keep the style checker happy. Removed the fence/sync code as that will now be part of a separate patch series.

v5: Removed forward declarations and white space. Added documentation. [Joonas Lahtinen]

Also squashed in later patch to add seqno information from the start. It was only being added in a separate patch due to historical reasons which have since gone away.

For: VIZ-1587

Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/i915_scheduler.c | 302 +++++++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_scheduler.h |  15 ++
 2 files changed, 315 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_scheduler.c +300 -2
index f7f29d5..d0eed52 100644
@@ -40,6 +40,117 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
 	return dev_priv->scheduler != NULL;
 }
 
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
+{
+	static char	str[50];
+	char		*ptr = str;
+
+	*(ptr++) = node->bumped ? 'B' : '-',
+	*(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
+
+	*ptr = 0;
+
+	return str;
+}
+
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
+{
+	switch (status) {
+	case i915_sqs_none:
+	return 'N';
+
+	case i915_sqs_queued:
+	return 'Q';
+
+	case i915_sqs_popped:
+	return 'X';
+
+	case i915_sqs_flying:
+	return 'F';
+
+	case i915_sqs_complete:
+	return 'C';
+
+	case i915_sqs_dead:
+	return 'D';
+
+	default:
+	break;
+	}
+
+	return '?';
+}
+
+const char *i915_scheduler_queue_status_str(
+				enum i915_scheduler_queue_status status)
+{
+	static char	str[50];
+
+	switch (status) {
+	case i915_sqs_none:
+	return "None";
+
+	case i915_sqs_queued:
+	return "Queued";
+
+	case i915_sqs_popped:
+	return "Popped";
+
+	case i915_sqs_flying:
+	return "Flying";
+
+	case i915_sqs_complete:
+	return "Complete";
+
+	case i915_sqs_dead:
+	return "Dead";
+
+	default:
+	break;
+	}
+
+	sprintf(str, "[Unknown_%d!]", status);
+	return str;
+}
+
+const char *i915_scheduler_flag_str(uint32_t flags)
+{
+	static char str[100];
+	char *ptr = str;
+
+	*ptr = 0;
+
+#define TEST_FLAG(flag, msg)						\
+	do {								\
+		if (flags & (flag)) {					\
+			strcpy(ptr, msg);				\
+			ptr += strlen(ptr);				\
+			flags &= ~(flag);				\
+		}							\
+	} while (0)
+
+	TEST_FLAG(i915_sf_interrupts_enabled, "IntOn|");
+	TEST_FLAG(i915_sf_submitting,         "Submitting|");
+	TEST_FLAG(i915_sf_dump_force,         "DumpForce|");
+	TEST_FLAG(i915_sf_dump_details,       "DumpDetails|");
+	TEST_FLAG(i915_sf_dump_dependencies,  "DumpDeps|");
+	TEST_FLAG(i915_sf_dump_seqno,         "DumpSeqno|");
+
+#undef TEST_FLAG
+
+	if (flags) {
+		sprintf(ptr, "Unknown_0x%X!", flags);
+		ptr += strlen(ptr);
+	}
+
+	if (ptr == str)
+		strcpy(str, "-");
+	else
+		ptr[-1] = 0;
+
+	return str;
+};
+
 /**
  * i915_scheduler_init - Initialise the scheduler.
  * @dev: DRM device
@@ -1024,6 +1135,193 @@ void i915_scheduler_work_handler(struct work_struct *work)
 		i915_scheduler_process_work(ring);
 }
 
+static int i915_scheduler_dump_locked(struct intel_engine_cs *ring,
+				      const char *msg)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	int flying = 0, queued = 0, complete = 0, other = 0;
+	static int old_flying = -1, old_queued = -1, old_complete = -1;
+	bool b_dump;
+	char brkt[2] = { '<', '>' };
+
+	if (!ring)
+		return -EINVAL;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			complete++;
+		else
+			other++;
+	}
+
+	b_dump = (flying != old_flying) ||
+		 (queued != old_queued) ||
+		 (complete != old_complete);
+	if (scheduler->flags[ring->id] & i915_sf_dump_force) {
+		if (!b_dump) {
+			b_dump = true;
+			brkt[0] = '{';
+			brkt[1] = '}';
+		}
+
+		scheduler->flags[ring->id] &= ~i915_sf_dump_force;
+	}
+
+	if (b_dump) {
+		old_flying   = flying;
+		old_queued   = queued;
+		old_complete = complete;
+		DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
+				 "Flags = %s, Next = %d:%d %c%s%c\n",
+				 ring->name, queued, flying, complete, other,
+				 i915_scheduler_flag_str(scheduler->flags[ring->id]),
+				 dev_priv->request_uniq, dev_priv->next_seqno,
+				 brkt[0], msg, brkt[1]);
+	} else {
+		/*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
+				 ", Flags = %s, Next = %d:%d [%s]\n",
+				 ring->name,
+				 queued, flying, complete, other,
+				 i915_scheduler_flag_str(scheduler->flags[ring->id]),
+				 dev_priv->request_uniq, dev_priv->next_seqno, msg); */
+
+		return 0;
+	}
+
+	if (scheduler->flags[ring->id] & i915_sf_dump_seqno) {
+		uint32_t seqno;
+
+		seqno = ring->get_seqno(ring, true);
+
+		DRM_DEBUG_DRIVER("<%s> Seqno = %d\n", ring->name, seqno);
+	}
+
+	if (scheduler->flags[ring->id] & i915_sf_dump_details) {
+		int i, deps;
+		uint32_t count, counts[i915_sqs_MAX];
+
+		memset(counts, 0x00, sizeof(counts));
+
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (node->status < i915_sqs_MAX) {
+				count = counts[node->status]++;
+			} else {
+				DRM_DEBUG_DRIVER("<%s>   Unknown status: %d!\n",
+						 ring->name, node->status);
+				count = -1;
+			}
+
+			deps = 0;
+			for (i = 0; i < node->num_deps; i++)
+				if (i915_scheduler_is_dependency_valid(node, i))
+					deps++;
+
+			DRM_DEBUG_DRIVER("<%s>   %c:%02d> uniq = %d, seqno"
+					 " = %d/%s, deps = %d / %d, %s [pri = "
+					 "%4d]\n", ring->name,
+					 i915_scheduler_queue_status_chr(node->status),
+					 count,
+					 node->params.request->uniq,
+					 node->params.request->seqno,
+					 node->params.ring->name,
+					 deps, node->num_deps,
+					 i915_qe_state_str(node),
+					 node->priority);
+
+			if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies)
+				== 0)
+				continue;
+
+			for (i = 0; i < node->num_deps; i++)
+				if (node->dep_list[i])
+					DRM_DEBUG_DRIVER("<%s>       |-%c:"
+						"%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
+						ring->name,
+						i915_scheduler_queue_status_chr(node->dep_list[i]->status),
+						i,
+						i915_scheduler_is_dependency_valid(node, i)
+							? '>' : '#',
+						node->dep_list[i]->params.request->uniq,
+						node->dep_list[i]->params.request->seqno,
+						node->dep_list[i]->params.ring->name,
+						i915_qe_state_str(node->dep_list[i]),
+						node->dep_list[i]->priority);
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * i915_scheduler_dump - dump the scheduler's internal state to the debug log.
+ * @ring: Ring to dump info for
+ * @msg: A reason why it is being dumped
+ * For debugging purposes, it can be very useful to see the internal state of
+ * the scheduler for a given ring.
+ */
+int i915_scheduler_dump(struct intel_engine_cs *ring, const char *msg)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	unsigned long flags;
+	int ret;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	ret = i915_scheduler_dump_locked(ring, msg);
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return ret;
+}
+
+static int i915_scheduler_dump_all_locked(struct drm_device *dev,
+					  const char *msg)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs *ring;
+	int i, r, ret = 0;
+
+	for_each_ring(ring, dev_priv, i) {
+		scheduler->flags[ring->id] |= i915_sf_dump_force   |
+					      i915_sf_dump_details |
+					      i915_sf_dump_seqno   |
+					      i915_sf_dump_dependencies;
+		r = i915_scheduler_dump_locked(ring, msg);
+		if (ret == 0)
+			ret = r;
+	}
+
+	return ret;
+}
+
+/**
+ * i915_scheduler_dump_all - dump the scheduler's internal state to the debug
+ * log.
+ * @dev: DRM device
+ * @msg: A reason why it is being dumped
+ * For debugging purposes, it can be very useful to see the internal state of
+ * the scheduler.
+ */
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler *scheduler = dev_priv->scheduler;
+	unsigned long flags;
+	int ret;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	ret = i915_scheduler_dump_all_locked(dev, msg);
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return ret;
+}
+
 static int i915_scheduler_submit_max_priority(struct intel_engine_cs *ring,
 					      bool is_locked)
 {
@@ -1246,10 +1544,10 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 				continue;
 
 			if (!I915_SQS_IS_COMPLETE(node))
-				DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+				DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%s on %s\n",
 						 node->params.request->uniq,
 						 node->params.request->seqno,
-						 node->status,
+						 i915_qe_state_str(node),
 						 ring->name);
 
 			i915_scheduler_file_queue_dec(node->params.file);
drivers/gpu/drm/i915/i915_scheduler.h +15 -0
index b78de12..a071ebb 100644
@@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
 	/* Limit value for use with arrays/loops */
 	i915_sqs_MAX
 };
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
+const char *i915_scheduler_queue_status_str(
+				enum i915_scheduler_queue_status status);
 
 #define I915_SQS_IS_QUEUED(node)	(((node)->status == i915_sqs_queued))
 #define I915_SQS_IS_FLYING(node)	(((node)->status == i915_sqs_flying))
@@ -65,6 +68,7 @@ struct i915_scheduler_queue_entry {
 	unsigned long                       stamp;
 	struct list_head                    link;
 };
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
 
 struct i915_scheduler {
 	struct list_head    node_queue[I915_NUM_RINGS];
@@ -82,9 +86,17 @@ struct i915_scheduler {
 
 /* Flag bits for i915_scheduler::flags */
 enum {
+	/* Internal state */
 	i915_sf_interrupts_enabled  = (1 << 0),
 	i915_sf_submitting          = (1 << 1),
+
+	/* Dump/debug flags */
+	i915_sf_dump_force          = (1 << 8),
+	i915_sf_dump_details        = (1 << 9),
+	i915_sf_dump_dependencies   = (1 << 10),
+	i915_sf_dump_seqno          = (1 << 11),
 };
+const char *i915_scheduler_flag_str(uint32_t flags);
 
 bool i915_scheduler_is_enabled(struct drm_device *dev);
 int i915_scheduler_init(struct drm_device *dev);
@@ -99,6 +111,9 @@ void i915_scheduler_work_handler(struct work_struct *work);
 int i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
 int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
 			       unsigned long stamp, bool is_locked);
+int i915_scheduler_dump(struct intel_engine_cs *ring,
+			const char *msg);
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
 bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 				       bool *completed, bool *busy);
 bool i915_scheduler_file_queue_is_full(struct drm_file *file);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx