Skip to contentSkip to conversation

GPU scheduler for i915 driver

PATCH In review [email protected] 8 participants 143 messages Nov 23, 2015 – Feb 19, 2016 Page 1 of 2

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover GPU scheduler for i915 driver

From: John Harrison <[email protected]>

Implemented a batch buffer submission scheduler for the i915 DRM driver.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code assigns a unique seqno value and then packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

As each batch buffer completes, it raises an interrupt which wakes up the scheduler. Note that it is possible for multiple buffers to complete before the IRQ handler gets to run. Further, the seqno values of the individual buffers are not necessary incrementing as the scheduler may have re-ordered their submission. However, the scheduler keeps the list of executing buffers in order of hardware submission. Thus it can scan through the list until a matching seqno is found and then mark all in flight nodes from that point on as completed.

A deferred work queue is also poked by the interrupt handler. When this wakes up it can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.

The scheduler also allows high priority batch buffers (e.g. from a desktop compositor) to jump ahead of whatever is already running if the underlying hardware supports pre-emption. In this situation, any work that was pre-empted is returned to the queued list ready to be resubmitted when no more high priority work is outstanding.

Various IGT tests are in progress to test the scheduler's operation and will follow.

v2: Updated for changes in struct fence patch series and other changes to underlying tree (e.g. removal of cliprects). Also changed priority levels to be signed +/-1023 range and reduced mutex lock usage.

[Patches against drm-intel-nightly tree fetched 17/11/2015 with struct fence conversion patches applied]

Dave Gordon (2):
drm/i915: Updating assorted register and status page definitions drm/i915: Add scheduling priority to per-context parameters

John Harrison (37):
drm/i915: Add total count to context status debugfs output drm/i915: Explicit power enable during deferred context initialisation drm/i915: Prelude to splitting i915_gem_do_execbuffer in two drm/i915: Split i915_dem_do_execbuffer() in half
drm/i915: Re-instate request->uniq because it is extremely useful drm/i915: Start of GPU scheduler
drm/i915: Prepare retire_requests to handle out-of-order seqnos drm/i915: Disable hardware semaphores when GPU scheduler is enabled drm/i915: Force MMIO flips when scheduler enabled
drm/i915: Added scheduler hook when closing DRM file handles drm/i915: Added scheduler hook into i915_gem_request_notify() drm/i915: Added deferred work handler for scheduler
drm/i915: Redirect execbuffer_final() via scheduler
drm/i915: Keep the reserved space mechanism happy
drm/i915: Added tracking/locking of batch buffer objects drm/i915: Hook scheduler node clean up into retire requests drm/i915: Added scheduler support to __wait_request() calls drm/i915: Added scheduler support to page fault handler drm/i915: Added scheduler flush calls to ring throttle and idle functions drm/i915: Added a module parameter for allowing scheduler overrides drm/i915: Support for 'unflushed' ring idle
drm/i915: Defer seqno allocation until actual hardware submission time drm/i915: Added immediate submission override to scheduler drm/i915: Add sync wait support to scheduler
drm/i915: Connecting execbuff fences to scheduler
drm/i915: Added trace points to scheduler
drm/i915: Added scheduler queue throttling by DRM file handle drm/i915: Added debugfs interface to scheduler tuning parameters drm/i915: Added debug state dump facilities to scheduler drm/i915: Add early exit to execbuff_final() if insufficient ring space drm/i915: Added scheduler statistic reporting to debugfs drm/i915: Added seqno values to scheduler status dump drm/i915: Add scheduler support functions for TDR
drm/i915: GPU priority bumping to prevent starvation drm/i915: Scheduler state dump via debugfs
drm/i915: Enable GPU scheduler by default
drm/i915: Allow scheduler to manage inter-ring object synchronisation

 drivers/gpu/drm/i915/Makefile              |    1 +
 drivers/gpu/drm/i915/i915_debugfs.c        |  283 +++++
 drivers/gpu/drm/i915/i915_dma.c            |    6 +
 drivers/gpu/drm/i915/i915_drv.c            |    9 +
 drivers/gpu/drm/i915/i915_drv.h            |   56 +-
 drivers/gpu/drm/i915/i915_gem.c            |  174 ++-
 drivers/gpu/drm/i915/i915_gem_context.c    |   24 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  341 ++++--
 drivers/gpu/drm/i915/i915_params.c         |    4 +
 drivers/gpu/drm/i915/i915_reg.h            |   30 +-
 drivers/gpu/drm/i915/i915_scheduler.c      | 1697 ++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  176 +++
 drivers/gpu/drm/i915/i915_trace.h          |  215 +++-
 drivers/gpu/drm/i915/intel_display.c       |   10 +-
 drivers/gpu/drm/i915/intel_lrc.c           |  162 ++-
 drivers/gpu/drm/i915/intel_lrc.h           |    1 +
 drivers/gpu/drm/i915/intel_ringbuffer.c    |   47 +-
 drivers/gpu/drm/i915/intel_ringbuffer.h    |   35 +-
 include/uapi/drm/i915_drm.h                |    1 +
 19 files changed, 3110 insertions(+), 162 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 1/39 drm/i915: Add total count to context status debugfs output

From: John Harrison <[email protected]>

When there are lots and lots and even more lots of contexts (e.g. when running with execlists) it is useful to be able to immediately see what the total context count is.

Change-Id: If9726d4df86567100ecf53867b43f4753f08bf84
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +4 -0
index 18dfb56..ecab9b8 100644
@@ -1953,6 +1953,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
 	struct drm_i915_private *dev_priv = dev->dev_private;
 	struct intel_engine_cs *ring;
 	struct intel_context *ctx;
+	uint32_t count = 0;
 	int ret, i;
 
 	ret = mutex_lock_interruptible(&dev->struct_mutex);
@@ -1966,6 +1967,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
 
 		seq_puts(m, "HW context ");
 		describe_ctx(m, ctx);
+		count++;
 		for_each_ring(ring, dev_priv, i) {
 			if (ring->default_context == ctx)
 				seq_printf(m, "(default context %s) ",
@@ -1994,6 +1996,8 @@ static int i915_context_status(struct seq_file *m, void *unused)
 		seq_putc(m, '\n');
 	}
 
+	seq_printf(m, "Total: %d contexts\n", count);
+
 	mutex_unlock(&dev->struct_mutex);
 
 	return 0;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer Reviewed-by on v1 1/39
#2 lore raw

Re: [PATCH 01/39] drm/i915: Add total count to context status debugfs output

Quoted 10 lines · On ma, 2015-11-23 at 11:38 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

When there are lots and lots and even more lots of contexts (e.g. when running
with execlists) it is useful to be able to immediately see what the total
context count is.

Change-Id: If9726d4df86567100ecf53867b43f4753f08bf84
For: VIZ-1587

With below notes;

Quoted 11 lines
---
 drivers/gpu/drm/i915/i915_debugfs.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +0 -0
index 18dfb56..ecab9b8 100644

drivers/gpu/drm/i915/i915_debugfs.c static int i915_context_status(struct seq_file · line 1957

Quoted 5 lines
drivers/gpu/drm/i915/i915_debugfs.c +1 -0
@@ -1953,6 +1953,7 @@ static int i915_context_status(struct seq_file
 *m, void *unused)
 	struct drm_i915_private *dev_priv = dev->dev_private;
 	struct intel_engine_cs *ring;
 	struct intel_context *ctx;
+	uint32_t count = 0;

Should really be just of type 'int'. Overall the driver code use 'int' for counting.

Quoted 14 lines
drivers/gpu/drm/i915/i915_debugfs.c +0 -0
 	int ret, i;
 
drivers/gpu/drm/i915/i915_debugfs.c +1 -0
 	ret = mutex_lock_interruptible(&dev->struct_mutex);
@@ -1966,6 +1967,7 @@ static int i915_context_status(struct seq_file
 *m, void *unused)
 
 		seq_puts(m, "HW context ");
 		describe_ctx(m, ctx);
+		count++;
 		for_each_ring(ring, dev_priv, i) {
 			if (ring->default_context == ctx)

seq_printf(m, "(default context %s)
",

drivers/gpu/drm/i915/i915_debugfs.c static int i915_context_status(struct seq_file · line 2001

Quoted 6 lines
drivers/gpu/drm/i915/i915_debugfs.c +2 -0
@@ -1994,6 +1996,8 @@ static int i915_context_status(struct seq_file
 *m, void *unused)
 		seq_putc(m, '\n');
 	}
 
+	seq_printf(m, "Total: %d contexts\n", count);
+

If the above variable is not changed, %d is wrong for uint32_t and unsi gned int, should be %u.

Quoted 3 lines
drivers/gpu/drm/i915/i915_debugfs.c +0 -0
 	mutex_unlock(&dev->struct_mutex);
 

return 0;

Signature · 8 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH 2/39 drm/i915: Updating assorted register and status page definitions

In reply to [email protected]

From: Dave Gordon <[email protected]>

Added various definitions that will be useful for the scheduler in general and pre-emptive context switching in particular.

Change-Id: Ica805b94160426def51f5d520f5ce51c60864a98
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_reg.h         | 30 +++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/intel_ringbuffer.h | 32 +++++++++++++++++++++++++++++---
 2 files changed, 58 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/i915_reg.h +29 -1
index db72f98..b986050 100644
@@ -259,6 +259,10 @@
 #define  MI_GLOBAL_GTT    (1<<22)
 
 #define MI_NOOP			MI_INSTR(0, 0)
+#define   MI_NOOP_WRITE_ID		(1<<22)
+#define   MI_NOOP_ID_MASK		((1<<22) - 1)
+#define   MI_NOOP_MID(id)		((id) & MI_NOOP_ID_MASK)
+#define MI_NOOP_WITH_ID(id)	MI_INSTR(0, MI_NOOP_WRITE_ID|MI_NOOP_MID(id))
 #define MI_USER_INTERRUPT	MI_INSTR(0x02, 0)
 #define MI_WAIT_FOR_EVENT       MI_INSTR(0x03, 0)
 #define   MI_WAIT_FOR_OVERLAY_FLIP	(1<<16)
@@ -276,6 +280,7 @@
 #define MI_ARB_ON_OFF		MI_INSTR(0x08, 0)
 #define   MI_ARB_ENABLE			(1<<0)
 #define   MI_ARB_DISABLE		(0<<0)
+#define MI_ARB_CHECK		MI_INSTR(0x05, 0)
 #define MI_BATCH_BUFFER_END	MI_INSTR(0x0a, 0)
 #define MI_SUSPEND_FLUSH	MI_INSTR(0x0b, 0)
 #define   MI_SUSPEND_FLUSH_EN	(1<<0)
@@ -325,6 +330,8 @@
 #define   MI_SEMAPHORE_SYNC_INVALID (3<<16)
 #define   MI_SEMAPHORE_SYNC_MASK    (3<<16)
 #define MI_SET_CONTEXT		MI_INSTR(0x18, 0)
+#define   MI_CONTEXT_ADDR_MASK		((~0)<<12)
+#define   MI_SET_CONTEXT_FLAG_MASK	((1<<12)-1)
 #define   MI_MM_SPACE_GTT		(1<<8)
 #define   MI_MM_SPACE_PHYSICAL		(0<<8)
 #define   MI_SAVE_EXT_STATE_EN		(1<<3)
@@ -344,6 +351,10 @@
 #define   MI_USE_GGTT		(1 << 22) /* g4x+ */
 #define MI_STORE_DWORD_INDEX	MI_INSTR(0x21, 1)
 #define   MI_STORE_DWORD_INDEX_SHIFT 2
+#define MI_STORE_REG_MEM	MI_INSTR(0x24, 1)
+#define   MI_STORE_REG_MEM_GTT		(1 << 22)
+#define   MI_STORE_REG_MEM_PREDICATE	(1 << 21)
+
 /* Official intel docs are somewhat sloppy concerning MI_LOAD_REGISTER_IMM:
  * - Always issue a MI_NOOP _before_ the MI_LOAD_REGISTER_IMM - otherwise hw
  *   simply ignores the register load under certain conditions.
@@ -358,7 +369,10 @@
 #define MI_FLUSH_DW		MI_INSTR(0x26, 1) /* for GEN6 */
 #define   MI_FLUSH_DW_STORE_INDEX	(1<<21)
 #define   MI_INVALIDATE_TLB		(1<<18)
+#define   MI_FLUSH_DW_OP_NONE		(0<<14)
 #define   MI_FLUSH_DW_OP_STOREDW	(1<<14)
+#define   MI_FLUSH_DW_OP_RSVD		(2<<14)
+#define   MI_FLUSH_DW_OP_STAMP		(3<<14)
 #define   MI_FLUSH_DW_OP_MASK		(3<<14)
 #define   MI_FLUSH_DW_NOTIFY		(1<<8)
 #define   MI_INVALIDATE_BSD		(1<<7)
@@ -1533,6 +1547,19 @@ enum skl_disp_power_wells {
 
 #define HSW_GTT_CACHE_EN	0x4024
 #define   GTT_CACHE_EN_ALL	0xF0007FFF
+
+/*
+ * Premption-related registers
+ */
+#define RING_UHPTR(base)	((base)+0x134)
+#define   UHPTR_GFX_ADDR_ALIGN		(0x7)
+#define   UHPTR_VALID			(0x1)
+#define RING_PREEMPT_ADDR	0x0214c
+#define   PREEMPT_BATCH_LEVEL_MASK	(0x3)
+#define BB_PREEMPT_ADDR		0x02148
+#define SBB_PREEMPT_ADDR	0x0213c
+#define RS_PREEMPT_STATUS	0x0215c
+
 #define GEN7_WR_WATERMARK	0x4028
 #define GEN7_GFX_PRIO_CTRL	0x402C
 #define ARB_MODE		0x4030
@@ -6736,7 +6763,8 @@ enum skl_disp_power_wells {
 #define  VLV_SPAREG2H				0xA194
 
 #define  GTFIFODBG				0x120000
-#define    GT_FIFO_SBDROPERR			(1<<6)
+#define    GT_FIFO_CPU_ERROR_MASK		0xf
+#define    GT_FIFO_SDDROPERR			(1<<6)
 #define    GT_FIFO_BLOBDROPERR			(1<<5)
 #define    GT_FIFO_SB_READ_ABORTERR		(1<<4)
 #define    GT_FIFO_DROPERR			(1<<3)
drivers/gpu/drm/i915/intel_ringbuffer.h +29 -3
index 1987abd..48f60cc 100644
@@ -49,6 +49,12 @@ struct  intel_hw_status_page {
 #define I915_READ_MODE(ring) I915_READ(RING_MI_MODE((ring)->mmio_base))
 #define I915_WRITE_MODE(ring, val) I915_WRITE(RING_MI_MODE((ring)->mmio_base), val)
 
+#define I915_READ_UHPTR(ring) \
+		I915_READ(RING_UHPTR((ring)->mmio_base))
+#define I915_WRITE_UHPTR(ring, val) \
+		I915_WRITE(RING_UHPTR((ring)->mmio_base), val)
+#define I915_READ_NOPID(ring) I915_READ(RING_NOPID((ring)->mmio_base))
+
 /* seqno size is actually only a uint32, but since we plan to use MI_FLUSH_DW to
  * do the writes, and that must have qw aligned offsets, simply pretend it's 8b.
  */
@@ -426,10 +432,30 @@ intel_write_status_page(struct intel_engine_cs *ring,
  * 0x20-0x2f: Reserved (Gen6+)
  *
  * The area from dword 0x30 to 0x3ff is available for driver usage.
+ *
+ * Note: in general the allocation of these indices is arbitrary, as long
+ * as they are all unique. But a few of them are used with instructions that
+ * have specific alignment requirements, those particular indices must be
+ * chosen carefully to meet those requirements. The list below shows the
+ * currently-known alignment requirements:
+ *
+ *	I915_GEM_SCRATCH_INDEX	    must be EVEN
+ */
+
+/*
+ * Tracking; these are updated by the GPU at the beginning and/or end of every
+ * batch. One pair for regular buffers, the other for preemptive ones.
  */
-#define I915_GEM_HWS_INDEX		0x30
-#define I915_GEM_HWS_SCRATCH_INDEX	0x40
-#define I915_GEM_HWS_SCRATCH_ADDR (I915_GEM_HWS_SCRATCH_INDEX << MI_STORE_DWORD_INDEX_SHIFT)
+#define I915_BATCH_DONE_SEQNO		0x30  /* Completed batch seqno        */
+#define I915_BATCH_ACTIVE_SEQNO		0x31  /* In progress batch seqno      */
+#define I915_PREEMPTIVE_DONE_SEQNO	0x32  /* Completed preemptive batch   */
+#define I915_PREEMPTIVE_ACTIVE_SEQNO	0x33  /* In progress preemptive batch */
+
+#define I915_GEM_HWS_INDEX		I915_BATCH_DONE_SEQNO	/* alias */
+#define I915_GEM_ACTIVE_SEQNO_INDEX	I915_BATCH_ACTIVE_SEQNO	/* alias */
+
+#define I915_GEM_HWS_SCRATCH_INDEX	0x38  /* QWord */
+#define I915_GEM_HWS_SCRATCH_ADDR	(I915_GEM_HWS_SCRATCH_INDEX << MI_STORE_DWORD_INDEX_SHIFT)
 
 struct intel_ringbuffer *
 intel_engine_create_ringbuffer(struct intel_engine_cs *engine, int size);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#4 lore raw

Re: [PATCH 02/39] drm/i915: Updating assorted register and status page definitions

Hi,

Comments below.

Quoted 22 lines · On ma, 2015-11-23 at 11:38 +0000, [email protected] wrote:

From: Dave Gordon <[email protected]>

Added various definitions that will be useful for the scheduler in general and
pre-emptive context switching in particular.

Change-Id: Ica805b94160426def51f5d520f5ce51c60864a98
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_reg.h         | 30
+++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/intel_ringbuffer.h | 32
+++++++++++++++++++++++++++++---
 2 files changed, 58 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/i915_reg.h +0 -0
index db72f98..b986050 100644

drivers/gpu/drm/i915/i915_reg.h line 264

Quoted 6 lines
drivers/gpu/drm/i915/i915_reg.h +3 -0
@@ -259,6 +259,10 @@
 #define  MI_GLOBAL_GTT    (1<<22)
 
 #define MI_NOOP			MI_INSTR(0, 0)
+#define   MI_NOOP_WRITE_ID		(1<<22)
+#define   MI_NOOP_ID_MASK		((1<<22) - 1)
+#define   MI_NOOP_MID(id)		((id) & MI_NOOP_ID_MASK)

I'd stay consistent with the style of other _MASK/_SHIFT pairs elsewhere in this header. I find this rather unintuitive compared to 0xff notation especially if it would be with nonzero shift. MI_NOOP_MID is extra, too.

Most of these new definitions are not used anywhere in the series? Partially due to that I would prefer these to be introduced in smaller patches prior to first use, at least to an extent that one patch per series, with only stuff being used. Would make reviewing much easier to know they have been actually tested too.

Daniel, how do you want to see the register updates? As much added at once as possible, or only needed?

Quoted 149 lines
drivers/gpu/drm/i915/i915_reg.h +1 -0
+#define MI_NOOP_WITH_ID(id)	MI_INSTR(0,
 MI_NOOP_WRITE_ID|MI_NOOP_MID(id))
 #define MI_USER_INTERRUPT	MI_INSTR(0x02, 0)
 #define MI_WAIT_FOR_EVENT       MI_INSTR(0x03, 0)

#define MI_WAIT_FOR_OVERLAY_FLIP (1<<16)

drivers/gpu/drm/i915/i915_reg.h +7 -0
@@ -276,6 +280,7 @@
 #define MI_ARB_ON_OFF		MI_INSTR(0x08, 0)
 #define   MI_ARB_ENABLE			(1<<0)
 #define   MI_ARB_DISABLE		(0<<0)
+#define MI_ARB_CHECK		MI_INSTR(0x05, 0)
 #define MI_BATCH_BUFFER_END	MI_INSTR(0x0a, 0)
 #define MI_SUSPEND_FLUSH	MI_INSTR(0x0b, 0)
 #define   MI_SUSPEND_FLUSH_EN	(1<<0)
@@ -325,6 +330,8 @@
 #define   MI_SEMAPHORE_SYNC_INVALID (3<<16)
 #define   MI_SEMAPHORE_SYNC_MASK    (3<<16)
 #define MI_SET_CONTEXT		MI_INSTR(0x18, 0)
+#define   MI_CONTEXT_ADDR_MASK		((~0)<<12)
+#define   MI_SET_CONTEXT_FLAG_MASK	((1<<12)-1)
 #define   MI_MM_SPACE_GTT		(1<<8)
 #define   MI_MM_SPACE_PHYSICAL		(0<<8)
 #define   MI_SAVE_EXT_STATE_EN		(1<<3)
@@ -344,6 +351,10 @@
 #define   MI_USE_GGTT		(1 << 22) /* g4x+ */
 #define MI_STORE_DWORD_INDEX	MI_INSTR(0x21, 1)
 #define   MI_STORE_DWORD_INDEX_SHIFT 2
+#define MI_STORE_REG_MEM	MI_INSTR(0x24, 1)
+#define   MI_STORE_REG_MEM_GTT		(1 << 22)
+#define   MI_STORE_REG_MEM_PREDICATE	(1 << 21)
+
 /* Official intel docs are somewhat sloppy concerning

MI_LOAD_REGISTER_IMM:
* - Always issue a MI_NOOP _before_ the MI_LOAD_REGISTER_IMM -
otherwise hw
* simply ignores the register load under certain conditions.

drivers/gpu/drm/i915/i915_reg.h +18 -1
@@ -358,7 +369,10 @@
 #define MI_FLUSH_DW		MI_INSTR(0x26, 1) /* for GEN6 */
 #define   MI_FLUSH_DW_STORE_INDEX	(1<<21)
 #define   MI_INVALIDATE_TLB		(1<<18)
+#define   MI_FLUSH_DW_OP_NONE		(0<<14)
 #define   MI_FLUSH_DW_OP_STOREDW	(1<<14)
+#define   MI_FLUSH_DW_OP_RSVD		(2<<14)
+#define   MI_FLUSH_DW_OP_STAMP		(3<<14)
 #define   MI_FLUSH_DW_OP_MASK		(3<<14)
 #define   MI_FLUSH_DW_NOTIFY		(1<<8)
 #define   MI_INVALIDATE_BSD		(1<<7)
@@ -1533,6 +1547,19 @@ enum skl_disp_power_wells {
 
 #define HSW_GTT_CACHE_EN	0x4024
 #define   GTT_CACHE_EN_ALL	0xF0007FFF
+
+/*
+ * Premption-related registers
+ */
+#define RING_UHPTR(base)	((base)+0x134)
+#define   UHPTR_GFX_ADDR_ALIGN		(0x7)
+#define   UHPTR_VALID			(0x1)
+#define RING_PREEMPT_ADDR	0x0214c
+#define   PREEMPT_BATCH_LEVEL_MASK	(0x3)
+#define BB_PREEMPT_ADDR		0x02148
+#define SBB_PREEMPT_ADDR	0x0213c
+#define RS_PREEMPT_STATUS	0x0215c
+
 #define GEN7_WR_WATERMARK	0x4028
 #define GEN7_GFX_PRIO_CTRL	0x402C
 #define ARB_MODE		0x4030
@@ -6736,7 +6763,8 @@ enum skl_disp_power_wells {
 #define  VLV_SPAREG2H				0xA194
 
 #define  GTFIFODBG				0x120000
-#define    GT_FIFO_SBDROPERR			(1<<6)
+#define    GT_FIFO_CPU_ERROR_MASK		0xf
+#define    GT_FIFO_SDDROPERR			(1<<6)
 #define    GT_FIFO_BLOBDROPERR			(1<<5)
 #define    GT_FIFO_SB_READ_ABORTERR		(1<<4)
 #define    GT_FIFO_DROPERR			(1<<3)
drivers/gpu/drm/i915/intel_ringbuffer.h +6 -3
index 1987abd..48f60cc 100644
@@ -49,6 +49,12 @@ struct  intel_hw_status_page {
 #define I915_READ_MODE(ring) I915_READ(RING_MI_MODE((ring)
->mmio_base))
 #define I915_WRITE_MODE(ring, val) I915_WRITE(RING_MI_MODE((ring)
->mmio_base), val)
 
+#define I915_READ_UHPTR(ring) \
+		I915_READ(RING_UHPTR((ring)->mmio_base))
+#define I915_WRITE_UHPTR(ring, val) \
+		I915_WRITE(RING_UHPTR((ring)->mmio_base), val)
+#define I915_READ_NOPID(ring) I915_READ(RING_NOPID((ring)
->mmio_base))
+
 /* seqno size is actually only a uint32, but since we plan to use

MI_FLUSH_DW to
* do the writes, and that must have qw aligned offsets, simply
pretend it's 8b.
*/

drivers/gpu/drm/i915/intel_ringbuffer.h +2 -0
@@ -426,10 +432,30 @@ intel_write_status_page(struct intel_engine_cs
 *ring,
  * 0x20-0x2f: Reserved (Gen6+)
  *
  * The area from dword 0x30 to 0x3ff is available for driver usage.
+ *
+ * Note: in general the allocation of these indices is arbitrary, as

long
+ * as they are all unique. But a few of them are used with instructions that
+ * have specific alignment requirements, those particular indices must be
+ * chosen carefully to meet those requirements. The list below shows the
+ * currently-known alignment requirements:
+ *
+ * I915_GEM_SCRATCH_INDEX must be EVEN
+ */

drivers/gpu/drm/i915/intel_ringbuffer.h +3 -0
+
+/*
+ * Tracking; these are updated by the GPU at the beginning and/or

end of every
+ * batch. One pair for regular buffers, the other for preemptive ones.
*/

drivers/gpu/drm/i915/intel_ringbuffer.h +0 -3
-#define I915_GEM_HWS_INDEX		0x30
-#define I915_GEM_HWS_SCRATCH_INDEX	0x40
-#define I915_GEM_HWS_SCRATCH_ADDR (I915_GEM_HWS_SCRATCH_INDEX <<

MI_STORE_DWORD_INDEX_SHIFT)
+#define I915_BATCH_DONE_SEQNO 0x30 /* Completed
batch seqno */
+#define I915_BATCH_ACTIVE_SEQNO 0x31 /* In progress
batch seqno */
+#define I915_PREEMPTIVE_DONE_SEQNO 0x32 /* Completed
preemptive batch */
+#define I915_PREEMPTIVE_ACTIVE_SEQNO 0x33 /* In progress
preemptive batch */

drivers/gpu/drm/i915/intel_ringbuffer.h +2 -0
+
+#define I915_GEM_HWS_INDEX		I915_BATCH_DONE_SEQNO	

/* alias */
+#define I915_GEM_ACTIVE_SEQNO_INDEX I915_BATCH_ACTIVE_SEQNO
/* alias */

drivers/gpu/drm/i915/intel_ringbuffer.h +3 -0
+
+#define I915_GEM_HWS_SCRATCH_INDEX	0x38  /* QWord */
+#define I915_GEM_HWS_SCRATCH_ADDR	(I915_GEM_HWS_SCRATCH_INDEX

<< MI_STORE_DWORD_INDEX_SHIFT)

struct intel_ringbuffer * intel_engine_create_ringbuffer(struct intel_engine_cs *engine, int
size);

Signature · 8 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [PATCH 02/39] drm/i915: Updating assorted register and status page definitions

Quoted 39 lines · Joonas Lahtinen

Hi,

Comments below.

Quoted 22 lines · On ma, 2015-11-23 at 11:38 +0000, [email protected] wrote:

From: Dave Gordon <[email protected]>

Added various definitions that will be useful for the scheduler in general and
pre-emptive context switching in particular.

Change-Id: Ica805b94160426def51f5d520f5ce51c60864a98
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_reg.h         | 30
+++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/intel_ringbuffer.h | 32
+++++++++++++++++++++++++++++---
 2 files changed, 58 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/i915_reg.h +0 -0
index db72f98..b986050 100644

drivers/gpu/drm/i915/i915_reg.h line 264

Quoted 6 lines
drivers/gpu/drm/i915/i915_reg.h +3 -0
@@ -259,6 +259,10 @@
 #define  MI_GLOBAL_GTT    (1<<22)
 
 #define MI_NOOP			MI_INSTR(0, 0)
+#define   MI_NOOP_WRITE_ID		(1<<22)
+#define   MI_NOOP_ID_MASK		((1<<22) - 1)
+#define   MI_NOOP_MID(id)		((id) & MI_NOOP_ID_MASK)

I'd stay consistent with the style of other _MASK/_SHIFT pairs elsewhere in this header. I find this rather unintuitive compared to 0xff notation especially if it would be with nonzero shift. MI_NOOP_MID is extra, too.

Quoted 7 lines

Most of these new definitions are not used anywhere in the series? Partially due to that I would prefer these to be introduced in smaller patches prior to first use, at least to an extent that one patch per series, with only stuff being used. Would make reviewing much easier to know they have been actually tested too.

Daniel, how do you want to see the register updates? As much added at once as possible, or only needed?

I'm fine with a separate patch to just update them all. We do that regularly for feature work.
-Daniel

Quoted 155 lines
Quoted 150 lines
drivers/gpu/drm/i915/i915_reg.h +1 -0
+#define MI_NOOP_WITH_ID(id)	MI_INSTR(0,
 MI_NOOP_WRITE_ID|MI_NOOP_MID(id))
 #define MI_USER_INTERRUPT	MI_INSTR(0x02, 0)
 #define MI_WAIT_FOR_EVENT       MI_INSTR(0x03, 0)

#define MI_WAIT_FOR_OVERLAY_FLIP (1<<16)

drivers/gpu/drm/i915/i915_reg.h +7 -0
@@ -276,6 +280,7 @@
 #define MI_ARB_ON_OFF		MI_INSTR(0x08, 0)
 #define   MI_ARB_ENABLE			(1<<0)
 #define   MI_ARB_DISABLE		(0<<0)
+#define MI_ARB_CHECK		MI_INSTR(0x05, 0)
 #define MI_BATCH_BUFFER_END	MI_INSTR(0x0a, 0)
 #define MI_SUSPEND_FLUSH	MI_INSTR(0x0b, 0)
 #define   MI_SUSPEND_FLUSH_EN	(1<<0)
@@ -325,6 +330,8 @@
 #define   MI_SEMAPHORE_SYNC_INVALID (3<<16)
 #define   MI_SEMAPHORE_SYNC_MASK    (3<<16)
 #define MI_SET_CONTEXT		MI_INSTR(0x18, 0)
+#define   MI_CONTEXT_ADDR_MASK		((~0)<<12)
+#define   MI_SET_CONTEXT_FLAG_MASK	((1<<12)-1)
 #define   MI_MM_SPACE_GTT		(1<<8)
 #define   MI_MM_SPACE_PHYSICAL		(0<<8)
 #define   MI_SAVE_EXT_STATE_EN		(1<<3)
@@ -344,6 +351,10 @@
 #define   MI_USE_GGTT		(1 << 22) /* g4x+ */
 #define MI_STORE_DWORD_INDEX	MI_INSTR(0x21, 1)
 #define   MI_STORE_DWORD_INDEX_SHIFT 2
+#define MI_STORE_REG_MEM	MI_INSTR(0x24, 1)
+#define   MI_STORE_REG_MEM_GTT		(1 << 22)
+#define   MI_STORE_REG_MEM_PREDICATE	(1 << 21)
+
 /* Official intel docs are somewhat sloppy concerning

MI_LOAD_REGISTER_IMM:
* - Always issue a MI_NOOP _before_ the MI_LOAD_REGISTER_IMM -
otherwise hw
* simply ignores the register load under certain conditions.

drivers/gpu/drm/i915/i915_reg.h +18 -1
@@ -358,7 +369,10 @@
 #define MI_FLUSH_DW		MI_INSTR(0x26, 1) /* for GEN6 */
 #define   MI_FLUSH_DW_STORE_INDEX	(1<<21)
 #define   MI_INVALIDATE_TLB		(1<<18)
+#define   MI_FLUSH_DW_OP_NONE		(0<<14)
 #define   MI_FLUSH_DW_OP_STOREDW	(1<<14)
+#define   MI_FLUSH_DW_OP_RSVD		(2<<14)
+#define   MI_FLUSH_DW_OP_STAMP		(3<<14)
 #define   MI_FLUSH_DW_OP_MASK		(3<<14)
 #define   MI_FLUSH_DW_NOTIFY		(1<<8)
 #define   MI_INVALIDATE_BSD		(1<<7)
@@ -1533,6 +1547,19 @@ enum skl_disp_power_wells {
 
 #define HSW_GTT_CACHE_EN	0x4024
 #define   GTT_CACHE_EN_ALL	0xF0007FFF
+
+/*
+ * Premption-related registers
+ */
+#define RING_UHPTR(base)	((base)+0x134)
+#define   UHPTR_GFX_ADDR_ALIGN		(0x7)
+#define   UHPTR_VALID			(0x1)
+#define RING_PREEMPT_ADDR	0x0214c
+#define   PREEMPT_BATCH_LEVEL_MASK	(0x3)
+#define BB_PREEMPT_ADDR		0x02148
+#define SBB_PREEMPT_ADDR	0x0213c
+#define RS_PREEMPT_STATUS	0x0215c
+
 #define GEN7_WR_WATERMARK	0x4028
 #define GEN7_GFX_PRIO_CTRL	0x402C
 #define ARB_MODE		0x4030
@@ -6736,7 +6763,8 @@ enum skl_disp_power_wells {
 #define  VLV_SPAREG2H				0xA194
 
 #define  GTFIFODBG				0x120000
-#define    GT_FIFO_SBDROPERR			(1<<6)
+#define    GT_FIFO_CPU_ERROR_MASK		0xf
+#define    GT_FIFO_SDDROPERR			(1<<6)
 #define    GT_FIFO_BLOBDROPERR			(1<<5)
 #define    GT_FIFO_SB_READ_ABORTERR		(1<<4)
 #define    GT_FIFO_DROPERR			(1<<3)
drivers/gpu/drm/i915/intel_ringbuffer.h +6 -3
index 1987abd..48f60cc 100644
@@ -49,6 +49,12 @@ struct  intel_hw_status_page {
 #define I915_READ_MODE(ring) I915_READ(RING_MI_MODE((ring)
->mmio_base))
 #define I915_WRITE_MODE(ring, val) I915_WRITE(RING_MI_MODE((ring)
->mmio_base), val)
 
+#define I915_READ_UHPTR(ring) \
+		I915_READ(RING_UHPTR((ring)->mmio_base))
+#define I915_WRITE_UHPTR(ring, val) \
+		I915_WRITE(RING_UHPTR((ring)->mmio_base), val)
+#define I915_READ_NOPID(ring) I915_READ(RING_NOPID((ring)
->mmio_base))
+
 /* seqno size is actually only a uint32, but since we plan to use

MI_FLUSH_DW to
* do the writes, and that must have qw aligned offsets, simply
pretend it's 8b.
*/

drivers/gpu/drm/i915/intel_ringbuffer.h +2 -0
@@ -426,10 +432,30 @@ intel_write_status_page(struct intel_engine_cs
 *ring,
  * 0x20-0x2f: Reserved (Gen6+)
  *
  * The area from dword 0x30 to 0x3ff is available for driver usage.
+ *
+ * Note: in general the allocation of these indices is arbitrary, as

long
+ * as they are all unique. But a few of them are used with instructions that
+ * have specific alignment requirements, those particular indices must be
+ * chosen carefully to meet those requirements. The list below shows the
+ * currently-known alignment requirements:
+ *
+ * I915_GEM_SCRATCH_INDEX must be EVEN
+ */

drivers/gpu/drm/i915/intel_ringbuffer.h +3 -0
+
+/*
+ * Tracking; these are updated by the GPU at the beginning and/or

end of every
+ * batch. One pair for regular buffers, the other for preemptive ones.
*/

drivers/gpu/drm/i915/intel_ringbuffer.h +0 -3
-#define I915_GEM_HWS_INDEX		0x30
-#define I915_GEM_HWS_SCRATCH_INDEX	0x40
-#define I915_GEM_HWS_SCRATCH_ADDR (I915_GEM_HWS_SCRATCH_INDEX <<

MI_STORE_DWORD_INDEX_SHIFT)
+#define I915_BATCH_DONE_SEQNO 0x30 /* Completed
batch seqno */
+#define I915_BATCH_ACTIVE_SEQNO 0x31 /* In progress
batch seqno */
+#define I915_PREEMPTIVE_DONE_SEQNO 0x32 /* Completed
preemptive batch */
+#define I915_PREEMPTIVE_ACTIVE_SEQNO 0x33 /* In progress
preemptive batch */

drivers/gpu/drm/i915/intel_ringbuffer.h +2 -0
+
+#define I915_GEM_HWS_INDEX		I915_BATCH_DONE_SEQNO	

/* alias */
+#define I915_GEM_ACTIVE_SEQNO_INDEX I915_BATCH_ACTIVE_SEQNO
/* alias */

drivers/gpu/drm/i915/intel_ringbuffer.h +3 -0
+
+#define I915_GEM_HWS_SCRATCH_INDEX	0x38  /* QWord */
+#define I915_GEM_HWS_SCRATCH_ADDR	(I915_GEM_HWS_SCRATCH_INDEX

<< MI_STORE_DWORD_INDEX_SHIFT)

struct intel_ringbuffer * intel_engine_create_ringbuffer(struct intel_engine_cs *engine, int
size);

Signature · 3 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#6 lore raw

PATCH 3/39 drm/i915: Explicit power enable during deferred context initialisation

In reply to [email protected]

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

While that is a good thing in general, it causes a problem with the deferred context initialisation. That is done quite early on in the execbuf code path - it happens at context validation time rather than context switch time. Some of the deferred work requires the power to be enabled. Hence this patch adds an explicit power reference count to the deferred initialisation code itself.

Change-Id: Id7b1535dfd8809a2bd5546272de2bbec39da2868
Issue: GMINL-5159

---
 drivers/gpu/drm/i915/intel_lrc.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/intel_lrc.c +8 -1
index 76fc245..e510730 100644
@@ -2470,6 +2470,8 @@ int intel_lr_context_deferred_alloc(struct intel_context *ctx,
 	WARN_ON(ctx->legacy_hw_ctx.rcs_state != NULL);
 	WARN_ON(ctx->engine[ring->id].state);
 
+	intel_runtime_pm_get(dev->dev_private);
+
 	context_size = round_up(get_lr_context_size(ring), 4096);
 
 	/* One extra page as the sharing data between driver and GuC */
@@ -2478,7 +2480,8 @@ int intel_lr_context_deferred_alloc(struct intel_context *ctx,
 	ctx_obj = i915_gem_alloc_object(dev, context_size);
 	if (!ctx_obj) {
 		DRM_DEBUG_DRIVER("Alloc LRC backing obj failed.\n");
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto error_pm;
 	}
 
 	ringbuf = intel_engine_create_ringbuffer(ring, 4 * PAGE_SIZE);
@@ -2524,6 +2527,8 @@ int intel_lr_context_deferred_alloc(struct intel_context *ctx,
 		}
 		i915_add_request_no_flush(req);
 	}
+
+	intel_runtime_pm_put(dev->dev_private);
 	return 0;
 
 error_ringbuf:
@@ -2532,6 +2537,8 @@ error_deref_obj:
 	drm_gem_object_unreference(&ctx_obj->base);
 	ctx->engine[ring->id].ringbuf = NULL;
 	ctx->engine[ring->id].state = NULL;
+error_pm:
+	intel_runtime_pm_put(dev->dev_private);
 	return ret;
 }
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer Reviewed-by on v1 3/39
#7 lore raw

Re: [PATCH 03/39] drm/i915: Explicit power enable during deferred context initialisation

Quoted 17 lines · On ma, 2015-11-23 at 11:38 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware
submission part in the back half.

While that is a good thing in general, it causes a problem with the deferred context initialisation. That is done quite early on in the execbuf code path - it happens at context validation time rather than context switch time. Some of the deferred work requires the power to be enabled. Hence this patch adds an explicit power reference count to
the deferred initialisation code itself.

Quoted 2 lines

Change-Id: Id7b1535dfd8809a2bd5546272de2bbec39da2868
Issue: GMINL-5159

Quoted 52 lines
---
 drivers/gpu/drm/i915/intel_lrc.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/intel_lrc.c +2 -0
index 76fc245..e510730 100644
@@ -2470,6 +2470,8 @@ int intel_lr_context_deferred_alloc(struct
 intel_context *ctx,
 	WARN_ON(ctx->legacy_hw_ctx.rcs_state != NULL);
 	WARN_ON(ctx->engine[ring->id].state);
 
+	intel_runtime_pm_get(dev->dev_private);
+
 	context_size = round_up(get_lr_context_size(ring), 4096);
 

/* One extra page as the sharing data between driver and GuC
*/

drivers/gpu/drm/i915/intel_lrc.c +2 -1
@@ -2478,7 +2480,8 @@ int intel_lr_context_deferred_alloc(struct
 intel_context *ctx,
 	ctx_obj = i915_gem_alloc_object(dev, context_size);
 	if (!ctx_obj) {
 		DRM_DEBUG_DRIVER("Alloc LRC backing obj failed.\n");
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto error_pm;
 	}
 

ringbuf = intel_engine_create_ringbuffer(ring, 4 *
PAGE_SIZE);

drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -2524,6 +2527,8 @@ int intel_lr_context_deferred_alloc(struct
 intel_context *ctx,
 		}
 		i915_add_request_no_flush(req);
 	}
+
+	intel_runtime_pm_put(dev->dev_private);
 	return 0;
 

error_ringbuf:

drivers/gpu/drm/i915/intel_lrc.c +2 -0
@@ -2532,6 +2537,8 @@ error_deref_obj:
 	drm_gem_object_unreference(&ctx_obj->base);
 	ctx->engine[ring->id].ringbuf = NULL;
 	ctx->engine[ring->id].state = NULL;
+error_pm:
+	intel_runtime_pm_put(dev->dev_private);
 	return ret;
 }
 
Signature · 8 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#8 lore raw

PATCH 4/39 drm/i915: Prelude to splitting i915_gem_do_execbuffer in two

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with their submission to the hardware. This basically means splitting the execbuffer() function in half. This change rearranges some code ready for the split to occur.

Change-Id: Icc9c8afaac18821f3eb8a151a49f918f90c068a3
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 64 ++++++++++++++++++------------
 drivers/gpu/drm/i915/intel_lrc.c           | 18 ++++++---
 2 files changed, 51 insertions(+), 31 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +38 -26
index 5f629f8..f7f1057 100644
@@ -935,10 +935,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return intel_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 static bool
@@ -1191,17 +1188,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	u32 instp_mask;
 	int ret;
 
-	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
-	if (ret)
-		return ret;
-
-	ret = i915_switch_context(params->request);
-	if (ret)
-		return ret;
-
-	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
-	     "%s didn't clear reload\n", ring->name);
-
 	instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
 	instp_mask = I915_EXEC_CONSTANTS_MASK;
 	switch (instp_mode) {
@@ -1235,11 +1221,37 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 		return -EINVAL;
 	}
 
+	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
+	if (ret)
+		return ret;
+
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	intel_runtime_pm_get(dev_priv);
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = intel_ring_invalidate_all_caches(params->request);
+	if (ret)
+		goto error;
+
+	/* Switch to the correct context for the batch */
+	ret = i915_switch_context(params->request);
+	if (ret)
+		goto error;
+
+	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
+	     "%s didn't clear reload\n", ring->name);
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_ring_begin(params->request, 4);
 		if (ret)
-			return ret;
+			goto error;
 
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
@@ -1253,7 +1265,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
 		ret = i915_reset_gen7_sol_offsets(dev, params->request);
 		if (ret)
-			return ret;
+			goto error;
 	}
 
 	exec_len   = args->batch_len;
@@ -1264,14 +1276,20 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto error;
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
-	return 0;
+error:
+	/*
+	 * intel_gpu_busy should also get a ref, so it will free when the device
+	 * is really idle.
+	 */
+	intel_runtime_pm_put(dev_priv);
+
+	return ret;
 }
 
 /**
@@ -1480,8 +1498,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 		args->flags &= ~I915_EXEC_WAIT_FENCE;
 	}
 
-	intel_runtime_pm_get(dev_priv);
-
 	ret = i915_mutex_lock_interruptible(dev);
 	if (ret)
 		goto pre_mutex_err;
@@ -1688,10 +1704,6 @@ err:
 	mutex_unlock(&dev->struct_mutex);
 
 pre_mutex_err:
-	/* intel_gpu_busy should also get a ref, so it will free when the device
-	 * is really idle. */
-	intel_runtime_pm_put(dev_priv);
-
 	return ret;
 }
 
drivers/gpu/drm/i915/intel_lrc.c +13 -5
index e510730..4bf0ee6 100644
@@ -647,10 +647,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return logical_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -913,6 +910,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 	if (ret)
 		return ret;
 
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = logical_ring_invalidate_all_caches(params->request);
+	if (ret)
+		return ret;
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_logical_ring_begin(params->request, 4);
@@ -937,7 +946,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#9 lore raw

PATCH 5/39 drm/i915: Split i915_dem_do_execbuffer() in half

In reply to [email protected]

From: John Harrison <[email protected]>

Split the execbuffer() function in half. The first half collects and validates all the information requried to process the batch buffer. It also does all the object pinning, relocations, active list management, etc - basically anything that must be done upfront before the IOCTL returns and allows the user land side to start changing/freeing things. The second half does the actual ring submission.

This change implements the split but leaves the back half being called directly from the end of the front half.

v2: Updated due to changes in underlying tree - addition of sync fence support and removal of cliprects.

Change-Id: I5e1c77639ce526ab2401b0323186c518bf13da0a
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  11 +++
 drivers/gpu/drm/i915/i915_gem.c            |   2 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 130 ++++++++++++++++++++---------
 drivers/gpu/drm/i915/intel_lrc.c           |  58 +++++++++----
 drivers/gpu/drm/i915/intel_lrc.h           |   1 +
 5 files changed, 146 insertions(+), 56 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
index 194bca0..eb00454 100644
@@ -1679,10 +1679,18 @@ struct i915_execbuffer_params {
 	struct drm_device               *dev;
 	struct drm_file                 *file;
 	uint32_t                        dispatch_flags;
+	uint32_t                        args_flags;
 	uint32_t                        args_batch_start_offset;
+	uint32_t                        args_batch_len;
+	uint32_t                        args_num_cliprects;
+	uint32_t                        args_DR1;
+	uint32_t                        args_DR4;
 	uint64_t                        batch_obj_vm_offset;
 	struct intel_engine_cs          *ring;
 	struct drm_i915_gem_object      *batch_obj;
+	struct drm_clip_rect            *cliprects;
+	uint32_t                        instp_mask;
+	int                             instp_mode;
 	struct intel_context            *ctx;
 	struct drm_i915_gem_request     *request;
 };
@@ -1944,6 +1952,7 @@ struct drm_i915_private {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
 				      struct drm_i915_gem_execbuffer2 *args,
 				      struct list_head *vmas);
+		int (*execbuf_final)(struct i915_execbuffer_params *params);
 		int (*init_rings)(struct drm_device *dev);
 		void (*cleanup_ring)(struct intel_engine_cs *ring);
 		void (*stop_ring)(struct intel_engine_cs *ring);
@@ -2798,9 +2807,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
 void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
 					struct drm_i915_gem_request *req);
 void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj);
 int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 				   struct drm_i915_gem_execbuffer2 *args,
 				   struct list_head *vmas);
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params);
 int i915_gem_execbuffer(struct drm_device *dev, void *data,
 			struct drm_file *file_priv);
 int i915_gem_execbuffer2(struct drm_device *dev, void *data,
drivers/gpu/drm/i915/i915_gem.c +2 -0
index 0e41386..8ecc1f4 100644
@@ -5256,11 +5256,13 @@ int i915_gem_init(struct drm_device *dev)
 
 	if (!i915.enable_execlists) {
 		dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
+		dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final;
 		dev_priv->gt.init_rings = i915_gem_init_rings;
 		dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer;
 		dev_priv->gt.stop_ring = intel_stop_ring_buffer;
 	} else {
 		dev_priv->gt.execbuf_submit = intel_execlists_submission;
+		dev_priv->gt.execbuf_final = intel_execlists_submission_final;
 		dev_priv->gt.init_rings = intel_logical_rings_init;
 		dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup;
 		dev_priv->gt.stop_ring = intel_logical_ring_stop;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +91 -39

214 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.c +41 -17

104 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.h +1 -0
index 4e60d54..8d9bad7 100644
@@ -93,6 +93,7 @@ struct i915_execbuffer_params;
 int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas);
+int intel_execlists_submission_final(struct i915_execbuffer_params *params);
 u32 intel_execlists_ctx_id(struct drm_i915_gem_object *ctx_obj);
 
 void intel_lrc_irq_handler(struct intel_engine_cs *ring);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#10 lore raw

PATCH 5/40 drm/i915: Split i915_dem_do_execbuffer() in half

From: John Harrison <[email protected]>

Split the execbuffer() function in half. The first half collects and validates all the information requried to process the batch buffer. It also does all the object pinning, relocations, active list management, etc - basically anything that must be done upfront before the IOCTL returns and allows the user land side to start changing/freeing things. The second half does the actual ring submission.

This change implements the split but leaves the back half being called directly from the end of the front half.

v2: Updated due to changes in underlying tree - addition of sync fence support and removal of cliprects.

v3: Moved local 'ringbuf' variable to make later patches in the series a bit neater.

Change-Id: I5e1c77639ce526ab2401b0323186c518bf13da0a
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  11 +++
 drivers/gpu/drm/i915/i915_gem.c            |   2 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 130 ++++++++++++++++++++---------
 drivers/gpu/drm/i915/intel_lrc.c           |  57 +++++++++----
 drivers/gpu/drm/i915/intel_lrc.h           |   1 +
 5 files changed, 145 insertions(+), 56 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
index 194bca0..eb00454 100644
@@ -1679,10 +1679,18 @@ struct i915_execbuffer_params {
 	struct drm_device               *dev;
 	struct drm_file                 *file;
 	uint32_t                        dispatch_flags;
+	uint32_t                        args_flags;
 	uint32_t                        args_batch_start_offset;
+	uint32_t                        args_batch_len;
+	uint32_t                        args_num_cliprects;
+	uint32_t                        args_DR1;
+	uint32_t                        args_DR4;
 	uint64_t                        batch_obj_vm_offset;
 	struct intel_engine_cs          *ring;
 	struct drm_i915_gem_object      *batch_obj;
+	struct drm_clip_rect            *cliprects;
+	uint32_t                        instp_mask;
+	int                             instp_mode;
 	struct intel_context            *ctx;
 	struct drm_i915_gem_request     *request;
 };
@@ -1944,6 +1952,7 @@ struct drm_i915_private {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
 				      struct drm_i915_gem_execbuffer2 *args,
 				      struct list_head *vmas);
+		int (*execbuf_final)(struct i915_execbuffer_params *params);
 		int (*init_rings)(struct drm_device *dev);
 		void (*cleanup_ring)(struct intel_engine_cs *ring);
 		void (*stop_ring)(struct intel_engine_cs *ring);
@@ -2798,9 +2807,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
 void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
 					struct drm_i915_gem_request *req);
 void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj);
 int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 				   struct drm_i915_gem_execbuffer2 *args,
 				   struct list_head *vmas);
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params);
 int i915_gem_execbuffer(struct drm_device *dev, void *data,
 			struct drm_file *file_priv);
 int i915_gem_execbuffer2(struct drm_device *dev, void *data,
drivers/gpu/drm/i915/i915_gem.c +2 -0
index 3c88678..b9501ca 100644
@@ -5257,11 +5257,13 @@ int i915_gem_init(struct drm_device *dev)
 
 	if (!i915.enable_execlists) {
 		dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
+		dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final;
 		dev_priv->gt.init_rings = i915_gem_init_rings;
 		dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer;
 		dev_priv->gt.stop_ring = intel_stop_ring_buffer;
 	} else {
 		dev_priv->gt.execbuf_submit = intel_execlists_submission;
+		dev_priv->gt.execbuf_final = intel_execlists_submission_final;
 		dev_priv->gt.init_rings = intel_logical_rings_init;
 		dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup;
 		dev_priv->gt.stop_ring = intel_logical_ring_stop;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +91 -39

214 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.c +40 -17

103 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.h +1 -0
index 4e60d54..8d9bad7 100644
@@ -93,6 +93,7 @@ struct i915_execbuffer_params;
 int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas);
+int intel_execlists_submission_final(struct i915_execbuffer_params *params);
 u32 intel_execlists_ctx_id(struct drm_i915_gem_object *ctx_obj);
 
 void intel_lrc_irq_handler(struct intel_engine_cs *ring);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#11 lore raw

PATCH 6/39 drm/i915: Re-instate request->uniq because it is extremely useful

In reply to [email protected]

From: John Harrison <[email protected]>

The seqno value cannot always be used when debugging issues via trace points. This is because it can be reset back to start, especially during TDR type tests. Also, when the scheduler arrives the seqno is only valid while a given request is executing on the hardware. While the request is simply queued waiting for submission, it's seqno value will be zero (meaning invalid).

For: VIZ-5115

---
 drivers/gpu/drm/i915/i915_drv.h   |  5 +++++
 drivers/gpu/drm/i915/i915_gem.c   |  3 ++-
 drivers/gpu/drm/i915/i915_trace.h | 25 +++++++++++++++++--------
 3 files changed, 24 insertions(+), 9 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +5 -0
index eb00454..5d390d9 100644
@@ -1963,6 +1963,8 @@ struct drm_i915_private {
 	/* perform PHY state sanity checks? */
 	bool chv_phy_assert[2];
 
+	uint32_t request_uniq;
+
 	/*
 	 * NOTE: This is the dri1/ums dungeon, don't add stuff here. Your patch
 	 * will be rejected. Instead look for a better place.
@@ -2210,6 +2212,9 @@ struct drm_i915_gem_request {
 	/** GEM sequence number associated with this request. */
 	uint32_t seqno;
 
+	/* Unique identifier which can be used for trace points & debug */
+	uint32_t uniq;
+
 	/** Position in the ringbuffer of the start of the request */
 	u32 head;
 
drivers/gpu/drm/i915/i915_gem.c +2 -1
index 8ecc1f4..b23746a 100644
@@ -2862,7 +2862,7 @@ static void i915_gem_request_fence_value_str(struct fence *req_fence, char *str,
 
 	req = container_of(req_fence, typeof(*req), fence);
 
-	snprintf(str, size, "%d [%d]", req->fence.seqno, req->seqno);
+	snprintf(str, size, "%d [%d:%d]", req->fence.seqno, req->uniq, req->seqno);
 }
 
 static const struct fence_ops i915_gem_request_fops = {
@@ -3015,6 +3015,7 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
 
 	req->i915 = dev_priv;
 	req->ring = ring;
+	req->uniq = dev_priv->request_uniq++;
 	req->ctx  = ctx;
 	i915_gem_context_reference(req->ctx);
 
drivers/gpu/drm/i915/i915_trace.h +17 -8
index 41a026d..76b08fa 100644
@@ -469,6 +469,7 @@ TRACE_EVENT(i915_gem_ring_sync_to,
 			     __field(u32, dev)
 			     __field(u32, sync_from)
 			     __field(u32, sync_to)
+			     __field(u32, uniq_to)
 			     __field(u32, seqno)
 			     ),
 
@@ -476,13 +477,14 @@ TRACE_EVENT(i915_gem_ring_sync_to,
 			   __entry->dev = from->dev->primary->index;
 			   __entry->sync_from = from->id;
 			   __entry->sync_to = to_req->ring->id;
+			   __entry->uniq_to = to_req->uniq;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   ),
 
-	    TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u",
+	    TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u, to_uniq=%u",
 		      __entry->dev,
 		      __entry->sync_from, __entry->sync_to,
-		      __entry->seqno)
+		      __entry->seqno, __entry->uniq_to)
 );
 
 TRACE_EVENT(i915_gem_ring_dispatch,
@@ -517,6 +519,7 @@ TRACE_EVENT(i915_gem_ring_flush,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, invalidate)
 			     __field(u32, flush)
 			     ),
@@ -524,12 +527,13 @@ TRACE_EVENT(i915_gem_ring_flush,
 	    TP_fast_assign(
 			   __entry->dev = req->ring->dev->primary->index;
 			   __entry->ring = req->ring->id;
+			   __entry->uniq = req->uniq;
 			   __entry->invalidate = invalidate;
 			   __entry->flush = flush;
 			   ),
 
-	    TP_printk("dev=%u, ring=%x, invalidate=%04x, flush=%04x",
-		      __entry->dev, __entry->ring,
+	    TP_printk("dev=%u, ring=%x, request=%u, invalidate=%04x, flush=%04x",
+		      __entry->dev, __entry->ring, __entry->uniq,
 		      __entry->invalidate, __entry->flush)
 );
 
@@ -540,6 +544,7 @@ DECLARE_EVENT_CLASS(i915_gem_request,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, seqno)
 			     ),
 
@@ -548,11 +553,13 @@ DECLARE_EVENT_CLASS(i915_gem_request,
 						i915_gem_request_get_ring(req);
 			   __entry->dev = ring->dev->primary->index;
 			   __entry->ring = ring->id;
+			   __entry->uniq = req ? req->uniq : 0;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   ),
 
-	    TP_printk("dev=%u, ring=%u, seqno=%u",
-		      __entry->dev, __entry->ring, __entry->seqno)
+	    TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u",
+		      __entry->dev, __entry->ring, __entry->uniq,
+		      __entry->seqno)
 );
 
 DEFINE_EVENT(i915_gem_request, i915_gem_request_add,
@@ -600,6 +607,7 @@ TRACE_EVENT(i915_gem_request_wait_begin,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, seqno)
 			     __field(bool, blocking)
 			     ),
@@ -615,13 +623,14 @@ TRACE_EVENT(i915_gem_request_wait_begin,
 						i915_gem_request_get_ring(req);
 			   __entry->dev = ring->dev->primary->index;
 			   __entry->ring = ring->id;
+			   __entry->uniq = req ? req->uniq : 0;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   __entry->blocking =
 				     mutex_is_locked(&ring->dev->struct_mutex);
 			   ),
 
-	    TP_printk("dev=%u, ring=%u, seqno=%u, blocking=%s",
-		      __entry->dev, __entry->ring,
+	    TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, blocking=%s",
+		      __entry->dev, __entry->ring, __entry->uniq,
 		      __entry->seqno, __entry->blocking ?  "yes (NB)" : "no")
 );
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#12 lore raw

PATCH 7/39 drm/i915: Start of GPU scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in byte size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code assigns a unique request and then packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

As each batch buffer completes, it raises an interrupt which wakes up the scheduler. Note that it is possible for multiple buffers to complete before the IRQ handler gets to run. Further, it is possible for the seqno values to be un-ordered (particularly once pre-emption is enabled). However, the scheduler keeps the list of executing buffers in order of hardware submission. Thus it can scan through the list until a matching seqno is found and then mark all in flight nodes from that point on as completed.

A deferred work queue is also poked by the interrupt handler. When this wakes up it can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

Change-Id: I1e08f59e650a3c2bbaaa9de7627da33849b06106
For: VIZ-1587

---
 drivers/gpu/drm/i915/Makefile         |   1 +
 drivers/gpu/drm/i915/i915_drv.h       |   4 +
 drivers/gpu/drm/i915/i915_gem.c       |   5 +
 drivers/gpu/drm/i915/i915_scheduler.c | 740 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  90 +++++
 5 files changed, 840 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
 i915-y := i915_drv.o \
 	  i915_irq.o \
 	  i915_params.o \
+	  i915_scheduler.o \
           i915_suspend.o \
 	  i915_sysfs.o \
 	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +4 -0
index 5d390d9..23aed32 100644
@@ -1695,6 +1695,8 @@ struct i915_execbuffer_params {
 	struct drm_i915_gem_request     *request;
 };
 
+struct i915_scheduler;
+
 /* used in computing the new watermarks state */
 struct intel_wm_config {
 	unsigned int num_pipes_active;
@@ -1947,6 +1949,8 @@ struct drm_i915_private {
 
 	struct i915_runtime_pm pm;
 
+	struct i915_scheduler *scheduler;
+
 	/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
 	struct {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
drivers/gpu/drm/i915/i915_gem.c +5 -0
index b23746a..5239340 100644
@@ -38,6 +38,7 @@
 #include <linux/pci.h>
 #include <linux/dma-buf.h>
 #include <../drivers/android/sync.h>
+#include "i915_scheduler.h"
 
 #define RQ_BUG_ON(expr)
 
@@ -5277,6 +5278,10 @@ int i915_gem_init(struct drm_device *dev)
 	 */
 	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
 
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+
 	ret = i915_gem_init_userptr(dev);
 	if (ret)
 		goto out_unlock;
drivers/gpu/drm/i915/i915_scheduler.c new file +740 -0

740 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h new file +90 -0

90 lines not on this page · raw message

Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#13 lore raw

PATCH 8/40 drm/i915: Start of GPU scheduler

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in byte size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code assigns a unique request and then packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

As each batch buffer completes, it raises an interrupt which wakes up the scheduler. Note that it is possible for multiple buffers to complete before the IRQ handler gets to run. Further, it is possible for the seqno values to be un-ordered (particularly once pre-emption is enabled). However, the scheduler keeps the list of executing buffers in order of hardware submission. Thus it can scan through the list until a matching seqno is found and then mark all in flight nodes from that point on as completed.

A deferred work queue is also poked by the interrupt handler. When this wakes up it can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

Change-Id: I1e08f59e650a3c2bbaaa9de7627da33849b06106
For: VIZ-1587

---
 drivers/gpu/drm/i915/Makefile         |   1 +
 drivers/gpu/drm/i915/i915_drv.h       |   4 +
 drivers/gpu/drm/i915/i915_gem.c       |   5 +
 drivers/gpu/drm/i915/i915_scheduler.c | 763 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  91 ++++
 5 files changed, 864 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
 i915-y := i915_drv.o \
 	  i915_irq.o \
 	  i915_params.o \
+	  i915_scheduler.o \
           i915_suspend.o \
 	  i915_sysfs.o \
 	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +4 -0
index 5d390d9..23aed32 100644
@@ -1695,6 +1695,8 @@ struct i915_execbuffer_params {
 	struct drm_i915_gem_request     *request;
 };
 
+struct i915_scheduler;
+
 /* used in computing the new watermarks state */
 struct intel_wm_config {
 	unsigned int num_pipes_active;
@@ -1947,6 +1949,8 @@ struct drm_i915_private {
 
 	struct i915_runtime_pm pm;
 
+	struct i915_scheduler *scheduler;
+
 	/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
 	struct {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
drivers/gpu/drm/i915/i915_gem.c +5 -0
index 280cff0..36a11d2 100644
@@ -38,6 +38,7 @@
 #include <linux/pci.h>
 #include <linux/dma-buf.h>
 #include <../drivers/android/sync.h>
+#include "i915_scheduler.h"
 
 #define RQ_BUG_ON(expr)
 
@@ -5278,6 +5279,10 @@ int i915_gem_init(struct drm_device *dev)
 	 */
 	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
 
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+
 	ret = i915_gem_init_userptr(dev);
 	if (ret)
 		goto out_unlock;
drivers/gpu/drm/i915/i915_scheduler.c new file +763 -0

763 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h new file +91 -0

91 lines not on this page · raw message

Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#14 lore raw

PATCH 8/39 drm/i915: Prepare retire_requests to handle out-of-order seqnos

In reply to [email protected]

From: John Harrison <[email protected]>

A major point of the GPU scheduler is that it re-orders batch buffers after they have been submitted to the driver. This leads to requests completing out of order. In turn, this means that the retire processing can no longer assume that all completed entries are at the front of the list. Rather than attempting to re-order the request list on a regular basis, it is better to simply scan the entire list.

v2: Removed deferred free code as no longer necessary due to request handling updates.

Change-Id: I7eb6793581d9d28eb832e0e94c116b7202fa1b26
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +13 -18
index 5239340..d38e013 100644
@@ -3218,6 +3218,7 @@ void i915_gem_reset(struct drm_device *dev)
 void
 i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
 {
+	struct drm_i915_gem_object *obj, *obj_next;
 	struct drm_i915_gem_request *req, *req_next;
 	LIST_HEAD(list_head);
 	unsigned long flags;
@@ -3231,37 +3232,31 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
 	 */
 	i915_gem_request_notify(ring, false);
 
+	/*
+	 * Note that request entries might be out of order due to rescheduling
+	 * and pre-emption. Thus both lists must be processed in their entirety
+	 * rather than stopping at the first non-complete entry.
+	 */
+
 	/* Retire requests first as we use it above for the early return.
 	 * If we retire requests last, we may use a later seqno and so clear
 	 * the requests lists without clearing the active list, leading to
 	 * confusion.
 	 */
-	while (!list_empty(&ring->request_list)) {
-		struct drm_i915_gem_request *request;
-
-		request = list_first_entry(&ring->request_list,
-					   struct drm_i915_gem_request,
-					   list);
-
-		if (!i915_gem_request_completed(request))
-			break;
+	list_for_each_entry_safe(req, req_next, &ring->request_list, list) {
+		if (!i915_gem_request_completed(req))
+			continue;
 
-		i915_gem_request_retire(request);
+		i915_gem_request_retire(req);
 	}
 
 	/* Move any buffers on the active list that are no longer referenced
 	 * by the ringbuffer to the flushing/inactive lists as appropriate,
 	 * before we free the context associated with the requests.
 	 */
-	while (!list_empty(&ring->active_list)) {
-		struct drm_i915_gem_object *obj;
-
-		obj = list_first_entry(&ring->active_list,
-				      struct drm_i915_gem_object,
-				      ring_list[ring->id]);
-
+	list_for_each_entry_safe(obj, obj_next, &ring->active_list, ring_list[ring->id]) {
 		if (!list_empty(&obj->last_read_req[ring->id]->list))
-			break;
+			continue;
 
 		i915_gem_object_retire__read(obj, ring->id);
 	}
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#15 lore raw

PATCH 9/39 drm/i915: Disable hardware semaphores when GPU scheduler is enabled

In reply to [email protected]

From: John Harrison <[email protected]>

Hardware sempahores require seqno values to be continuously incrementing. However, the scheduler's reordering of batch buffers means that the seqno values going through the hardware could be out of order. Thus semaphores can not be used.

On the other hand, the scheduler superceeds the need for hardware semaphores anyway. Having one ring stall waiting for something to complete on another ring is inefficient if that ring could be working on some other, independent task. This is what the scheduler is meant to do - keep the hardware as busy as possible by reordering batch buffers to avoid dependency stalls.

Change-Id: I95d1fceacd370455a9720d7dca55cfd0a1f6beaa
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.c         | 9 +++++++++
 drivers/gpu/drm/i915/i915_scheduler.c   | 7 +++++++
 drivers/gpu/drm/i915/i915_scheduler.h   | 1 +
 drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++
 4 files changed, 21 insertions(+)
drivers/gpu/drm/i915/i915_drv.c +9 -0
index 858d58c..e6be8f5 100644
@@ -34,6 +34,7 @@
 #include "i915_drv.h"
 #include "i915_trace.h"
 #include "intel_drv.h"
+#include "i915_scheduler.h"
 
 #include <linux/console.h>
 #include <linux/module.h>
@@ -581,6 +582,14 @@ void intel_detect_pch(struct drm_device *dev)
 
 bool i915_semaphore_is_enabled(struct drm_device *dev)
 {
+	/* Hardware semaphores are not compatible with the scheduler due to the
+	 * seqno values being potentially out of order. However, semaphores are
+	 * also not required as the scheduler will handle interring dependencies
+	 * and try do so in a way that does not cause dead time on the hardware.
+	 */
+	if (i915_scheduler_is_enabled(dev))
+		return false;
+
 	if (INTEL_INFO(dev)->gen < 6)
 		return false;
 
drivers/gpu/drm/i915/i915_scheduler.c +7 -0
index 76973ab..6386d1c 100644
@@ -38,6 +38,13 @@ static int         i915_scheduler_priority_bump(struct i915_scheduler *scheduler
 						struct i915_scheduler_queue_entry *target,
 						uint32_t bump);
 
+bool i915_scheduler_is_enabled(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+
+	return dev_priv->scheduler != NULL;
+}
+
 int i915_scheduler_init(struct drm_device *dev)
 {
 	struct drm_i915_private *dev_priv = dev->dev_private;
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index ee39b77..93faf40 100644
@@ -83,6 +83,7 @@ enum {
 	i915_sf_submitting          = (1 << 1),
 };
 
+bool        i915_scheduler_is_enabled(struct drm_device *dev);
 int         i915_scheduler_init(struct drm_device *dev);
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
drivers/gpu/drm/i915/intel_ringbuffer.c +4 -0
index 1dec252..34fbe68 100644
@@ -32,6 +32,7 @@
 #include <drm/i915_drm.h>
 #include "i915_trace.h"
 #include "intel_drv.h"
+#include "i915_scheduler.h"
 
 bool
 intel_ring_initialized(struct intel_engine_cs *ring)
@@ -1411,6 +1412,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
 	u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];
 	int ret;
 
+	/* Arithmetic on sequence numbers is unreliable with a scheduler. */
+	BUG_ON(i915_scheduler_is_enabled(signaller->dev));
+
 	/* Throughout all of the GEM code, seqno passed implies our current
 	 * seqno is >= the last seqno executed. However for hardware the
 	 * comparison is strictly greater than.
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#16 lore raw

PATCH 10/39 drm/i915: Force MMIO flips when scheduler enabled

In reply to [email protected]

From: John Harrison <[email protected]>

Change-Id: Ice071af6d88306b0d1c53bdb651a1a3e20bdc1af
For: VIZ-1587

---
 drivers/gpu/drm/i915/intel_display.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/intel_display.c +3 -0
index 9291a1d..c00bc50 100644
@@ -44,6 +44,7 @@
 #include <drm/drm_plane_helper.h>
 #include <drm/drm_rect.h>
 #include <linux/dma_remapping.h>
+#include "i915_scheduler.h"
 
 /* Primary plane formats for gen <= 3 */
 static const uint32_t i8xx_primary_formats[] = {
@@ -11135,6 +11136,8 @@ static bool use_mmio_flip(struct intel_engine_cs *ring,
 		return true;
 	else if (i915.enable_execlists)
 		return true;
+	else if (i915_scheduler_is_enabled(ring->dev))
+		return true;
 	else
 		return ring != i915_gem_request_get_ring(obj->last_write_req);
 }
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#17 lore raw

PATCH 11/39 drm/i915: Added scheduler hook when closing DRM file handles

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to submit work, then close the DRM handle and free up all the resources that piece of work wishes to use before the work has even been submitted to the hardware. To prevent this, the scheduler needs to be informed of the DRM close event so that it can force through any outstanding work attributed to that file handle.

Change-Id: I24ac056c062b075ff1cc5e2ed2d3fa8e17e85951
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_dma.c       |  3 ++
 drivers/gpu/drm/i915/i915_scheduler.c | 66 +++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  2 ++
 3 files changed, 71 insertions(+)
drivers/gpu/drm/i915/i915_dma.c +3 -0
index 731cf31..c2f9c03 100644
@@ -46,6 +46,7 @@
 #include <linux/vga_switcheroo.h>
 #include <linux/slab.h>
 #include <acpi/video.h>
+#include "i915_scheduler.h"
 #include <linux/pm.h>
 #include <linux/pm_runtime.h>
 #include <linux/oom.h>
@@ -1250,6 +1251,8 @@ void i915_driver_lastclose(struct drm_device *dev)
 
 void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
 {
+	i915_scheduler_closefile(dev, file);
+
 	mutex_lock(&dev->struct_mutex);
 	i915_gem_context_close(dev, file);
 	i915_gem_release(dev, file);
drivers/gpu/drm/i915/i915_scheduler.c +66 -0

69 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 93faf40..9736b8d 100644
@@ -85,6 +85,8 @@ enum {
 
 bool        i915_scheduler_is_enabled(struct drm_device *dev);
 int         i915_scheduler_init(struct drm_device *dev);
+int         i915_scheduler_closefile(struct drm_device *dev,
+				     struct drm_file *file);
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw

PATCH 12/40 drm/i915: Added scheduler hook when closing DRM file handles

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with submission of batch buffers to the hardware. Thus it is possible for an application to close its DRM file handle while there is still work outstanding. That means the scheduler needs to know about file close events so it can remove the file pointer from such orphaned batch buffers and not attempt to dereference it later.

v3: Updated to not wait for outstanding work to complete but merely remove the file handle reference. The wait was getting excessively complicated with inter-ring dependencies, pre-emption, and other such issues.

Change-Id: I24ac056c062b075ff1cc5e2ed2d3fa8e17e85951
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_dma.c       |  3 +++
 drivers/gpu/drm/i915/i915_scheduler.c | 35 +++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  2 ++
 3 files changed, 40 insertions(+)
drivers/gpu/drm/i915/i915_dma.c +3 -0
index 731cf31..c2f9c03 100644
@@ -46,6 +46,7 @@
 #include <linux/vga_switcheroo.h>
 #include <linux/slab.h>
 #include <acpi/video.h>
+#include "i915_scheduler.h"
 #include <linux/pm.h>
 #include <linux/pm_runtime.h>
 #include <linux/oom.h>
@@ -1250,6 +1251,8 @@ void i915_driver_lastclose(struct drm_device *dev)
 
 void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
 {
+	i915_scheduler_closefile(dev, file);
+
 	mutex_lock(&dev->struct_mutex);
 	i915_gem_context_close(dev, file);
 	i915_gem_release(dev, file);
drivers/gpu/drm/i915/i915_scheduler.c +35 -0
index 344760e..5aafc96 100644
@@ -768,3 +768,38 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
 
 	return 0;
 }
+
+int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
+{
+	struct i915_scheduler_queue_entry  *node;
+	struct drm_i915_private            *dev_priv = dev->dev_private;
+	struct i915_scheduler              *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs  *ring;
+	int                     i;
+	unsigned long           flags;
+
+	if (!scheduler)
+		return 0;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	for_each_ring(ring, dev_priv, i) {
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (node->params.file != file)
+				continue;
+
+			if(!I915_SQS_IS_COMPLETE(node))
+				DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+						 node->params.request->uniq,
+						 node->params.request->seqno,
+						 node->status,
+						 ring->name);
+
+			node->params.file = NULL;
+		}
+	}
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return 0;
+}
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 2d50d83..02ac6f2 100644
@@ -86,6 +86,8 @@ enum {
 
 bool        i915_scheduler_is_enabled(struct drm_device *dev);
 int         i915_scheduler_init(struct drm_device *dev);
+int         i915_scheduler_closefile(struct drm_device *dev,
+				     struct drm_file *file);
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#19 lore raw

PATCH 12/39 drm/i915: Added scheduler hook into i915_gem_request_notify()

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler needs to know when requests have completed so that it can keep its own internal state up to date and can submit new requests to the hardware from its queue.

v2: Updated due to changes in request handling. The operation is now reversed from before. Rather than the scheduler being in control of completion events, it is now the request code itself. The scheduler merely receives a notification event. It can then optionally request it's worker thread be woken up after all completion processing is complete.

Change-Id: I149250a8f9382586514ca324aba1c53063b83e19
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h       |  2 ++
 drivers/gpu/drm/i915/i915_gem.c       | 16 ++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.c | 28 ++++++++++++++++++----------
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 4 files changed, 37 insertions(+), 10 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index 23aed32..8c576fe 100644
@@ -2262,6 +2262,8 @@ struct drm_i915_gem_request {
 	/** process identifier submitting this request */
 	struct pid *pid;
 
+	struct i915_scheduler_queue_entry	*scheduler_qe;
+
 	/**
 	 * The ELSP only accepts two elements at a time, so we queue
 	 * context/tail pairs on a given queue (ring->execlist_queue) until the
drivers/gpu/drm/i915/i915_gem.c +16 -0
index d38e013..8a2fc7c 100644
@@ -2785,6 +2785,7 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 {
 	struct drm_i915_gem_request *req, *req_next;
 	unsigned long flags;
+	bool wake_sched = false;
 	u32 seqno;
 
 	if (list_empty(&ring->fence_signal_list)) {
@@ -2813,6 +2814,14 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 		 */
 		list_del_init(&req->signal_link);
 
+		/*
+		 * NB: Must notify the scheduler before signalling
+		 * the node. Otherwise the node can get retired first
+		 * and call scheduler_clean() while the scheduler
+		 * thinks it is still active.
+		 */
+		wake_sched |= i915_scheduler_notify_request(req);
+
 		if (!req->cancelled) {
 			fence_signal_locked(&req->fence);
 			trace_i915_gem_request_complete(req);
@@ -2829,6 +2838,13 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 
 	if (!fence_locked)
 		spin_unlock_irqrestore(&ring->fence_lock, flags);
+
+	/* Necessary? Or does the fence_signal() call do an implicit wakeup? */
+	wake_up_all(&ring->irq_queue);
+
+	/* Final scheduler processing after all individual updates are done. */
+	if (wake_sched)
+		i915_scheduler_wakeup(ring->dev);
 }
 
 static const char *i915_gem_request_get_driver_name(struct fence *req_fence)
drivers/gpu/drm/i915/i915_scheduler.c +18 -10
index 25e7ade..dd9c710 100644
@@ -120,6 +120,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	node->stamp  = jiffies;
 	i915_gem_request_reference(node->params.request);
 
+	BUG_ON(node->params.request->scheduler_qe);
+	node->params.request->scheduler_qe = node;
+
 	/* Need to determine the number of incomplete entries in the list as
 	 * that will be the maximum size of the dependency list.
 	 *
@@ -319,14 +322,16 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
  * code has mapped it back to a request and will mark that request complete.
  * It also calls this function to notify the scheduler about the completion
  * so the scheduler's node can be updated appropriately.
- * Returns true if the request is scheduler managed, false if not.
+ * Returns true if the request is scheduler managed, false if not. The return
+ * value is combined for all freshly completed requests and if any were true
+ * then i915_scheduler_wakeup() is called so the scheduler can do further
+ * processing (submit more work) at the end.
  */
 bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 {
 	struct drm_i915_private *dev_priv  = to_i915(req->ring->dev);
 	struct i915_scheduler   *scheduler = dev_priv->scheduler;
-	/* XXX: Need to map back from request to node */
-	struct i915_scheduler_queue_entry *node = NULL;
+	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
 	unsigned long       flags;
 
 	if (!node)
@@ -344,16 +349,18 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
-	/*
-	 * XXX: If the in-flight list is now empty then new work should be
-	 * submitted. However, this function is called from interrupt context
-	 * and thus cannot acquire mutex locks and other such things that are
-	 * necessary for fresh submission.
-	 */
-
 	return true;
 }
 
+/*
+ * Called at the end of seqno interrupt processing if any request has
+ * completed that corresponds to a scheduler node.
+ */
+void i915_scheduler_wakeup(struct drm_device *dev)
+{
+	/* XXX: Need to call i915_scheduler_remove() via work handler. */
+}
+
 int i915_scheduler_remove(struct intel_engine_cs *ring)
 {
 	struct drm_i915_private *dev_priv = ring->dev->dev_private;
@@ -468,6 +475,7 @@ int i915_scheduler_remove(struct intel_engine_cs *ring)
 			i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
 
 		/* Free everything that is owned by the node: */
+		node->params.request->scheduler_qe = NULL;
 		i915_gem_request_unreference(node->params.request);
 		kfree(node->params.cliprects);
 		kfree(node->dep_list);
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 9736b8d..a0b38b8 100644
@@ -89,5 +89,6 @@ int         i915_scheduler_closefile(struct drm_device *dev,
 				     struct drm_file *file);
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
+void        i915_scheduler_wakeup(struct drm_device *dev);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#20 lore raw

PATCH 13/39 drm/i915: Added deferred work handler for scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler needs to do interrupt triggered work that is too complex to do in the interrupt handler. Thus it requires a deferred work handler to process such tasks asynchronously.

v2: Updated to reduce mutex lock usage. The lock is now only held for the minimum time within the remove function rather than for the whole of the worker thread's operation.

Change-Id: I0f7cc2b6f034a50bf8f7e368b60ad8bafd00f993
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_dma.c       |  3 +++
 drivers/gpu/drm/i915/i915_drv.h       | 10 ++++++++++
 drivers/gpu/drm/i915/i915_gem.c       |  2 ++
 drivers/gpu/drm/i915/i915_scheduler.c | 21 +++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 5 files changed, 35 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_dma.c +3 -0
index c2f9c03..8c9246f 100644
@@ -1149,6 +1149,9 @@ int i915_driver_unload(struct drm_device *dev)
 	WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
 	unregister_shrinker(&dev_priv->mm.shrinker);
 
+	/* Cancel the scheduler work handler, which should be idle now. */
+	cancel_work_sync(&dev_priv->mm.scheduler_work);
+
 	io_mapping_free(dev_priv->gtt.mappable);
 	arch_phys_wc_del(dev_priv->gtt.mtrr);
 
drivers/gpu/drm/i915/i915_drv.h +10 -0
index 8c576fe..9a67f7c 100644
@@ -1285,6 +1285,16 @@ struct i915_gem_mm {
 	struct delayed_work retire_work;
 
 	/**
+	 * New scheme is to get an interrupt after every work packet
+	 * in order to allow the low latency scheduling of pending
+	 * packets. The idea behind adding new packets to a pending
+	 * queue rather than directly into the hardware ring buffer
+	 * is to allow high priority packets to over take low priority
+	 * ones.
+	 */
+	struct work_struct scheduler_work;
+
+	/**
 	 * When we detect an idle GPU, we want to turn on
 	 * powersaving features. So once we see that there
 	 * are no more requests outstanding and no more
drivers/gpu/drm/i915/i915_gem.c +2 -0
index 8a2fc7c..f625d88 100644
@@ -5389,6 +5389,8 @@ i915_gem_load(struct drm_device *dev)
 			  i915_gem_retire_work_handler);
 	INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
 			  i915_gem_idle_work_handler);
+	INIT_WORK(&dev_priv->mm.scheduler_work,
+				i915_gem_scheduler_work_handler);
 	init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
 
 	dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
drivers/gpu/drm/i915/i915_scheduler.c +19 -2
index dd9c710..7521338 100644
@@ -358,10 +358,12 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
  */
 void i915_scheduler_wakeup(struct drm_device *dev)
 {
-	/* XXX: Need to call i915_scheduler_remove() via work handler. */
+	struct drm_i915_private *dev_priv  = to_i915(dev);
+
+	queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
 }
 
-int i915_scheduler_remove(struct intel_engine_cs *ring)
+static int i915_scheduler_remove(struct intel_engine_cs *ring)
 {
 	struct drm_i915_private *dev_priv = ring->dev->dev_private;
 	struct i915_scheduler   *scheduler = dev_priv->scheduler;
@@ -487,6 +489,21 @@ int i915_scheduler_remove(struct intel_engine_cs *ring)
 	return ret;
 }
 
+void i915_gem_scheduler_work_handler(struct work_struct *work)
+{
+	struct intel_engine_cs  *ring;
+	struct drm_i915_private *dev_priv;
+	struct drm_device       *dev;
+	int                     i;
+
+	dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
+	dev = dev_priv->dev;
+
+	for_each_ring(ring, dev_priv, i) {
+		i915_scheduler_remove(ring);
+	}
+}
+
 static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
 {
 	struct i915_scheduler_queue_entry *node;
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index a0b38b8..da095f9 100644
@@ -90,5 +90,6 @@ int         i915_scheduler_closefile(struct drm_device *dev,
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
+void        i915_gem_scheduler_work_handler(struct work_struct *work);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#21 lore raw

PATCH 14/39 drm/i915: Redirect execbuffer_final() via scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

Updated the execbuffer() code to pass the packaged up batch buffer information to the scheduler rather than calling execbuffer_final() directly. The scheduler queue() code is currently a stub which simply chains on to _final() immediately.

Change-Id: I2a19062a9e66845f2e886332fc4b5fc7ac992864
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++-----------
 drivers/gpu/drm/i915/intel_lrc.c           | 12 ++++--------
 2 files changed, 11 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -11
index 05c9de6..cd4aa64 100644
@@ -35,6 +35,7 @@
 #include <linux/dma_remapping.h>
 #include <linux/uaccess.h>
 #include <../drivers/android/sync.h>
+#include "i915_scheduler.h"
 
 #define  __EXEC_OBJECT_HAS_PIN (1<<31)
 #define  __EXEC_OBJECT_HAS_FENCE (1<<30)
@@ -1180,6 +1181,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas)
 {
+	struct i915_scheduler_queue_entry *qe;
 	struct drm_device *dev = params->dev;
 	struct intel_engine_cs *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -1224,17 +1226,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	ret = dev_priv->gt.execbuf_final(params);
+	qe = container_of(params, typeof(*qe), params);
+	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
 		return ret;
 
-	/*
-	 * Free everything that was stored in the QE structure (until the
-	 * scheduler arrives and does it instead):
-	 */
-	if (params->dispatch_flags & I915_DISPATCH_SECURE)
-		i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
 	return 0;
 }
 
@@ -1410,8 +1406,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	struct intel_engine_cs *ring;
 	struct intel_context *ctx;
 	struct i915_address_space *vm;
-	struct i915_execbuffer_params params_master; /* XXX: will be removed later */
-	struct i915_execbuffer_params *params = &params_master;
+	struct i915_scheduler_queue_entry qe;
+	struct i915_execbuffer_params *params = &qe.params;
 	const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
 	u32 dispatch_flags;
 	int ret;
@@ -1539,7 +1535,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	else
 		vm = &dev_priv->gtt.base;
 
-	memset(&params_master, 0x00, sizeof(params_master));
+	memset(&qe, 0x00, sizeof(qe));
 
 	eb = eb_create(args);
 	if (eb == NULL) {
drivers/gpu/drm/i915/intel_lrc.c +4 -8
index 4aae31c..2f49fb9 100644
@@ -136,6 +136,7 @@
 #include <drm/i915_drm.h>
 #include "i915_drv.h"
 #include "intel_mocs.h"
+#include "i915_scheduler.h"
 
 #define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
 #define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -866,6 +867,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas)
 {
+	struct i915_scheduler_queue_entry *qe;
 	struct drm_device       *dev = params->dev;
 	struct intel_engine_cs  *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -908,17 +910,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	ret = dev_priv->gt.execbuf_final(params);
+	qe = container_of(params, typeof(*qe), params);
+	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
 		return ret;
 
-	/*
-	 * Free everything that was stored in the QE structure (until the
-	 * scheduler arrives and does it instead):
-	 */
-	if (params->dispatch_flags & I915_DISPATCH_SECURE)
-		i915_gem_execbuff_release_batch_obj(params->batch_obj);
-
 	return 0;
 }
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#22 lore raw

PATCH 15/39 drm/i915: Keep the reserved space mechanism happy

In reply to [email protected]

From: John Harrison <[email protected]>

Ring space is reserved when constructing a request to ensure that the subsequent 'add_request()' call cannot fail due to waiting for space on a busy or broken GPU. However, the scheduler jumps in to the middle of the execbuffer process between request creation and request submission. Thus it needs to cancel the reserved space when the request is simply added to the scheduler's queue and not yet submitted. Similarly, it needs to re-reserve the space when it finally does want to send the batch buffer to the hardware.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  7 +++++++
 drivers/gpu/drm/i915/i915_scheduler.c      |  4 ++++
 drivers/gpu/drm/i915/intel_lrc.c           | 13 +++++++++++--
 3 files changed, 22 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -0
index cd4aa64..12b34d7 100644
@@ -1248,6 +1248,10 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_ring_reserve_space(params->request);
+	if (ret)
+		return ret;
+
 	intel_runtime_pm_get(dev_priv);
 
 	/*
@@ -1308,6 +1312,9 @@ error:
 	 */
 	intel_runtime_pm_put(dev_priv);
 
+	if (ret)
+		intel_ring_reserved_space_cancel(params->request->ringbuf);
+
 	return ret;
 }
 
drivers/gpu/drm/i915/i915_scheduler.c +4 -0
index 7521338..79d02d7 100644
@@ -91,6 +91,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	if (1/*i915.scheduler_override & i915_so_direct_submit*/) {
 		int ret;
 
+		intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
+
 		scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
 		ret = dev_priv->gt.execbuf_final(&qe->params);
 		scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
@@ -120,6 +122,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	node->stamp  = jiffies;
 	i915_gem_request_reference(node->params.request);
 
+	intel_ring_reserved_space_cancel(node->params.request->ringbuf);
+
 	BUG_ON(node->params.request->scheduler_qe);
 	node->params.request->scheduler_qe = node;
 
drivers/gpu/drm/i915/intel_lrc.c +11 -2
index 2f49fb9..16c5302 100644
@@ -932,13 +932,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_logical_ring_reserve_space(params->request);
+	if (ret)
+		return ret;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
 	ret = logical_ring_invalidate_all_caches(params->request);
 	if (ret)
-		return ret;
+		goto err;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
@@ -962,13 +966,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 
 	ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto err;
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;
+
+err:
+	intel_ring_reserved_space_cancel(params->request->ringbuf);
+
+	return ret;
 }
 
 void intel_execlists_retire_requests(struct intel_engine_cs *ring)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#23 lore raw

PATCH 16/40 drm/i915: Keep the reserved space mechanism happy

From: John Harrison <[email protected]>

Ring space is reserved when constructing a request to ensure that the subsequent 'add_request()' call cannot fail due to waiting for space on a busy or broken GPU. However, the scheduler jumps in to the middle of the execbuffer process between request creation and request submission. Thus it needs to cancel the reserved space when the request is simply added to the scheduler's queue and not yet submitted. Similarly, it needs to re-reserve the space when it finally does want to send the batch buffer to the hardware.

v3: Updated to use locally cached request pointer.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  7 +++++++
 drivers/gpu/drm/i915/i915_scheduler.c      |  4 ++++
 drivers/gpu/drm/i915/intel_lrc.c           | 13 +++++++++++--
 3 files changed, 22 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -0
index b5d618a..2c7a395 100644
@@ -1249,6 +1249,10 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_ring_reserve_space(req);
+	if (ret)
+		return ret;
+
 	intel_runtime_pm_get(dev_priv);
 
 	/*
@@ -1309,6 +1313,9 @@ error:
 	 */
 	intel_runtime_pm_put(dev_priv);
 
+	if (ret)
+		intel_ring_reserved_space_cancel(req->ringbuf);
+
 	return ret;
 }
 
drivers/gpu/drm/i915/i915_scheduler.c +4 -0
index 0e657cf..9d1475f 100644
@@ -145,6 +145,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	if (1/*i915.scheduler_override & i915_so_direct_submit*/) {
 		int ret;
 
+		intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
+
 		scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
 		ret = dev_priv->gt.execbuf_final(&qe->params);
 		scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
@@ -174,6 +176,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	node->stamp  = jiffies;
 	i915_gem_request_reference(node->params.request);
 
+	intel_ring_reserved_space_cancel(node->params.request->ringbuf);
+
 	BUG_ON(node->params.request->scheduler_qe);
 	node->params.request->scheduler_qe = node;
 
drivers/gpu/drm/i915/intel_lrc.c +11 -2
index f14d9b2..ebc951e 100644
@@ -934,13 +934,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	ret = intel_logical_ring_reserve_space(req);
+	if (ret)
+		return ret;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
 	ret = logical_ring_invalidate_all_caches(req);
 	if (ret)
-		return ret;
+		goto err;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
@@ -962,13 +966,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 
 	ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto err;
 
 	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;
+
+err:
+	intel_ring_reserved_space_cancel(params->request->ringbuf);
+
+	return ret;
 }
 
 void intel_execlists_retire_requests(struct intel_engine_cs *ring)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#24 lore raw

PATCH 16/39 drm/i915: Added tracking/locking of batch buffer objects

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler needs to track interdependencies between batch buffers. These are calculated by analysing the object lists of the buffers and looking for commonality. The scheduler also needs to keep those buffers locked long after the initial IOCTL call has returned to user land.

Change-Id: I31e3677ecfc2c9b5a908bda6acc4850432d55f1e
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 48 ++++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.c      | 33 ++++++++++++++++++--
 2 files changed, 76 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +46 -2
index 12b34d7..fe90571 100644
@@ -1417,7 +1417,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	struct i915_execbuffer_params *params = &qe.params;
 	const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
 	u32 dispatch_flags;
-	int ret;
+	int ret, i;
 	bool need_relocs;
 	int fd_fence_complete = -1;
 	int fd_fence_wait = lower_32_bits(args->rsvd2);
@@ -1552,6 +1552,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 		goto pre_mutex_err;
 	}
 
+	qe.saved_objects = kzalloc(
+			sizeof(*qe.saved_objects) * args->buffer_count,
+			GFP_KERNEL);
+	if (!qe.saved_objects) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
 	/* Look up object handles */
 	ret = eb_lookup_vmas(eb, exec, args, vm, file);
 	if (ret)
@@ -1672,7 +1680,26 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	params->args_DR1                = args->DR1;
 	params->args_DR4                = args->DR4;
 	params->batch_obj               = batch_obj;
-	params->ctx                     = ctx;
+
+	/*
+	 * Save away the list of objects used by this batch buffer for the
+	 * purpose of tracking inter-buffer dependencies.
+	 */
+	for (i = 0; i < args->buffer_count; i++) {
+		/*
+		 * NB: 'drm_gem_object_lookup()' increments the object's
+		 * reference count and so must be matched by a
+		 * 'drm_gem_object_unreference' call.
+		 */
+		qe.saved_objects[i].obj =
+			to_intel_bo(drm_gem_object_lookup(dev, file,
+							  exec[i].handle));
+	}
+	qe.num_objs = i;
+
+	/* Lock and save the context object as well. */
+	i915_gem_context_reference(ctx);
+	params->ctx = ctx;
 
 	if (args->flags & I915_EXEC_CREATE_FENCE) {
 		/*
@@ -1737,6 +1764,23 @@ err:
 	i915_gem_context_unreference(ctx);
 	eb_destroy(eb);
 
+	if (qe.saved_objects) {
+		/* Need to release the objects: */
+		for (i = 0; i < qe.num_objs; i++) {
+			if (!qe.saved_objects[i].obj)
+				continue;
+
+			drm_gem_object_unreference(
+					&qe.saved_objects[i].obj->base);
+		}
+
+		kfree(qe.saved_objects);
+
+		/* Context too */
+		if (params->ctx)
+			i915_gem_context_unreference(params->ctx);
+	}
+
 	/*
 	 * If the request was created but not successfully submitted then it
 	 * must be freed again. If it was submitted then it is being tracked
drivers/gpu/drm/i915/i915_scheduler.c +30 -3
index 79d02d7..cd69f53 100644
@@ -104,7 +104,23 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 		if (ret)
 			return ret;
 
-		/* Free everything that is owned by the QE structure: */
+		/* Need to release the objects: */
+		for (i = 0; i < qe->num_objs; i++) {
+			if (!qe->saved_objects[i].obj)
+				continue;
+
+			drm_gem_object_unreference(&qe->saved_objects[i].obj->base);
+		}
+
+		kfree(qe->saved_objects);
+		qe->saved_objects = NULL;
+		qe->num_objs = 0;
+
+		/* Free the context object too: */
+		if (qe->params.ctx)
+			i915_gem_context_unreference(qe->params.ctx);
+
+		/* And anything else owned by the QE structure: */
 		kfree(qe->params.cliprects);
 		if (qe->params.dispatch_flags & I915_DISPATCH_SECURE)
 			i915_gem_execbuff_release_batch_obj(qe->params.batch_obj);
@@ -376,7 +392,7 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 	int                 flying = 0, queued = 0;
 	int                 ret = 0;
 	bool                do_submit;
-	uint32_t            min_seqno;
+	uint32_t            i, min_seqno;
 	struct list_head    remove;
 
 	if (list_empty(&scheduler->node_queue[ring->id]))
@@ -480,7 +496,18 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 		if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
 			i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
 
-		/* Free everything that is owned by the node: */
+		/* Release the locked buffers: */
+		for (i = 0; i < node->num_objs; i++) {
+			drm_gem_object_unreference(
+					    &node->saved_objects[i].obj->base);
+		}
+		kfree(node->saved_objects);
+
+		/* Context too: */
+		if (node->params.ctx)
+			i915_gem_context_unreference(node->params.ctx);
+
+		/* And anything else owned by the node: */
 		node->params.request->scheduler_qe = NULL;
 		i915_gem_request_unreference(node->params.request);
 		kfree(node->params.cliprects);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#25 lore raw

PATCH 17/40 drm/i915: Added tracking/locking of batch buffer objects

From: John Harrison <[email protected]>

The scheduler needs to track interdependencies between batch buffers. These are calculated by analysing the object lists of the buffers and looking for commonality. The scheduler also needs to keep those buffers locked long after the initial IOCTL call has returned to user land.

v3: Updated to support read-read optimisation.

Change-Id: I31e3677ecfc2c9b5a908bda6acc4850432d55f1e
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 52 ++++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.c      | 33 +++++++++++++++++--
 2 files changed, 80 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +50 -2
index 2c7a395..0908699 100644
@@ -1418,7 +1418,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	struct i915_execbuffer_params *params = &qe.params;
 	const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
 	u32 dispatch_flags;
-	int ret;
+	int ret, i;
 	bool need_relocs;
 	int fd_fence_complete = -1;
 	int fd_fence_wait = lower_32_bits(args->rsvd2);
@@ -1553,6 +1553,14 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 		goto pre_mutex_err;
 	}
 
+	qe.saved_objects = kzalloc(
+			sizeof(*qe.saved_objects) * args->buffer_count,
+			GFP_KERNEL);
+	if (!qe.saved_objects) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
 	/* Look up object handles */
 	ret = eb_lookup_vmas(eb, exec, args, vm, file);
 	if (ret)
@@ -1673,7 +1681,30 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	params->args_DR1                = args->DR1;
 	params->args_DR4                = args->DR4;
 	params->batch_obj               = batch_obj;
-	params->ctx                     = ctx;
+
+	/*
+	 * Save away the list of objects used by this batch buffer for the
+	 * purpose of tracking inter-buffer dependencies.
+	 */
+	for (i = 0; i < args->buffer_count; i++) {
+		struct drm_i915_gem_object *obj;
+
+		/*
+		 * NB: 'drm_gem_object_lookup()' increments the object's
+		 * reference count and so must be matched by a
+		 * 'drm_gem_object_unreference' call.
+		 */
+		obj = to_intel_bo(drm_gem_object_lookup(dev, file,
+							  exec[i].handle));
+		qe.saved_objects[i].obj       = obj;
+		qe.saved_objects[i].read_only = obj->base.pending_write_domain == 0;
+
+	}
+	qe.num_objs = i;
+
+	/* Lock and save the context object as well. */
+	i915_gem_context_reference(ctx);
+	params->ctx = ctx;
 
 	if (args->flags & I915_EXEC_CREATE_FENCE) {
 		/*
@@ -1738,6 +1769,23 @@ err:
 	i915_gem_context_unreference(ctx);
 	eb_destroy(eb);
 
+	if (qe.saved_objects) {
+		/* Need to release the objects: */
+		for (i = 0; i < qe.num_objs; i++) {
+			if (!qe.saved_objects[i].obj)
+				continue;
+
+			drm_gem_object_unreference(
+					&qe.saved_objects[i].obj->base);
+		}
+
+		kfree(qe.saved_objects);
+
+		/* Context too */
+		if (params->ctx)
+			i915_gem_context_unreference(params->ctx);
+	}
+
 	/*
 	 * If the request was created but not successfully submitted then it
 	 * must be freed again. If it was submitted then it is being tracked
drivers/gpu/drm/i915/i915_scheduler.c +30 -3
index 9d1475f..300cd89 100644
@@ -158,7 +158,23 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 		if (ret)
 			return ret;
 
-		/* Free everything that is owned by the QE structure: */
+		/* Need to release the objects: */
+		for (i = 0; i < qe->num_objs; i++) {
+			if (!qe->saved_objects[i].obj)
+				continue;
+
+			drm_gem_object_unreference(&qe->saved_objects[i].obj->base);
+		}
+
+		kfree(qe->saved_objects);
+		qe->saved_objects = NULL;
+		qe->num_objs = 0;
+
+		/* Free the context object too: */
+		if (qe->params.ctx)
+			i915_gem_context_unreference(qe->params.ctx);
+
+		/* And anything else owned by the QE structure: */
 		kfree(qe->params.cliprects);
 		if (qe->params.dispatch_flags & I915_DISPATCH_SECURE)
 			i915_gem_execbuff_release_batch_obj(qe->params.batch_obj);
@@ -399,7 +415,7 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 	int                 flying = 0, queued = 0;
 	int                 ret = 0;
 	bool                do_submit;
-	uint32_t            min_seqno;
+	uint32_t            i, min_seqno;
 	struct list_head    remove;
 
 	if (list_empty(&scheduler->node_queue[ring->id]))
@@ -503,7 +519,18 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 		if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
 			i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
 
-		/* Free everything that is owned by the node: */
+		/* Release the locked buffers: */
+		for (i = 0; i < node->num_objs; i++) {
+			drm_gem_object_unreference(
+					    &node->saved_objects[i].obj->base);
+		}
+		kfree(node->saved_objects);
+
+		/* Context too: */
+		if (node->params.ctx)
+			i915_gem_context_unreference(node->params.ctx);
+
+		/* And anything else owned by the node: */
 		node->params.request->scheduler_qe = NULL;
 		i915_gem_request_unreference(node->params.request);
 		kfree(node->params.cliprects);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#26 lore raw

PATCH 17/39 drm/i915: Hook scheduler node clean up into retire requests

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler keeps its own lock on various DRM objects in order to guarantee safe access long after the original execbuff IOCTL has completed. This is especially important when pre-emption is enabled as the batch buffer might need to be submitted to the hardware multiple times. This patch hooks the clean up of these locks into the request retire function. The request can only be retired after it has completed on the hardware and thus is no longer eligible for re-submission. Thus there is no point holding on to the locks beyond that time.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c       |  3 +++
 drivers/gpu/drm/i915/i915_scheduler.c | 51 ++++++++++++++++++++++++-----------
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 3 files changed, 39 insertions(+), 16 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +3 -0
index f625d88..451ae6d 100644
@@ -1402,6 +1402,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
 		fence_signal_locked(&request->fence);
 	}
 
+	if (request->scheduler_qe)
+		i915_gem_scheduler_clean_node(request->scheduler_qe);
+
 	i915_gem_request_unreference(request);
 }
 
drivers/gpu/drm/i915/i915_scheduler.c +35 -16

69 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index da095f9..8469270 100644
@@ -87,6 +87,7 @@ bool        i915_scheduler_is_enabled(struct drm_device *dev);
 int         i915_scheduler_init(struct drm_device *dev);
 int         i915_scheduler_closefile(struct drm_device *dev,
 				     struct drm_file *file);
+void        i915_gem_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#27 lore raw

PATCH 18/40 drm/i915: Hook scheduler node clean up into retire requests

From: John Harrison <[email protected]>

The scheduler keeps its own lock on various DRM objects in order to guarantee safe access long after the original execbuff IOCTL has completed. This is especially important when pre-emption is enabled as the batch buffer might need to be submitted to the hardware multiple times. This patch hooks the clean up of these locks into the request retire function. The request can only be retired after it has completed on the hardware and thus is no longer eligible for re-submission. Thus there is no point holding on to the locks beyond that time.

v3: Updated to not WARN when cleaning a node that is being cancelled. The clean will happen later so skipping it at the point of cancellation is fine.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c       |  3 ++
 drivers/gpu/drm/i915/i915_scheduler.c | 54 ++++++++++++++++++++++++-----------
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 3 files changed, 42 insertions(+), 16 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +3 -0
index dc5f3fe..349ff58 100644
@@ -1402,6 +1402,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
 		fence_signal_locked(&request->fence);
 	}
 
+	if (request->scheduler_qe)
+		i915_gem_scheduler_clean_node(request->scheduler_qe);
+
 	i915_gem_request_unreference(request);
 }
 
drivers/gpu/drm/i915/i915_scheduler.c +38 -16

72 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 56f68e5..54d87fb 100644
@@ -88,6 +88,7 @@ bool        i915_scheduler_is_enabled(struct drm_device *dev);
 int         i915_scheduler_init(struct drm_device *dev);
 int         i915_scheduler_closefile(struct drm_device *dev,
 				     struct drm_file *file);
+void        i915_gem_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#28 lore raw

PATCH 18/39 drm/i915: Added scheduler support to __wait_request() calls

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.

This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of __wait_request() release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.

This change hooks the scheduler in to the __wait_request() code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.

Change-Id: I31fe6bc7e38f6ffdd843fcae16e7cc8b1e52a931
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h         |   3 +-
 drivers/gpu/drm/i915/i915_gem.c         |  37 +++++++++--
 drivers/gpu/drm/i915/i915_scheduler.c   | 111 ++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h   |   4 ++
 drivers/gpu/drm/i915/intel_display.c    |   5 +-
 drivers/gpu/drm/i915/intel_ringbuffer.c |   2 +-
 6 files changed, 151 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +2 -1
index 9a67f7c..5ed600c 100644
@@ -3029,7 +3029,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			unsigned reset_counter,
 			bool interruptible,
 			s64 *timeout,
-			struct intel_rps_client *rps);
+			struct intel_rps_client *rps,
+			bool is_locked);
 int __must_check i915_wait_request(struct drm_i915_gem_request *req);
 int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
 int __must_check
drivers/gpu/drm/i915/i915_gem.c +30 -7
index 451ae6d..3994b47 100644
@@ -1207,7 +1207,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			unsigned reset_counter,
 			bool interruptible,
 			s64 *timeout,
-			struct intel_rps_client *rps)
+			struct intel_rps_client *rps,
+			bool is_locked)
 {
 	struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
 	struct drm_device *dev = ring->dev;
@@ -1217,8 +1218,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 	DEFINE_WAIT(wait);
 	unsigned long timeout_expire;
 	s64 before, now;
-	int ret;
+	int ret = 0;
+	bool    busy;
 
+	might_sleep();
 	WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
 
 	if (i915_gem_request_completed(req))
@@ -1269,6 +1272,22 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			break;
 		}
 
+		if (is_locked) {
+			/* If this request is being processed by the scheduler
+			 * then it is unsafe to sleep with the mutex lock held
+			 * as the scheduler may require the lock in order to
+			 * progress the request. */
+			if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
+				if (busy) {
+					ret = -EAGAIN;
+					break;
+				}
+			}
+
+			/* If the request is not tracked by the scheduler then the
+			 * regular test can be done. */
+		}
+
 		if (i915_gem_request_completed(req)) {
 			ret = 0;
 			break;
@@ -1449,13 +1468,17 @@ i915_wait_request(struct drm_i915_gem_request *req)
 
 	BUG_ON(!mutex_is_locked(&dev->struct_mutex));
 
+	ret = i915_scheduler_flush_request(req, true);
+	if (ret < 0)
+		return ret;
+
 	ret = i915_gem_check_wedge(&dev_priv->gpu_error, interruptible);
 	if (ret)
 		return ret;
 
 	ret = __i915_wait_request(req,
 				  atomic_read(&dev_priv->gpu_error.reset_counter),
-				  interruptible, NULL, NULL);
+				  interruptible, NULL, NULL, true);
 	if (ret)
 		return ret;
 
@@ -1568,7 +1591,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
 	mutex_unlock(&dev->struct_mutex);
 	for (i = 0; ret == 0 && i < n; i++)
 		ret = __i915_wait_request(requests[i], reset_counter, true,
-					  NULL, rps);
+					  NULL, rps, false);
 	mutex_lock(&dev->struct_mutex);
 
 	for (i = 0; i < n; i++) {
@@ -3493,7 +3516,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		if (ret == 0)
 			ret = __i915_wait_request(req[i], reset_counter, true,
 						  args->timeout_ns > 0 ? &args->timeout_ns : NULL,
-						  file->driver_priv);
+						  file->driver_priv, false);
 		i915_gem_request_unreference(req[i]);
 	}
 	return ret;
@@ -3526,7 +3549,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
 					  atomic_read(&i915->gpu_error.reset_counter),
 					  i915->mm.interruptible,
 					  NULL,
-					  &i915->rps.semaphores);
+					  &i915->rps.semaphores, true);
 		if (ret)
 			return ret;
 
@@ -4485,7 +4508,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	if (target == NULL)
 		return 0;
 
-	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);
 	if (ret == 0)
 		queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
 
drivers/gpu/drm/i915/i915_scheduler.c +111 -0

135 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +4 -0
index 8469270..42371b3 100644
@@ -92,5 +92,9 @@ int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *q
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
 void        i915_gem_scheduler_work_handler(struct work_struct *work);
+int         i915_scheduler_flush_request(struct drm_i915_gem_request *req,
+					 bool is_locked);
+bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+					      bool *completed, bool *busy);
 
 #endif  /* _I915_SCHEDULER_H_ */
drivers/gpu/drm/i915/intel_display.c +3 -2
index c00bc50..58c464f 100644
@@ -11258,7 +11258,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
 		WARN_ON(__i915_wait_request(mmio_flip->req,
 					    mmio_flip->crtc->reset_counter,
 					    false, NULL,
-					    &mmio_flip->i915->rps.mmioflips));
+					    &mmio_flip->i915->rps.mmioflips,
+					    false));
 		i915_gem_request_unreference(mmio_flip->req);
 	}
 
@@ -13258,7 +13259,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
 
 			ret = __i915_wait_request(intel_plane_state->wait_req,
 						  reset_counter, true,
-						  NULL, NULL);
+						  NULL, NULL, false);
 
 			/* Swallow -EIO errors to allow updates during hw lockup. */
 			if (ret == -EIO)
drivers/gpu/drm/i915/intel_ringbuffer.c +1 -1
index 34fbe68..4d77886 100644
@@ -2294,7 +2294,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
 	return __i915_wait_request(req,
 				   atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
 				   to_i915(ring->dev)->mm.interruptible,
-				   NULL, NULL);
+				   NULL, NULL, true);
 }
 
 int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#29 lore raw

PATCH 19/40 drm/i915: Added scheduler support to __wait_request() calls

From: John Harrison <[email protected]>

The scheduler can cause batch buffers, and hence requests, to be submitted to the ring out of order and asynchronously to their submission to the driver. Thus at the point of waiting for the completion of a given request, it is not even guaranteed that the request has actually been sent to the hardware yet. Even it is has been sent, it is possible that it could be pre-empted and thus 'unsent'.

This means that it is necessary to be able to submit requests to the hardware during the wait call itself. Unfortunately, while some callers of __wait_request() release the mutex lock first, others do not (and apparently can not). Hence there is the ability to deadlock as the wait stalls for submission but the asynchronous submission is stalled for the mutex lock.

This change hooks the scheduler in to the __wait_request() code to ensure correct behaviour. That is, flush the target batch buffer through to the hardware and do not deadlock waiting for something that cannot currently be submitted. Instead, the wait call must return EAGAIN at least as far back as necessary to release the mutex lock and allow the scheduler's asynchronous processing to get in and handle the pre-emption operation and eventually (re-)submit the work.

v3: Removed the explicit scheduler flush from i915_wait_request(). This is no longer necessary and was causing unintended changes to the scheduler priority level which broke a validation team test.

Change-Id: I31fe6bc7e38f6ffdd843fcae16e7cc8b1e52a931
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h         |  3 ++-
 drivers/gpu/drm/i915/i915_gem.c         | 33 ++++++++++++++++++++++++++-------
 drivers/gpu/drm/i915/i915_scheduler.c   | 20 ++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h   |  2 ++
 drivers/gpu/drm/i915/intel_display.c    |  5 +++--
 drivers/gpu/drm/i915/intel_ringbuffer.c |  2 +-
 6 files changed, 54 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +2 -1
index 9a67f7c..5ed600c 100644
@@ -3029,7 +3029,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			unsigned reset_counter,
 			bool interruptible,
 			s64 *timeout,
-			struct intel_rps_client *rps);
+			struct intel_rps_client *rps,
+			bool is_locked);
 int __must_check i915_wait_request(struct drm_i915_gem_request *req);
 int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
 int __must_check
drivers/gpu/drm/i915/i915_gem.c +26 -7
index 349ff58..784000b 100644
@@ -1207,7 +1207,8 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			unsigned reset_counter,
 			bool interruptible,
 			s64 *timeout,
-			struct intel_rps_client *rps)
+			struct intel_rps_client *rps,
+			bool is_locked)
 {
 	struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
 	struct drm_device *dev = ring->dev;
@@ -1217,8 +1218,10 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 	DEFINE_WAIT(wait);
 	unsigned long timeout_expire;
 	s64 before, now;
-	int ret;
+	int ret = 0;
+	bool    busy;
 
+	might_sleep();
 	WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
 
 	if (i915_gem_request_completed(req))
@@ -1269,6 +1272,22 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
 			break;
 		}
 
+		if (is_locked) {
+			/* If this request is being processed by the scheduler
+			 * then it is unsafe to sleep with the mutex lock held
+			 * as the scheduler may require the lock in order to
+			 * progress the request. */
+			if (i915_scheduler_is_request_tracked(req, NULL, &busy)) {
+				if (busy) {
+					ret = -EAGAIN;
+					break;
+				}
+			}
+
+			/* If the request is not tracked by the scheduler then the
+			 * regular test can be done. */
+		}
+
 		if (i915_gem_request_completed(req)) {
 			ret = 0;
 			break;
@@ -1455,7 +1474,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
 
 	ret = __i915_wait_request(req,
 				  atomic_read(&dev_priv->gpu_error.reset_counter),
-				  interruptible, NULL, NULL);
+				  interruptible, NULL, NULL, true);
 	if (ret)
 		return ret;
 
@@ -1568,7 +1587,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
 	mutex_unlock(&dev->struct_mutex);
 	for (i = 0; ret == 0 && i < n; i++)
 		ret = __i915_wait_request(requests[i], reset_counter, true,
-					  NULL, rps);
+					  NULL, rps, false);
 	mutex_lock(&dev->struct_mutex);
 
 	for (i = 0; i < n; i++) {
@@ -3494,7 +3513,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		if (ret == 0)
 			ret = __i915_wait_request(req[i], reset_counter, true,
 						  args->timeout_ns > 0 ? &args->timeout_ns : NULL,
-						  file->driver_priv);
+						  file->driver_priv, false);
 		i915_gem_request_unreference(req[i]);
 	}
 	return ret;
@@ -3527,7 +3546,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
 					  atomic_read(&i915->gpu_error.reset_counter),
 					  i915->mm.interruptible,
 					  NULL,
-					  &i915->rps.semaphores);
+					  &i915->rps.semaphores, true);
 		if (ret)
 			return ret;
 
@@ -4486,7 +4505,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	if (target == NULL)
 		return 0;
 
-	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+	ret = __i915_wait_request(target, reset_counter, true, NULL, NULL, false);
 	if (ret == 0)
 		queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
 
drivers/gpu/drm/i915/i915_scheduler.c +20 -0
index f88c871..386f157 100644
@@ -847,6 +847,26 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
 	return 0;
 }
 
+bool i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+				       bool *completed, bool *busy)
+{
+	struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	if (!scheduler)
+		return false;
+
+	if (req->scheduler_qe == NULL)
+		return false;
+
+	if (completed)
+		*completed = I915_SQS_IS_COMPLETE(req->scheduler_qe);
+	if (busy)
+		*busy      = I915_SQS_IS_QUEUED(req->scheduler_qe);
+
+	return true;
+}
+
 int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 {
 	struct i915_scheduler_queue_entry  *node;
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index 54d87fb..c3e7ac6 100644
@@ -93,5 +93,7 @@ int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *q
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
 void        i915_gem_scheduler_work_handler(struct work_struct *work);
+bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
+					      bool *completed, bool *busy);
 
 #endif  /* _I915_SCHEDULER_H_ */
drivers/gpu/drm/i915/intel_display.c +3 -2
index c00bc50..58c464f 100644
@@ -11258,7 +11258,8 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
 		WARN_ON(__i915_wait_request(mmio_flip->req,
 					    mmio_flip->crtc->reset_counter,
 					    false, NULL,
-					    &mmio_flip->i915->rps.mmioflips));
+					    &mmio_flip->i915->rps.mmioflips,
+					    false));
 		i915_gem_request_unreference(mmio_flip->req);
 	}
 
@@ -13258,7 +13259,7 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
 
 			ret = __i915_wait_request(intel_plane_state->wait_req,
 						  reset_counter, true,
-						  NULL, NULL);
+						  NULL, NULL, false);
 
 			/* Swallow -EIO errors to allow updates during hw lockup. */
 			if (ret == -EIO)
drivers/gpu/drm/i915/intel_ringbuffer.c +1 -1
index 34fbe68..4d77886 100644
@@ -2294,7 +2294,7 @@ int intel_ring_idle(struct intel_engine_cs *ring)
 	return __i915_wait_request(req,
 				   atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
 				   to_i915(ring->dev)->mm.interruptible,
-				   NULL, NULL);
+				   NULL, NULL, true);
 }
 
 int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#30 lore raw

PATCH 19/39 drm/i915: Added scheduler support to page fault handler

In reply to [email protected]

From: John Harrison <[email protected]>

GPU page faults can now require scheduler operation in order to complete. For example, in order to free up sufficient memory to handle the fault the handler must wait for a batch buffer to complete that has not even been sent to the hardware yet. Thus EAGAIN no longer means a GPU hang, it can occur under normal operation.

Change-Id: Iff6bd2744ef12bb7405fbcd6b43c286caad4141f
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +7 -2
index 3994b47..067f213 100644
@@ -1916,10 +1916,15 @@ out:
 		}
 	case -EAGAIN:
 		/*
-		 * EAGAIN means the gpu is hung and we'll wait for the error
-		 * handler to reset everything when re-faulting in
+		 * EAGAIN can mean the gpu is hung and we'll have to wait for
+		 * the error handler to reset everything when re-faulting in
 		 * i915_mutex_lock_interruptible.
+		 *
+		 * It can also indicate various other nonfatal errors for which
+		 * the best response is to give other threads a chance to run,
+		 * and then retry the failing operation in its entirety.
 		 */
+		/*FALLTHRU*/
 	case 0:
 	case -ERESTARTSYS:
 	case -EINTR:
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#31 lore raw

PATCH 20/39 drm/i915: Added scheduler flush calls to ring throttle and idle functions

In reply to [email protected]

From: John Harrison <[email protected]>

When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.

v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.

Change-Id: I95dcc2a2ee5c1a844748621c333994ddd6cf6a66
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c       | 24 ++++++++-
 drivers/gpu/drm/i915/i915_scheduler.c | 92 +++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  3 ++
 3 files changed, 118 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +23 -1
index 067f213..16bd82c 100644
@@ -3768,6 +3768,10 @@ int i915_gpu_idle(struct drm_device *dev)
 
 	/* Flush everything onto the inactive list. */
 	for_each_ring(ring, dev_priv, i) {
+		ret = i915_scheduler_flush(ring, true);
+		if (ret < 0)
+			return ret;
+
 		if (!i915.enable_execlists) {
 			struct drm_i915_gem_request *req;
 
@@ -4481,7 +4485,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
 	struct drm_i915_gem_request *request, *target = NULL;
 	unsigned reset_counter;
-	int ret;
+	int i, ret;
+	struct intel_engine_cs *ring;
 
 	ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
 	if (ret)
@@ -4491,6 +4496,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	if (ret)
 		return ret;
 
+	for_each_ring(ring, dev_priv, i) {
+		/*
+		 * Flush out scheduler entries that are getting 'stale'. Note
+		 * that the following recent_enough test will only check
+		 * against the time at which the request was submitted to the
+		 * hardware (i.e. when it left the scheduler) not the time it
+		 * was submitted to the driver.
+		 *
+		 * Also, there is not much point worring about busy return
+		 * codes from the scheduler flush call. Even if more work
+		 * cannot be submitted right now for whatever reason, we
+		 * still want to throttle against stale work that has already
+		 * been submitted.
+		 */
+		i915_scheduler_flush_stamp(ring, recent_enough, false);
+	}
+
 	spin_lock(&file_priv->mm.lock);
 	list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
 		if (time_after_eq(request->emitted_jiffies, recent_enough))
drivers/gpu/drm/i915/i915_scheduler.c +92 -0

98 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +3 -0
index 42371b3..29f7248 100644
@@ -92,6 +92,9 @@ int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *q
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
 void        i915_gem_scheduler_work_handler(struct work_struct *work);
+int         i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
+int         i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+				       unsigned long stamp, bool is_locked);
 int         i915_scheduler_flush_request(struct drm_i915_gem_request *req,
 					 bool is_locked);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#32 lore raw

PATCH 21/40 drm/i915: Added scheduler flush calls to ring throttle and idle functions

From: John Harrison <[email protected]>

When requesting that all GPU work is completed, it is now necessary to get the scheduler involved in order to flush out work that queued and not yet submitted.

v2: Updated to add support for flushing the scheduler queue by time stamp rather than just doing a blanket flush.

v3: Moved submit_max_priority() to this patch from an earlier patch is it is no longer required in the other.

Change-Id: I95dcc2a2ee5c1a844748621c333994ddd6cf6a66
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c       |  24 ++++++-
 drivers/gpu/drm/i915/i915_scheduler.c | 132 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |   3 +
 3 files changed, 158 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +23 -1
index 1a05c97..541ed9a 100644
@@ -3765,6 +3765,10 @@ int i915_gpu_idle(struct drm_device *dev)
 
 	/* Flush everything onto the inactive list. */
 	for_each_ring(ring, dev_priv, i) {
+		ret = i915_scheduler_flush(ring, true);
+		if (ret < 0)
+			return ret;
+
 		if (!i915.enable_execlists) {
 			struct drm_i915_gem_request *req;
 
@@ -4478,7 +4482,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
 	struct drm_i915_gem_request *request, *target = NULL;
 	unsigned reset_counter;
-	int ret;
+	int i, ret;
+	struct intel_engine_cs *ring;
 
 	ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
 	if (ret)
@@ -4488,6 +4493,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
 	if (ret)
 		return ret;
 
+	for_each_ring(ring, dev_priv, i) {
+		/*
+		 * Flush out scheduler entries that are getting 'stale'. Note
+		 * that the following recent_enough test will only check
+		 * against the time at which the request was submitted to the
+		 * hardware (i.e. when it left the scheduler) not the time it
+		 * was submitted to the driver.
+		 *
+		 * Also, there is not much point worring about busy return
+		 * codes from the scheduler flush call. Even if more work
+		 * cannot be submitted right now for whatever reason, we
+		 * still want to throttle against stale work that has already
+		 * been submitted.
+		 */
+		i915_scheduler_flush_stamp(ring, recent_enough, false);
+	}
+
 	spin_lock(&file_priv->mm.lock);
 	list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
 		if (time_after_eq(request->emitted_jiffies, recent_enough))
drivers/gpu/drm/i915/i915_scheduler.c +132 -0

150 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +3 -0
index c3e7ac6..7654013 100644
@@ -93,6 +93,9 @@ int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *q
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
 void        i915_gem_scheduler_work_handler(struct work_struct *work);
+int         i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
+int         i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
+				       unsigned long stamp, bool is_locked);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 					      bool *completed, bool *busy);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#33 lore raw

PATCH 21/39 drm/i915: Added a module parameter for allowing scheduler overrides

In reply to [email protected]

From: John Harrison <[email protected]>

It can be useful to be able to disable certain features (e.g. the entire scheduler) via a module parameter for debugging purposes. A parameter has the advantage of not being a compile time switch but without implying that it can be changed dynamically at runtime.

Change-Id: I92f4c832be88f5b34b49b90d6a9903fac68f7004
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h       | 1 +
 drivers/gpu/drm/i915/i915_params.c    | 4 ++++
 drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++-
 drivers/gpu/drm/i915/i915_scheduler.h | 5 +++++
 4 files changed, 14 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 5ed600c..5b893a6 100644
@@ -2717,6 +2717,7 @@ struct i915_params {
 	bool verbose_state_checks;
 	bool nuclear_pageflip;
 	int edp_vswing;
+	int scheduler_override;
 };
 extern struct i915_params i915 __read_mostly;
 
drivers/gpu/drm/i915/i915_params.c +4 -0
index ba7274a..be2e16c 100644
@@ -54,6 +54,7 @@ struct i915_params i915 __read_mostly = {
 	.edp_vswing = 0,
 	.enable_guc_submission = true,
 	.guc_log_level = -1,
+	.scheduler_override = 1,
 };
 
 module_param_named(modeset, i915.modeset, int, 0400);
@@ -191,3 +192,6 @@ MODULE_PARM_DESC(enable_guc_submission, "Enable GuC submission (default:false)")
 module_param_named(guc_log_level, i915.guc_log_level, int, 0400);
 MODULE_PARM_DESC(guc_log_level,
 	"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
+
+module_param_named(scheduler_override, i915.scheduler_override, int, 0600);
+MODULE_PARM_DESC(scheduler_override, "Scheduler override mask (0 = none, 1 = direct submission [default])");
drivers/gpu/drm/i915/i915_scheduler.c +4 -1
index 232f48a..f51676b 100644
@@ -44,6 +44,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
 {
 	struct drm_i915_private *dev_priv = dev->dev_private;
 
+	if (i915.scheduler_override & i915_so_direct_submit)
+		return false;
+
 	return dev_priv->scheduler != NULL;
 }
 
@@ -90,7 +93,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
 	BUG_ON(!scheduler);
 
-	if (1/*i915.scheduler_override & i915_so_direct_submit*/) {
+	if (i915.scheduler_override & i915_so_direct_submit) {
 		int ret;
 
 		intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
drivers/gpu/drm/i915/i915_scheduler.h +5 -0
index 29f7248..7e85f80 100644
@@ -83,6 +83,11 @@ enum {
 	i915_sf_submitting          = (1 << 1),
 };
 
+/* Options for 'scheduler_override' module parameter: */
+enum {
+	i915_so_direct_submit       = (1 << 0),
+};
+
 bool        i915_scheduler_is_enabled(struct drm_device *dev);
 int         i915_scheduler_init(struct drm_device *dev);
 int         i915_scheduler_closefile(struct drm_device *dev,
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#34 lore raw

PATCH 22/39 drm/i915: Support for 'unflushed' ring idle

In reply to [email protected]

From: John Harrison <[email protected]>

When the seqno wraps around zero, the entire GPU is forced to be idle for some reason (possibly only to work around issues with hardware semaphores but no-one seems too sure!). This causes a problem if the force idle occurs at an inopportune moment such as in the middle of submitting a batch buffer. Specifically, it would lead to recursive submits - submitting work requires a new seqno, the new seqno requires idling the ring, idling the ring requires submitting work, submitting work requires a new seqno...

This change adds a 'flush' parameter to the idle function call which specifies whether the scheduler queues should be flushed out. I.e. is the call intended to just idle the ring as it is right now (no flush) or is it intended to force all outstanding work out of the system (with flush).

In the seqno wrap case, pending work is not an issue because the next operation will be to submit it. However, in other cases, the intention is to make sure everything that could be done has been done.

Change-Id: I182e9a5853666c64ecc9e84d8a8b820a7f8e8836
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem.c         |  4 ++--
 drivers/gpu/drm/i915/intel_lrc.c        |  2 +-
 drivers/gpu/drm/i915/intel_ringbuffer.c | 17 +++++++++++++++--
 drivers/gpu/drm/i915/intel_ringbuffer.h |  2 +-
 4 files changed, 19 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +2 -2
index 16bd82c..ebb8e0c 100644
@@ -2477,7 +2477,7 @@ i915_gem_init_seqno(struct drm_device *dev, u32 seqno)
 
 	/* Carefully retire all requests without writing to the rings */
 	for_each_ring(ring, dev_priv, i) {
-		ret = intel_ring_idle(ring);
+		ret = intel_ring_idle(ring, false);
 		if (ret)
 			return ret;
 	}
@@ -3788,7 +3788,7 @@ int i915_gpu_idle(struct drm_device *dev)
 			i915_add_request_no_flush(req);
 		}
 
-		ret = intel_ring_idle(ring);
+		ret = intel_ring_idle(ring, true);
 		if (ret)
 			return ret;
 	}
drivers/gpu/drm/i915/intel_lrc.c +1 -1
index 16c5302..191a118 100644
@@ -1014,7 +1014,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *ring)
 	if (!intel_ring_initialized(ring))
 		return;
 
-	ret = intel_ring_idle(ring);
+	ret = intel_ring_idle(ring, true);
 	if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
 		DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
 			  ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.c +15 -2
index 4d77886..2d8dc54 100644
@@ -2278,9 +2278,22 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)
 	intel_ring_update_space(ringbuf);
 }
 
-int intel_ring_idle(struct intel_engine_cs *ring)
+int intel_ring_idle(struct intel_engine_cs *ring, bool flush)
 {
 	struct drm_i915_gem_request *req;
+	int ret;
+
+	/*
+	 * NB: Must not flush the scheduler if this idle request is from
+	 * within an execbuff submission (i.e. due to 'get_seqno' calling
+	 * 'wrap_seqno' calling 'idle'). As that would lead to recursive
+	 * flushes!
+	 */
+	if (flush) {
+		ret = i915_scheduler_flush(ring, true);
+		if (ret)
+			return ret;
+	}
 
 	/* Wait upon the last request to be completed */
 	if (list_empty(&ring->request_list))
@@ -3085,7 +3098,7 @@ intel_stop_ring_buffer(struct intel_engine_cs *ring)
 	if (!intel_ring_initialized(ring))
 		return;
 
-	ret = intel_ring_idle(ring);
+	ret = intel_ring_idle(ring, true);
 	if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
 		DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
 			  ring->name, ret);
drivers/gpu/drm/i915/intel_ringbuffer.h +1 -1
index 48f60cc..d31c94f 100644
@@ -488,7 +488,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
 int intel_ring_space(struct intel_ringbuffer *ringbuf);
 bool intel_ring_stopped(struct intel_engine_cs *ring);
 
-int __must_check intel_ring_idle(struct intel_engine_cs *ring);
+int __must_check intel_ring_idle(struct intel_engine_cs *ring, bool flush);
 void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno);
 int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
 int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#35 lore raw

PATCH 23/39 drm/i915: Defer seqno allocation until actual hardware submission time

In reply to [email protected]

From: John Harrison <[email protected]>

The seqno value is now only used for the final test for completion of a request. It is no longer used to track the request through the software stack. Thus it is no longer necessary to allocate the seqno immediately with the request. Instead, it can be done lazily and left until the request is actually sent to the hardware. This is particular advantageous with a GPU scheduler as the requests can then be re-ordered between their creation and their hardware submission without having out of order seqnos.

v2: i915_add_request() can't fail!

v3: combine with 'drm/i915: Assign seqno at start of exec_final()' Various bits of code during the execbuf code path need a seqno value to be assigned to the request. This change makes this assignment explicit at the start of submission_final() rather than relying on an auto-generated seqno to have happened already. This is in preparation for a future patch which changes seqno values to be assigned lazily (during add_request).

Change-Id: I0d922b84c517611a79fa6c2b9e730d4fe3671d6a
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  1 +
 drivers/gpu/drm/i915/i915_gem.c            | 21 ++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           | 13 +++++++++++++
 4 files changed, 47 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 5b893a6..15dee41 100644
@@ -2225,6 +2225,7 @@ struct drm_i915_gem_request {
 
 	/** GEM sequence number associated with this request. */
 	uint32_t seqno;
+	uint32_t reserved_seqno;
 
 	/* Unique identifier which can be used for trace points & debug */
 	uint32_t uniq;
drivers/gpu/drm/i915/i915_gem.c +20 -1
index ebb8e0c..9c08035 100644
@@ -2529,6 +2529,9 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)
 
 	/* reserve 0 for non-seqno */
 	if (dev_priv->next_seqno == 0) {
+		/* Why is the full re-initialisation required? Is it only for
+		 * hardware semaphores? If so, could skip it in the case where
+		 * semaphores are disabled? */
 		int ret = i915_gem_init_seqno(dev, 0);
 		if (ret)
 			return ret;
@@ -2586,6 +2589,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,
 		WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
 	}
 
+	/* Make the request's seqno 'live': */
+	if(!request->seqno) {
+		request->seqno = request->reserved_seqno;
+		WARN_ON(request->seqno != dev_priv->last_seqno);
+	}
+
 	/* Record the position of the start of the request so that
 	 * should we detect the updated seqno part-way through the
 	 * GPU processing the request, we never over-estimate the
@@ -2835,6 +2844,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 
 	list_for_each_entry_safe(req, req_next, &ring->fence_signal_list, signal_link) {
 		if (!req->cancelled) {
+			/* How can this happen? */
+			WARN_ON(req->seqno == 0);
+
 			if (!i915_seqno_passed(seqno, req->seqno))
 				break;
 		}
@@ -3057,7 +3069,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
 	if (req == NULL)
 		return -ENOMEM;
 
-	ret = i915_gem_get_seqno(ring->dev, &req->seqno);
+	/*
+	 * Assign an identifier to track this request through the hardware
+	 * but don't make it live yet. It could change in the future if this
+	 * request gets overtaken. However, it still needs to be allocated
+	 * in advance because the point of submission must not fail and seqno
+	 * allocation can fail.
+	 */
+	ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
 	if (ret)
 		goto err;
 
drivers/gpu/drm/i915/i915_gem_execbuffer.c +13 -0
index fe90571..1c249d0 100644
@@ -1248,6 +1248,19 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	/* Make sure the request's seqno is the latest and greatest: */
+	if(params->request->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &params->request->reserved_seqno);
+		if (ret)
+			return ret;
+	}
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front. */
+	WARN_ON(params->request->seqno);
+	params->request->seqno = params->request->reserved_seqno;
+	WARN_ON(params->request->seqno != dev_priv->last_seqno);
+
 	ret = intel_ring_reserve_space(params->request);
 	if (ret)
 		return ret;
drivers/gpu/drm/i915/intel_lrc.c +13 -0
index 191a118..41e20ba 100644
@@ -932,6 +932,19 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	/* Make sure the request's seqno is the latest and greatest: */
+	if(params->request->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &params->request->reserved_seqno);
+		if (ret)
+			return ret;
+	}
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front. */
+	WARN_ON(params->request->seqno);
+	params->request->seqno = params->request->reserved_seqno;
+	WARN_ON(params->request->seqno != dev_priv->last_seqno);
+
 	ret = intel_logical_ring_reserve_space(params->request);
 	if (ret)
 		return ret;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#36 lore raw

PATCH 24/40 drm/i915: Defer seqno allocation until actual hardware submission time

From: John Harrison <[email protected]>

The seqno value is now only used for the final test for completion of a request. It is no longer used to track the request through the software stack. Thus it is no longer necessary to allocate the seqno immediately with the request. Instead, it can be done lazily and left until the request is actually sent to the hardware. This is particular advantageous with a GPU scheduler as the requests can then be re-ordered between their creation and their hardware submission without having out of order seqnos.

v2: i915_add_request() can't fail!

Combine with 'drm/i915: Assign seqno at start of exec_final()' Various bits of code during the execbuf code path need a seqno value to be assigned to the request. This change makes this assignment explicit at the start of submission_final() rather than relying on an auto-generated seqno to have happened already. This is in preparation for a future patch which changes seqno values to be assigned lazily (during add_request).

v3: Updated to use locally cached request pointer.

Change-Id: I0d922b84c517611a79fa6c2b9e730d4fe3671d6a
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  1 +
 drivers/gpu/drm/i915/i915_gem.c            | 21 ++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           | 13 +++++++++++++
 4 files changed, 47 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 5b893a6..15dee41 100644
@@ -2225,6 +2225,7 @@ struct drm_i915_gem_request {
 
 	/** GEM sequence number associated with this request. */
 	uint32_t seqno;
+	uint32_t reserved_seqno;
 
 	/* Unique identifier which can be used for trace points & debug */
 	uint32_t uniq;
drivers/gpu/drm/i915/i915_gem.c +20 -1
index 99e5b1d0..1fb45c2 100644
@@ -2525,6 +2525,9 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)
 
 	/* reserve 0 for non-seqno */
 	if (dev_priv->next_seqno == 0) {
+		/* Why is the full re-initialisation required? Is it only for
+		 * hardware semaphores? If so, could skip it in the case where
+		 * semaphores are disabled? */
 		int ret = i915_gem_init_seqno(dev, 0);
 		if (ret)
 			return ret;
@@ -2582,6 +2585,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,
 		WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
 	}
 
+	/* Make the request's seqno 'live': */
+	if(!request->seqno) {
+		request->seqno = request->reserved_seqno;
+		WARN_ON(request->seqno != dev_priv->last_seqno);
+	}
+
 	/* Record the position of the start of the request so that
 	 * should we detect the updated seqno part-way through the
 	 * GPU processing the request, we never over-estimate the
@@ -2830,6 +2839,9 @@ void i915_gem_request_notify(struct intel_engine_cs *ring, bool fence_locked)
 
 	list_for_each_entry_safe(req, req_next, &ring->fence_signal_list, signal_link) {
 		if (!req->cancelled) {
+			/* How can this happen? */
+			WARN_ON(req->seqno == 0);
+
 			if (!i915_seqno_passed(seqno, req->seqno))
 				break;
 		}
@@ -3054,7 +3066,14 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
 	if (req == NULL)
 		return -ENOMEM;
 
-	ret = i915_gem_get_seqno(ring->dev, &req->seqno);
+	/*
+	 * Assign an identifier to track this request through the hardware
+	 * but don't make it live yet. It could change in the future if this
+	 * request gets overtaken. However, it still needs to be allocated
+	 * in advance because the point of submission must not fail and seqno
+	 * allocation can fail.
+	 */
+	ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
 	if (ret)
 		goto err;
 
drivers/gpu/drm/i915/i915_gem_execbuffer.c +13 -0
index 0908699..7970958 100644
@@ -1249,6 +1249,19 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	/* Make sure the request's seqno is the latest and greatest: */
+	if(req->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+		if (ret)
+			return ret;
+	}
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front. */
+	WARN_ON(req->seqno);
+	req->seqno = req->reserved_seqno;
+	WARN_ON(req->seqno != dev_priv->last_seqno);
+
 	ret = intel_ring_reserve_space(req);
 	if (ret)
 		return ret;
drivers/gpu/drm/i915/intel_lrc.c +13 -0
index 85bef51..d187f9a 100644
@@ -934,6 +934,19 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
+	/* Make sure the request's seqno is the latest and greatest: */
+	if(req->reserved_seqno != dev_priv->last_seqno) {
+		ret = i915_gem_get_seqno(ring->dev, &req->reserved_seqno);
+		if (ret)
+			return ret;
+	}
+	/*
+	 * And make it live because some of the execbuff submission code
+	 * requires the seqno to be available up front. */
+	WARN_ON(req->seqno);
+	req->seqno = req->reserved_seqno;
+	WARN_ON(req->seqno != dev_priv->last_seqno);
+
 	ret = intel_logical_ring_reserve_space(req);
 	if (ret)
 		return ret;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#37 lore raw

PATCH 24/39 drm/i915: Added immediate submission override to scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

To aid with debugging issues related to the scheduler, it can be useful to ensure that all batch buffers are submitted immediately rather than queued until later. This change adds an override flag via the module parameter to force instant submission.

Change-Id: I7652df53e2d3c3d77d78bebcf99856e2c53f2801
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_scheduler.c | 7 +++++--
 drivers/gpu/drm/i915/i915_scheduler.h | 1 +
 2 files changed, 6 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_scheduler.c +5 -2
index f51676b..d4d2466 100644
@@ -235,8 +235,11 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
 	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
 
-	not_flying = i915_scheduler_count_flying(scheduler, ring) <
-						 scheduler->min_flying;
+	if (i915.scheduler_override & i915_so_submit_on_queue)
+		not_flying = true;
+	else
+		not_flying = i915_scheduler_count_flying(scheduler, ring) <
+							 scheduler->min_flying;
 
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 7e85f80..fc58d49 100644
@@ -86,6 +86,7 @@ enum {
 /* Options for 'scheduler_override' module parameter: */
 enum {
 	i915_so_direct_submit       = (1 << 0),
+	i915_so_submit_on_queue     = (1 << 1),
 };
 
 bool        i915_scheduler_is_enabled(struct drm_device *dev);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#38 lore raw

PATCH 25/39 drm/i915: Add sync wait support to scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

There is a sync framework to allow work for multiple independent systems to be synchronised with each other but without stalling the CPU whether in the application or the driver. This patch adds support for this framework to the GPU scheduler.

Batch buffers can now have sync framework fence objects associated with them. The scheduler will look at this fence when deciding what to submit next to the hardware. If the fence is outstanding then that batch buffer will be passed over in preference of one that is ready to run. If no other batches are ready then the scheduler will queue an asynchronous callback to be woken up when the fence has been signalled. The callback will wake the scheduler and submit the now ready batch buffer.

v2: Updated to use the new sync_fence_is_signaled() API rather than having to know about the internals of a fence object.

Also removed the spin lock from the fence wait callback function. This was used to clear the 'waiting' flag. However, it causes a problem with the TDR code. Specifically, when cancelling work packets due to a TDR there is a code path where the fence can be signalled while the spinlock is already held. It is not really necessary to clear the flag anyway as it's purpose is solely to prevent multiple waits being issued. Once the fence has been signalled, no further waits will be attempted so it doesn't matter whether the fence is marked as having an outstanding wait or not.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h       |   1 +
 drivers/gpu/drm/i915/i915_scheduler.c | 144 ++++++++++++++++++++++++++++++++--
 drivers/gpu/drm/i915/i915_scheduler.h |   6 ++
 3 files changed, 146 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 15dee41..4187e75 100644
@@ -1698,6 +1698,7 @@ struct i915_execbuffer_params {
 	uint64_t                        batch_obj_vm_offset;
 	struct intel_engine_cs          *ring;
 	struct drm_i915_gem_object      *batch_obj;
+	struct sync_fence               *fence_wait;
 	struct drm_clip_rect            *cliprects;
 	uint32_t                        instp_mask;
 	int                             instp_mode;
drivers/gpu/drm/i915/i915_scheduler.c +139 -5

216 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +6 -0
index fc58d49..adfa1e0 100644
@@ -51,6 +51,11 @@ struct i915_scheduler_obj_entry {
 	struct drm_i915_gem_object          *obj;
 };
 
+enum i915_scheduler_queue_entry_flags {
+	/* Fence is being waited on */
+	i915_qef_fence_waiting              = (1 << 0),
+};
+
 struct i915_scheduler_queue_entry {
 	struct i915_execbuffer_params       params;
 	/* -1023 = lowest priority, 0 = default, 1023 = highest */
@@ -63,6 +68,7 @@ struct i915_scheduler_queue_entry {
 	enum i915_scheduler_queue_status    status;
 	unsigned long                       stamp;
 	struct list_head                    link;
+	uint32_t                            flags;
 };
 
 struct i915_scheduler {
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#39 lore raw

PATCH 26/39 drm/i915: Connecting execbuff fences to scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler now supports sync framework fences being associated with batch buffers. The execbuff IOCTL allows such fences to be passed in from user land. This patch wires the two together so that the IOCTL no longer needs to stall on the fence immediately. Instead the stall is now swallowed by the scheduler's scheduling algorithm.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 19 ++++++++++++++++++-
 drivers/gpu/drm/i915/i915_scheduler.c      |  3 +++
 2 files changed, 21 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +18 -1
index 1c249d0..a247d50 100644
@@ -1529,7 +1529,9 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	/*
 	 * Without a GPU scheduler, any fence waits must be done up front.
 	 */
-	if (args->flags & I915_EXEC_WAIT_FENCE) {
+	if ((args->flags & I915_EXEC_WAIT_FENCE) &&
+	    (i915.scheduler_override & i915_so_direct_submit))
+	{
 		ret = i915_early_fence_wait(ring, fd_fence_wait);
 		if (ret < 0)
 			return ret;
@@ -1714,6 +1716,18 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	i915_gem_context_reference(ctx);
 	params->ctx = ctx;
 
+	if (args->flags & I915_EXEC_WAIT_FENCE) {
+		if (fd_fence_wait < 0) {
+			DRM_ERROR("Wait fence for ring %d has invalid id %d\n",
+				  (int) ring->id, fd_fence_wait);
+		} else {
+			params->fence_wait = sync_fence_fdget(fd_fence_wait);
+			if (params->fence_wait == NULL)
+				DRM_ERROR("Invalid wait fence %d\n",
+					  fd_fence_wait);
+		}
+	}
+
 	if (args->flags & I915_EXEC_CREATE_FENCE) {
 		/*
 		 * Caller has requested a sync fence.
@@ -1794,6 +1808,9 @@ err:
 			i915_gem_context_unreference(params->ctx);
 	}
 
+	if (params->fence_wait)
+		sync_fence_put(params->fence_wait);
+
 	/*
 	 * If the request was created but not successfully submitted then it
 	 * must be freed again. If it was submitted then it is being tracked
drivers/gpu/drm/i915/i915_scheduler.c +3 -0
index 939dc2b..ddd21b2 100644
@@ -915,6 +915,9 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		else
 			signalled = true;
 
+		if (!signalled)
+			signalled = i915_safe_to_ignore_fence(ring, node->params.fence_wait);
+
 		has_local  = false;
 		has_remote = false;
 		for (i = 0; i < node->num_deps; i++) {
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#40 lore raw

PATCH 27/39 drm/i915: Added trace points to scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.

v2: Updated for new request completion tracking implementation.

Change-Id: I9886390cfc7897bc1faf50a104bc651d8baed8a5
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |   2 +
 drivers/gpu/drm/i915/i915_scheduler.c      |  24 ++++
 drivers/gpu/drm/i915/i915_trace.h          | 190 +++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           |   2 +
 4 files changed, 218 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +2 -0
index a247d50..e748fd3 100644
@@ -1226,6 +1226,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +24 -0
index ddd21b2..098f69a 100644
@@ -97,6 +97,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	if (i915.scheduler_override & i915_so_direct_submit) {
 		int ret;
 
+		trace_i915_scheduler_queue(qe->params.ring, qe);
+
 		WARN_ON(qe->params.fence_wait &&
 			(!sync_fence_is_signaled(qe->params.fence_wait)));
 
@@ -248,6 +250,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 		not_flying = i915_scheduler_count_flying(scheduler, ring) <
 							 scheduler->min_flying;
 
+	trace_i915_scheduler_queue(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	if (not_flying)
@@ -275,6 +280,9 @@ static int i915_scheduler_fly_node(struct i915_scheduler_queue_entry *node)
 
 	node->status = i915_sqs_flying;
 
+	trace_i915_scheduler_fly(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
 		bool    success = true;
 
@@ -340,6 +348,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
 
 	node->status = i915_sqs_queued;
 	node->params.request->seqno = 0;
+	trace_i915_scheduler_unfly(node->params.ring, node);
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /* Give up on a popped node completely. For example, because it is causing the
@@ -350,6 +360,8 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
 	BUG_ON(!I915_SQS_IS_FLYING(node));
 
 	node->status = i915_sqs_dead;
+	trace_i915_scheduler_unfly(node->params.ring, node);
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /*
@@ -369,6 +381,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
 	unsigned long       flags;
 
+	trace_i915_scheduler_landing(req);
+
 	if (!node)
 		return false;
 
@@ -382,6 +396,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	else
 		node->status = i915_sqs_complete;
 
+	trace_i915_scheduler_node_state_change(req->ring, node);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	return true;
@@ -524,6 +540,8 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 	/* Launch more packets now? */
 	do_submit = (queued > 0) && (flying < scheduler->min_flying);
 
+	trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	if (!do_submit && list_empty(&remove))
@@ -538,6 +556,8 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 		node = list_first_entry(&remove, typeof(*node), link);
 		list_del(&node->link);
 
+		trace_i915_scheduler_destroy(ring, node);
+
 		if (node->params.fence_wait)
 			sync_fence_put(node->params.fence_wait);
 
@@ -952,6 +972,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		INIT_LIST_HEAD(&best->link);
 		best->status  = i915_sqs_popped;
 
+		trace_i915_scheduler_node_state_change(ring, best);
+
 		ret = 0;
 	} else {
 		/* Can only get here if:
@@ -1005,6 +1027,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		spin_lock_irqsave(&scheduler->lock, *flags);
 	}
 
+	trace_i915_scheduler_pop_from_queue(ring, best);
+
 	*pop_node = best;
 	return ret;
 }
drivers/gpu/drm/i915/i915_trace.h +190 -0

202 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.c +2 -0
index 41e20ba..6c01f6c 100644
@@ -910,6 +910,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#41 lore raw

PATCH 28/40 drm/i915: Added trace points to scheduler

From: John Harrison <[email protected]>

Added trace points to the scheduler to track all the various events, node state transitions and other interesting things that occur.

v2: Updated for new request completion tracking implementation.

v3: Updated for changes to node kill code.

Change-Id: I9886390cfc7897bc1faf50a104bc651d8baed8a5
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |   2 +
 drivers/gpu/drm/i915/i915_scheduler.c      |  26 ++++
 drivers/gpu/drm/i915/i915_trace.h          | 190 +++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_lrc.c           |   2 +
 4 files changed, 220 insertions(+)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +2 -0
index fdaede3..b358b21 100644
@@ -1226,6 +1226,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
drivers/gpu/drm/i915/i915_scheduler.c +26 -0
index 39aa702..4736f0f 100644
@@ -151,6 +151,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 	if (i915.scheduler_override & i915_so_direct_submit) {
 		int ret;
 
+		trace_i915_scheduler_queue(qe->params.ring, qe);
+
 		WARN_ON(qe->params.fence_wait &&
 			(!sync_fence_is_signaled(qe->params.fence_wait)));
 
@@ -271,6 +273,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 		not_flying = i915_scheduler_count_flying(scheduler, ring) <
 							 scheduler->min_flying;
 
+	trace_i915_scheduler_queue(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	if (not_flying)
@@ -298,6 +303,9 @@ static int i915_scheduler_fly_node(struct i915_scheduler_queue_entry *node)
 
 	node->status = i915_sqs_flying;
 
+	trace_i915_scheduler_fly(ring, node);
+	trace_i915_scheduler_node_state_change(ring, node);
+
 	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
 		bool    success = true;
 
@@ -363,6 +371,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
 
 	node->status = i915_sqs_queued;
 	node->params.request->seqno = 0;
+	trace_i915_scheduler_unfly(node->params.ring, node);
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /* Give up on a node completely. For example, because it is causing the
@@ -372,7 +382,11 @@ static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
 	BUG_ON(!node);
 	BUG_ON(I915_SQS_IS_COMPLETE(node));
 
+	if (I915_SQS_IS_FLYING(node))
+		trace_i915_scheduler_unfly(node->params.ring, node);
+
 	node->status = i915_sqs_dead;
+	trace_i915_scheduler_node_state_change(node->params.ring, node);
 }
 
 /*
@@ -392,6 +406,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	struct i915_scheduler_queue_entry *node = req->scheduler_qe;
 	unsigned long       flags;
 
+	trace_i915_scheduler_landing(req);
+
 	if (!node)
 		return false;
 
@@ -405,6 +421,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
 	else
 		node->status = i915_sqs_complete;
 
+	trace_i915_scheduler_node_state_change(req->ring, node);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	return true;
@@ -550,6 +568,8 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 	/* Launch more packets now? */
 	do_submit = (queued > 0) && (flying < scheduler->min_flying);
 
+	trace_i915_scheduler_remove(ring, min_seqno, do_submit);
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	if (!do_submit && list_empty(&remove))
@@ -564,6 +584,8 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 		node = list_first_entry(&remove, typeof(*node), link);
 		list_del(&node->link);
 
+		trace_i915_scheduler_destroy(ring, node);
+
 		if (node->params.fence_wait)
 			sync_fence_put(node->params.fence_wait);
 
@@ -927,6 +949,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		INIT_LIST_HEAD(&best->link);
 		best->status  = i915_sqs_popped;
 
+		trace_i915_scheduler_node_state_change(ring, best);
+
 		ret = 0;
 	} else {
 		/* Can only get here if:
@@ -980,6 +1004,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
 		spin_lock_irqsave(&scheduler->lock, *flags);
 	}
 
+	trace_i915_scheduler_pop_from_queue(ring, best);
+
 	*pop_node = best;
 	return ret;
 }
drivers/gpu/drm/i915/i915_trace.h +190 -0

202 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.c +2 -0
index d187f9a..1fa3228 100644
@@ -910,6 +910,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
+	trace_i915_gem_ring_queue(ring, params);
+
 	qe = container_of(params, typeof(*qe), params);
 	ret = i915_scheduler_queue_execbuffer(qe);
 	if (ret)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#42 lore raw

PATCH 28/39 drm/i915: Added scheduler queue throttling by DRM file handle

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.

Change-Id: I83258240aec7c810db08c006a3062d46aa91363f
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  2 ++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  8 +++++++
 drivers/gpu/drm/i915/i915_scheduler.c      | 34 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  2 ++
 4 files changed, 46 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index 4187e75..4ecb6e4 100644
@@ -342,6 +342,8 @@ struct drm_i915_file_private {
 	} rps;
 
 	struct intel_engine_cs *bsd_ring;
+
+	u32 scheduler_queue_length;
 };
 
 enum intel_dpll_id {
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8 -0
index e748fd3..8aca374 100644
@@ -1857,6 +1857,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	/* Copy in the exec list from userland */
 	exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
 	exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
@@ -1940,6 +1944,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
 			     GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
 	if (exec2_list == NULL)
drivers/gpu/drm/i915/i915_scheduler.c +34 -0
index 098f69a..ff6460e 100644
@@ -40,6 +40,8 @@ static void        i915_scheduler_priority_bump_clear(struct i915_scheduler *sch
 static int         i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
 						struct i915_scheduler_queue_entry *target,
 						uint32_t bump);
+static void        i915_scheduler_file_queue_inc(struct drm_file *file);
+static void        i915_scheduler_file_queue_dec(struct drm_file *file);
 
 bool i915_scheduler_is_enabled(struct drm_device *dev)
 {
@@ -74,6 +76,7 @@ int i915_scheduler_init(struct drm_device *dev)
 	scheduler->priority_level_max     = 1023;
 	scheduler->priority_level_preempt = 900;
 	scheduler->min_flying             = 2;
+	scheduler->file_queue_max         = 64;
 
 	dev_priv->scheduler = scheduler;
 
@@ -244,6 +247,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
 	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
 
+	i915_scheduler_file_queue_inc(node->params.file);
+
 	if (i915.scheduler_override & i915_so_submit_on_queue)
 		not_flying = true;
 	else
@@ -523,6 +528,12 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 		/* Strip the dependency info while the mutex is still locked */
 		i915_scheduler_remove_dependent(scheduler, node);
 
+		/* Likewise clean up the file descriptor before it might disappear. */
+		if (node->params.file) {
+			i915_scheduler_file_queue_dec(node->params.file);
+			node->params.file = NULL;
+		}
+
 		continue;
 	}
 
@@ -1256,3 +1267,26 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 
 	return 0;
 }
+
+bool i915_scheduler_file_queue_is_full(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+	struct drm_i915_private      *dev_priv  = file_priv->dev_priv;
+	struct i915_scheduler        *scheduler = dev_priv->scheduler;
+
+	return file_priv->scheduler_queue_length >= scheduler->file_queue_max;
+}
+
+static void i915_scheduler_file_queue_inc(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length++;
+}
+
+static void i915_scheduler_file_queue_dec(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length--;
+}
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index adfa1e0..74d7321 100644
@@ -81,6 +81,7 @@ struct i915_scheduler {
 	int32_t             priority_level_max;
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
+	uint32_t            file_queue_max;
 };
 
 /* Flag bits for i915_scheduler::flags */
@@ -111,5 +112,6 @@ int         i915_scheduler_flush_request(struct drm_i915_gem_request *req,
 					 bool is_locked);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 					      bool *completed, bool *busy);
+bool        i915_scheduler_file_queue_is_full(struct drm_file *file);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#43 lore raw

PATCH 29/40 drm/i915: Added scheduler queue throttling by DRM file handle

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver from their subsequent submission to the hardware. This means that an application which is continuously submitting buffers as fast as it can could potentialy flood the driver. To prevent this, the driver now tracks how many buffers are in progress (queued in software or executing in hardware) and limits this to a given (tunable) number. If this number is exceeded then the queue to the driver will return EAGAIN and thus prevent the scheduler's queue becoming arbitrarily large.

v3: Added a missing decrement of the file queue counter.

Change-Id: I83258240aec7c810db08c006a3062d46aa91363f
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  2 ++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  8 +++++++
 drivers/gpu/drm/i915/i915_scheduler.c      | 35 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  2 ++
 4 files changed, 47 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index 4187e75..4ecb6e4 100644
@@ -342,6 +342,8 @@ struct drm_i915_file_private {
 	} rps;
 
 	struct intel_engine_cs *bsd_ring;
+
+	u32 scheduler_queue_length;
 };
 
 enum intel_dpll_id {
drivers/gpu/drm/i915/i915_gem_execbuffer.c +8 -0
index b358b21..8ba426f 100644
@@ -1862,6 +1862,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	/* Copy in the exec list from userland */
 	exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
 	exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
@@ -1945,6 +1949,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* Throttle batch requests per device file */
+	if (i915_scheduler_file_queue_is_full(file))
+		return -EAGAIN;
+
 	exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
 			     GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
 	if (exec2_list == NULL)
drivers/gpu/drm/i915/i915_scheduler.c +35 -0
index 4736f0f..e6e1bd967 100644
@@ -40,6 +40,8 @@ static void        i915_scheduler_priority_bump_clear(struct i915_scheduler *sch
 static int         i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
 						struct i915_scheduler_queue_entry *target,
 						uint32_t bump);
+static void        i915_scheduler_file_queue_inc(struct drm_file *file);
+static void        i915_scheduler_file_queue_dec(struct drm_file *file);
 
 bool i915_scheduler_is_enabled(struct drm_device *dev)
 {
@@ -74,6 +76,7 @@ int i915_scheduler_init(struct drm_device *dev)
 	scheduler->priority_level_max     = 1023;
 	scheduler->priority_level_preempt = 900;
 	scheduler->min_flying             = 2;
+	scheduler->file_queue_max         = 64;
 
 	dev_priv->scheduler = scheduler;
 
@@ -267,6 +270,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
 
 	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
 
+	i915_scheduler_file_queue_inc(node->params.file);
+
 	if (i915.scheduler_override & i915_so_submit_on_queue)
 		not_flying = true;
 	else
@@ -551,6 +556,12 @@ static int i915_scheduler_remove(struct intel_engine_cs *ring)
 		/* Strip the dependency info while the mutex is still locked */
 		i915_scheduler_remove_dependent(scheduler, node);
 
+		/* Likewise clean up the file descriptor before it might disappear. */
+		if (node->params.file) {
+			i915_scheduler_file_queue_dec(node->params.file);
+			node->params.file = NULL;
+		}
+
 		continue;
 	}
 
@@ -1194,6 +1205,7 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 						 node->status,
 						 ring->name);
 
+			i915_scheduler_file_queue_dec(node->params.file);
 			node->params.file = NULL;
 		}
 	}
@@ -1202,3 +1214,26 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 
 	return 0;
 }
+
+bool i915_scheduler_file_queue_is_full(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+	struct drm_i915_private      *dev_priv  = file_priv->dev_priv;
+	struct i915_scheduler        *scheduler = dev_priv->scheduler;
+
+	return file_priv->scheduler_queue_length >= scheduler->file_queue_max;
+}
+
+static void i915_scheduler_file_queue_inc(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length++;
+}
+
+static void i915_scheduler_file_queue_dec(struct drm_file *file)
+{
+	struct drm_i915_file_private *file_priv = file->driver_priv;
+
+	file_priv->scheduler_queue_length--;
+}
drivers/gpu/drm/i915/i915_scheduler.h +2 -0
index b10f30c..5de0506 100644
@@ -82,6 +82,7 @@ struct i915_scheduler {
 	int32_t             priority_level_max;
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
+	uint32_t            file_queue_max;
 };
 
 /* Flag bits for i915_scheduler::flags */
@@ -110,5 +111,6 @@ int         i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
 				       unsigned long stamp, bool is_locked);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 					      bool *completed, bool *busy);
+bool        i915_scheduler_file_queue_is_full(struct drm_file *file);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#44 lore raw

PATCH 29/39 drm/i915: Added debugfs interface to scheduler tuning parameters

In reply to [email protected]

From: John Harrison <[email protected]>

There are various parameters within the scheduler which can be tuned to improve performance, reduce memory footprint, etc. This change adds support for altering these via debugfs.

v2: Updated for priorities now being signed values.

Change-Id: I6c26765269ae7173ff4d3a5c20921eaaca7c36ed
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c | 141 ++++++++++++++++++++++++++++++++++++
 1 file changed, 141 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +141 -0
index ecab9b8..8f1c10c 100644
@@ -39,6 +39,7 @@
 #include "intel_ringbuffer.h"
 #include <drm/i915_drm.h>
 #include "i915_drv.h"
+#include "i915_scheduler.h"
 
 enum {
 	ACTIVE_LIST,
@@ -1122,6 +1123,141 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
 			i915_next_seqno_get, i915_next_seqno_set,
 			"0x%llx\n");
 
+static int
+i915_scheduler_priority_min_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_min;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_min_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_min = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
+			i915_scheduler_priority_min_get,
+			i915_scheduler_priority_min_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_max;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_max = (int32_t) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
+			i915_scheduler_priority_max_get,
+			i915_scheduler_priority_max_set,
+			"%lld\n");
+
+static int
+i915_scheduler_priority_preempt_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_preempt;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_preempt_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_preempt = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
+			i915_scheduler_priority_preempt_get,
+			i915_scheduler_priority_preempt_set,
+			"%lld\n");
+
+static int
+i915_scheduler_min_flying_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->min_flying;
+	return 0;
+}
+
+static int
+i915_scheduler_min_flying_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->min_flying = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
+			i915_scheduler_min_flying_get,
+			i915_scheduler_min_flying_set,
+			"%llu\n");
+
+static int
+i915_scheduler_file_queue_max_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->file_queue_max;
+	return 0;
+}
+
+static int
+i915_scheduler_file_queue_max_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->file_queue_max = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
+			i915_scheduler_file_queue_max_get,
+			i915_scheduler_file_queue_max_set,
+			"%llu\n");
+
 static int i915_frequency_info(struct seq_file *m, void *unused)
 {
 	struct drm_info_node *node = m->private;
@@ -5457,6 +5593,11 @@ static const struct i915_debugfs_files {
 	{"i915_gem_drop_caches", &i915_drop_caches_fops},
 	{"i915_error_state", &i915_error_state_fops},
 	{"i915_next_seqno", &i915_next_seqno_fops},
+	{"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
+	{"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+	{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
+	{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
+	{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
 	{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
 	{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
 	{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#45 lore raw

PATCH 30/39 drm/i915: Added debug state dump facilities to scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

When debugging batch buffer submission issues, it is useful to be able to see what the current state of the scheduler is. This change adds functions for decoding the internal scheduler state and reporting it.

Change-Id: I0634168e3f3465ff023f5a673165c90b07e535b6
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_scheduler.c | 276 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  14 ++
 2 files changed, 290 insertions(+)
drivers/gpu/drm/i915/i915_scheduler.c +276 -0
index ff6460e..5eec217 100644
@@ -36,6 +36,9 @@ static int         i915_scheduler_submit_max_priority(struct intel_engine_cs *ri
 						      bool is_locked);
 static uint32_t    i915_scheduler_count_flying(struct i915_scheduler *scheduler,
 					       struct intel_engine_cs *ring);
+static int         i915_scheduler_dump_locked(struct intel_engine_cs *ring,
+					      const char *msg);
+static int         i915_scheduler_dump_all_locked(struct drm_device *dev, const char *msg);
 static void        i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler);
 static int         i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
 						struct i915_scheduler_queue_entry *target,
@@ -53,6 +56,116 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
 	return dev_priv->scheduler != NULL;
 }
 
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
+{
+	static char	str[50];
+	char		*ptr = str;
+
+	*(ptr++) = node->bumped ? 'B' : '-',
+	*(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
+
+	*ptr = 0;
+
+	return str;
+}
+
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
+{
+	switch (status) {
+	case i915_sqs_none:
+	return 'N';
+
+	case i915_sqs_queued:
+	return 'Q';
+
+	case i915_sqs_popped:
+	return 'X';
+
+	case i915_sqs_flying:
+	return 'F';
+
+	case i915_sqs_complete:
+	return 'C';
+
+	case i915_sqs_dead:
+	return 'D';
+
+	default:
+	break;
+	}
+
+	return '?';
+}
+
+const char *i915_scheduler_queue_status_str(
+				enum i915_scheduler_queue_status status)
+{
+	static char	str[50];
+
+	switch (status) {
+	case i915_sqs_none:
+	return "None";
+
+	case i915_sqs_queued:
+	return "Queued";
+
+	case i915_sqs_popped:
+	return "Popped";
+
+	case i915_sqs_flying:
+	return "Flying";
+
+	case i915_sqs_complete:
+	return "Complete";
+
+	case i915_sqs_dead:
+	return "Dead";
+
+	default:
+	break;
+	}
+
+	sprintf(str, "[Unknown_%d!]", status);
+	return str;
+}
+
+const char *i915_scheduler_flag_str(uint32_t flags)
+{
+	static char     str[100];
+	char           *ptr = str;
+
+	*ptr = 0;
+
+#define TEST_FLAG(flag, msg)						\
+	do {								\
+		if (flags & (flag)) {					\
+			strcpy(ptr, msg);				\
+			ptr += strlen(ptr);				\
+			flags &= ~(flag);				\
+		}							\
+	} while (0)
+
+	TEST_FLAG(i915_sf_interrupts_enabled, "IntOn|");
+	TEST_FLAG(i915_sf_submitting,         "Submitting|");
+	TEST_FLAG(i915_sf_dump_force,         "DumpForce|");
+	TEST_FLAG(i915_sf_dump_details,       "DumpDetails|");
+	TEST_FLAG(i915_sf_dump_dependencies,  "DumpDeps|");
+
+#undef TEST_FLAG
+
+	if (flags) {
+		sprintf(ptr, "Unknown_0x%X!", flags);
+		ptr += strlen(ptr);
+	}
+
+	if (ptr == str)
+		strcpy(str, "-");
+	else
+		ptr[-1] = 0;
+
+	return str;
+};
+
 int i915_scheduler_init(struct drm_device *dev)
 {
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -603,6 +716,169 @@ void i915_gem_scheduler_work_handler(struct work_struct *work)
 	}
 }
 
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	unsigned long   flags;
+	int             ret;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	ret = i915_scheduler_dump_all_locked(dev, msg);
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return ret;
+}
+
+static int i915_scheduler_dump_all_locked(struct drm_device *dev, const char *msg)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs  *ring;
+	int                     i, r, ret = 0;
+
+	for_each_ring(ring, dev_priv, i) {
+		scheduler->flags[ring->id] |= i915_sf_dump_force   |
+					      i915_sf_dump_details |
+					      i915_sf_dump_dependencies;
+		r = i915_scheduler_dump_locked(ring, msg);
+		if (ret == 0)
+			ret = r;
+	}
+
+	return ret;
+}
+
+int i915_scheduler_dump(struct intel_engine_cs *ring, const char *msg)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	unsigned long   flags;
+	int             ret;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	ret = i915_scheduler_dump_locked(ring, msg);
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return ret;
+}
+
+static int i915_scheduler_dump_locked(struct intel_engine_cs *ring, const char *msg)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry  *node;
+	int                 flying = 0, queued = 0, complete = 0, other = 0;
+	static int          old_flying = -1, old_queued = -1, old_complete = -1;
+	bool                b_dump;
+	char                brkt[2] = { '<', '>' };
+
+	if (!ring)
+		return -EINVAL;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			complete++;
+		else
+			other++;
+	}
+
+	b_dump = (flying != old_flying) ||
+		 (queued != old_queued) ||
+		 (complete != old_complete);
+	if (scheduler->flags[ring->id] & i915_sf_dump_force) {
+		if (!b_dump) {
+			b_dump = true;
+			brkt[0] = '{';
+			brkt[1] = '}';
+		}
+
+		scheduler->flags[ring->id] &= ~i915_sf_dump_force;
+	}
+
+	if (b_dump) {
+		old_flying   = flying;
+		old_queued   = queued;
+		old_complete = complete;
+		DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
+				 "Flags = %s, Next = %d:%d %c%s%c\n",
+				 ring->name, queued, flying, complete, other,
+				 i915_scheduler_flag_str(scheduler->flags[ring->id]),
+				 dev_priv->request_uniq, dev_priv->next_seqno,
+				 brkt[0], msg, brkt[1]);
+	} else {
+		/*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
+				 ", Flags = %s, Next = %d:%d [%s]\n",
+				 ring->name,
+				 queued, flying, complete, other,
+				 i915_scheduler_flag_str(scheduler->flags[ring->id]),
+				 dev_priv->request_uniq, dev_priv->next_seqno, msg); */
+
+		return 0;
+	}
+
+	if (scheduler->flags[ring->id] & i915_sf_dump_details) {
+		int         i, deps;
+		uint32_t    count, counts[i915_sqs_MAX];
+
+		memset(counts, 0x00, sizeof(counts));
+
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (node->status < i915_sqs_MAX) {
+				count = counts[node->status]++;
+			} else {
+				DRM_DEBUG_DRIVER("<%s>   Unknown status: %d!\n",
+						 ring->name, node->status);
+				count = -1;
+			}
+
+			deps = 0;
+			for (i = 0; i < node->num_deps; i++)
+				if (i915_scheduler_is_dependency_valid(node, i))
+					deps++;
+
+			DRM_DEBUG_DRIVER("<%s>   %c:%02d> uniq = %d, seqno"
+					 " = %d/%s, deps = %d / %d, fence = %p/%d, %s [pri = "
+					 "%4d]\n", ring->name,
+					 i915_scheduler_queue_status_chr(node->status),
+					 count,
+					 node->params.request->uniq,
+					 node->params.request->seqno,
+					 node->params.ring->name,
+					 deps, node->num_deps,
+					 node->params.fence_wait,
+					 node->params.fence_wait ? sync_fence_is_signaled(node->params.fence_wait) : 0,
+					 i915_qe_state_str(node),
+					 node->priority);
+
+			if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies)
+				== 0)
+				continue;
+
+			for (i = 0; i < node->num_deps; i++)
+				if (node->dep_list[i])
+					DRM_DEBUG_DRIVER("<%s>       |-%c:"
+						"%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
+						ring->name,
+						i915_scheduler_queue_status_chr(node->dep_list[i]->status),
+						i,
+						i915_scheduler_is_dependency_valid(node, i)
+							? '>' : '#',
+						node->dep_list[i]->params.request->uniq,
+						node->dep_list[i]->params.request->seqno,
+						node->dep_list[i]->params.ring->name,
+						i915_qe_state_str(node->dep_list[i]),
+						node->dep_list[i]->priority);
+		}
+	}
+
+	return 0;
+}
+
 int i915_scheduler_flush_request(struct drm_i915_gem_request *req,
 				 bool is_locked)
 {
drivers/gpu/drm/i915/i915_scheduler.h +14 -0
index 74d7321..b5518b5 100644
@@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
 	/* Limit value for use with arrays/loops */
 	i915_sqs_MAX
 };
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
+const char *i915_scheduler_queue_status_str(
+				enum i915_scheduler_queue_status status);
 
 #define I915_SQS_IS_QUEUED(node)	(((node)->status == i915_sqs_queued))
 #define I915_SQS_IS_FLYING(node)	(((node)->status == i915_sqs_flying))
@@ -70,6 +73,7 @@ struct i915_scheduler_queue_entry {
 	struct list_head                    link;
 	uint32_t                            flags;
 };
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
 
 struct i915_scheduler {
 	struct list_head    node_queue[I915_NUM_RINGS];
@@ -86,9 +90,16 @@ struct i915_scheduler {
 
 /* Flag bits for i915_scheduler::flags */
 enum {
+	/* Internal state */
 	i915_sf_interrupts_enabled  = (1 << 0),
 	i915_sf_submitting          = (1 << 1),
+
+	/* Dump/debug flags */
+	i915_sf_dump_force          = (1 << 8),
+	i915_sf_dump_details        = (1 << 9),
+	i915_sf_dump_dependencies   = (1 << 10),
 };
+const char *i915_scheduler_flag_str(uint32_t flags);
 
 /* Options for 'scheduler_override' module parameter: */
 enum {
@@ -110,6 +121,9 @@ int         i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
 				       unsigned long stamp, bool is_locked);
 int         i915_scheduler_flush_request(struct drm_i915_gem_request *req,
 					 bool is_locked);
+int         i915_scheduler_dump(struct intel_engine_cs *ring,
+				const char *msg);
+int         i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 					      bool *completed, bool *busy);
 bool        i915_scheduler_file_queue_is_full(struct drm_file *file);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#46 lore raw

PATCH 31/40 drm/i915: Added debug state dump facilities to scheduler

From: John Harrison <[email protected]>

When debugging batch buffer submission issues, it is useful to be able to see what the current state of the scheduler is. This change adds functions for decoding the internal scheduler state and reporting it.

v3: Updated a debug message with the new state_str() function.

Change-Id: I0634168e3f3465ff023f5a673165c90b07e535b6
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_scheduler.c | 280 +++++++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/i915_scheduler.h |  14 ++
 2 files changed, 292 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_scheduler.c +278 -2
index e6e1bd967..be2430d 100644
@@ -36,6 +36,9 @@ static int         i915_scheduler_submit_max_priority(struct intel_engine_cs *ri
 						      bool is_locked);
 static uint32_t    i915_scheduler_count_flying(struct i915_scheduler *scheduler,
 					       struct intel_engine_cs *ring);
+static int         i915_scheduler_dump_locked(struct intel_engine_cs *ring,
+					      const char *msg);
+static int         i915_scheduler_dump_all_locked(struct drm_device *dev, const char *msg);
 static void        i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler);
 static int         i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
 						struct i915_scheduler_queue_entry *target,
@@ -53,6 +56,116 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
 	return dev_priv->scheduler != NULL;
 }
 
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
+{
+	static char	str[50];
+	char		*ptr = str;
+
+	*(ptr++) = node->bumped ? 'B' : '-',
+	*(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
+
+	*ptr = 0;
+
+	return str;
+}
+
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
+{
+	switch (status) {
+	case i915_sqs_none:
+	return 'N';
+
+	case i915_sqs_queued:
+	return 'Q';
+
+	case i915_sqs_popped:
+	return 'X';
+
+	case i915_sqs_flying:
+	return 'F';
+
+	case i915_sqs_complete:
+	return 'C';
+
+	case i915_sqs_dead:
+	return 'D';
+
+	default:
+	break;
+	}
+
+	return '?';
+}
+
+const char *i915_scheduler_queue_status_str(
+				enum i915_scheduler_queue_status status)
+{
+	static char	str[50];
+
+	switch (status) {
+	case i915_sqs_none:
+	return "None";
+
+	case i915_sqs_queued:
+	return "Queued";
+
+	case i915_sqs_popped:
+	return "Popped";
+
+	case i915_sqs_flying:
+	return "Flying";
+
+	case i915_sqs_complete:
+	return "Complete";
+
+	case i915_sqs_dead:
+	return "Dead";
+
+	default:
+	break;
+	}
+
+	sprintf(str, "[Unknown_%d!]", status);
+	return str;
+}
+
+const char *i915_scheduler_flag_str(uint32_t flags)
+{
+	static char     str[100];
+	char           *ptr = str;
+
+	*ptr = 0;
+
+#define TEST_FLAG(flag, msg)						\
+	do {								\
+		if (flags & (flag)) {					\
+			strcpy(ptr, msg);				\
+			ptr += strlen(ptr);				\
+			flags &= ~(flag);				\
+		}							\
+	} while (0)
+
+	TEST_FLAG(i915_sf_interrupts_enabled, "IntOn|");
+	TEST_FLAG(i915_sf_submitting,         "Submitting|");
+	TEST_FLAG(i915_sf_dump_force,         "DumpForce|");
+	TEST_FLAG(i915_sf_dump_details,       "DumpDetails|");
+	TEST_FLAG(i915_sf_dump_dependencies,  "DumpDeps|");
+
+#undef TEST_FLAG
+
+	if (flags) {
+		sprintf(ptr, "Unknown_0x%X!", flags);
+		ptr += strlen(ptr);
+	}
+
+	if (ptr == str)
+		strcpy(str, "-");
+	else
+		ptr[-1] = 0;
+
+	return str;
+};
+
 int i915_scheduler_init(struct drm_device *dev)
 {
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -631,6 +744,169 @@ void i915_gem_scheduler_work_handler(struct work_struct *work)
 	}
 }
 
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	unsigned long   flags;
+	int             ret;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	ret = i915_scheduler_dump_all_locked(dev, msg);
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return ret;
+}
+
+static int i915_scheduler_dump_all_locked(struct drm_device *dev, const char *msg)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs  *ring;
+	int                     i, r, ret = 0;
+
+	for_each_ring(ring, dev_priv, i) {
+		scheduler->flags[ring->id] |= i915_sf_dump_force   |
+					      i915_sf_dump_details |
+					      i915_sf_dump_dependencies;
+		r = i915_scheduler_dump_locked(ring, msg);
+		if (ret == 0)
+			ret = r;
+	}
+
+	return ret;
+}
+
+int i915_scheduler_dump(struct intel_engine_cs *ring, const char *msg)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	unsigned long   flags;
+	int             ret;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	ret = i915_scheduler_dump_locked(ring, msg);
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return ret;
+}
+
+static int i915_scheduler_dump_locked(struct intel_engine_cs *ring, const char *msg)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry  *node;
+	int                 flying = 0, queued = 0, complete = 0, other = 0;
+	static int          old_flying = -1, old_queued = -1, old_complete = -1;
+	bool                b_dump;
+	char                brkt[2] = { '<', '>' };
+
+	if (!ring)
+		return -EINVAL;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			complete++;
+		else
+			other++;
+	}
+
+	b_dump = (flying != old_flying) ||
+		 (queued != old_queued) ||
+		 (complete != old_complete);
+	if (scheduler->flags[ring->id] & i915_sf_dump_force) {
+		if (!b_dump) {
+			b_dump = true;
+			brkt[0] = '{';
+			brkt[1] = '}';
+		}
+
+		scheduler->flags[ring->id] &= ~i915_sf_dump_force;
+	}
+
+	if (b_dump) {
+		old_flying   = flying;
+		old_queued   = queued;
+		old_complete = complete;
+		DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
+				 "Flags = %s, Next = %d:%d %c%s%c\n",
+				 ring->name, queued, flying, complete, other,
+				 i915_scheduler_flag_str(scheduler->flags[ring->id]),
+				 dev_priv->request_uniq, dev_priv->next_seqno,
+				 brkt[0], msg, brkt[1]);
+	} else {
+		/*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
+				 ", Flags = %s, Next = %d:%d [%s]\n",
+				 ring->name,
+				 queued, flying, complete, other,
+				 i915_scheduler_flag_str(scheduler->flags[ring->id]),
+				 dev_priv->request_uniq, dev_priv->next_seqno, msg); */
+
+		return 0;
+	}
+
+	if (scheduler->flags[ring->id] & i915_sf_dump_details) {
+		int         i, deps;
+		uint32_t    count, counts[i915_sqs_MAX];
+
+		memset(counts, 0x00, sizeof(counts));
+
+		list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+			if (node->status < i915_sqs_MAX) {
+				count = counts[node->status]++;
+			} else {
+				DRM_DEBUG_DRIVER("<%s>   Unknown status: %d!\n",
+						 ring->name, node->status);
+				count = -1;
+			}
+
+			deps = 0;
+			for (i = 0; i < node->num_deps; i++)
+				if (i915_scheduler_is_dependency_valid(node, i))
+					deps++;
+
+			DRM_DEBUG_DRIVER("<%s>   %c:%02d> uniq = %d, seqno"
+					 " = %d/%s, deps = %d / %d, fence = %p/%d, %s [pri = "
+					 "%4d]\n", ring->name,
+					 i915_scheduler_queue_status_chr(node->status),
+					 count,
+					 node->params.request->uniq,
+					 node->params.request->seqno,
+					 node->params.ring->name,
+					 deps, node->num_deps,
+					 node->params.fence_wait,
+					 node->params.fence_wait ? sync_fence_is_signaled(node->params.fence_wait) : 0,
+					 i915_qe_state_str(node),
+					 node->priority);
+
+			if ((scheduler->flags[ring->id] & i915_sf_dump_dependencies)
+				== 0)
+				continue;
+
+			for (i = 0; i < node->num_deps; i++)
+				if (node->dep_list[i])
+					DRM_DEBUG_DRIVER("<%s>       |-%c:"
+						"%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
+						ring->name,
+						i915_scheduler_queue_status_chr(node->dep_list[i]->status),
+						i,
+						i915_scheduler_is_dependency_valid(node, i)
+							? '>' : '#',
+						node->dep_list[i]->params.request->uniq,
+						node->dep_list[i]->params.request->seqno,
+						node->dep_list[i]->params.ring->name,
+						i915_qe_state_str(node->dep_list[i]),
+						node->dep_list[i]->priority);
+		}
+	}
+
+	return 0;
+}
+
 int i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
 			       unsigned long target,
 			       bool is_locked)
@@ -1199,10 +1475,10 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 				continue;
 
 			if(!I915_SQS_IS_COMPLETE(node))
-				DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+				DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%s on %s\n",
 						 node->params.request->uniq,
 						 node->params.request->seqno,
-						 node->status,
+						 i915_qe_state_str(node),
 						 ring->name);
 
 			i915_scheduler_file_queue_dec(node->params.file);
drivers/gpu/drm/i915/i915_scheduler.h +14 -0
index 5de0506..1bf2fa6 100644
@@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
 	/* Limit value for use with arrays/loops */
 	i915_sqs_MAX
 };
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
+const char *i915_scheduler_queue_status_str(
+				enum i915_scheduler_queue_status status);
 
 #define I915_SQS_IS_QUEUED(node)	(((node)->status == i915_sqs_queued))
 #define I915_SQS_IS_FLYING(node)	(((node)->status == i915_sqs_flying))
@@ -71,6 +74,7 @@ struct i915_scheduler_queue_entry {
 	struct list_head                    link;
 	uint32_t                            flags;
 };
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
 
 struct i915_scheduler {
 	struct list_head    node_queue[I915_NUM_RINGS];
@@ -87,9 +91,16 @@ struct i915_scheduler {
 
 /* Flag bits for i915_scheduler::flags */
 enum {
+	/* Internal state */
 	i915_sf_interrupts_enabled  = (1 << 0),
 	i915_sf_submitting          = (1 << 1),
+
+	/* Dump/debug flags */
+	i915_sf_dump_force          = (1 << 8),
+	i915_sf_dump_details        = (1 << 9),
+	i915_sf_dump_dependencies   = (1 << 10),
 };
+const char *i915_scheduler_flag_str(uint32_t flags);
 
 /* Options for 'scheduler_override' module parameter: */
 enum {
@@ -109,6 +120,9 @@ void        i915_gem_scheduler_work_handler(struct work_struct *work);
 int         i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
 int         i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
 				       unsigned long stamp, bool is_locked);
+int         i915_scheduler_dump(struct intel_engine_cs *ring,
+				const char *msg);
+int         i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 					      bool *completed, bool *busy);
 bool        i915_scheduler_file_queue_is_full(struct drm_file *file);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#47 lore raw

PATCH 31/39 drm/i915: Add early exit to execbuff_final() if insufficient ring space

In reply to [email protected]

From: John Harrison <[email protected]>

One of the major purposes of the GPU scheduler is to avoid stalling the CPU when the GPU is busy and unable to accept more work. This change adds support to the ring submission code to allow a ring space check to be performed before attempting to submit a batch buffer to the hardware. If insufficient space is available then the scheduler can go away and come back later, letting the CPU get on with other work, rather than stalling and waiting for the hardware to catch up.

Change-Id: I267159ce1150cb6714d34a49b841bcbe4bf66326
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 42 ++++++++++++++++------
 drivers/gpu/drm/i915/intel_lrc.c           | 57 +++++++++++++++++++++++++++---
 drivers/gpu/drm/i915/intel_ringbuffer.c    | 24 +++++++++++++
 drivers/gpu/drm/i915/intel_ringbuffer.h    |  1 +
 4 files changed, 109 insertions(+), 15 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +31 -11
index 8aca374..ca57147 100644
@@ -1101,25 +1101,19 @@ i915_reset_gen7_sol_offsets(struct drm_device *dev,
 {
 	struct intel_engine_cs *ring = req->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
-	int ret, i;
+	int i;
 
 	if (!IS_GEN7(dev) || ring != &dev_priv->ring[RCS]) {
 		DRM_DEBUG("sol reset is gen7/rcs only\n");
 		return -EINVAL;
 	}
 
-	ret = intel_ring_begin(req, 4 * 3);
-	if (ret)
-		return ret;
-
 	for (i = 0; i < 4; i++) {
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
 		intel_ring_emit(ring, GEN7_SO_WRITE_OFFSET(i));
 		intel_ring_emit(ring, 0);
 	}
 
-	intel_ring_advance(ring);
-
 	return 0;
 }
 
@@ -1246,6 +1240,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	struct intel_engine_cs  *ring = params->ring;
 	u64 exec_start, exec_len;
 	int ret;
+	uint32_t min_space;
 
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
@@ -1267,8 +1262,36 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	if (ret)
 		return ret;
 
+	/*
+	 * It would be a bad idea to run out of space while writing commands
+	 * to the ring. One of the major aims of the scheduler is to not stall
+	 * at any point for any reason. However, doing an early exit half way
+	 * through submission could result in a partial sequence being written
+	 * which would leave the engine in an unknown state. Therefore, check in
+	 * advance that there will be enough space for the entire submission
+	 * whether emitted by the code below OR by any other functions that may
+	 * be executed before the end of final().
+	 *
+	 * NB: This test deliberately overestimates, because that's easier than
+	 * tracing every potential path that could be taken!
+	 *
+	 * Current measurements suggest that we may need to emit up to 744 bytes
+	 * (186 dwords), so this is rounded up to 256 dwords here. Then we double
+	 * that to get the free space requirement, because the block isn't allowed
+	 * to span the transition from the end to the beginning of the ring.
+	 */
+#define I915_BATCH_EXEC_MAX_LEN         256	/* max dwords emitted here	*/
+	min_space = I915_BATCH_EXEC_MAX_LEN * 2 * sizeof(uint32_t);
+	ret = intel_ring_test_space(params->request->ringbuf, min_space);
+	if (ret)
+		goto early_error;
+
 	intel_runtime_pm_get(dev_priv);
 
+	ret = intel_ring_begin(params->request, I915_BATCH_EXEC_MAX_LEN);
+	if (ret)
+		goto error;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
@@ -1287,10 +1310,6 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_ring_begin(params->request, 4);
-		if (ret)
-			goto error;
-
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
 		intel_ring_emit(ring, INSTPM);
@@ -1327,6 +1346,7 @@ error:
 	 */
 	intel_runtime_pm_put(dev_priv);
 
+early_error:
 	if (ret)
 		intel_ring_reserved_space_cancel(params->request->ringbuf);
 
drivers/gpu/drm/i915/intel_lrc.c +53 -4

81 lines not on this page · raw message

drivers/gpu/drm/i915/intel_ringbuffer.c +24 -0
index 2d8dc54..97b9dfa 100644
@@ -2472,6 +2472,30 @@ int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
 	return 0;
 }
 
+/* Test to see if the ring has sufficient space to submit a given piece of work
+ * without causing a stall */
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space)
+{
+	struct drm_i915_private *dev_priv = ringbuf->ring->dev->dev_private;
+
+	/* There is a separate LRC version of this code. */
+	BUG_ON(i915.enable_execlists);
+
+	if (ringbuf->space < min_space) {
+		/* Need to update the actual ring space. Otherwise, the system
+		 * hangs forever testing a software copy of the space value that
+		 * never changes!
+		 */
+		ringbuf->head  = I915_READ_HEAD(ringbuf->ring);
+		ringbuf->space = intel_ring_space(ringbuf);
+
+		if (ringbuf->space < min_space)
+			return -EAGAIN;
+	}
+
+	return 0;
+}
+
 void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno)
 {
 	struct drm_device *dev = ring->dev;
drivers/gpu/drm/i915/intel_ringbuffer.h +1 -0
index d31c94f..c472f8a 100644
@@ -469,6 +469,7 @@ void intel_cleanup_ring_buffer(struct intel_engine_cs *ring);
 
 int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
 
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space);
 int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
 int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
 static inline void intel_ring_emit(struct intel_engine_cs *ring,
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#48 lore raw

PATCH 32/40 drm/i915: Add early exit to execbuff_final() if insufficient ring space

From: John Harrison <[email protected]>

One of the major purposes of the GPU scheduler is to avoid stalling the CPU when the GPU is busy and unable to accept more work. This change adds support to the ring submission code to allow a ring space check to be performed before attempting to submit a batch buffer to the hardware. If insufficient space is available then the scheduler can go away and come back later, letting the CPU get on with other work, rather than stalling and waiting for the hardware to catch up.

v3: Updated to use locally cached request pointer.

Change-Id: I267159ce1150cb6714d34a49b841bcbe4bf66326
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 42 ++++++++++++++++------
 drivers/gpu/drm/i915/intel_lrc.c           | 57 +++++++++++++++++++++++++++---
 drivers/gpu/drm/i915/intel_ringbuffer.c    | 24 +++++++++++++
 drivers/gpu/drm/i915/intel_ringbuffer.h    |  1 +
 4 files changed, 109 insertions(+), 15 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +31 -11
index 8ba426f..bf9d804 100644
@@ -1101,25 +1101,19 @@ i915_reset_gen7_sol_offsets(struct drm_device *dev,
 {
 	struct intel_engine_cs *ring = req->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
-	int ret, i;
+	int i;
 
 	if (!IS_GEN7(dev) || ring != &dev_priv->ring[RCS]) {
 		DRM_DEBUG("sol reset is gen7/rcs only\n");
 		return -EINVAL;
 	}
 
-	ret = intel_ring_begin(req, 4 * 3);
-	if (ret)
-		return ret;
-
 	for (i = 0; i < 4; i++) {
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
 		intel_ring_emit(ring, GEN7_SO_WRITE_OFFSET(i));
 		intel_ring_emit(ring, 0);
 	}
 
-	intel_ring_advance(ring);
-
 	return 0;
 }
 
@@ -1247,6 +1241,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	struct intel_engine_cs  *ring = params->ring;
 	u64 exec_start, exec_len;
 	int ret;
+	uint32_t min_space;
 
 	/* The mutex must be acquired before calling this function */
 	BUG_ON(!mutex_is_locked(&params->dev->struct_mutex));
@@ -1268,8 +1263,36 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	if (ret)
 		return ret;
 
+	/*
+	 * It would be a bad idea to run out of space while writing commands
+	 * to the ring. One of the major aims of the scheduler is to not stall
+	 * at any point for any reason. However, doing an early exit half way
+	 * through submission could result in a partial sequence being written
+	 * which would leave the engine in an unknown state. Therefore, check in
+	 * advance that there will be enough space for the entire submission
+	 * whether emitted by the code below OR by any other functions that may
+	 * be executed before the end of final().
+	 *
+	 * NB: This test deliberately overestimates, because that's easier than
+	 * tracing every potential path that could be taken!
+	 *
+	 * Current measurements suggest that we may need to emit up to 744 bytes
+	 * (186 dwords), so this is rounded up to 256 dwords here. Then we double
+	 * that to get the free space requirement, because the block isn't allowed
+	 * to span the transition from the end to the beginning of the ring.
+	 */
+#define I915_BATCH_EXEC_MAX_LEN         256	/* max dwords emitted here	*/
+	min_space = I915_BATCH_EXEC_MAX_LEN * 2 * sizeof(uint32_t);
+	ret = intel_ring_test_space(req->ringbuf, min_space);
+	if (ret)
+		goto early_error;
+
 	intel_runtime_pm_get(dev_priv);
 
+	ret = intel_ring_begin(req, I915_BATCH_EXEC_MAX_LEN);
+	if (ret)
+		goto error;
+
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
@@ -1288,10 +1311,6 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_ring_begin(req, 4);
-		if (ret)
-			goto error;
-
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
 		intel_ring_emit(ring, INSTPM);
@@ -1328,6 +1347,7 @@ error:
 	 */
 	intel_runtime_pm_put(dev_priv);
 
+early_error:
 	if (ret)
 		intel_ring_reserved_space_cancel(req->ringbuf);
 
drivers/gpu/drm/i915/intel_lrc.c +53 -4

81 lines not on this page · raw message

drivers/gpu/drm/i915/intel_ringbuffer.c +24 -0
index 2d8dc54..97b9dfa 100644
@@ -2472,6 +2472,30 @@ int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
 	return 0;
 }
 
+/* Test to see if the ring has sufficient space to submit a given piece of work
+ * without causing a stall */
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space)
+{
+	struct drm_i915_private *dev_priv = ringbuf->ring->dev->dev_private;
+
+	/* There is a separate LRC version of this code. */
+	BUG_ON(i915.enable_execlists);
+
+	if (ringbuf->space < min_space) {
+		/* Need to update the actual ring space. Otherwise, the system
+		 * hangs forever testing a software copy of the space value that
+		 * never changes!
+		 */
+		ringbuf->head  = I915_READ_HEAD(ringbuf->ring);
+		ringbuf->space = intel_ring_space(ringbuf);
+
+		if (ringbuf->space < min_space)
+			return -EAGAIN;
+	}
+
+	return 0;
+}
+
 void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno)
 {
 	struct drm_device *dev = ring->dev;
drivers/gpu/drm/i915/intel_ringbuffer.h +1 -0
index d31c94f..c472f8a 100644
@@ -469,6 +469,7 @@ void intel_cleanup_ring_buffer(struct intel_engine_cs *ring);
 
 int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
 
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space);
 int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
 int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
 static inline void intel_ring_emit(struct intel_engine_cs *ring,
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#49 lore raw

PATCH 32/39 drm/i915: Added scheduler statistic reporting to debugfs

In reply to [email protected]

From: John Harrison <[email protected]>

It is useful for know what the scheduler is doing for both debugging and performance analysis purposes. This change adds a bunch of counters and such that keep track of various scheduler operations (batches submitted, completed, flush requests, etc.). The data can then be read in userland via the debugfs mechanism.

v2: Updated to match changes to scheduler implementation.

Change-Id: I3266c631cd70c9eeb2c235f88f493e60462f85d7
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c        | 77 ++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 11 +++-
 drivers/gpu/drm/i915/i915_scheduler.c      | 87 +++++++++++++++++++++++++++---
 drivers/gpu/drm/i915/i915_scheduler.h      | 36 +++++++++++++
 4 files changed, 203 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/i915_debugfs.c +77 -0
index 8f1c10c..9e7d67d 100644
@@ -3603,6 +3603,82 @@ static int i915_drrs_status(struct seq_file *m, void *unused)
 	return 0;
 }
 
+static int i915_scheduler_info(struct seq_file *m, void *unused)
+{
+	struct drm_info_node *node = (struct drm_info_node *) m->private;
+	struct drm_device *dev = node->minor->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_stats *stats = scheduler->stats;
+	struct i915_scheduler_stats_nodes node_stats[I915_NUM_RINGS];
+	struct intel_engine_cs *ring;
+	char   str[50 * (I915_NUM_RINGS + 1)], name[50], *ptr;
+	int ret, i, r;
+
+	ret = mutex_lock_interruptible(&dev->mode_config.mutex);
+	if (ret)
+		return ret;
+
+#define PRINT_VAR(name, fmt, var)					\
+	do {								\
+		sprintf(str, "%-22s", name);				\
+		ptr = str + strlen(str);				\
+		for_each_ring(ring, dev_priv, r) {			\
+			sprintf(ptr, " %10" fmt, var);			\
+			ptr += strlen(ptr);				\
+		}							\
+		seq_printf(m, "%s\n", str);				\
+	} while (0)
+
+	PRINT_VAR("Ring name:",             "s", dev_priv->ring[r].name);
+	PRINT_VAR("  Ring seqno",           "d", ring->get_seqno(ring, false));
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Batch submissions:\n");
+	PRINT_VAR("  Queued",               "u", stats[r].queued);
+	PRINT_VAR("  Submitted",            "u", stats[r].submitted);
+	PRINT_VAR("  Completed",            "u", stats[r].completed);
+	PRINT_VAR("  Expired",              "u", stats[r].expired);
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Flush counts:\n");
+	PRINT_VAR("  By object",            "u", stats[r].flush_obj);
+	PRINT_VAR("  By request",           "u", stats[r].flush_req);
+	PRINT_VAR("  By stamp",             "u", stats[r].flush_stamp);
+	PRINT_VAR("  Blanket",              "u", stats[r].flush_all);
+	PRINT_VAR("  Entries bumped",       "u", stats[r].flush_bump);
+	PRINT_VAR("  Entries submitted",    "u", stats[r].flush_submit);
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Miscellaneous:\n");
+	PRINT_VAR("  ExecEarly retry",      "u", stats[r].exec_early);
+	PRINT_VAR("  ExecFinal requeue",    "u", stats[r].exec_again);
+	PRINT_VAR("  ExecFinal killed",     "u", stats[r].exec_dead);
+	PRINT_VAR("  Fence wait",           "u", stats[r].fence_wait);
+	PRINT_VAR("  Fence wait again",     "u", stats[r].fence_again);
+	PRINT_VAR("  Fence wait ignore",    "u", stats[r].fence_ignore);
+	PRINT_VAR("  Fence supplied",       "u", stats[r].fence_got);
+	PRINT_VAR("  Hung flying",          "u", stats[r].kill_flying);
+	PRINT_VAR("  Hung queued",          "u", stats[r].kill_queued);
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Queue contents:\n");
+	for_each_ring(ring, dev_priv, i)
+		i915_scheduler_query_stats(ring, node_stats + ring->id);
+
+	for (i = 0; i < (i915_sqs_MAX + 1); i++) {
+		sprintf(name, "  %s", i915_scheduler_queue_status_str(i));
+		PRINT_VAR(name, "d", node_stats[r].counts[i]);
+	}
+	seq_putc(m, '\n');
+
+#undef PRINT_VAR
+
+	mutex_unlock(&dev->mode_config.mutex);
+
+	return 0;
+}
+
 struct pipe_crc_info {
 	const char *name;
 	struct drm_device *dev;
@@ -5571,6 +5647,7 @@ static const struct drm_info_list i915_debugfs_list[] = {
 	{"i915_semaphore_status", i915_semaphore_status, 0},
 	{"i915_shared_dplls_info", i915_shared_dplls_info, 0},
 	{"i915_dp_mst_info", i915_dp_mst_info, 0},
+	{"i915_scheduler_info", i915_scheduler_info, 0},
 	{"i915_wa_registers", i915_wa_registers, 0},
 	{"i915_ddb_info", i915_ddb_info, 0},
 	{"i915_sseu_status", i915_sseu_status, 0},
drivers/gpu/drm/i915/i915_gem_execbuffer.c +10 -1
index ca57147..dfd9c29 100644
@@ -1422,13 +1422,20 @@ static int i915_early_fence_wait(struct intel_engine_cs *ring, int fence_fd)
 	}
 
 	if (!sync_fence_is_signaled(fence)) {
+		struct drm_i915_private *dev_priv = ring->dev->dev_private;
+		struct i915_scheduler *scheduler = dev_priv->scheduler;
+
 		/*
 		 * Wait forever for the fence to be signalled. This is safe
 		 * because the the mutex lock has not yet been acquired and
 		 * the wait is interruptible.
 		 */
-		if (!i915_safe_to_ignore_fence(ring, fence))
+		if (i915_safe_to_ignore_fence(ring, fence))
+			scheduler->stats[ring->id].fence_ignore++;
+		else {
+			scheduler->stats[ring->id].fence_wait++;
 			ret = sync_fence_wait(fence, -1);
+		}
 	}
 
 	sync_fence_put(fence);
@@ -1844,6 +1851,8 @@ err:
 	mutex_unlock(&dev->struct_mutex);
 
 pre_mutex_err:
+	dev_priv->scheduler->stats[ring->id].exec_early++;
+
 	return ret;
 }
 
drivers/gpu/drm/i915/i915_scheduler.c +80 -7

248 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +36 -0
index b5518b5..ea70a3f 100644
@@ -75,6 +75,37 @@ struct i915_scheduler_queue_entry {
 };
 const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
 
+struct i915_scheduler_stats_nodes {
+	uint32_t	counts[i915_sqs_MAX + 1];
+};
+
+struct i915_scheduler_stats {
+	/* Batch buffer counts: */
+	uint32_t            queued;
+	uint32_t            submitted;
+	uint32_t            completed;
+	uint32_t            expired;
+
+	/* Other stuff: */
+	uint32_t            flush_obj;
+	uint32_t            flush_req;
+	uint32_t            flush_stamp;
+	uint32_t            flush_all;
+	uint32_t            flush_bump;
+	uint32_t            flush_submit;
+
+	uint32_t            exec_early;
+	uint32_t            exec_again;
+	uint32_t            exec_dead;
+	uint32_t            kill_flying;
+	uint32_t            kill_queued;
+
+	uint32_t            fence_wait;
+	uint32_t            fence_again;
+	uint32_t            fence_ignore;
+	uint32_t            fence_got;
+};
+
 struct i915_scheduler {
 	struct list_head    node_queue[I915_NUM_RINGS];
 	uint32_t            flags[I915_NUM_RINGS];
@@ -86,6 +117,9 @@ struct i915_scheduler {
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
 	uint32_t            file_queue_max;
+
+	/* Statistics: */
+	struct i915_scheduler_stats     stats[I915_NUM_RINGS];
 };
 
 /* Flag bits for i915_scheduler::flags */
@@ -126,6 +160,8 @@ int         i915_scheduler_dump(struct intel_engine_cs *ring,
 int         i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 					      bool *completed, bool *busy);
+int         i915_scheduler_query_stats(struct intel_engine_cs *ring,
+				       struct i915_scheduler_stats_nodes *stats);
 bool        i915_scheduler_file_queue_is_full(struct drm_file *file);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#50 lore raw

PATCH 33/40 drm/i915: Added scheduler statistic reporting to debugfs

From: John Harrison <[email protected]>

It is useful for know what the scheduler is doing for both debugging and performance analysis purposes. This change adds a bunch of counters and such that keep track of various scheduler operations (batches submitted, completed, flush requests, etc.). The data can then be read in userland via the debugfs mechanism.

v2: Updated to match changes to scheduler implementation.

v3: Updated for changes to kill code and flush code.

Change-Id: I3266c631cd70c9eeb2c235f88f493e60462f85d7
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c        | 77 +++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 11 +++-
 drivers/gpu/drm/i915/i915_scheduler.c      | 85 +++++++++++++++++++++++++++---
 drivers/gpu/drm/i915/i915_scheduler.h      | 36 +++++++++++++
 4 files changed, 200 insertions(+), 9 deletions(-)
drivers/gpu/drm/i915/i915_debugfs.c +77 -0
index 8f1c10c..9e7d67d 100644
@@ -3603,6 +3603,82 @@ static int i915_drrs_status(struct seq_file *m, void *unused)
 	return 0;
 }
 
+static int i915_scheduler_info(struct seq_file *m, void *unused)
+{
+	struct drm_info_node *node = (struct drm_info_node *) m->private;
+	struct drm_device *dev = node->minor->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_stats *stats = scheduler->stats;
+	struct i915_scheduler_stats_nodes node_stats[I915_NUM_RINGS];
+	struct intel_engine_cs *ring;
+	char   str[50 * (I915_NUM_RINGS + 1)], name[50], *ptr;
+	int ret, i, r;
+
+	ret = mutex_lock_interruptible(&dev->mode_config.mutex);
+	if (ret)
+		return ret;
+
+#define PRINT_VAR(name, fmt, var)					\
+	do {								\
+		sprintf(str, "%-22s", name);				\
+		ptr = str + strlen(str);				\
+		for_each_ring(ring, dev_priv, r) {			\
+			sprintf(ptr, " %10" fmt, var);			\
+			ptr += strlen(ptr);				\
+		}							\
+		seq_printf(m, "%s\n", str);				\
+	} while (0)
+
+	PRINT_VAR("Ring name:",             "s", dev_priv->ring[r].name);
+	PRINT_VAR("  Ring seqno",           "d", ring->get_seqno(ring, false));
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Batch submissions:\n");
+	PRINT_VAR("  Queued",               "u", stats[r].queued);
+	PRINT_VAR("  Submitted",            "u", stats[r].submitted);
+	PRINT_VAR("  Completed",            "u", stats[r].completed);
+	PRINT_VAR("  Expired",              "u", stats[r].expired);
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Flush counts:\n");
+	PRINT_VAR("  By object",            "u", stats[r].flush_obj);
+	PRINT_VAR("  By request",           "u", stats[r].flush_req);
+	PRINT_VAR("  By stamp",             "u", stats[r].flush_stamp);
+	PRINT_VAR("  Blanket",              "u", stats[r].flush_all);
+	PRINT_VAR("  Entries bumped",       "u", stats[r].flush_bump);
+	PRINT_VAR("  Entries submitted",    "u", stats[r].flush_submit);
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Miscellaneous:\n");
+	PRINT_VAR("  ExecEarly retry",      "u", stats[r].exec_early);
+	PRINT_VAR("  ExecFinal requeue",    "u", stats[r].exec_again);
+	PRINT_VAR("  ExecFinal killed",     "u", stats[r].exec_dead);
+	PRINT_VAR("  Fence wait",           "u", stats[r].fence_wait);
+	PRINT_VAR("  Fence wait again",     "u", stats[r].fence_again);
+	PRINT_VAR("  Fence wait ignore",    "u", stats[r].fence_ignore);
+	PRINT_VAR("  Fence supplied",       "u", stats[r].fence_got);
+	PRINT_VAR("  Hung flying",          "u", stats[r].kill_flying);
+	PRINT_VAR("  Hung queued",          "u", stats[r].kill_queued);
+	seq_putc(m, '\n');
+
+	seq_puts(m, "Queue contents:\n");
+	for_each_ring(ring, dev_priv, i)
+		i915_scheduler_query_stats(ring, node_stats + ring->id);
+
+	for (i = 0; i < (i915_sqs_MAX + 1); i++) {
+		sprintf(name, "  %s", i915_scheduler_queue_status_str(i));
+		PRINT_VAR(name, "d", node_stats[r].counts[i]);
+	}
+	seq_putc(m, '\n');
+
+#undef PRINT_VAR
+
+	mutex_unlock(&dev->mode_config.mutex);
+
+	return 0;
+}
+
 struct pipe_crc_info {
 	const char *name;
 	struct drm_device *dev;
@@ -5571,6 +5647,7 @@ static const struct drm_info_list i915_debugfs_list[] = {
 	{"i915_semaphore_status", i915_semaphore_status, 0},
 	{"i915_shared_dplls_info", i915_shared_dplls_info, 0},
 	{"i915_dp_mst_info", i915_dp_mst_info, 0},
+	{"i915_scheduler_info", i915_scheduler_info, 0},
 	{"i915_wa_registers", i915_wa_registers, 0},
 	{"i915_ddb_info", i915_ddb_info, 0},
 	{"i915_sseu_status", i915_sseu_status, 0},
drivers/gpu/drm/i915/i915_gem_execbuffer.c +10 -1
index bf9d804..c40ceea 100644
@@ -1423,13 +1423,20 @@ static int i915_early_fence_wait(struct intel_engine_cs *ring, int fence_fd)
 	}
 
 	if (!sync_fence_is_signaled(fence)) {
+		struct drm_i915_private *dev_priv = ring->dev->dev_private;
+		struct i915_scheduler *scheduler = dev_priv->scheduler;
+
 		/*
 		 * Wait forever for the fence to be signalled. This is safe
 		 * because the the mutex lock has not yet been acquired and
 		 * the wait is interruptible.
 		 */
-		if (!i915_safe_to_ignore_fence(ring, fence))
+		if (i915_safe_to_ignore_fence(ring, fence))
+			scheduler->stats[ring->id].fence_ignore++;
+		else {
+			scheduler->stats[ring->id].fence_wait++;
 			ret = sync_fence_wait(fence, -1);
+		}
 	}
 
 	sync_fence_put(fence);
@@ -1849,6 +1856,8 @@ err:
 	mutex_unlock(&dev->struct_mutex);
 
 pre_mutex_err:
+	dev_priv->scheduler->stats[ring->id].exec_early++;
+
 	return ret;
 }
 
drivers/gpu/drm/i915/i915_scheduler.c +77 -8

229 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h +36 -0
index 1bf2fa6..9396ab4 100644
@@ -76,6 +76,37 @@ struct i915_scheduler_queue_entry {
 };
 const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
 
+struct i915_scheduler_stats_nodes {
+	uint32_t	counts[i915_sqs_MAX + 1];
+};
+
+struct i915_scheduler_stats {
+	/* Batch buffer counts: */
+	uint32_t            queued;
+	uint32_t            submitted;
+	uint32_t            completed;
+	uint32_t            expired;
+
+	/* Other stuff: */
+	uint32_t            flush_obj;
+	uint32_t            flush_req;
+	uint32_t            flush_stamp;
+	uint32_t            flush_all;
+	uint32_t            flush_bump;
+	uint32_t            flush_submit;
+
+	uint32_t            exec_early;
+	uint32_t            exec_again;
+	uint32_t            exec_dead;
+	uint32_t            kill_flying;
+	uint32_t            kill_queued;
+
+	uint32_t            fence_wait;
+	uint32_t            fence_again;
+	uint32_t            fence_ignore;
+	uint32_t            fence_got;
+};
+
 struct i915_scheduler {
 	struct list_head    node_queue[I915_NUM_RINGS];
 	uint32_t            flags[I915_NUM_RINGS];
@@ -87,6 +118,9 @@ struct i915_scheduler {
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
 	uint32_t            file_queue_max;
+
+	/* Statistics: */
+	struct i915_scheduler_stats     stats[I915_NUM_RINGS];
 };
 
 /* Flag bits for i915_scheduler::flags */
@@ -125,6 +159,8 @@ int         i915_scheduler_dump(struct intel_engine_cs *ring,
 int         i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
 bool        i915_scheduler_is_request_tracked(struct drm_i915_gem_request *req,
 					      bool *completed, bool *busy);
+int         i915_scheduler_query_stats(struct intel_engine_cs *ring,
+				       struct i915_scheduler_stats_nodes *stats);
 bool        i915_scheduler_file_queue_is_full(struct drm_file *file);
 
 #endif  /* _I915_SCHEDULER_H_ */
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#51 lore raw

PATCH 33/39 drm/i915: Added seqno values to scheduler status dump

In reply to [email protected]

From: John Harrison <[email protected]>

It is useful to be able to see what seqnos have actually popped out of the hardware when viewing the scheduler status.

Change-Id: Ie93e51c64328be2606b8b43440f6344d5f225426
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_scheduler.c | 10 ++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 2 files changed, 11 insertions(+)
drivers/gpu/drm/i915/i915_scheduler.c +10 -0
index 2acc798..ebb9ab3 100644
@@ -153,6 +153,7 @@ const char *i915_scheduler_flag_str(uint32_t flags)
 	TEST_FLAG(i915_sf_dump_force,         "DumpForce|");
 	TEST_FLAG(i915_sf_dump_details,       "DumpDetails|");
 	TEST_FLAG(i915_sf_dump_dependencies,  "DumpDeps|");
+	TEST_FLAG(i915_sf_dump_seqno,         "DumpSeqno|");
 
 #undef TEST_FLAG
 
@@ -760,6 +761,7 @@ static int i915_scheduler_dump_all_locked(struct drm_device *dev, const char *ms
 	for_each_ring(ring, dev_priv, i) {
 		scheduler->flags[ring->id] |= i915_sf_dump_force   |
 					      i915_sf_dump_details |
+					      i915_sf_dump_seqno   |
 					      i915_sf_dump_dependencies;
 		r = i915_scheduler_dump_locked(ring, msg);
 		if (ret == 0)
@@ -841,6 +843,14 @@ static int i915_scheduler_dump_locked(struct intel_engine_cs *ring, const char *
 		return 0;
 	}
 
+	if (scheduler->flags[ring->id] & i915_sf_dump_seqno) {
+		uint32_t    seqno;
+
+		seqno    = ring->get_seqno(ring, true);
+
+		DRM_DEBUG_DRIVER("<%s> Seqno = %d\n", ring->name, seqno);
+	}
+
 	if (scheduler->flags[ring->id] & i915_sf_dump_details) {
 		int         i, deps;
 		uint32_t    count, counts[i915_sqs_MAX];
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index ea70a3f..46d2503 100644
@@ -132,6 +132,7 @@ enum {
 	i915_sf_dump_force          = (1 << 8),
 	i915_sf_dump_details        = (1 << 9),
 	i915_sf_dump_dependencies   = (1 << 10),
+	i915_sf_dump_seqno          = (1 << 11),
 };
 const char *i915_scheduler_flag_str(uint32_t flags);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#52 lore raw

PATCH 34/39 drm/i915: Add scheduler support functions for TDR

In reply to [email protected]

From: John Harrison <[email protected]>

Change-Id: I720463f01c4edd3579ce52e315a85e4d7874d7e5
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_scheduler.c | 31 +++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 2 files changed, 32 insertions(+)
drivers/gpu/drm/i915/i915_scheduler.c +31 -0
index ebb9ab3..1f1dfa78 100644
@@ -1627,6 +1627,37 @@ int i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
 	return 0;
 }
 
+/*
+ * Used by TDR to distinguish hung rings (not moving but with work to do)
+ * from idle rings (not moving because there is nothing to do).
+ */
+bool i915_scheduler_is_ring_flying(struct intel_engine_cs *ring)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry *node;
+	unsigned long   flags;
+	bool            found = false;
+
+	/* With the scheduler in bypass mode, no information can be returned. */
+	if (i915.scheduler_override & i915_so_direct_submit) {
+		return true;
+	}
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_FLYING(node)) {
+			found = true;
+			break;
+		}
+	}
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	return found;
+}
+
 bool i915_scheduler_file_queue_is_full(struct drm_file *file)
 {
 	struct drm_i915_file_private *file_priv = file->driver_priv;
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index 46d2503..adaefa6 100644
@@ -150,6 +150,7 @@ void        i915_gem_scheduler_clean_node(struct i915_scheduler_queue_entry *nod
 int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
 bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
 void        i915_scheduler_wakeup(struct drm_device *dev);
+bool        i915_scheduler_is_ring_flying(struct intel_engine_cs *ring);
 void        i915_gem_scheduler_work_handler(struct work_struct *work);
 int         i915_scheduler_flush(struct intel_engine_cs *ring, bool is_locked);
 int         i915_scheduler_flush_stamp(struct intel_engine_cs *ring,
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#53 lore raw

PATCH 35/39 drm/i915: GPU priority bumping to prevent starvation

In reply to [email protected]

From: John Harrison <[email protected]>

If a high priority task was to continuously submit batch buffers to the driver, it could starve out any lower priority task from getting any GPU time at all. To prevent this, the priority of a queued batch buffer is bumped each time it does not get submitted to the hardware.

v2: Updated for signed priority values.

Change-Id: I0319c7d2f306c61a283f03edda9b5d09a6d3b621
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c   | 28 ++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.c | 14 ++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  1 +
 3 files changed, 43 insertions(+)
drivers/gpu/drm/i915/i915_debugfs.c +28 -0
index 9e7d67d..66a2202 100644
@@ -1178,6 +1178,33 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
 			"%lld\n");
 
 static int
+i915_scheduler_priority_bump_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = (u64) scheduler->priority_level_bump;
+	return 0;
+}
+
+static int
+i915_scheduler_priority_bump_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->priority_level_bump = (u32) val;
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
+			i915_scheduler_priority_bump_get,
+			i915_scheduler_priority_bump_set,
+			"%lld\n");
+
+static int
 i915_scheduler_priority_preempt_get(void *data, u64 *val)
 {
 	struct drm_device       *dev       = data;
@@ -5672,6 +5699,7 @@ static const struct i915_debugfs_files {
 	{"i915_next_seqno", &i915_next_seqno_fops},
 	{"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
 	{"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+	{"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
 	{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
 	{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
 	{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
drivers/gpu/drm/i915/i915_scheduler.c +14 -0
index 1f1dfa78..9035cdb 100644
@@ -191,6 +191,7 @@ int i915_scheduler_init(struct drm_device *dev)
 	/* Default tuning values: */
 	scheduler->priority_level_min     = -1023;
 	scheduler->priority_level_max     = 1023;
+	scheduler->priority_level_bump    = 50;
 	scheduler->priority_level_preempt = 900;
 	scheduler->min_flying             = 2;
 	scheduler->file_queue_max         = 64;
@@ -1502,6 +1503,19 @@ static int i915_scheduler_submit(struct intel_engine_cs *ring, bool was_locked)
 		ret = i915_scheduler_pop_from_queue_locked(ring, &node, &flags);
 	} while (ret == 0);
 
+	/*
+	 * Bump the priority of everything that was not submitted to prevent
+	 * starvation of low priority tasks by a spamming high priority task.
+	 */
+	i915_scheduler_priority_bump_clear(scheduler);
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+
+		i915_scheduler_priority_bump(scheduler, node,
+					     scheduler->priority_level_bump);
+	}
+
 	spin_unlock_irqrestore(&scheduler->lock, flags);
 
 	if (!was_locked)
drivers/gpu/drm/i915/i915_scheduler.h +1 -0
index adaefa6..6e0dfa9 100644
@@ -114,6 +114,7 @@ struct i915_scheduler {
 	/* Tuning parameters: */
 	int32_t             priority_level_min;
 	int32_t             priority_level_max;
+	int32_t             priority_level_bump;
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
 	uint32_t            file_queue_max;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#54 lore raw

PATCH 36/39 drm/i915: Scheduler state dump via debugfs

In reply to [email protected]

From: John Harrison <[email protected]>

Added a facility for triggering the scheduler state dump via a debugfs entry.

v2: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c   | 33 +++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.c |  9 +++++----
 drivers/gpu/drm/i915/i915_scheduler.h |  6 ++++++
 3 files changed, 44 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/i915_debugfs.c +33 -0
index 66a2202..1c1c4bd 100644
@@ -1285,6 +1285,38 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
 			i915_scheduler_file_queue_max_set,
 			"%llu\n");
 
+static int
+i915_scheduler_dump_flags_get(void *data, u64 *val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	*val = scheduler->dump_flags;
+
+	return 0;
+}
+
+static int
+i915_scheduler_dump_flags_set(void *data, u64 val)
+{
+	struct drm_device       *dev       = data;
+	struct drm_i915_private *dev_priv  = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+	scheduler->dump_flags = lower_32_bits(val) & i915_sf_dump_mask;
+
+	if (val & 1)
+		i915_scheduler_dump_all(dev, "DebugFS");
+
+	return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_dump_flags_fops,
+			i915_scheduler_dump_flags_get,
+			i915_scheduler_dump_flags_set,
+			"0x%llx\n");
+
 static int i915_frequency_info(struct seq_file *m, void *unused)
 {
 	struct drm_info_node *node = m->private;
@@ -5703,6 +5735,7 @@ static const struct i915_debugfs_files {
 	{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
 	{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
 	{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
+	{"i915_scheduler_dump_flags", &i915_scheduler_dump_flags_fops},
 	{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
 	{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
 	{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
drivers/gpu/drm/i915/i915_scheduler.c +5 -4
index 9035cdb..16d067e 100644
@@ -195,6 +195,10 @@ int i915_scheduler_init(struct drm_device *dev)
 	scheduler->priority_level_preempt = 900;
 	scheduler->min_flying             = 2;
 	scheduler->file_queue_max         = 64;
+	scheduler->dump_flags             = i915_sf_dump_force   |
+					    i915_sf_dump_details |
+					    i915_sf_dump_seqno   |
+					    i915_sf_dump_dependencies;
 
 	dev_priv->scheduler = scheduler;
 
@@ -760,10 +764,7 @@ static int i915_scheduler_dump_all_locked(struct drm_device *dev, const char *ms
 	int                     i, r, ret = 0;
 
 	for_each_ring(ring, dev_priv, i) {
-		scheduler->flags[ring->id] |= i915_sf_dump_force   |
-					      i915_sf_dump_details |
-					      i915_sf_dump_seqno   |
-					      i915_sf_dump_dependencies;
+		scheduler->flags[ring->id] |= scheduler->dump_flags & i915_sf_dump_mask;
 		r = i915_scheduler_dump_locked(ring, msg);
 		if (ret == 0)
 			ret = r;
drivers/gpu/drm/i915/i915_scheduler.h +6 -0
index 6e0dfa9..5257f5c 100644
@@ -118,6 +118,7 @@ struct i915_scheduler {
 	int32_t             priority_level_preempt;
 	uint32_t            min_flying;
 	uint32_t            file_queue_max;
+	uint32_t            dump_flags;
 
 	/* Statistics: */
 	struct i915_scheduler_stats     stats[I915_NUM_RINGS];
@@ -134,6 +135,11 @@ enum {
 	i915_sf_dump_details        = (1 << 9),
 	i915_sf_dump_dependencies   = (1 << 10),
 	i915_sf_dump_seqno          = (1 << 11),
+
+	i915_sf_dump_mask           = i915_sf_dump_force        |
+				      i915_sf_dump_details      |
+				      i915_sf_dump_dependencies |
+				      i915_sf_dump_seqno,
 };
 const char *i915_scheduler_flag_str(uint32_t flags);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#55 lore raw

PATCH 37/39 drm/i915: Enable GPU scheduler by default

In reply to [email protected]

From: John Harrison <[email protected]>

Now that all the scheduler patches have been applied, it is safe to enable.

Change-Id: I128042e85a30fca765ce1eb46c837c62dee66089
For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_params.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_params.c +2 -2
index be2e16c..7db0f83 100644
@@ -54,7 +54,7 @@ struct i915_params i915 __read_mostly = {
 	.edp_vswing = 0,
 	.enable_guc_submission = true,
 	.guc_log_level = -1,
-	.scheduler_override = 1,
+	.scheduler_override = 0,
 };
 
 module_param_named(modeset, i915.modeset, int, 0400);
@@ -194,4 +194,4 @@ MODULE_PARM_DESC(guc_log_level,
 	"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
 
 module_param_named(scheduler_override, i915.scheduler_override, int, 0600);
-MODULE_PARM_DESC(scheduler_override, "Scheduler override mask (0 = none, 1 = direct submission [default])");
+MODULE_PARM_DESC(scheduler_override, "Scheduler override mask (default: 0)");
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#56 lore raw

PATCH 38/39 drm/i915: Add scheduling priority to per-context parameters

In reply to [email protected]

From: Dave Gordon <[email protected]>

Added an interface for user land applications/libraries/services to set their GPU scheduler priority. This extends the existing context parameter IOCTL interface to add a scheduler priority parameter. The range is +/-1023 with +ve numbers meaning higher priority. Only system processes may set a higher priority than the default (zero), normal applications may only lower theirs.

v2: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            | 14 ++++++++++++++
 drivers/gpu/drm/i915/i915_gem_context.c    | 24 ++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  3 +++
 include/uapi/drm/i915_drm.h                |  1 +
 4 files changed, 42 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +14 -0
index 4ecb6e4..9de831b 100644
@@ -843,6 +843,19 @@ struct i915_ctx_hang_stats {
 	bool banned;
 };
 
+/*
+ * User-settable GFX scheduler priorities are on a scale of -1023 (I don't
+ * care about running) to +1023 (I'm the most important thing in existence)
+ * with zero being the default. Any process may decrease its scheduling
+ * priority, but only a sufficiently privileged process may increase it
+ * beyond zero.
+ */
+
+struct i915_ctx_sched_info {
+	/* Scheduling priority */
+	int32_t priority;
+};
+
 struct i915_fence_timeline {
 	char        name[32];
 	unsigned    fence_context;
@@ -883,6 +896,7 @@ struct intel_context {
 	int flags;
 	struct drm_i915_file_private *file_priv;
 	struct i915_ctx_hang_stats hang_stats;
+	struct i915_ctx_sched_info sched_info;
 	struct i915_hw_ppgtt *ppgtt;
 
 	/* Legacy ring buffer submission */
drivers/gpu/drm/i915/i915_gem_context.c +24 -0
index 2798ddc..b226e6d 100644
@@ -946,6 +946,9 @@ int i915_gem_context_getparam_ioctl(struct drm_device *dev, void *data,
 		else
 			args->value = to_i915(dev)->gtt.base.total;
 		break;
+	case I915_CONTEXT_PARAM_PRIORITY:
+		args->value = (__u64) ctx->sched_info.priority;
+		break;
 	default:
 		ret = -EINVAL;
 		break;
@@ -983,6 +986,7 @@ int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data,
 		else
 			ctx->hang_stats.ban_period_seconds = args->value;
 		break;
+
 	case I915_CONTEXT_PARAM_NO_ZEROMAP:
 		if (args->size) {
 			ret = -EINVAL;
@@ -991,6 +995,26 @@ int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data,
 			ctx->flags |= args->value ? CONTEXT_NO_ZEROMAP : 0;
 		}
 		break;
+
+	case I915_CONTEXT_PARAM_PRIORITY:
+	{
+		int32_t	priority = (int32_t) args->value;
+		struct drm_i915_private *dev_priv  = dev->dev_private;
+		struct i915_scheduler   *scheduler = dev_priv->scheduler;
+
+		if (args->size)
+			ret = -EINVAL;
+		else if ((priority > scheduler->priority_level_max) ||
+			 (priority < scheduler->priority_level_min))
+			ret = -EINVAL;
+		else if ((priority > 0) &&
+			 !capable(CAP_SYS_ADMIN))
+			ret = -EPERM;
+		else
+			ctx->sched_info.priority = priority;
+		break;
+	}
+
 	default:
 		ret = -EINVAL;
 		break;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +3 -0
index dfd9c29..8553297 100644
@@ -1725,6 +1725,9 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	params->args_DR4                = args->DR4;
 	params->batch_obj               = batch_obj;
 
+	/* Start with the context's priority level */
+	qe.priority = ctx->sched_info.priority;
+
 	/*
 	 * Save away the list of objects used by this batch buffer for the
 	 * purpose of tracking inter-buffer dependencies.
include/uapi/drm/i915_drm.h +1 -0
index 86f7921..5f35f89 100644
@@ -1138,6 +1138,7 @@ struct drm_i915_gem_context_param {
 #define I915_CONTEXT_PARAM_BAN_PERIOD	0x1
 #define I915_CONTEXT_PARAM_NO_ZEROMAP	0x2
 #define I915_CONTEXT_PARAM_GTT_SIZE	0x3
+#define I915_CONTEXT_PARAM_PRIORITY	0x4
 	__u64 value;
 };
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#57 lore raw

PATCH 39/39 drm/i915: Allow scheduler to manage inter-ring object synchronisation

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler has always tracked batch buffer dependencies based on DRM object usage. This means that it will not submit a batch on one ring that has outstanding dependencies still executing on other rings. This is exactly the same synchronisation performed by i915_gem_object_sync() using hardware semaphores where available and CPU stalls where not (e.g. in execlist mode and/or on Gen8 hardware).

Unfortunately, when a batch buffer is submitted to the driver the _object_sync() call happens first. Thus in case where hardware semaphores are disabled, the driver has already stalled until the dependency has been resolved.

This patch adds an optimisation to _object_sync() to ignore the synchronisation in the case where it will subsequently be handled by the scheduler. This removes the driver stall and (in the single application case) provides near hardware semaphore performance even when hardware semaphores are disabled. In a busy system where there is other work that can be executed on the stalling ring, it provides better than hardware semaphore performance as it removes the stall from both the driver and from the hardware. There is also a theory that this method should improve power usage as hardware semaphores are apparently not very power efficient - the stalled ring does not go into as low a power a state as when it is genuinely idle.

The optimisation is to check whether both ends of the synchronisation are batch buffer requests. If they are, then the scheduler will have the inter-dependency tracked and managed. If one or other end is not a batch buffer request (e.g. a page flip) then the code falls back to the CPU stall or hardware semaphore as appropriate.

To check whether the existing usage is a batch buffer, the code simply calls the 'are you tracking this request' function of the scheduler on the object's last_read_req member. To check whether the new usage is a batch buffer, a flag is passed in from the caller.

Change-Id: Idc16e19b5a4dc8b3782ce9db44dd3df445f396c1
Issue: VIZ-5566

---
 drivers/gpu/drm/i915/i915_drv.h            |  2 +-
 drivers/gpu/drm/i915/i915_gem.c            | 17 ++++++++++++++---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  2 +-
 drivers/gpu/drm/i915/intel_display.c       |  2 +-
 drivers/gpu/drm/i915/intel_lrc.c           |  2 +-
 5 files changed, 18 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +1 -1
index 9de831b..809816a 100644
@@ -2971,7 +2971,7 @@ int __must_check i915_mutex_lock_interruptible(struct drm_device *dev);
 #endif
 int i915_gem_object_sync(struct drm_i915_gem_object *obj,
 			 struct intel_engine_cs *to,
-			 struct drm_i915_gem_request **to_req);
+			 struct drm_i915_gem_request **to_req, bool to_batch);
 void i915_vma_move_to_active(struct i915_vma *vma,
 			     struct drm_i915_gem_request *req);
 int i915_gem_dumb_create(struct drm_file *file_priv,
drivers/gpu/drm/i915/i915_gem.c +14 -3
index 9c08035..ab6064e 100644
@@ -3555,7 +3555,7 @@ static int
 __i915_gem_object_sync(struct drm_i915_gem_object *obj,
 		       struct intel_engine_cs *to,
 		       struct drm_i915_gem_request *from_req,
-		       struct drm_i915_gem_request **to_req)
+		       struct drm_i915_gem_request **to_req, bool to_batch)
 {
 	struct intel_engine_cs *from;
 	int ret;
@@ -3567,6 +3567,15 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
 	if (i915_gem_request_completed(from_req))
 		return 0;
 
+	/*
+	 * The scheduler will manage inter-ring object dependencies
+	 * as long as both to and from requests are scheduler managed
+	 * (i.e. batch buffers).
+	 */
+	if (to_batch &&
+	    i915_scheduler_is_request_tracked(from_req, NULL, NULL))
+		return 0;
+
 	if (!i915_semaphore_is_enabled(obj->base.dev)) {
 		struct drm_i915_private *i915 = to_i915(obj->base.dev);
 		ret = __i915_wait_request(from_req,
@@ -3617,6 +3626,8 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
  * @to_req: request we wish to use the object for. See below.
  *          This will be allocated and returned if a request is
  *          required but not passed in.
+ * @to_batch: is the sync request on behalf of batch buffer submission?
+ * If so then the scheduler can (potentially) manage the synchronisation.
  *
  * This code is meant to abstract object synchronization with the GPU.
  * Calling with NULL implies synchronizing the object with the CPU
@@ -3647,7 +3658,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
 int
 i915_gem_object_sync(struct drm_i915_gem_object *obj,
 		     struct intel_engine_cs *to,
-		     struct drm_i915_gem_request **to_req)
+		     struct drm_i915_gem_request **to_req, bool to_batch)
 {
 	const bool readonly = obj->base.pending_write_domain == 0;
 	struct drm_i915_gem_request *req[I915_NUM_RINGS];
@@ -3669,7 +3680,7 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
 				req[n++] = obj->last_read_req[i];
 	}
 	for (i = 0; i < n; i++) {
-		ret = __i915_gem_object_sync(obj, to, req[i], to_req);
+		ret = __i915_gem_object_sync(obj, to, req[i], to_req, to_batch);
 		if (ret)
 			return ret;
 	}
drivers/gpu/drm/i915/i915_gem_execbuffer.c +1 -1
index 8553297..20693c3 100644
@@ -919,7 +919,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
 		struct drm_i915_gem_object *obj = vma->obj;
 
 		if (obj->active & other_rings) {
-			ret = i915_gem_object_sync(obj, req->ring, &req);
+			ret = i915_gem_object_sync(obj, req->ring, &req, true);
 			if (ret)
 				return ret;
 		}
drivers/gpu/drm/i915/intel_display.c +1 -1
index 58c464f..3612562 100644
@@ -11487,7 +11487,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
 	 * into the display plane and skip any waits.
 	 */
 	if (!mmio_flip) {
-		ret = i915_gem_object_sync(obj, ring, &request);
+		ret = i915_gem_object_sync(obj, ring, &request, false);
 		if (ret)
 			goto cleanup_pending;
 	}
drivers/gpu/drm/i915/intel_lrc.c +1 -1
index b5abea4..7a3fb219 100644
@@ -658,7 +658,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
 		struct drm_i915_gem_object *obj = vma->obj;
 
 		if (obj->active & other_rings) {
-			ret = i915_gem_object_sync(obj, req->ring, &req);
+			ret = i915_gem_object_sync(obj, req->ring, &req, true);
 			if (ret)
 				return ret;
 		}
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#58 lore raw

PATCH 6/40 drm/i915: Cache request pointer in *_submission_final()

In reply to [email protected]

From: Dave Gordon <[email protected]>

Keep a local copy of the request pointer in the _final() functions rather than dereferencing the params block repeatedly.

v3: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++------
 drivers/gpu/drm/i915/intel_lrc.c           | 11 ++++++-----
 2 files changed, 13 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -6
index 05c9de6..e38310f 100644
@@ -1245,6 +1245,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
+	struct drm_i915_gem_request *req = params->request;
 	struct intel_engine_cs  *ring = params->ring;
 	u64 exec_start, exec_len;
 	int ret;
@@ -1258,12 +1259,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = intel_ring_invalidate_all_caches(params->request);
+	ret = intel_ring_invalidate_all_caches(req);
 	if (ret)
 		goto error;
 
 	/* Switch to the correct context for the batch */
-	ret = i915_switch_context(params->request);
+	ret = i915_switch_context(req);
 	if (ret)
 		goto error;
 
@@ -1272,7 +1273,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_ring_begin(params->request, 4);
+		ret = intel_ring_begin(req, 4);
 		if (ret)
 			goto error;
 
@@ -1286,7 +1287,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	}
 
 	if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
-		ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
+		ret = i915_reset_gen7_sol_offsets(params->dev, req);
 		if (ret)
 			goto error;
 	}
@@ -1295,13 +1296,13 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 
-	ret = ring->dispatch_execbuffer(params->request,
+	ret = ring->dispatch_execbuffer(req,
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
 		goto error;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
drivers/gpu/drm/i915/intel_lrc.c +6 -5
index 88d57b7..b98ea3d 100644
@@ -929,7 +929,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
-	struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+	struct drm_i915_gem_request *req = params->request;
+	struct intel_ringbuffer *ringbuf = req->ringbuf;
 	struct intel_engine_cs *ring = params->ring;
 	u64 exec_start;
 	int ret;
@@ -941,13 +942,13 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = logical_ring_invalidate_all_caches(params->request);
+	ret = logical_ring_invalidate_all_caches(req);
 	if (ret)
 		return ret;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_logical_ring_begin(params->request, 4);
+		ret = intel_logical_ring_begin(req, 4);
 		if (ret)
 			return ret;
 
@@ -963,11 +964,11 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 
-	ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+	ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
 	if (ret)
 		return ret;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#59 lore raw

cover GPU scheduler for i915 driver

In reply to [email protected]

From: John Harrison <[email protected]>

Implemented a batch buffer submission scheduler for the i915 DRM driver.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code assigns a unique seqno value and then packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

As each batch buffer completes, it raises an interrupt which wakes up the scheduler. Note that it is possible for multiple buffers to complete before the IRQ handler gets to run. Further, the seqno values of the individual buffers are not necessary incrementing as the scheduler may have re-ordered their submission. However, the scheduler keeps the list of executing buffers in order of hardware submission. Thus it can scan through the list until a matching seqno is found and then mark all in flight nodes from that point on as completed.

A deferred work queue is also poked by the interrupt handler. When this wakes up it can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.

The scheduler also allows high priority batch buffers (e.g. from a desktop compositor) to jump ahead of whatever is already running if the underlying hardware supports pre-emption. In this situation, any work that was pre-empted is returned to the queued list ready to be resubmitted when no more high priority work is outstanding.

Various IGT tests are in progress to test the scheduler's operation and will follow.

v2: Updated for changes in struct fence patch series and other changes to underlying tree (e.g. removal of cliprects). Also changed priority levels to be signed +/-1023 range and reduced mutex lock usage.

v3: More reuse of cached pointers rather than repeated dereferencing (David Gordon).

Moved the dependency generation code out to a seperate function for easier readability. Also added in support for the read-read optimisation.

Major simplification of the DRM file close handler.

Fixed up an overzealous WARN.

Removed unnecessary flushing of the scheduler queue when waiting for a request.

[Patches against drm-intel-nightly tree fetched 17/11/2015 with struct fence conversion patches applied]

Dave Gordon (3):
drm/i915: Updating assorted register and status page definitions drm/i915: Cache request pointer in *_submission_final() drm/i915: Add scheduling priority to per-context parameters

John Harrison (37):
drm/i915: Add total count to context status debugfs output drm/i915: Explicit power enable during deferred context initialisation drm/i915: Prelude to splitting i915_gem_do_execbuffer in two drm/i915: Split i915_dem_do_execbuffer() in half
drm/i915: Re-instate request->uniq because it is extremely useful drm/i915: Start of GPU scheduler
drm/i915: Prepare retire_requests to handle out-of-order seqnos drm/i915: Disable hardware semaphores when GPU scheduler is enabled drm/i915: Force MMIO flips when scheduler enabled
drm/i915: Added scheduler hook when closing DRM file handles drm/i915: Added scheduler hook into i915_gem_request_notify() drm/i915: Added deferred work handler for scheduler
drm/i915: Redirect execbuffer_final() via scheduler
drm/i915: Keep the reserved space mechanism happy
drm/i915: Added tracking/locking of batch buffer objects drm/i915: Hook scheduler node clean up into retire requests drm/i915: Added scheduler support to __wait_request() calls drm/i915: Added scheduler support to page fault handler drm/i915: Added scheduler flush calls to ring throttle and idle functions drm/i915: Added a module parameter for allowing scheduler overrides drm/i915: Support for 'unflushed' ring idle
drm/i915: Defer seqno allocation until actual hardware submission time drm/i915: Added immediate submission override to scheduler drm/i915: Add sync wait support to scheduler
drm/i915: Connecting execbuff fences to scheduler
drm/i915: Added trace points to scheduler
drm/i915: Added scheduler queue throttling by DRM file handle drm/i915: Added debugfs interface to scheduler tuning parameters drm/i915: Added debug state dump facilities to scheduler drm/i915: Add early exit to execbuff_final() if insufficient ring space drm/i915: Added scheduler statistic reporting to debugfs drm/i915: Added seqno values to scheduler status dump drm/i915: Add scheduler support functions for TDR
drm/i915: GPU priority bumping to prevent starvation drm/i915: Scheduler state dump via debugfs
drm/i915: Enable GPU scheduler by default
drm/i915: Allow scheduler to manage inter-ring object synchronisation

 drivers/gpu/drm/i915/Makefile              |    1 +
 drivers/gpu/drm/i915/i915_debugfs.c        |  283 +++++
 drivers/gpu/drm/i915/i915_dma.c            |    6 +
 drivers/gpu/drm/i915/i915_drv.c            |    9 +
 drivers/gpu/drm/i915/i915_drv.h            |   56 +-
 drivers/gpu/drm/i915/i915_gem.c            |  170 ++-
 drivers/gpu/drm/i915/i915_gem_context.c    |   24 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  350 ++++--
 drivers/gpu/drm/i915/i915_params.c         |    4 +
 drivers/gpu/drm/i915/i915_reg.h            |   30 +-
 drivers/gpu/drm/i915/i915_scheduler.c      | 1640 ++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  175 +++
 drivers/gpu/drm/i915/i915_trace.h          |  215 +++-
 drivers/gpu/drm/i915/intel_display.c       |   10 +-
 drivers/gpu/drm/i915/intel_lrc.c           |  166 ++-
 drivers/gpu/drm/i915/intel_lrc.h           |    1 +
 drivers/gpu/drm/i915/intel_ringbuffer.c    |   47 +-
 drivers/gpu/drm/i915/intel_ringbuffer.h    |   35 +-
 include/uapi/drm/i915_drm.h                |    1 +
 19 files changed, 3057 insertions(+), 166 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#60 lore raw

v4 cover GPU scheduler for i915 driver

In reply to [email protected]

From: John Harrison <[email protected]>

Implemented a batch buffer submission scheduler for the i915 DRM driver.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code assigns a unique seqno value and then packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

As each batch buffer completes, it raises an interrupt which wakes up the scheduler. Note that it is possible for multiple buffers to complete before the IRQ handler gets to run. Further, the seqno values of the individual buffers are not necessary incrementing as the scheduler may have re-ordered their submission. However, the scheduler keeps the list of executing buffers in order of hardware submission. Thus it can scan through the list until a matching seqno is found and then mark all in flight nodes from that point on as completed.

A deferred work queue is also poked by the interrupt handler. When this wakes up it can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.

The scheduler also allows high priority batch buffers (e.g. from a desktop compositor) to jump ahead of whatever is already running if the underlying hardware supports pre-emption. In this situation, any work that was pre-empted is returned to the queued list ready to be resubmitted when no more high priority work is outstanding.

Various IGT tests are in progress to test the scheduler's operation and will follow.

v2: Updated for changes in struct fence patch series and other changes to underlying tree (e.g. removal of cliprects). Also changed priority levels to be signed +/-1023 range and reduced mutex lock usage.

v3: More reuse of cached pointers rather than repeated dereferencing (David Gordon).

Moved the dependency generation code out to a seperate function for easier readability. Also added in support for the read-read optimisation.

Major simplification of the DRM file close handler.

Fixed up an overzealous WARN.

Removed unnecessary flushing of the scheduler queue when waiting for a request.

v4: Removed user land fence/sync integration as this is dependent upon the de-staging of the Android sync code. That de-staging is now being done by someone else. The sync support will be added back in to the scheduler in a separate patch series which must wait for the de-staging to be landed first.

Added support for killing batches from contexts that were banned after the batches were submitted to the scheduler.

Changed various comments to fix typos, update to reflect changes to the code, correct formatting and line wrapping, etc. Also wrapped various long lines and twiddled white space to keep the style checker happy.

Changed a bunch of BUG_ONs to WARN_ONs as apparently the latter are preferred.

Used the correct array memory allocator function (kmalloc_array instead of kmalloc).

Fixed a variable type (review comment by Joonas).

Fixed a WARN_ON firing incorrectly when removing killed nodes from the scheduler's queue.

Dropped register definition update patch from this series. The changes are all for pre-emption so it makes more sense for it to be part of that series instead.

[Patches against drm-intel-nightly tree fetched 17/11/2015 with struct fence conversion patches applied]

Dave Gordon (2):
drm/i915: Cache request pointer in *_submission_final() drm/i915: Add scheduling priority to per-context parameters

John Harrison (36):
drm/i915: Add total count to context status debugfs output drm/i915: Explicit power enable during deferred context initialisation drm/i915: Prelude to splitting i915_gem_do_execbuffer in two drm/i915: Split i915_dem_do_execbuffer() in half
drm/i915: Re-instate request->uniq because it is extremely useful drm/i915: Start of GPU scheduler
drm/i915: Prepare retire_requests to handle out-of-order seqnos drm/i915: Disable hardware semaphores when GPU scheduler is enabled drm/i915: Force MMIO flips when scheduler enabled
drm/i915: Added scheduler hook when closing DRM file handles drm/i915: Added scheduler hook into i915_gem_request_notify() drm/i915: Added deferred work handler for scheduler
drm/i915: Redirect execbuffer_final() via scheduler
drm/i915: Keep the reserved space mechanism happy
drm/i915: Added tracking/locking of batch buffer objects drm/i915: Hook scheduler node clean up into retire requests drm/i915: Added scheduler support to __wait_request() calls drm/i915: Added scheduler support to page fault handler drm/i915: Added scheduler flush calls to ring throttle and idle functions drm/i915: Added a module parameter for allowing scheduler overrides drm/i915: Support for 'unflushed' ring idle
drm/i915: Defer seqno allocation until actual hardware submission time drm/i915: Added immediate submission override to scheduler drm/i915: Added trace points to scheduler
drm/i915: Added scheduler queue throttling by DRM file handle drm/i915: Added debugfs interface to scheduler tuning parameters drm/i915: Added debug state dump facilities to scheduler drm/i915: Add early exit to execbuff_final() if insufficient ring space drm/i915: Added scheduler statistic reporting to debugfs drm/i915: Added seqno values to scheduler status dump drm/i915: Add scheduler support functions for TDR
drm/i915: GPU priority bumping to prevent starvation drm/i915: Scheduler state dump via debugfs
drm/i915: Enable GPU scheduler by default
drm/i915: Add support for retro-actively banning batch buffers drm/i915: Allow scheduler to manage inter-ring object synchronisation

 drivers/gpu/drm/i915/Makefile              |    1 +
 drivers/gpu/drm/i915/i915_debugfs.c        |  280 ++++-
 drivers/gpu/drm/i915/i915_dma.c            |    6 +
 drivers/gpu/drm/i915/i915_drv.c            |    9 +
 drivers/gpu/drm/i915/i915_drv.h            |   55 +-
 drivers/gpu/drm/i915/i915_gem.c            |  177 +++-
 drivers/gpu/drm/i915/i915_gem_context.c    |   24 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |  301 ++++--
 drivers/gpu/drm/i915/i915_params.c         |    4 +
 drivers/gpu/drm/i915/i915_scheduler.c      | 1530 ++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h      |  164 +++
 drivers/gpu/drm/i915/i915_trace.h          |  218 +++-
 drivers/gpu/drm/i915/intel_display.c       |   10 +-
 drivers/gpu/drm/i915/intel_lrc.c           |  170 +++-
 drivers/gpu/drm/i915/intel_lrc.h           |    1 +
 drivers/gpu/drm/i915/intel_ringbuffer.c    |   49 +-
 drivers/gpu/drm/i915/intel_ringbuffer.h    |    3 +-
 include/uapi/drm/i915_drm.h                |    1 +
 18 files changed, 2856 insertions(+), 147 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#61 lore raw

PATCH v4 1/38 drm/i915: Add total count to context status debugfs output

From: John Harrison <[email protected]>

When there are lots and lots and even more lots of contexts (e.g. when running with execlists) it is useful to be able to immediately see what the total context count is.

v4: Re-typed a variable (review feedback from Joonas)

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_debugfs.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_debugfs.c +4 -1
index b54d99e..e55294c 100644
@@ -1953,7 +1953,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
 	struct drm_i915_private *dev_priv = dev->dev_private;
 	struct intel_engine_cs *ring;
 	struct intel_context *ctx;
-	int ret, i;
+	int ret, i, count = 0;
 
 	ret = mutex_lock_interruptible(&dev->struct_mutex);
 	if (ret)
@@ -1966,6 +1966,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
 
 		seq_puts(m, "HW context ");
 		describe_ctx(m, ctx);
+		count++;
 		for_each_ring(ring, dev_priv, i) {
 			if (ring->default_context == ctx)
 				seq_printf(m, "(default context %s) ",
@@ -1994,6 +1995,8 @@ static int i915_context_status(struct seq_file *m, void *unused)
 		seq_putc(m, '\n');
 	}
 
+	seq_printf(m, "Total: %d contexts\n", count);
+
 	mutex_unlock(&dev->struct_mutex);
 
 	return 0;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#62 lore raw

PATCH v4 2/38 drm/i915: Explicit power enable during deferred context initialisation

In reply to [email protected]

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

While that is a good thing in general, it causes a problem with the deferred context initialisation. That is done quite early on in the execbuf code path - it happens at context validation time rather than context switch time. Some of the deferred work requires the power to be enabled. Hence this patch adds an explicit power reference count to the deferred initialisation code itself.

Issue: GMINL-5159

---
 drivers/gpu/drm/i915/intel_lrc.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/intel_lrc.c +8 -1
index 76fc245..e510730 100644
@@ -2470,6 +2470,8 @@ int intel_lr_context_deferred_alloc(struct intel_context *ctx,
 	WARN_ON(ctx->legacy_hw_ctx.rcs_state != NULL);
 	WARN_ON(ctx->engine[ring->id].state);
 
+	intel_runtime_pm_get(dev->dev_private);
+
 	context_size = round_up(get_lr_context_size(ring), 4096);
 
 	/* One extra page as the sharing data between driver and GuC */
@@ -2478,7 +2480,8 @@ int intel_lr_context_deferred_alloc(struct intel_context *ctx,
 	ctx_obj = i915_gem_alloc_object(dev, context_size);
 	if (!ctx_obj) {
 		DRM_DEBUG_DRIVER("Alloc LRC backing obj failed.\n");
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto error_pm;
 	}
 
 	ringbuf = intel_engine_create_ringbuffer(ring, 4 * PAGE_SIZE);
@@ -2524,6 +2527,8 @@ int intel_lr_context_deferred_alloc(struct intel_context *ctx,
 		}
 		i915_add_request_no_flush(req);
 	}
+
+	intel_runtime_pm_put(dev->dev_private);
 	return 0;
 
 error_ringbuf:
@@ -2532,6 +2537,8 @@ error_deref_obj:
 	drm_gem_object_unreference(&ctx_obj->base);
 	ctx->engine[ring->id].ringbuf = NULL;
 	ctx->engine[ring->id].state = NULL;
+error_pm:
+	intel_runtime_pm_put(dev->dev_private);
 	return ret;
 }
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#63 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 7 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

However, as you haven't fixed the ordering issue that requires rpm_get before struct_mutex, this is broken. When we have rpm fixed, you don't need this patch as we can take the wakeref around the GSM access itself. We still want the prolonged rpm wakeref for the GPU activity, ofc. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#64 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 10 lines · Chris Wilson
Quoted 7 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

However, as you haven't fixed the ordering issue that requires rpm_get before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held? It has certainly not complained at me about trying to do stuff without it.

Quoted 5 lines

When we have rpm fixed, you don't
need this patch as we can take the wakeref around the GSM access itself. We still want the prolonged rpm wakeref for the GPU activity, ofc. -Chris

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#65 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 9 lines · John Harrison
Quoted 8 lines · Chris Wilson
Quoted 7 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

Quoted 5 lines
Quoted 2 lines

However, as you haven't fixed the ordering issue that requires rpm_get before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held? It has certainly not complained at me about trying to do stuff without it.

Because it depends upon the struct_mutex and rpm doesn't have sufficient lockdep integration to be able to warn about using rpm from the incorrect contexts.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#66 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 10 lines · Chris Wilson
Quoted 9 lines · John Harrison
Quoted 8 lines · Chris Wilson
Quoted 7 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

However, as you haven't fixed the ordering issue that requires rpm_get before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held? It has certainly not complained at me about trying to do stuff without it.

Because it depends upon the struct_mutex and rpm doesn't have sufficient lockdep integration to be able to warn about using rpm from the incorrect contexts.

Where? What does the 'pm_runtime_get_sync' call turn into? There are already other places in the driver which call intel_runtime_pm_get() immediately after grabbing the mutex lock. Also, the description comment for _pm_get() does not mention anything about mutexes at all.

Quoted 2 lines

-Chris

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#67 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 16 lines · John Harrison
Quoted 15 lines · Chris Wilson
Quoted 14 lines · John Harrison
Quoted 10 lines · Chris Wilson
Quoted 7 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

However, as you haven't fixed the ordering issue that requires rpm_get before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held? It has certainly not complained at me about trying to do stuff without it.

Quoted 7 lines
Quoted 3 lines

Because it depends upon the struct_mutex and rpm doesn't have sufficient lockdep integration to be able to warn about using rpm from the incorrect contexts.

Where? What does the 'pm_runtime_get_sync' call turn into? There are already other places in the driver which call intel_runtime_pm_get() immediately after grabbing the mutex lock. Also, the description comment for _pm_get() does not mention anything about mutexes at all.

If you nest rpm_get within dev->struct_mutex that's a bug and could deadlock. Where does this happen? And for any such place we need a new subtest in pm_rpm.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#68 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 20 lines · Daniel Vetter
Quoted 19 lines · John Harrison
Quoted 18 lines · Chris Wilson
Quoted 9 lines · John Harrison
Quoted 8 lines · Chris Wilson
Quoted 7 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

Quoted 5 lines
Quoted 2 lines

However, as you haven't fixed the ordering issue that requires rpm_get before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held? It has certainly not complained at me about trying to do stuff without it.

Because it depends upon the struct_mutex and rpm doesn't have sufficient lockdep integration to be able to warn about using rpm from the incorrect contexts.

Quoted 7 lines
Quoted 4 lines

Where? What does the 'pm_runtime_get_sync' call turn into? There are already other places in the driver which call intel_runtime_pm_get() immediately after grabbing the mutex lock. Also, the description comment for _pm_get() does not mention anything about mutexes at all.

If you nest rpm_get within dev->struct_mutex that's a bug and could deadlock. Where does this happen? And for any such place we need a new subtest in pm_rpm.

The first two hits when grepping the driver are in 'i915_gem_seqno_info()' and 'i915_interrupt_info()' in i915_debugfs.c. Both say:

    ret = mutex_lock_interruptible(&dev->struct_mutex);
    if (ret)
        return ret;
    intel_runtime_pm_get(dev_priv);
Quoted 1 line

-Daniel

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#69 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 29 lines · John Harrison
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · John Harrison
Quoted 18 lines · Chris Wilson
Quoted 9 lines · John Harrison
Quoted 8 lines · Chris Wilson
Quoted 7 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission code. Part of that is to reduce the scope of a pm_get/put pair. Specifically, they previously wrapped the entire submission path from the very start to the very end, now they only wrap the actual hardware submission part in the back half.

Quoted 5 lines
Quoted 2 lines

However, as you haven't fixed the ordering issue that requires rpm_get before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held? It has certainly not complained at me about trying to do stuff without it.

Because it depends upon the struct_mutex and rpm doesn't have sufficient lockdep integration to be able to warn about using rpm from the incorrect contexts.

Quoted 7 lines
Quoted 4 lines

Where? What does the 'pm_runtime_get_sync' call turn into? There are already other places in the driver which call intel_runtime_pm_get() immediately after grabbing the mutex lock. Also, the description comment for _pm_get() does not mention anything about mutexes at all.

If you nest rpm_get within dev->struct_mutex that's a bug and could deadlock. Where does this happen? And for any such place we need a new subtest in pm_rpm.

Quoted 6 lines

The first two hits when grepping the driver are in 'i915_gem_seqno_info()' and 'i915_interrupt_info()' in i915_debugfs.c. Both say:

    ret = mutex_lock_interruptible(&dev->struct_mutex);
    if (ret)
        return ret;
    intel_runtime_pm_get(dev_priv);

Yeah that's totally bonkers and will deadlock if the device is actually suspend.

/me goes and files JIRAs

Thanks, Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#70 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 54 lines · Daniel Vetter
Quoted 45 lines · John Harrison
Quoted 38 lines · Daniel Vetter
Quoted 30 lines · John Harrison
Quoted 24 lines · Chris Wilson
Quoted 14 lines · John Harrison
Quoted 13 lines · Chris Wilson
Quoted 11 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, John.C.Harrison @Intel.com wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission
code. Part of that is to reduce the scope of a pm_get/put pair.
Specifically, they previously wrapped the entire submission path from
the very start to the very end, now they only wrap the actual hardware
submission part in the back half.

Quoted 8 lines
Quoted 3 lines

However, as you haven't fixed the ordering issue that requires rpm_get
before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held?
It has certainly not complained at me about trying to do stuff
without it.

Quoted 5 lines

Because it depends upon the struct_mutex and rpm doesn't have sufficient
lockdep integration to be able to warn about using rpm from the
incorrect contexts.

Quoted 7 lines

Where? What does the 'pm_runtime_get_sync' call turn into? There are already
other places in the driver which call intel_runtime_pm_get() immediately
after grabbing the mutex lock. Also, the description comment for _pm_get()
does not mention anything about mutexes at all.

Quoted 5 lines

If you nest rpm_get within dev->struct_mutex that's a bug and could
deadlock. Where does this happen? And for any such place we need a new
subtest in pm_rpm.

Quoted 7 lines

The first two hits when grepping the driver are in 'i915_gem_seqno_info()'
and 'i915_interrupt_info()' in i915_debugfs.c. Both say:
    ret = mutex_lock_interruptible(&dev->struct_mutex); if (ret)
        return ret; intel_runtime_pm_get(dev_priv);

Quoted 3 lines

Yeah that's totally bonkers and will deadlock if the device is actually
suspend.

It won't deadlock, runtime resume doesn't need struct mutex. Runtime suspend needs it, but we return -EAGAIN from there if it's already held. That in turn will delay the suspend.

--Imre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#71 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 55 lines · Imre Deak
Quoted 54 lines · Daniel Vetter
Quoted 52 lines · John Harrison
Quoted 38 lines · Daniel Vetter
Quoted 30 lines · John Harrison
Quoted 24 lines · Chris Wilson
Quoted 14 lines · John Harrison
Quoted 13 lines · Chris Wilson
Quoted 11 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, John.C.Harrison @Intel.com wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission
code. Part of that is to reduce the scope of a pm_get/put pair.
Specifically, they previously wrapped the entire submission path from
the very start to the very end, now they only wrap the actual hardware
submission part in the back half.

Quoted 8 lines
Quoted 3 lines

However, as you haven't fixed the ordering issue that requires rpm_get
before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held?
It has certainly not complained at me about trying to do stuff
without it.

Quoted 5 lines

Because it depends upon the struct_mutex and rpm doesn't have sufficient
lockdep integration to be able to warn about using rpm from the
incorrect contexts.

Quoted 7 lines

Where? What does the 'pm_runtime_get_sync' call turn into? There are already
other places in the driver which call intel_runtime_pm_get() immediately
after grabbing the mutex lock. Also, the description comment for _pm_get()
does not mention anything about mutexes at all.

Quoted 5 lines

If you nest rpm_get within dev->struct_mutex that's a bug and could
deadlock. Where does this happen? And for any such place we need a new
subtest in pm_rpm.

The first two hits when grepping the driver are in 'i915_gem_seqno_info()'
and 'i915_interrupt_info()' in i915_debugfs.c. Both say:
    ret = mutex_lock_interruptible(&dev->struct_mutex); if (ret)
        return ret; intel_runtime_pm_get(dev_priv);

Quoted 7 lines
Quoted 3 lines

Yeah that's totally bonkers and will deadlock if the device is actually
suspend.

It won't deadlock, runtime resume doesn't need struct mutex. Runtime suspend needs it, but we return -EAGAIN from there if it's already held. That in turn will delay the suspend.

You're right, why do I always mix this up. But this just got me thinking about possible lockdep annotations. We could place a might_lock(rpm_lockdep_key) within rpm_get. And fake-acquire that around the actual ->resume callback. We could even do this in the rpm core ...

That would put all my worries on this topic at easy I think, and I could finally stop confusing everyone ;-)

Or did I again miss the obvious?
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#72 lore raw

Re: [PATCH v4 02/38] drm/i915: Explicit power enable during deferred context initialisation

Quoted 60 lines · Daniel Vetter
Quoted 59 lines · Imre Deak
Quoted 57 lines · Daniel Vetter
Quoted 45 lines · John Harrison
Quoted 38 lines · Daniel Vetter
Quoted 30 lines · John Harrison
Quoted 24 lines · Chris Wilson
Quoted 14 lines · John Harrison
Quoted 13 lines · Chris Wilson
Quoted 11 lines · On Mon, Jan 11, 2016 at 06:42:31PM +0000, John.C.Harrison @Intel.com wrote:

From: John Harrison <[email protected]>

A later patch in this series re-organises the batch buffer submission
code. Part of that is to reduce the scope of a pm_get/put pair.
Specifically, they previously wrapped the entire submission path from
the very start to the very end, now they only wrap the actual hardware
submission part in the back half.

Quoted 8 lines
Quoted 3 lines

However, as you haven't fixed the ordering issue that requires rpm_get
before struct_mutex, this is broken.

Why does 'intel_runtime_pm_get' require the struct mutex to be held?
It has certainly not complained at me about trying to do stuff
without it.

Quoted 5 lines

Because it depends upon the struct_mutex and rpm doesn't have sufficient
lockdep integration to be able to warn about using rpm from the
incorrect contexts.

Quoted 7 lines

Where? What does the 'pm_runtime_get_sync' call turn into? There are already
other places in the driver which call intel_runtime_pm_get() immediately
after grabbing the mutex lock. Also, the description comment for _pm_get()
does not mention anything about mutexes at all.

Quoted 5 lines

If you nest rpm_get within dev->struct_mutex that's a bug and could
deadlock. Where does this happen? And for any such place we need a new
subtest in pm_rpm.

Quoted 7 lines

The first two hits when grepping the driver are in 'i915_gem_seqno_info()'
and 'i915_interrupt_info()' in i915_debugfs.c. Both say:
    ret = mutex_lock_interruptible(&dev->struct_mutex); if (ret)
        return ret; intel_runtime_pm_get(dev_priv);

Yeah that's totally bonkers and will deadlock if the device is actually
suspend.

Quoted 8 lines
Quoted 3 lines

It won't deadlock, runtime resume doesn't need struct mutex. Runtime suspend needs it, but we return -EAGAIN from there if it's already held. That in turn will delay the suspend.

You're right, why do I always mix this up. But this just got me thinking about possible lockdep annotations. We could place a might_lock(rpm_lockdep_key) within rpm_get. And fake-acquire that around the actual ->resume callback. We could even do this in the rpm core ...

Or we could make progress on eliminating the need for struct_mutex in rpm resume/suspend/whenevner :)
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#73 lore raw

PATCH v4 3/38 drm/i915: Prelude to splitting i915_gem_do_execbuffer in two

In reply to [email protected]

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with their submission to the hardware. This basically means splitting the execbuffer() function in half. This change rearranges some code ready for the split to occur.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 63 ++++++++++++++++++------------
 drivers/gpu/drm/i915/intel_lrc.c           | 18 ++++++---
 2 files changed, 51 insertions(+), 30 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +38 -25
index bfc4c17..0eca2b6 100644
@@ -933,10 +933,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return intel_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 static bool
@@ -1189,17 +1186,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	u32 instp_mask;
 	int ret;
 
-	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
-	if (ret)
-		return ret;
-
-	ret = i915_switch_context(params->request);
-	if (ret)
-		return ret;
-
-	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
-	     "%s didn't clear reload\n", ring->name);
-
 	instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
 	instp_mask = I915_EXEC_CONSTANTS_MASK;
 	switch (instp_mode) {
@@ -1233,11 +1219,37 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 		return -EINVAL;
 	}
 
+	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
+	if (ret)
+		return ret;
+
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	intel_runtime_pm_get(dev_priv);
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = intel_ring_invalidate_all_caches(params->request);
+	if (ret)
+		goto error;
+
+	/* Switch to the correct context for the batch */
+	ret = i915_switch_context(params->request);
+	if (ret)
+		goto error;
+
+	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
+	     "%s didn't clear reload\n", ring->name);
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_ring_begin(params->request, 4);
 		if (ret)
-			return ret;
+			goto error;
 
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
@@ -1251,7 +1263,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
 		ret = i915_reset_gen7_sol_offsets(dev, params->request);
 		if (ret)
-			return ret;
+			goto error;
 	}
 
 	exec_len   = args->batch_len;
@@ -1262,14 +1274,20 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto error;
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
-	return 0;
+error:
+	/*
+	 * intel_gpu_busy should also get a ref, so it will free when the device
+	 * is really idle.
+	 */
+	intel_runtime_pm_put(dev_priv);
+
+	return ret;
 }
 
 /**
@@ -1424,8 +1442,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 		dispatch_flags |= I915_DISPATCH_RS;
 	}
 
-	intel_runtime_pm_get(dev_priv);
-
 	ret = i915_mutex_lock_interruptible(dev);
 	if (ret)
 		goto pre_mutex_err;
@@ -1599,9 +1615,6 @@ err:
 	mutex_unlock(&dev->struct_mutex);
 
 pre_mutex_err:
-	/* intel_gpu_busy should also get a ref, so it will free when the device
-	 * is really idle. */
-	intel_runtime_pm_put(dev_priv);
 	return ret;
 }
 
drivers/gpu/drm/i915/intel_lrc.c +13 -5
index e510730..4bf0ee6 100644
@@ -647,10 +647,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return logical_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -913,6 +910,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 	if (ret)
 		return ret;
 
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = logical_ring_invalidate_all_caches(params->request);
+	if (ret)
+		return ret;
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_logical_ring_begin(params->request, 4);
@@ -937,7 +946,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v4 3/38
#74 lore raw

Re: [PATCH v4 03/38] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two

Quoted 176 lines · On 01/11/2016 10:42 AM, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with their submission to the hardware. This basically means splitting the execbuffer() function in half. This change rearranges some code ready for the split to occur.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 63 ++++++++++++++++++------------
 drivers/gpu/drm/i915/intel_lrc.c           | 18 ++++++---
 2 files changed, 51 insertions(+), 30 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +38 -25
index bfc4c17..0eca2b6 100644
@@ -933,10 +933,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return intel_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 static bool
@@ -1189,17 +1186,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	u32 instp_mask;
 	int ret;
 
-	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
-	if (ret)
-		return ret;
-
-	ret = i915_switch_context(params->request);
-	if (ret)
-		return ret;
-
-	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
-	     "%s didn't clear reload\n", ring->name);
-
 	instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
 	instp_mask = I915_EXEC_CONSTANTS_MASK;
 	switch (instp_mode) {
@@ -1233,11 +1219,37 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 		return -EINVAL;
 	}
 
+	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
+	if (ret)
+		return ret;
+
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	intel_runtime_pm_get(dev_priv);
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = intel_ring_invalidate_all_caches(params->request);
+	if (ret)
+		goto error;
+
+	/* Switch to the correct context for the batch */
+	ret = i915_switch_context(params->request);
+	if (ret)
+		goto error;
+
+	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
+	     "%s didn't clear reload\n", ring->name);
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_ring_begin(params->request, 4);
 		if (ret)
-			return ret;
+			goto error;
 
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
@@ -1251,7 +1263,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
 		ret = i915_reset_gen7_sol_offsets(dev, params->request);
 		if (ret)
-			return ret;
+			goto error;
 	}
 
 	exec_len   = args->batch_len;
@@ -1262,14 +1274,20 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
-		return ret;
+		goto error;
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
-	return 0;
+error:
+	/*
+	 * intel_gpu_busy should also get a ref, so it will free when the device
+	 * is really idle.
+	 */
+	intel_runtime_pm_put(dev_priv);
+
+	return ret;
 }
 
 /**
@@ -1424,8 +1442,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 		dispatch_flags |= I915_DISPATCH_RS;
 	}
 
-	intel_runtime_pm_get(dev_priv);
-
 	ret = i915_mutex_lock_interruptible(dev);
 	if (ret)
 		goto pre_mutex_err;
@@ -1599,9 +1615,6 @@ err:
 	mutex_unlock(&dev->struct_mutex);
 
 pre_mutex_err:
-	/* intel_gpu_busy should also get a ref, so it will free when the device
-	 * is really idle. */
-	intel_runtime_pm_put(dev_priv);
 	return ret;
 }
 
drivers/gpu/drm/i915/intel_lrc.c +13 -4
index e510730..4bf0ee6 100644
@@ -647,10 +647,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
 	if (flush_domains & I915_GEM_DOMAIN_GTT)
 		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return logical_ring_invalidate_all_caches(req);
+	return 0;
 }
 
 int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -913,6 +910,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 	if (ret)
 		return ret;
 
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = logical_ring_invalidate_all_caches(params->request);
+	if (ret)
+		return ret;
+
 	if (ring == &dev_priv->ring[RCS] &&
 	    instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_logical_ring_begin(params->request, 4);

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 951

Quoted 7 lines
drivers/gpu/drm/i915/intel_lrc.c +0 -1
@@ -937,7 +946,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
 	i915_gem_execbuffer_retire_commands(params);
 
 	return 0;

Do we need to do anything if the cache invalidation fails like move the buffers back off the active list? The order changed here, so I'm wondering.

If that's not a problem, then:

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#75 lore raw

Re: [PATCH v4 03/38] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two

Quoted 177 lines · Jesse Barnes
Quoted 176 lines · On 01/11/2016 10:42 AM, [email protected] wrote:

From: John Harrison <[email protected]>

The scheduler decouples the submission of batch buffers to the driver with their submission to the hardware. This basically means splitting the execbuffer() function in half. This change rearranges some code ready for the split to occur.

For: VIZ-1587

---
  drivers/gpu/drm/i915/i915_gem_execbuffer.c | 63 ++++++++++++++++++------------
  drivers/gpu/drm/i915/intel_lrc.c           | 18 ++++++---
  2 files changed, 51 insertions(+), 30 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +38 -25
index bfc4c17..0eca2b6 100644
@@ -933,10 +933,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
  	if (flush_domains & I915_GEM_DOMAIN_GTT)
  		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return intel_ring_invalidate_all_caches(req);
+	return 0;
  }
 
  static bool
@@ -1189,17 +1186,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
  	u32 instp_mask;
  	int ret;
 
-	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
-	if (ret)
-		return ret;
-
-	ret = i915_switch_context(params->request);
-	if (ret)
-		return ret;
-
-	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
-	     "%s didn't clear reload\n", ring->name);
-
  	instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
  	instp_mask = I915_EXEC_CONSTANTS_MASK;
  	switch (instp_mode) {
@@ -1233,11 +1219,37 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
  		return -EINVAL;
  	}
 
+	ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
+	if (ret)
+		return ret;
+
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	intel_runtime_pm_get(dev_priv);
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = intel_ring_invalidate_all_caches(params->request);
+	if (ret)
+		goto error;
+
+	/* Switch to the correct context for the batch */
+	ret = i915_switch_context(params->request);
+	if (ret)
+		goto error;
+
+	WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
+	     "%s didn't clear reload\n", ring->name);
+
  	if (ring == &dev_priv->ring[RCS] &&
  	    instp_mode != dev_priv->relative_constants_mode) {
  		ret = intel_ring_begin(params->request, 4);
  		if (ret)
-			return ret;
+			goto error;
 
  		intel_ring_emit(ring, MI_NOOP);
  		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
@@ -1251,7 +1263,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
  	if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
  		ret = i915_reset_gen7_sol_offsets(dev, params->request);
  		if (ret)
-			return ret;
+			goto error;
  	}
 
  	exec_len   = args->batch_len;
@@ -1262,14 +1274,20 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
  					exec_start, exec_len,
  					params->dispatch_flags);
  	if (ret)
-		return ret;
+		goto error;
 
  	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
  	i915_gem_execbuffer_retire_commands(params);
 
-	return 0;
+error:
+	/*
+	 * intel_gpu_busy should also get a ref, so it will free when the device
+	 * is really idle.
+	 */
+	intel_runtime_pm_put(dev_priv);
+
+	return ret;
  }
 
  /**
@@ -1424,8 +1442,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
  		dispatch_flags |= I915_DISPATCH_RS;
  	}
 
-	intel_runtime_pm_get(dev_priv);
-
  	ret = i915_mutex_lock_interruptible(dev);
  	if (ret)
  		goto pre_mutex_err;
@@ -1599,9 +1615,6 @@ err:
  	mutex_unlock(&dev->struct_mutex);
 
  pre_mutex_err:
-	/* intel_gpu_busy should also get a ref, so it will free when the device
-	 * is really idle. */
-	intel_runtime_pm_put(dev_priv);
  	return ret;
  }
 
drivers/gpu/drm/i915/intel_lrc.c +13 -4
index e510730..4bf0ee6 100644
@@ -647,10 +647,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
  	if (flush_domains & I915_GEM_DOMAIN_GTT)
  		wmb();
 
-	/* Unconditionally invalidate gpu caches and ensure that we do flush
-	 * any residual writes from the previous batch.
-	 */
-	return logical_ring_invalidate_all_caches(req);
+	return 0;
  }
 
  int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -913,6 +910,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
  	if (ret)
  		return ret;
 
+	i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+	/* To be split into two functions here... */
+
+	/*
+	 * Unconditionally invalidate gpu caches and ensure that we do flush
+	 * any residual writes from the previous batch.
+	 */
+	ret = logical_ring_invalidate_all_caches(params->request);
+	if (ret)
+		return ret;
+
  	if (ring == &dev_priv->ring[RCS] &&
  	    instp_mode != dev_priv->relative_constants_mode) {
  		ret = intel_logical_ring_begin(params->request, 4);
@@ -937,7 +946,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
Quoted 8 lines

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission(struct i915_execbuffer_params *params, · line 951

Quoted 7 lines
drivers/gpu/drm/i915/intel_lrc.c +0 -1
  	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
 
-	i915_gem_execbuffer_move_to_active(vmas, params->request);
  	i915_gem_execbuffer_retire_commands(params);
 
  	return 0;

Do we need to do anything if the cache invalidation fails like move the buffers back off the active list? The order changed here, so I'm wondering.

I don't believe so. The objects are all tracked by request. The request itself is cancelled rather than added to the request list. So the next run of i915_gem_retire_requests_ring() will do all the necessary clean up. Potentially we could do something explicit here to speed the process up. However, this is only an intermediate step and there is no nice and simple move_to_inactive function to call. Once the scheduler arrives, a failed backend submission would be retried by the scheduler. So objects must not be cleaned up as that would break the re-submission.

Quoted 2 lines

If that's not a problem, then:

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#76 lore raw

PATCH v4 4/38 drm/i915: Split i915_dem_do_execbuffer() in half

In reply to [email protected]

From: John Harrison <[email protected]>

Split the execbuffer() function in half. The first half collects and validates all the information required to process the batch buffer. It also does all the object pinning, relocations, active list management, etc - basically anything that must be done upfront before the IOCTL returns and allows the user land side to start changing/freeing things. The second half does the actual ring submission.

This change implements the split but leaves the back half being called directly from the end of the front half.

v2: Updated due to changes in underlying tree - addition of sync fence support and removal of cliprects.

v3: Moved local 'ringbuf' variable to make later patches in the series a bit neater.

v4: Corrected a typo in the commit message and downgraded a BUG_ON to a WARN_ON as the latter is preferred. Also removed all the external sync/fence support as that will now be a separate patch series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  11 ++++
 drivers/gpu/drm/i915/i915_gem.c            |   2 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 101 ++++++++++++++++++++++-------
 drivers/gpu/drm/i915/intel_lrc.c           |  57 +++++++++++-----
 drivers/gpu/drm/i915/intel_lrc.h           |   1 +
 5 files changed, 130 insertions(+), 42 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
index acfe25f..2520459 100644
@@ -1679,10 +1679,18 @@ struct i915_execbuffer_params {
 	struct drm_device               *dev;
 	struct drm_file                 *file;
 	uint32_t                        dispatch_flags;
+	uint32_t                        args_flags;
 	uint32_t                        args_batch_start_offset;
+	uint32_t                        args_batch_len;
+	uint32_t                        args_num_cliprects;
+	uint32_t                        args_DR1;
+	uint32_t                        args_DR4;
 	uint64_t                        batch_obj_vm_offset;
 	struct intel_engine_cs          *ring;
 	struct drm_i915_gem_object      *batch_obj;
+	struct drm_clip_rect            *cliprects;
+	uint32_t                        instp_mask;
+	int                             instp_mode;
 	struct intel_context            *ctx;
 	struct drm_i915_gem_request     *request;
 };
@@ -1944,6 +1952,7 @@ struct drm_i915_private {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
 				      struct drm_i915_gem_execbuffer2 *args,
 				      struct list_head *vmas);
+		int (*execbuf_final)(struct i915_execbuffer_params *params);
 		int (*init_rings)(struct drm_device *dev);
 		void (*cleanup_ring)(struct intel_engine_cs *ring);
 		void (*stop_ring)(struct intel_engine_cs *ring);
@@ -2790,9 +2799,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
 void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
 					struct drm_i915_gem_request *req);
 void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj);
 int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 				   struct drm_i915_gem_execbuffer2 *args,
 				   struct list_head *vmas);
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params);
 int i915_gem_execbuffer(struct drm_device *dev, void *data,
 			struct drm_file *file_priv);
 int i915_gem_execbuffer2(struct drm_device *dev, void *data,
drivers/gpu/drm/i915/i915_gem.c +2 -0
index e8ec49e..5bf7da6 100644
@@ -5220,11 +5220,13 @@ int i915_gem_init(struct drm_device *dev)
 
 	if (!i915.enable_execlists) {
 		dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
+		dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final;
 		dev_priv->gt.init_rings = i915_gem_init_rings;
 		dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer;
 		dev_priv->gt.stop_ring = intel_stop_ring_buffer;
 	} else {
 		dev_priv->gt.execbuf_submit = intel_execlists_submission;
+		dev_priv->gt.execbuf_final = intel_execlists_submission_final;
 		dev_priv->gt.init_rings = intel_logical_rings_init;
 		dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup;
 		dev_priv->gt.stop_ring = intel_logical_ring_stop;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +76 -25

179 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.c +40 -17

103 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.h +1 -0
index 4e60d54..8d9bad7 100644
@@ -93,6 +93,7 @@ struct i915_execbuffer_params;
 int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas);
+int intel_execlists_submission_final(struct i915_execbuffer_params *params);
 u32 intel_execlists_ctx_id(struct drm_i915_gem_object *ctx_obj);
 
 void intel_lrc_irq_handler(struct intel_engine_cs *ring);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#77 lore raw

Re: [PATCH v4 04/38] drm/i915: Split i915_dem_do_execbuffer() in half

Quoted 8 lines · On Mon, Jan 11, 2016 at 06:42:33PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

Split the execbuffer() function in half. The first half collects and validates all the information required to process the batch buffer. It also does all the object pinning, relocations, active list management, etc - basically anything that must be done upfront before the IOCTL returns and allows the user land side to start changing/freeing things. The second half does the actual ring submission.

I don't get this at all. The point of requests is that GEM constructed a request, which could be used to pass along all the implicit GEM synchronisation points and the explict ones, along with the ringbuffer to execute, to an engine that could then submit it. For legacy, the request was inline and so added immediately to the ring (but that is an implementation detail, there is nothing stopping us from using a chained batch to implement a ring per context), for execlists the request is queued for future execution. A scheduler was meant to sit in the middle and determine the order in which requests were executed, but that should be almost transparent to the high level code tracking the requests. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v4 4/38
#78 lore raw

Re: [PATCH v4 04/38] drm/i915: Split i915_dem_do_execbuffer() in half

In reply to [email protected]
Quoted 398 lines · On 01/11/2016 10:42 AM, [email protected] wrote:

From: John Harrison <[email protected]>

Split the execbuffer() function in half. The first half collects and validates all the information required to process the batch buffer. It also does all the object pinning, relocations, active list management, etc - basically anything that must be done upfront before the IOCTL returns and allows the user land side to start changing/freeing things. The second half does the actual ring submission.

This change implements the split but leaves the back half being called directly from the end of the front half.

v2: Updated due to changes in underlying tree - addition of sync fence support and removal of cliprects.

v3: Moved local 'ringbuf' variable to make later patches in the series a bit neater.

v4: Corrected a typo in the commit message and downgraded a BUG_ON to a WARN_ON as the latter is preferred. Also removed all the external sync/fence support as that will now be a separate patch series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_drv.h            |  11 ++++
 drivers/gpu/drm/i915/i915_gem.c            |   2 +
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 101 ++++++++++++++++++++++-------
 drivers/gpu/drm/i915/intel_lrc.c           |  57 +++++++++++-----
 drivers/gpu/drm/i915/intel_lrc.h           |   1 +
 5 files changed, 130 insertions(+), 42 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +11 -0
index acfe25f..2520459 100644
@@ -1679,10 +1679,18 @@ struct i915_execbuffer_params {
 	struct drm_device               *dev;
 	struct drm_file                 *file;
 	uint32_t                        dispatch_flags;
+	uint32_t                        args_flags;
 	uint32_t                        args_batch_start_offset;
+	uint32_t                        args_batch_len;
+	uint32_t                        args_num_cliprects;
+	uint32_t                        args_DR1;
+	uint32_t                        args_DR4;
 	uint64_t                        batch_obj_vm_offset;
 	struct intel_engine_cs          *ring;
 	struct drm_i915_gem_object      *batch_obj;
+	struct drm_clip_rect            *cliprects;
+	uint32_t                        instp_mask;
+	int                             instp_mode;
 	struct intel_context            *ctx;
 	struct drm_i915_gem_request     *request;
 };
@@ -1944,6 +1952,7 @@ struct drm_i915_private {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
 				      struct drm_i915_gem_execbuffer2 *args,
 				      struct list_head *vmas);
+		int (*execbuf_final)(struct i915_execbuffer_params *params);
 		int (*init_rings)(struct drm_device *dev);
 		void (*cleanup_ring)(struct intel_engine_cs *ring);
 		void (*stop_ring)(struct intel_engine_cs *ring);
@@ -2790,9 +2799,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
 void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
 					struct drm_i915_gem_request *req);
 void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj);
 int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 				   struct drm_i915_gem_execbuffer2 *args,
 				   struct list_head *vmas);
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params);
 int i915_gem_execbuffer(struct drm_device *dev, void *data,
 			struct drm_file *file_priv);
 int i915_gem_execbuffer2(struct drm_device *dev, void *data,
drivers/gpu/drm/i915/i915_gem.c +2 -0
index e8ec49e..5bf7da6 100644
@@ -5220,11 +5220,13 @@ int i915_gem_init(struct drm_device *dev)
 
 	if (!i915.enable_execlists) {
 		dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
+		dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final;
 		dev_priv->gt.init_rings = i915_gem_init_rings;
 		dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer;
 		dev_priv->gt.stop_ring = intel_stop_ring_buffer;
 	} else {
 		dev_priv->gt.execbuf_submit = intel_execlists_submission;
+		dev_priv->gt.execbuf_final = intel_execlists_submission_final;
 		dev_priv->gt.init_rings = intel_logical_rings_init;
 		dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup;
 		dev_priv->gt.stop_ring = intel_logical_ring_stop;
drivers/gpu/drm/i915/i915_gem_execbuffer.c +76 -25
index 0eca2b6..eff171d 100644
@@ -1181,41 +1181,38 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	struct drm_device *dev = params->dev;
 	struct intel_engine_cs *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
-	u64 exec_start, exec_len;
-	int instp_mode;
-	u32 instp_mask;
 	int ret;
 
-	instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
-	instp_mask = I915_EXEC_CONSTANTS_MASK;
-	switch (instp_mode) {
+	params->instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
+	params->instp_mask = I915_EXEC_CONSTANTS_MASK;
+	switch (params->instp_mode) {
 	case I915_EXEC_CONSTANTS_REL_GENERAL:
 	case I915_EXEC_CONSTANTS_ABSOLUTE:
 	case I915_EXEC_CONSTANTS_REL_SURFACE:
-		if (instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
+		if (params->instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
 			DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
 			return -EINVAL;
 		}
 
-		if (instp_mode != dev_priv->relative_constants_mode) {
+		if (params->instp_mode != dev_priv->relative_constants_mode) {
 			if (INTEL_INFO(dev)->gen < 4) {
 				DRM_DEBUG("no rel constants on pre-gen4\n");
 				return -EINVAL;
 			}
 
 			if (INTEL_INFO(dev)->gen > 5 &&
-			    instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
+			    params->instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
 				DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
 				return -EINVAL;
 			}
 
 			/* The HW changed the meaning on this bit on gen6 */
 			if (INTEL_INFO(dev)->gen >= 6)
-				instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
+				params->instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
 		}
 		break;
 	default:
-		DRM_DEBUG("execbuf with unknown constants: %d\n", instp_mode);
+		DRM_DEBUG("execbuf with unknown constants: %d\n", params->instp_mode);
 		return -EINVAL;
 	}
 
@@ -1225,7 +1222,33 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	/* To be split into two functions here... */
+	ret = dev_priv->gt.execbuf_final(params);
+	if (ret)
+		return ret;
+
+	/*
+	 * Free everything that was stored in the QE structure (until the
+	 * scheduler arrives and does it instead):
+	 */
+	if (params->dispatch_flags & I915_DISPATCH_SECURE)
+		i915_gem_execbuff_release_batch_obj(params->batch_obj);
+
+	return 0;
+}
+
+/*
+ * This is the main function for adding a batch to the ring.
+ * It is called from the scheduler, with the struct_mutex already held.
+ */
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
+{
+	struct drm_i915_private *dev_priv = params->dev->dev_private;
+	struct intel_engine_cs  *ring = params->ring;
+	u64 exec_start, exec_len;
+	int ret;
+
+	/* The mutex must be acquired before calling this function */
+	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
 	intel_runtime_pm_get(dev_priv);
 
@@ -1246,7 +1269,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 	     "%s didn't clear reload\n", ring->name);
 
 	if (ring == &dev_priv->ring[RCS] &&
-	    instp_mode != dev_priv->relative_constants_mode) {
+	    params->instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_ring_begin(params->request, 4);
 		if (ret)
 			goto error;
@@ -1254,19 +1277,19 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 		intel_ring_emit(ring, MI_NOOP);
 		intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
 		intel_ring_emit(ring, INSTPM);
-		intel_ring_emit(ring, instp_mask << 16 | instp_mode);
+		intel_ring_emit(ring, params->instp_mask << 16 | params->instp_mode);
 		intel_ring_advance(ring);
 
-		dev_priv->relative_constants_mode = instp_mode;
+		dev_priv->relative_constants_mode = params->instp_mode;
 	}
 
-	if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
-		ret = i915_reset_gen7_sol_offsets(dev, params->request);
+	if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
+		ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
 		if (ret)
 			goto error;
 	}
 
-	exec_len   = args->batch_len;
+	exec_len   = params->args_batch_len;
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 
@@ -1584,23 +1607,40 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
 	params->file                    = file;
 	params->ring                    = ring;
 	params->dispatch_flags          = dispatch_flags;
+	params->args_flags              = args->flags;
+	params->args_batch_len          = args->batch_len;
+	params->args_num_cliprects      = args->num_cliprects;
+	params->args_DR1                = args->DR1;
+	params->args_DR4                = args->DR4;
 	params->batch_obj               = batch_obj;
 	params->ctx                     = ctx;
 
 	ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
+	if (ret)
+		goto err_batch_unpin;
+
+	/* the request owns the ref now */
+	i915_gem_context_unreference(ctx);
 
-err_batch_unpin:
 	/*
-	 * FIXME: We crucially rely upon the active tracking for the (ppgtt)
-	 * batch vma for correctness. For less ugly and less fragility this
-	 * needs to be adjusted to also track the ggtt batch vma properly as
-	 * active.
+	 * The eb list is no longer required. The scheduler has extracted all
+	 * the information than needs to persist.
 	 */
+	eb_destroy(eb);
+
+	/*
+	 * Don't clean up everything that is now saved away in the queue.
+	 * Just unlock and return immediately.
+	 */
+	mutex_unlock(&dev->struct_mutex);
+
+	return 0;
+
+err_batch_unpin:
 	if (dispatch_flags & I915_DISPATCH_SECURE)
-		i915_gem_object_ggtt_unpin(batch_obj);
+		i915_gem_execbuff_release_batch_obj(batch_obj);
 
 err:
-	/* the request owns the ref now */
 	i915_gem_context_unreference(ctx);
 	eb_destroy(eb);
 
@@ -1618,6 +1658,17 @@ pre_mutex_err:
 	return ret;
 }
 
+void i915_gem_execbuff_release_batch_obj(struct drm_i915_gem_object *batch_obj)
+{
+	/*
+	 * FIXME: We crucially rely upon the active tracking for the (ppgtt)
+	 * batch vma for correctness. For less ugly and less fragility this
+	 * needs to be adjusted to also track the ggtt batch vma properly as
+	 * active.
+	 */
+	i915_gem_object_ggtt_unpin(batch_obj);
+}
+
 /*
  * Legacy execbuffer just creates an exec2 list from the original exec object
  * list array and passes it to the real function.
drivers/gpu/drm/i915/intel_lrc.c +40 -17
index 4bf0ee6..014180c 100644
@@ -869,35 +869,31 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 	struct drm_device       *dev = params->dev;
 	struct intel_engine_cs  *ring = params->ring;
 	struct drm_i915_private *dev_priv = dev->dev_private;
-	struct intel_ringbuffer *ringbuf = params->ctx->engine[ring->id].ringbuf;
-	u64 exec_start;
-	int instp_mode;
-	u32 instp_mask;
 	int ret;
 
-	instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
-	instp_mask = I915_EXEC_CONSTANTS_MASK;
-	switch (instp_mode) {
+	params->instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
+	params->instp_mask = I915_EXEC_CONSTANTS_MASK;
+	switch (params->instp_mode) {
 	case I915_EXEC_CONSTANTS_REL_GENERAL:
 	case I915_EXEC_CONSTANTS_ABSOLUTE:
 	case I915_EXEC_CONSTANTS_REL_SURFACE:
-		if (instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
+		if (params->instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
 			DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
 			return -EINVAL;
 		}
 
-		if (instp_mode != dev_priv->relative_constants_mode) {
-			if (instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
+		if (params->instp_mode != dev_priv->relative_constants_mode) {
+			if (params->instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
 				DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
 				return -EINVAL;
 			}
 
 			/* The HW changed the meaning on this bit on gen6 */
-			instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
+			params->instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
 		}
 		break;
 	default:
-		DRM_DEBUG("execbuf with unknown constants: %d\n", instp_mode);
+		DRM_DEBUG("execbuf with unknown constants: %d\n", params->instp_mode);
 		return -EINVAL;
 	}
 
@@ -912,7 +908,34 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 
 	i915_gem_execbuffer_move_to_active(vmas, params->request);
 
-	/* To be split into two functions here... */
+	ret = dev_priv->gt.execbuf_final(params);
+	if (ret)
+		return ret;
+
+	/*
+	 * Free everything that was stored in the QE structure (until the
+	 * scheduler arrives and does it instead):
+	 */
+	if (params->dispatch_flags & I915_DISPATCH_SECURE)
+		i915_gem_execbuff_release_batch_obj(params->batch_obj);
+
+	return 0;
+}
+
+/*
+ * This is the main function for adding a batch to the ring.
+ * It is called from the scheduler, with the struct_mutex already held.
+ */
+int intel_execlists_submission_final(struct i915_execbuffer_params *params)
+{
+	struct drm_i915_private *dev_priv = params->dev->dev_private;
+	struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+	struct intel_engine_cs *ring = params->ring;
+	u64 exec_start;
+	int ret;
+
+	/* The mutex must be acquired before calling this function */
+	WARN_ON(!mutex_is_locked(&params->dev->struct_mutex));
 
 	/*
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
@@ -923,7 +946,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 		return ret;
 
 	if (ring == &dev_priv->ring[RCS] &&
-	    instp_mode != dev_priv->relative_constants_mode) {
+	    params->instp_mode != dev_priv->relative_constants_mode) {
 		ret = intel_logical_ring_begin(params->request, 4);
 		if (ret)
 			return ret;
@@ -931,14 +954,14 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 		intel_logical_ring_emit(ringbuf, MI_NOOP);
 		intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
 		intel_logical_ring_emit(ringbuf, INSTPM);
-		intel_logical_ring_emit(ringbuf, instp_mask << 16 | instp_mode);
+		intel_logical_ring_emit(ringbuf, params->instp_mask << 16 | params->instp_mode);
 		intel_logical_ring_advance(ringbuf);
 
-		dev_priv->relative_constants_mode = instp_mode;
+		dev_priv->relative_constants_mode = params->instp_mode;
 	}
 
 	exec_start = params->batch_obj_vm_offset +
-		     args->batch_start_offset;
+		     params->args_batch_start_offset;
 
 	ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
 	if (ret)
drivers/gpu/drm/i915/intel_lrc.h +0 -0
index 4e60d54..8d9bad7 100644

drivers/gpu/drm/i915/intel_lrc.h struct i915_execbuffer_params; · line 99

Quoted 7 lines
drivers/gpu/drm/i915/intel_lrc.h +1 -0
@@ -93,6 +93,7 @@ struct i915_execbuffer_params;
 int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas);
+int intel_execlists_submission_final(struct i915_execbuffer_params *params);
 u32 intel_execlists_ctx_id(struct drm_i915_gem_object *ctx_obj);
 
 void intel_lrc_irq_handler(struct intel_engine_cs *ring);

Just a nitpick on naming; I think prepare/commit might be better than submit/submit_final. But you don't have to respin just for that.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#79 lore raw

PATCH v4 5/38 drm/i915: Cache request pointer in *_submission_final()

In reply to [email protected]

From: Dave Gordon <[email protected]>

Keep a local copy of the request pointer in the _final() functions rather than dereferencing the params block repeatedly.

v3: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++------
 drivers/gpu/drm/i915/intel_lrc.c           | 11 ++++++-----
 2 files changed, 13 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -6
index eff171d..0ad32f6 100644
@@ -1243,6 +1243,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
+	struct drm_i915_gem_request *req = params->request;
 	struct intel_engine_cs  *ring = params->ring;
 	u64 exec_start, exec_len;
 	int ret;
@@ -1256,12 +1257,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = intel_ring_invalidate_all_caches(params->request);
+	ret = intel_ring_invalidate_all_caches(req);
 	if (ret)
 		goto error;
 
 	/* Switch to the correct context for the batch */
-	ret = i915_switch_context(params->request);
+	ret = i915_switch_context(req);
 	if (ret)
 		goto error;
 
@@ -1270,7 +1271,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_ring_begin(params->request, 4);
+		ret = intel_ring_begin(req, 4);
 		if (ret)
 			goto error;
 
@@ -1284,7 +1285,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	}
 
 	if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
-		ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
+		ret = i915_reset_gen7_sol_offsets(params->dev, req);
 		if (ret)
 			goto error;
 	}
@@ -1293,13 +1294,13 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 
-	ret = ring->dispatch_execbuffer(params->request,
+	ret = ring->dispatch_execbuffer(req,
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
 		goto error;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
drivers/gpu/drm/i915/intel_lrc.c +6 -5
index 014180c..a344a3a 100644
@@ -929,7 +929,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
-	struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+	struct drm_i915_gem_request *req = params->request;
+	struct intel_ringbuffer *ringbuf = req->ringbuf;
 	struct intel_engine_cs *ring = params->ring;
 	u64 exec_start;
 	int ret;
@@ -941,13 +942,13 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = logical_ring_invalidate_all_caches(params->request);
+	ret = logical_ring_invalidate_all_caches(req);
 	if (ret)
 		return ret;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_logical_ring_begin(params->request, 4);
+		ret = intel_logical_ring_begin(req, 4);
 		if (ret)
 			return ret;
 
@@ -963,11 +964,11 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 
-	ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+	ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
 	if (ret)
 		return ret;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Reviewed-by on v4 5/38
#80 lore raw

Re: [PATCH v4 05/38] drm/i915: Cache request pointer in *_submission_final()

Quoted 111 lines · On 01/11/2016 10:42 AM, [email protected] wrote:

From: Dave Gordon <[email protected]>

Keep a local copy of the request pointer in the _final() functions rather than dereferencing the params block repeatedly.

v3: New patch in series.

For: VIZ-1587

---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c | 13 +++++++------
 drivers/gpu/drm/i915/intel_lrc.c           | 11 ++++++-----
 2 files changed, 13 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_gem_execbuffer.c +7 -6
index eff171d..0ad32f6 100644
@@ -1243,6 +1243,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
 int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
+	struct drm_i915_gem_request *req = params->request;
 	struct intel_engine_cs  *ring = params->ring;
 	u64 exec_start, exec_len;
 	int ret;
@@ -1256,12 +1257,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = intel_ring_invalidate_all_caches(params->request);
+	ret = intel_ring_invalidate_all_caches(req);
 	if (ret)
 		goto error;
 
 	/* Switch to the correct context for the batch */
-	ret = i915_switch_context(params->request);
+	ret = i915_switch_context(req);
 	if (ret)
 		goto error;
 
@@ -1270,7 +1271,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_ring_begin(params->request, 4);
+		ret = intel_ring_begin(req, 4);
 		if (ret)
 			goto error;
 
@@ -1284,7 +1285,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	}
 
 	if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
-		ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
+		ret = i915_reset_gen7_sol_offsets(params->dev, req);
 		if (ret)
 			goto error;
 	}
@@ -1293,13 +1294,13 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 
-	ret = ring->dispatch_execbuffer(params->request,
+	ret = ring->dispatch_execbuffer(req,
 					exec_start, exec_len,
 					params->dispatch_flags);
 	if (ret)
 		goto error;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 
drivers/gpu/drm/i915/intel_lrc.c +4 -3
index 014180c..a344a3a 100644
@@ -929,7 +929,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 {
 	struct drm_i915_private *dev_priv = params->dev->dev_private;
-	struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+	struct drm_i915_gem_request *req = params->request;
+	struct intel_ringbuffer *ringbuf = req->ringbuf;
 	struct intel_engine_cs *ring = params->ring;
 	u64 exec_start;
 	int ret;
@@ -941,13 +942,13 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	 * Unconditionally invalidate gpu caches and ensure that we do flush
 	 * any residual writes from the previous batch.
 	 */
-	ret = logical_ring_invalidate_all_caches(params->request);
+	ret = logical_ring_invalidate_all_caches(req);
 	if (ret)
 		return ret;
 
 	if (ring == &dev_priv->ring[RCS] &&
 	    params->instp_mode != dev_priv->relative_constants_mode) {
-		ret = intel_logical_ring_begin(params->request, 4);
+		ret = intel_logical_ring_begin(req, 4);
 		if (ret)
 			return ret;
 
@@ -963,11 +964,11 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
 	exec_start = params->batch_obj_vm_offset +
 		     params->args_batch_start_offset;
 

drivers/gpu/drm/i915/intel_lrc.c int intel_execlists_submission_final(struct i915_execbuffer_params *params) · line 974

Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +2 -2
-	ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+	ret = ring->emit_bb_start(req, exec_start, params->dispatch_flags);
 	if (ret)
 		return ret;
 
-	trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+	trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
 
 	i915_gem_execbuffer_retire_commands(params);
 

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#81 lore raw

PATCH v4 6/38 drm/i915: Re-instate request->uniq because it is extremely useful

In reply to [email protected]

From: John Harrison <[email protected]>

The seqno value cannot always be used when debugging issues via trace points. This is because it can be reset back to start, especially during TDR type tests. Also, when the scheduler arrives the seqno is only valid while a given request is executing on the hardware. While the request is simply queued waiting for submission, it's seqno value will be zero (meaning invalid).

v4: Wrapped a long line to keep the style checker happy.

For: VIZ-5115

---
 drivers/gpu/drm/i915/i915_drv.h   |  5 +++++
 drivers/gpu/drm/i915/i915_gem.c   |  4 +++-
 drivers/gpu/drm/i915/i915_trace.h | 25 +++++++++++++++++--------
 3 files changed, 25 insertions(+), 9 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +5 -0
index 2520459..10dffdd 100644
@@ -1963,6 +1963,8 @@ struct drm_i915_private {
 	/* perform PHY state sanity checks? */
 	bool chv_phy_assert[2];
 
+	uint32_t request_uniq;
+
 	/*
 	 * NOTE: This is the dri1/ums dungeon, don't add stuff here. Your patch
 	 * will be rejected. Instead look for a better place.
@@ -2210,6 +2212,9 @@ struct drm_i915_gem_request {
 	/** GEM sequence number associated with this request. */
 	uint32_t seqno;
 
+	/* Unique identifier which can be used for trace points & debug */
+	uint32_t uniq;
+
 	/** Position in the ringbuffer of the start of the request */
 	u32 head;
 
drivers/gpu/drm/i915/i915_gem.c +3 -1
index 5bf7da6..cff3768 100644
@@ -2904,7 +2904,8 @@ static void i915_gem_request_fence_value_str(struct fence *req_fence,
 
 	req = container_of(req_fence, typeof(*req), fence);
 
-	snprintf(str, size, "%d [%d]", req->fence.seqno, req->seqno);
+	snprintf(str, size, "%d [%d:%d]", req->fence.seqno, req->uniq,
+		 req->seqno);
 }
 
 static const struct fence_ops i915_gem_request_fops = {
@@ -2980,6 +2981,7 @@ int i915_gem_request_alloc(struct intel_engine_cs *ring,
 
 	req->i915 = dev_priv;
 	req->ring = ring;
+	req->uniq = dev_priv->request_uniq++;
 	req->ctx  = ctx;
 	i915_gem_context_reference(req->ctx);
 
drivers/gpu/drm/i915/i915_trace.h +17 -8
index b3ae894..4c03754 100644
@@ -469,6 +469,7 @@ TRACE_EVENT(i915_gem_ring_sync_to,
 			     __field(u32, dev)
 			     __field(u32, sync_from)
 			     __field(u32, sync_to)
+			     __field(u32, uniq_to)
 			     __field(u32, seqno)
 			     ),
 
@@ -476,13 +477,14 @@ TRACE_EVENT(i915_gem_ring_sync_to,
 			   __entry->dev = from->dev->primary->index;
 			   __entry->sync_from = from->id;
 			   __entry->sync_to = to_req->ring->id;
+			   __entry->uniq_to = to_req->uniq;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   ),
 
-	    TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u",
+	    TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u, to_uniq=%u",
 		      __entry->dev,
 		      __entry->sync_from, __entry->sync_to,
-		      __entry->seqno)
+		      __entry->seqno, __entry->uniq_to)
 );
 
 TRACE_EVENT(i915_gem_ring_dispatch,
@@ -517,6 +519,7 @@ TRACE_EVENT(i915_gem_ring_flush,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, invalidate)
 			     __field(u32, flush)
 			     ),
@@ -524,12 +527,13 @@ TRACE_EVENT(i915_gem_ring_flush,
 	    TP_fast_assign(
 			   __entry->dev = req->ring->dev->primary->index;
 			   __entry->ring = req->ring->id;
+			   __entry->uniq = req->uniq;
 			   __entry->invalidate = invalidate;
 			   __entry->flush = flush;
 			   ),
 
-	    TP_printk("dev=%u, ring=%x, invalidate=%04x, flush=%04x",
-		      __entry->dev, __entry->ring,
+	    TP_printk("dev=%u, ring=%x, request=%u, invalidate=%04x, flush=%04x",
+		      __entry->dev, __entry->ring, __entry->uniq,
 		      __entry->invalidate, __entry->flush)
 );
 
@@ -540,6 +544,7 @@ DECLARE_EVENT_CLASS(i915_gem_request,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, seqno)
 			     ),
 
@@ -548,11 +553,13 @@ DECLARE_EVENT_CLASS(i915_gem_request,
 						i915_gem_request_get_ring(req);
 			   __entry->dev = ring->dev->primary->index;
 			   __entry->ring = ring->id;
+			   __entry->uniq = req ? req->uniq : 0;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   ),
 
-	    TP_printk("dev=%u, ring=%u, seqno=%u",
-		      __entry->dev, __entry->ring, __entry->seqno)
+	    TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u",
+		      __entry->dev, __entry->ring, __entry->uniq,
+		      __entry->seqno)
 );
 
 DEFINE_EVENT(i915_gem_request, i915_gem_request_add,
@@ -601,6 +608,7 @@ TRACE_EVENT(i915_gem_request_wait_begin,
 	    TP_STRUCT__entry(
 			     __field(u32, dev)
 			     __field(u32, ring)
+			     __field(u32, uniq)
 			     __field(u32, seqno)
 			     __field(bool, blocking)
 			     ),
@@ -616,13 +624,14 @@ TRACE_EVENT(i915_gem_request_wait_begin,
 						i915_gem_request_get_ring(req);
 			   __entry->dev = ring->dev->primary->index;
 			   __entry->ring = ring->id;
+			   __entry->uniq = req ? req->uniq : 0;
 			   __entry->seqno = i915_gem_request_get_seqno(req);
 			   __entry->blocking =
 				     mutex_is_locked(&ring->dev->struct_mutex);
 			   ),
 
-	    TP_printk("dev=%u, ring=%u, seqno=%u, blocking=%s",
-		      __entry->dev, __entry->ring,
+	    TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, blocking=%s",
+		      __entry->dev, __entry->ring, __entry->uniq,
 		      __entry->seqno, __entry->blocking ?  "yes (NB)" : "no")
 );
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#82 lore raw

Re: [PATCH v4 06/38] drm/i915: Re-instate request->uniq because it is extremely useful

Quoted 8 lines · On Mon, Jan 11, 2016 at 06:42:35PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The seqno value cannot always be used when debugging issues via trace points. This is because it can be reset back to start, especially during TDR type tests. Also, when the scheduler arrives the seqno is only valid while a given request is executing on the hardware. While the request is simply queued waiting for submission, it's seqno value will be zero (meaning invalid).

Even with per-context seqno that can be assigned before execution as we know that requests within a context cannot be reordered? -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#83 lore raw

Re: [PATCH v4 06/38] drm/i915: Re-instate request->uniq because it is extremely useful

Quoted 9 lines · Chris Wilson
Quoted 8 lines · On Mon, Jan 11, 2016 at 06:42:35PM +0000, [email protected] wrote:

From: John Harrison <[email protected]>

The seqno value cannot always be used when debugging issues via trace points. This is because it can be reset back to start, especially during TDR type tests. Also, when the scheduler arrives the seqno is only valid while a given request is executing on the hardware. While the request is simply queued waiting for submission, it's seqno value will be zero (meaning invalid).

Quoted 4 lines

Even with per-context seqno that can be assigned before execution as we know that requests within a context cannot be reordered? -Chris

Firstly, we do not have per-context seqno values at the moment. However, even if we did that would make a per request unique value even more useful as the only way to identify a given request then would be through a context pointer and seqno pair which would make scanning through debug traces even worse.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#84 lore raw

PATCH v4 7/38 drm/i915: Start of GPU scheduler

In reply to [email protected]

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in to the driver yet. It also leaves the scheduler code in 'pass through' mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the information required to execute the batch buffer at a later time. This package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

The scheduler is notified when each batch buffer completes and updates its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed, the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and a new node popped from the list if appropriate.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

v4: Updated the commit message to better reflect the current state of the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.

Fixed a WARN_ON when killing nodes. The dependency removal code checks that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.

Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.

For: VIZ-1587

---
 drivers/gpu/drm/i915/Makefile         |   1 +
 drivers/gpu/drm/i915/i915_drv.h       |   4 +
 drivers/gpu/drm/i915/i915_gem.c       |   5 +
 drivers/gpu/drm/i915/i915_scheduler.c | 797 ++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  91 ++++
 5 files changed, 898 insertions(+)
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
 i915-y := i915_drv.o \
 	  i915_irq.o \
 	  i915_params.o \
+	  i915_scheduler.o \
           i915_suspend.o \
 	  i915_sysfs.o \
 	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +4 -0
index 10dffdd..38f423b 100644
@@ -1695,6 +1695,8 @@ struct i915_execbuffer_params {
 	struct drm_i915_gem_request     *request;
 };
 
+struct i915_scheduler;
+
 /* used in computing the new watermarks state */
 struct intel_wm_config {
 	unsigned int num_pipes_active;
@@ -1947,6 +1949,8 @@ struct drm_i915_private {
 
 	struct i915_runtime_pm pm;
 
+	struct i915_scheduler *scheduler;
+
 	/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
 	struct {
 		int (*execbuf_submit)(struct i915_execbuffer_params *params,
drivers/gpu/drm/i915/i915_gem.c +5 -0
index cff3768..47aa85b 100644
@@ -37,6 +37,7 @@
 #include <linux/swap.h>
 #include <linux/pci.h>
 #include <linux/dma-buf.h>
+#include "i915_scheduler.h"
 
 #define RQ_BUG_ON(expr)
 
@@ -5242,6 +5243,10 @@ int i915_gem_init(struct drm_device *dev)
 	 */
 	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
 
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+
 	ret = i915_gem_init_userptr(dev);
 	if (ret)
 		goto out_unlock;
drivers/gpu/drm/i915/i915_scheduler.c new file +797 -0

797 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scheduler.h new file +91 -0

91 lines not on this page · raw message

Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#85 lore raw

Re: [PATCH v4 07/38] drm/i915: Start of GPU scheduler

Hi,

Comments below this pre text.

Many of the comments are related to the indent and style of the code. That stuff is important to fix for future maintainability. In order for the future review to be more effective, I'd like to next see a v5 of the series where the code quality concerns have been addressed, patches squashed to be actual reviewable chunks and appropriate kerneldoc being added.

To give an idea of proper slicing of patches, first produce a no-op scheduler, adding the extra function calls where needed and still keeping the scheduling completely linear. Second patch could introduce out of order submitting, third one priority bumping, fourth pre-empting and so on. That way, each patch extends the functionality and is itself already mergeable. That way I've been able to go through and understand the existing code, and I can actually review (other than just nag about indent and coding style) if the changes are appropriate to bring in the functionality desired.

In the current split, for me or anyone who did not participate writing the code, it is otherwise too confusing to try to guess what future changes might make each piece of code make sense, and which will be redundant in the future too. There is no value in splitting code to chunks that are not itself functional.

Regards, Joonas

Quoted 140 lines · On Mon, 2016-01-11 at 18:42 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in
to the driver yet. It also leaves the scheduler code in 'pass through'
mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the information required to execute the batch buffer at a later time. This
package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in
use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

The scheduler is notified when each batch buffer completes and updates
its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed,
the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the
queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and
a new node popped from the list if appropriate.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

v4: Updated the commit message to better reflect the current state of the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.

Fixed a WARN_ON when killing nodes. The dependency removal code checks
that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.

Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.

For: VIZ-1587

---
 drivers/gpu/drm/i915/Makefile         |   1 +
 drivers/gpu/drm/i915/i915_drv.h       |   4 +
 drivers/gpu/drm/i915/i915_gem.c       |   5 +
 drivers/gpu/drm/i915/i915_scheduler.c | 797
++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_scheduler.h |  91 ++++

 5 files changed, 898 insertions(+)

 create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h

drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
  i915-y := i915_drv.o \
  	  i915_irq.o \
  	  i915_params.o \
+	  i915_scheduler.o \
            i915_suspend.o \
  	  i915_sysfs.o \
  	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +0 -0
index 10dffdd..38f423b 100644
@@ -1695,6 +1695,8 @@ struct i915_execbuffer_params {
  	struct drm_i915_gem_request     *request;
  };

drivers/gpu/drm/i915/i915_drv.h struct i915_execbuffer_params { · line 1699

Quoted 2 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
+struct i915_scheduler;
+

Rather add "i915_scheduler.h" include at the top and eliminate circular include dependencies. This is needed for the next comment.

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1953

Quoted 9 lines

 /* used in computing the new watermarks state */
 struct intel_wm_config {
  unsigned int num_pipes_active;

drivers/gpu/drm/i915/i915_drv.h +0 -0
@@ -1947,6 +1949,8 @@ struct drm_i915_private {


  struct i915_runtime_pm pm;

drivers/gpu/drm/i915/i915_drv.h +2 -0
+	struct i915_scheduler *scheduler;
+

As the scheduler is going to be enabled on all platforms to an extent, no point in making it a pointer. Just making it member like "pm" is the best, this also requires the above include change.

Quoted 12 lines

  /* Abstract the submission mechanism (legacy ringbuffer or execlists) away */ struct {
  int (*execbuf_submit)(struct i915_execbuffer_params *params,

drivers/gpu/drm/i915/i915_gem.c +1 -0
index cff3768..47aa85b 100644
@@ -37,6 +37,7 @@
  #include 
  #include 
  #include 
+#include "i915_scheduler.h"
Quoted 1 line

This should go before any <linux/...> not to mask missing includes from the header itself, so correct place would be right after #include "i915_trace.h"

Quoted 36 lines

 #define RQ_BUG_ON(expr)

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -5242,6 +5243,10 @@ int i915_gem_init(struct drm_device *dev)
  	 */
  	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);

drivers/gpu/drm/i915/i915_gem.c +4 -0
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+

  ret = i915_gem_init_userptr(dev);
  if (ret)
  goto out_unlock;

drivers/gpu/drm/i915/i915_scheduler.c new file +17 -0
new file mode 100644
index 0000000..8cb9063
@@ -0,0 +1,797 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL

drivers/gpu/drm/i915/i915_scheduler.c line 27

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+#include "intel_drv.h"
+#include "i915_scheduler.h"

Again, this include should be the top one, should not require any includes before it.

drivers/gpu/drm/i915/i915_scheduler.c line 39

Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+
+static int         i915_scheduler_fly_node(struct i915_scheduler_queue_entry *node);
+static int         i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+						   struct i915_scheduler_queue_entry *remove);
+static int         i915_scheduler_submit(struct intel_engine_cs *ring,
+					 bool is_locked);
+static uint32_t    i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+					       struct intel_engine_cs *ring);
+static void        i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler);
+static int         i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+						struct i915_scheduler_queue_entry *target,
+						uint32_t bump);

Do not indent the function names like this, it becomes unmaintainable and messy very fast if somebody adds a new function with a more complex return type, which is a very likely thing to happen.

What I would do is move all the helper functions here at the top and order them so that the forward declarations are not needed, at least i915_scheduler_fly_node is not used before its definition.

drivers/gpu/drm/i915/i915_scheduler.c line 45

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+
+int i915_scheduler_init(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	int                     r;

Again, this indent is a no-go. I'll not mention it on further functions, assume it to be fixed for next revision.

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+
+	if (scheduler)
+		return 0;
+
+	scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
+	if (!scheduler)
+		return -ENOMEM;
+

drivers/gpu/drm/i915/i915_scheduler.c line 63

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	spin_lock_init(&scheduler->lock);
+
+	for (r = 0; r < I915_NUM_RINGS; r++)
+		INIT_LIST_HEAD(&scheduler->node_queue[r]);
+
+	/* Default tuning values: */
+	scheduler->priority_level_min     = -1023;
+	scheduler->priority_level_max     = 1023;
+	scheduler->priority_level_preempt = 900;
+	scheduler->min_flying             = 2;

This kind of indent is tolerable because it is a contained code block, but not needed either.

Quoted 19 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +19 -0
+
+	dev_priv->scheduler = scheduler;
+
+	return 0;
+}
+
+static void i915_generate_dependencies(struct i915_scheduler *scheduler,
+				       struct i915_scheduler_queue_entry *node,
+				       uint32_t ring)
+{
+	struct i915_scheduler_obj_entry *this, *that;
+	struct i915_scheduler_queue_entry  *test;
+	int                 i, j;
+	bool                found;
+
+	list_for_each_entry(test, &scheduler->node_queue[ring], link) {
+		if (I915_SQS_IS_COMPLETE(test))
+			continue;
+

drivers/gpu/drm/i915/i915_scheduler.c line 92

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		/*
+		 * Batches on the same ring for the same
+		 * context must be kept in order.
+		 */
+		found = (node->params.ctx == test->params.ctx) &&
+			(node->params.ring == test->params.ring);
+
+		if (found)
+			goto depends;
+

This is not needed.

drivers/gpu/drm/i915/i915_scheduler.c line 97

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+		/*
+		 * Batches working on the same objects must
+		 * be kept in order.
+		 */
+		for (i = 0; (i < node->num_objs) && !found; i++) {

As the test is here already                         ---^

drivers/gpu/drm/i915/i915_scheduler.c line 104

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+			this = node->saved_objects + i;
+
+			for (j = 0; j < test->num_objs; j++) {
+				that = test->saved_objects + j;
+
+				if (this->obj != that->obj)
+					continue;

How about VMAs? There might be multiple mappings to an object, isn't it enough to depend on the required VMA instead of the whole object?

drivers/gpu/drm/i915/i915_scheduler.c line 114

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+
+				/* Only need to worry about writes */
+				if (this->read_only && that->read_only)
+					continue;
+
+				found = true;
+				break;
+			}
+		}
+

The following block is not needed.

drivers/gpu/drm/i915/i915_scheduler.c line 118

Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+		if (!found)
+			continue;
+
+depends:

Rather like this, in order to avoid a goto label;
if (found) {
...

drivers/gpu/drm/i915/i915_scheduler.c line 123

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+		node->dep_list[node->num_deps] = test;
+		node->num_deps++;
+	}
+}
+

Please add a brief kerneldoc above each function in the header, it's required. Adding it to non-trivial inline helper functions too will make reviewing much easier.

Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
+{
+	struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;

drivers/gpu/drm/i915/i915_scheduler.c line 137

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	struct intel_engine_cs  *ring = qe->params.ring;
+	struct i915_scheduler_queue_entry  *node;
+	struct i915_scheduler_queue_entry  *test;
+	unsigned long       flags;
+	bool                not_flying;
+	int                 i, r;
+	int                 incomplete = 0;
+
+	WARN_ON(!scheduler);
+

This kind of situations should have a be a BUG_ON, because scheduler being zero is literally going to cause an OOPS in the next dereference which is going to happen unconditionally. WARN + OOPS is kind of what BUG_ON should be used avoid. But this should be removed anyway after scheduler is made a data member of dev_priv.

drivers/gpu/drm/i915/i915_scheduler.c line 138

Quoted 1 line
drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	if (1/*i915.scheduler_override & i915_so_direct_submit*/) {

I assume this is going to be addressed in a future commit. Could have been introduced in this patch, too.

drivers/gpu/drm/i915/i915_scheduler.c line 144

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+		int ret;
+
+		scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
+		ret = dev_priv->gt.execbuf_final(&qe->params);
+		scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
+

The kerneldoc should mention locking requirements of this function.

drivers/gpu/drm/i915/i915_scheduler.c line 157

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +13 -0
+		/*
+		 * Don't do any clean up on failure because the caller will
+		 * do it all anyway.
+		 */
+		if (ret)
+			return ret;
+
+		/* Free everything that is owned by the QE structure: */
+		kfree(qe->params.cliprects);
+		if (qe->params.dispatch_flags & I915_DISPATCH_SECURE)
+			i915_gem_execbuff_release_batch_obj(qe->params.batch_obj);
+
+		return 0;

Above piece of code looks like its own function, so it should probably be one.

drivers/gpu/drm/i915/i915_scheduler.c line 164

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	}
+
+	node = kmalloc(sizeof(*node), GFP_KERNEL);
+	if (!node)
+		return -ENOMEM;
+
+	*node = *qe;

Any reason we can't simply move ownership of qe? If not, I'd rather make a clone function

Quoted 16 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +16 -0
+	INIT_LIST_HEAD(&node->link);
+	node->status = i915_sqs_queued;
+	node->stamp  = jiffies;
+	i915_gem_request_reference(node->params.request);
+
+	/* Need to determine the number of incomplete entries in the list as
+	 * that will be the maximum size of the dependency list.
+	 *
+	 * Note that the allocation must not be made with the spinlock acquired
+	 * as kmalloc can sleep. However, the unlock/relock is safe because no
+	 * new entries can be queued up during the unlock as the i915 driver
+	 * mutex is still held. Entries could be removed from the list but that
+	 * just means the dep_list will be over-allocated which is fine.
+	 */
+	spin_lock_irqsave(&scheduler->lock, flags);
+	for (r = 0; r < I915_NUM_RINGS; r++) {

drivers/gpu/drm/i915/i915_scheduler.c line 190

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		list_for_each_entry(test, &scheduler->node_queue[r], link) {
+			if (I915_SQS_IS_COMPLETE(test))
+				continue;
+
+			incomplete++;
+		}
+	}
+
+	/* Temporarily unlock to allocate memory: */
+	spin_unlock_irqrestore(&scheduler->lock, flags);

I'd make the above piece of code a helper, these stats are to be counted for debugfs anyway, too?

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	if (incomplete) {
+		node->dep_list = kmalloc_array(incomplete,
+					       sizeof(*node->dep_list),
+					       GFP_KERNEL);
+		if (!node->dep_list) {
+			kfree(node);
+			return -ENOMEM;
+		}
+	} else

drivers/gpu/drm/i915/i915_scheduler.c line 209

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		node->dep_list = NULL;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	node->num_deps = 0;
+
+	if (node->dep_list) {
+		for (r = 0; r < I915_NUM_RINGS; r++)
+			i915_generate_dependencies(scheduler, node, r);
+
+		WARN_ON(node->num_deps > incomplete);

Wouldn't this condition again lead to a crash? If so, should be BUG_ON to cause that crash as early as possible. WARN_ON is only good if there is a way of coping with the situation and no imminent system crash is bound to happen.

drivers/gpu/drm/i915/i915_scheduler.c line 216

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	}
+
+	if (node->priority > scheduler->priority_level_max)
+		node->priority = scheduler->priority_level_max;
+	else if (node->priority < scheduler->priority_level_min)
+		node->priority = scheduler->priority_level_min;
+

There is clamp_val macro in linux/kernel.h .

Quoted 20 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +20 -0
+	if ((node->priority > 0) && node->num_deps) {
+		i915_scheduler_priority_bump_clear(scheduler);
+
+		for (i = 0; i < node->num_deps; i++)
+			i915_scheduler_priority_bump(scheduler,
+					node->dep_list[i], node->priority);
+	}
+
+	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
+
+	not_flying = i915_scheduler_count_flying(scheduler, ring) <
+						 scheduler->min_flying;
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	if (not_flying)
+		i915_scheduler_submit(ring, true);
+
+	return 0;
+}

drivers/gpu/drm/i915/i915_scheduler.c line 246

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+
+static int i915_scheduler_fly_node(struct i915_scheduler_queue_entry *node)
+{
+	struct drm_i915_private *dev_priv = node->params.dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs  *ring;
+
+	WARN_ON(!scheduler);
+	WARN_ON(!node);
+	WARN_ON(node->status != i915_sqs_popped);

Other states had their I915_SQS_IS_* macro, why some don't?

Quoted 51 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +51 -0
+
+	ring = node->params.ring;
+
+	/*
+	 * Add the node (which should currently be in state popped) to the
+	 * front of the queue. This ensure that flying nodes are always held
+	 * in hardware submission order.
+	 */
+	list_add(&node->link, &scheduler->node_queue[ring->id]);
+
+	node->status = i915_sqs_flying;
+
+	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+		bool    success = true;
+
+		success = ring->irq_get(ring);
+		if (success)
+			scheduler->flags[ring->id] |= i915_sf_interrupts_enabled;
+		else
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+/*
+ * Nodes are considered valid dependencies if they are queued on any ring or
+ * if they are in flight on a different ring. In flight on the same ring is no
+ * longer interesting for non-premptive nodes as the ring serialises execution.
+ * For pre-empting nodes, all in flight dependencies are valid as they must not
+ * be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */
+static inline bool i915_scheduler_is_dependency_valid(
+			struct i915_scheduler_queue_entry *node, uint32_t idx)
+{
+	struct i915_scheduler_queue_entry *dep;
+
+	dep = node->dep_list[idx];
+	if (!dep)
+		return false;
+
+	if (I915_SQS_IS_QUEUED(dep))
+		return true;
+
+	if (I915_SQS_IS_FLYING(dep)) {
+		if (node->params.ring != dep->params.ring)
+			return true;
+	}
+

drivers/gpu/drm/i915/i915_scheduler.c line 307

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	return false;
+}
+
+static uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+					    struct intel_engine_cs *ring)
+{
+	struct i915_scheduler_queue_entry *node;
+	uint32_t                          flying = 0;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link)

This loop keeps popping up, it could use a define similar to the ones in i915_drv.h ;

#define for_each_hpd_pin(__pin) \

Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+		if (I915_SQS_IS_FLYING(node))
+			flying++;
+
+	return flying;

drivers/gpu/drm/i915/i915_scheduler.c line 321

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+}
+
+/*
+ * Add a popped node back in to the queue. For example, because the ring was
+ * hung when execfinal() was called and thus the ring submission needs to be
+ * retried later.
+ */
+static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);
	if (WARN_ON(!node))
		return;

Or rather no check at all, it's going to crash anyway even in the calling function if there's NULL, and it's internal function. It's relevant to check if the userspace.

Quoted 26 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +26 -0
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Seqno will be reassigned on relaunch */
+	node->params.request->seqno = 0;
+	node->status = i915_sqs_queued;
+}
+
+/*
+ * Give up on a node completely. For example, because it is causing the
+ * ring to hang or is using some resource that no longer exists.
+ */
+static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);
+	WARN_ON(I915_SQS_IS_COMPLETE(node));
+
+	node->status = i915_sqs_dead;
+}
+
+/*
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not.
+ */

drivers/gpu/drm/i915/i915_scheduler.c line 357

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
+{
+	struct drm_i915_private *dev_priv  = to_i915(req->ring->dev);
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	/* XXX: Need to map back from request to node */
+	struct i915_scheduler_queue_entry *node = NULL;
+	unsigned long       flags;
+
+	if (!node)
+		return false;

Not so sure if slicing the series down to an extent that functions are impossible to review, was a good idea. Idea is to slice things down, to reviewable pieces. It's hard to predict or keep looking forward the series what is going to come.

Quoted 32 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +32 -0
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Node was in flight so mark it as complete. */
+	if (req->cancelled)
+		node->status = i915_sqs_dead;
+	else
+		node->status = i915_sqs_complete;
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	/*
+	 * XXX: If the in-flight list is now empty then new work should be
+	 * submitted. However, this function is called from interrupt context
+	 * and thus cannot acquire mutex locks and other such things that are
+	 * necessary for fresh submission.
+	 */
+
+	return true;
+}
+
+int i915_scheduler_remove(struct intel_engine_cs *ring)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry  *node, *node_next;
+	unsigned long       flags;
+	int                 flying = 0, queued = 0;
+	int                 ret = 0;
+	bool                do_submit;

drivers/gpu/drm/i915/i915_scheduler.c line 399

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	uint32_t            min_seqno;
+	struct list_head    remove;
+
+	if (list_empty(&scheduler->node_queue[ring->id]))
+		return 0;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	/* /i915_scheduler_dump_locked(ring, "remove/pre");/ */
+

This should not be here at all.

Quoted 18 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +18 -0
+	/*
+	 * In the case where the system is idle, starting 'min_seqno' from a big
+	 * number will cause all nodes to be removed as they are now back to
+	 * being in-order. However, this will be a problem if the last one to
+	 * complete was actually out-of-order as the ring seqno value will be
+	 * lower than one or more completed buffers. Thus code looking for the
+	 * completion of said buffers will wait forever.
+	 * Instead, use the hardware seqno as the starting point. This means
+	 * that some buffers might be kept around even in a completely idle
+	 * system but it should guarantee that no-one ever gets confused when
+	 * waiting for buffer completion.
+	 */
+	min_seqno = ring->get_seqno(ring, true);
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))

drivers/gpu/drm/i915/i915_scheduler.c line 427

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->params.request->seqno == 0)
+			continue;
+
+		if (!i915_seqno_passed(node->params.request->seqno, min_seqno))
+			min_seqno = node->params.request->seqno;
+	}

Couldn't these values be kept cached, instead of counting them at each function?

Quoted 45 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +45 -0
+
+	INIT_LIST_HEAD(&remove);
+	list_for_each_entry_safe(node, node_next, &scheduler->node_queue[ring->id], link) {
+		/*
+		 * Only remove completed nodes which have a lower seqno than
+		 * all pending nodes. While there is the possibility of the
+		 * ring's seqno counting backwards, all higher buffers must
+		 * be remembered so that the 'i915_seqno_passed()' test can
+		 * report that they have in fact passed.
+		 *
+		 * NB: This is not true for 'dead' nodes. The GPU reset causes
+		 * the software seqno to restart from its initial value. Thus
+		 * the dead nodes must be removed even though their seqno values
+		 * are potentially vastly greater than the current ring seqno.
+		 */
+		if (!I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->status != i915_sqs_dead) {
+			if (i915_seqno_passed(node->params.request->seqno, min_seqno) &&
+			    (node->params.request->seqno != min_seqno))
+				continue;
+		}
+
+		list_del(&node->link);
+		list_add(&node->link, &remove);
+
+		/* Strip the dependency info while the mutex is still locked */
+		i915_scheduler_remove_dependent(scheduler, node);
+
+		continue;
+	}
+
+	/*
+	 * Release the interrupt reference count if there are no longer any
+	 * nodes to worry about.
+	 */
+	if (!flying && !queued &&
+	    (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+		ring->irq_put(ring);
+		scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
+	}
+
+	/* Launch more packets now? */
+	do_submit = (queued > 0) && (flying < scheduler->min_flying);

drivers/gpu/drm/i915/i915_scheduler.c line 482

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	if (!do_submit && list_empty(&remove))
+		return ret;
+
+	mutex_lock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		ret = i915_scheduler_submit(ring, true);

Confusing to have this at a remove function. Function naming needs to be reconsidered or moved out from here.

Quoted 86 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +86 -0
+
+	while (!list_empty(&remove)) {
+		node = list_first_entry(&remove, typeof(*node), link);
+		list_del(&node->link);
+
+		/*
+		 * The batch buffer must be unpinned before it is unreferenced
+		 * otherwise the unpin fails with a missing vma!?
+		 */priority_bump_clear
+		if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
+			i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
+
+		/* Free everything that is owned by the node: */
+		i915_gem_request_unreference(node->params.request);
+		kfree(node->params.cliprects);
+		kfree(node->dep_list);
+		kfree(node);
+	}
+
+	mutex_unlock(&ring->dev->struct_mutex);
+
+	return ret;
+}
+
+static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
+{
+	struct i915_scheduler_queue_entry *node;
+	int i;
+
+	/*
+	 * Ensure circular dependencies don't cause problems and that a bump
+	 * by object usage only bumps each using buffer once:
+	 */
+	for (i = 0; i < I915_NUM_RINGS; i++) {
+		list_for_each_entry(node, &scheduler->node_queue[i], link)
+			node->bumped = false;
+	}
+}
+
+static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *target,
+				uint32_t bump)
+{
+	uint32_t new_priority;
+	int      i, count;
+
+	if (target->priority >= scheduler->priority_level_max)
+		return 1;
+
+	if (target->bumped)
+		return 0;
+
+	new_priority = target->priority + bump;
+	if ((new_priority <= target->priority) ||
+	    (new_priority > scheduler->priority_level_max))
+		target->priority = scheduler->priority_level_max;
+	else
+		target->priority = new_priority;
+
+	count = 1;
+	target->bumped = true;
+
+	for (i = 0; i < target->num_deps; i++) {
+		if (!target->dep_list[i])
+			continue;
+
+		if (target->dep_list[i]->bumped)
+			continue;
+
+		count += i915_scheduler_priority_bump(scheduler,
+						      target->dep_list[i],
+						      bump);
+	}
+
+	return count;
+}
+
+static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
+				struct i915_scheduler_queue_entry **pop_node,
+				unsigned long *flags)
+{
+	struct drm_i915_private            *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler              *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry  *best;
+	struct i915_scheduler_queue_entry  *node;
+	int     ret;

drivers/gpu/drm/i915/i915_scheduler.c line 578

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	int     i;
+	bool	any_queued;
+	bool	has_local, has_remote, only_remote;
+
+	*pop_node = NULL;
+	ret = -ENODATA;
+
+	any_queued = false;
+	only_remote = false;
+	best = NULL;

These should just be initialized in-place. But looking at the code forward.

Quoted 62 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +62 -0
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+		any_queued = true;
+
+		has_local  = false;
+		has_remote = false;
+		for (i = 0; i < node->num_deps; i++) {
+			if (!i915_scheduler_is_dependency_valid(node, i))
+				continue;
+
+			if (node->dep_list[i]->params.ring == node->params.ring)
+				has_local = true;
+			else
+				has_remote = true;
+		}
+
+		if (has_remote && !has_local)
+			only_remote = true;
+
+		if (!has_local && !has_remote) {
+			if (!best ||
+			    (node->priority > best->priority))
+				best = node;
+		}
+	}
+
+	if (best) {
+		list_del(&best->link);
+
+		INIT_LIST_HEAD(&best->link);
+		best->status  = i915_sqs_popped;
+
+		ret = 0;
+	} else {
+		/* Can only get here if:
+		 * (a) there are no buffers in the queue
+		 * (b) all queued buffers are dependent on other buffers
+		 *     e.g. on a buffer that is in flight on a different ring
+		 */
+		if (only_remote) {
+			/* The only dependent buffers are on another ring. */
+			ret = -EAGAIN;
+		} else if (any_queued) {
+			/* It seems that something has gone horribly wrong! */
+			DRM_ERROR("Broken dependency tracking on ring %d!\n",
+				  (int) ring->id);
+		}
+	}
+
+	/* i915_scheduler_dump_queue_pop(ring, best); */
+
+	*pop_node = best;
+	return ret;
+}
+
+static int i915_scheduler_submit(struct intel_engine_cs *ring, bool was_locked)
+{
+	struct drm_device   *dev = ring->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;

drivers/gpu/drm/i915/i915_scheduler.c line 650

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	struct i915_scheduler_queue_entry  *node;
+	unsigned long       flags;
+	int                 ret = 0, count = 0;
+
+	if (!was_locked) {
+		ret = i915_mutex_lock_interruptible(dev);
+		if (ret)
+			return ret;
+	}
+

I don't really fancy this construct. Should be moved outside of this function for proper lockdep tracking.

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	/* First time around, complain if anything unexpected occurs: */

drivers/gpu/drm/i915/i915_scheduler.c line 665

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	ret = i915_scheduler_pop_from_queue_locked(ring, &node, &flags);
+	if (ret) {
+		spin_unlock_irqrestore(&scheduler->lock, flags);
+
+		if (!was_locked)
+			mutex_unlock(&dev->struct_mutex);
+
+		return ret;
+	}
+

Dropping the was_locked stuff, this should become a proper goto error label. e.g. out_unlock

drivers/gpu/drm/i915/i915_scheduler.c line 676

Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +11 -0
+	do {
+		WARN_ON(!node);
+		WARN_ON(node->params.ring != ring);
+		WARN_ON(node->status != i915_sqs_popped);
+		count++;
+
+		/*
+		 * The call to pop above will have removed the node from the
+		 * list. So add it back in and mark it as in flight.
+		 */
+		i915_scheduler_fly_node(node);

Why do we want to pull an object out of the list inside spin lock and push it back immediately in our critical code path? Seems like a waste for no obvious gain at this point. Why do not we rather just select an entry and modify it in-place, if it's going to stay in the same queue anyway.

drivers/gpu/drm/i915/i915_scheduler.c line 685

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+
+		scheduler->flags[ring->id] |= i915_sf_submitting;
+		spin_unlock_irqrestore(&scheduler->lock, flags);
+		ret = dev_priv->gt.execbuf_final(&node->params);
+		spin_lock_irqsave(&scheduler->lock, flags);
+		scheduler->flags[ring->id] &= ~i915_sf_submitting;
+
+		if (ret) {
+			int requeue = 1;

Multipurpose variable, not really a good idea. And as commented further, should not exist at all.

drivers/gpu/drm/i915/i915_scheduler.c line 698

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +13 -0
+
+			/*
+			 * Oh dear! Either the node is broken or the ring is
+			 * busy. So need to kill the node or requeue it and try
+			 * again later as appropriate.
+			 */
+
+			switch (-ret) {
+			case ENODEV:
+			case ENOENT:
+				/* Fatal errors. Kill the node. */
+				requeue = -1;
+			break;

"break" indent is wrong.

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+
+			case EAGAIN:
+			case EBUSY:
+			case EIO:
+			case ENOMEM:
+			case ERESTARTSYS:
+			case EINTR:

drivers/gpu/drm/i915/i915_scheduler.c line 715

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+				/* Supposedly recoverable errors. */
+			break;
+
+			default:
+				/*
+				 * Assume the error is recoverable and hope
+				 * for the best.
+				 */
+				DRM_DEBUG_DRIVER("<%s> Got unexpected error from execfinal(): %d!\n",
+						 ring->name, ret);

There's MISSING_CASE macro, should use it.

drivers/gpu/drm/i915/i915_scheduler.c line 114

Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+			break;
+			}
+

Just move the code below this point to the switch, no point having a switch to categorize your options and then doing bunch of ifs to execute code that could be in switch.

Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+			/*
+			 * Check that the watchdog/reset code has not nuked
+			 * the node while we weren't looking:
+			 */
+			if (node->status == i915_sqs_dead)
+				requeue = 0;
+
+			if (requeue == 1) {
+				i915_scheduler_node_requeue(node);

drivers/gpu/drm/i915/i915_scheduler.c line 737

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+				/*
+				 * No point spinning if the ring is currently
+				 * unavailable so just give up and come back
+				 * later.
+				 */
+				break;
+			} else if (requeue == -1)
+				i915_scheduler_node_kill(node);
+		}
+

Ending here, this actual submission of a single node could go to its own helper function, these functions now become too long to follow, although they really are not doing anything complicated.

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+		/* Keep launching until the sky is sufficiently full. */
+		if (i915_scheduler_count_flying(scheduler, ring) >=
+						scheduler->min_flying)
+			break;
+
+		ret = i915_scheduler_pop_from_queue_locked(ring, &node, &flags);
+	} while (ret == 0);
+

drivers/gpu/drm/i915/i915_scheduler.c line 755

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	if (!was_locked)
+		mutex_unlock(&dev->struct_mutex);
+
+	/* Don't complain about not being able to submit extra entries */
+	if (ret == -ENODATA)
+		ret = 0;
+
+	return (ret < 0) ? ret : count;

This is a combined error and success path, keeping to the convention of kernel drivers is preferred;

	if (ret != -ENODATA)
		goto out_foo;

	return count;

out_unlock:

	spin_unlock(...);
out_foo:
	return ret;
Quoted 81 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +42 -0
+}
+
+static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *remove)
+{
+	struct i915_scheduler_queue_entry  *node;
+	int     i, r;
+	int     count = 0;
+
+	/*
+	 * Ensure that a node is not being removed which is still dependent
+	 * upon other (not completed) work. If that happens, it implies
+	 * something has gone very wrong with the dependency tracking! Note
+	 * that there is no need to worry if this node has been explicitly
+	 * killed for some reason - it might be being killed before it got
+	 * sent to the hardware.
+	 */
+	if (remove->status != i915_sqs_dead) {
+		for (i = 0; i < remove->num_deps; i++)
+			if ((remove->dep_list[i]) &&
+			    (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
+				count++;
+		WARN_ON(count);
+	}
+
+	/*
+	 * Remove this node from the dependency lists of any other node which
+	 * might be waiting on it.
+	 */
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(node, &scheduler->node_queue[r], link) {
+			for (i = 0; i < node->num_deps; i++) {
+				if (node->dep_list[i] != remove)
+					continue;
+
+				node->dep_list[i] = NULL;
+			}
+		}
+	}
+
+	return 0;
+}
drivers/gpu/drm/i915/i915_scheduler.h new file +33 -0
new file mode 100644
index 0000000..00dc7f3
@@ -0,0 +1,91 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _I915_SCHEDULER_H_
+#define _I915_SCHEDULER_H_
+
+enum i915_scheduler_queue_status {
+	/* Limbo: */
+	i915_sqs_none = 0,
+	/* Not yet submitted to hardware: */
+	i915_sqs_queued,
+	/* Popped from queue, ready to fly: */

drivers/gpu/drm/i915/i915_scheduler.h line 43

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	i915_sqs_popped,
+	/* Sent to hardware for processing: */
+	i915_sqs_flying,
+	/* Finished processing on the hardware: */
+	i915_sqs_complete,
+	/* Killed by watchdog or catastrophic submission failure: */
+	i915_sqs_dead,
+	/* Limit value for use with arrays/loops */
+	i915_sqs_MAX
+};

These should be UPPERCASE_FOR_ENUM_VALUES . See i915_drv.h for samples.

drivers/gpu/drm/i915/i915_scheduler.h line 49

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +6 -0
+
+#define I915_SQS_IS_QUEUED(node)	(((node)->status == i915_sqs_queued))
+#define I915_SQS_IS_FLYING(node)	(((node)->status == i915_sqs_flying))
+#define I915_SQS_IS_COMPLETE(node)	(((node)->status == i915_sqs_complete) || \
+					 ((node)->status == i915_sqs_dead))
+

Might be slightly confusing that name is IS_COMPLETE and there is actual COMPLETE value. Rather have the test like IS_DONE and then test for COMPLETE or DEAD, no confusion. Also, some states have their IS_* macro and others don't, is there going to be more?

Quoted 25 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +25 -0
+struct i915_scheduler_obj_entry {
+	struct drm_i915_gem_object          *obj;
+	bool                                read_only;
+};
+
+struct i915_scheduler_queue_entry {
+	struct i915_execbuffer_params       params;
+	/* -1023 = lowest priority, 0 = default, 1023 = highest */
+	int32_t                             priority;
+	struct i915_scheduler_obj_entry     *saved_objects;
+	int                                 num_objs;
+	bool                                bumped;
+	struct i915_scheduler_queue_entry   **dep_list;
+	int                                 num_deps;
+	enum i915_scheduler_queue_status    status;
+	unsigned long                       stamp;
+	struct list_head                    link;
+};
+
+struct i915_scheduler {
+	struct list_head    node_queue[I915_NUM_RINGS];
+	uint32_t            flags[I915_NUM_RINGS];
+	spinlock_t          lock;
+
+	/* Tuning parameters: */

drivers/gpu/drm/i915/i915_scheduler.h line 84

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	int32_t             priority_level_min;
+	int32_t             priority_level_max;
+	int32_t             priority_level_preempt;
+	uint32_t            min_flying;
+};
+
+/* Flag bits for i915_scheduler::flags */
+enum {
+	i915_sf_interrupts_enabled  = (1 << 0),
+	i915_sf_submitting          = (1 << 1),

Again, should be uppercase. Also, enums to the beginning of file.

drivers/gpu/drm/i915/i915_scheduler.h line 91

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +7 -0
+};
+
+int         i915_scheduler_init(struct drm_device *dev);
+int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
+
+#endif  /* _I915_SCHEDULER_H_ */

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#86 lore raw

Re: [PATCH v4 07/38] drm/i915: Start of GPU scheduler

Quoted 326 lines · Joonas Lahtinen

Hi,

Comments below this pre text.

Many of the comments are related to the indent and style of the code. That stuff is important to fix for future maintainability. In order for the future review to be more effective, I'd like to next see a v5 of the series where the code quality concerns have been addressed, patches squashed to be actual reviewable chunks and appropriate kerneldoc being added.

To give an idea of proper slicing of patches, first produce a no-op scheduler, adding the extra function calls where needed and still keeping the scheduling completely linear. Second patch could introduce out of order submitting, third one priority bumping, fourth pre-empting and so on. That way, each patch extends the functionality and is itself already mergeable. That way I've been able to go through and understand the existing code, and I can actually review (other than just nag about indent and coding style) if the changes are appropriate to bring in the functionality desired.

In the current split, for me or anyone who did not participate writing the code, it is otherwise too confusing to try to guess what future changes might make each piece of code make sense, and which will be redundant in the future too. There is no value in splitting code to chunks that are not itself functional.

Regards, Joonas

Quoted 137 lines · On Mon, 2016-01-11 at 18:42 +0000, [email protected] wrote:

From: John Harrison <[email protected]>

Initial creation of scheduler source files. Note that this patch implements most of the scheduler functionality but does not hook it in
to the driver yet. It also leaves the scheduler code in 'pass through'
mode so that even when it is hooked in, it will not actually do very much. This allows the hooks to be added one at a time in bite size chunks and only when the scheduler is finally enabled at the end does anything start happening.

The general theory of operation is that when batch buffers are submitted to the driver, the execbuffer() code packages up all the information required to execute the batch buffer at a later time. This
package is given over to the scheduler which adds it to an internal node list. The scheduler also scans the list of objects associated with the batch buffer and compares them against the objects already in
use by other buffers in the node list. If matches are found then the new batch buffer node is marked as being dependent upon the matching node. The same is done for the context object. The scheduler also bumps up the priority of such matching nodes on the grounds that the more dependencies a given batch buffer has the more important it is likely to be.

The scheduler aims to have a given (tuneable) number of batch buffers in flight on the hardware at any given time. If fewer than this are currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added to the queue and the driver returns back to user land.

The scheduler is notified when each batch buffer completes and updates
its internal tracking accordingly. At the end of the completion interrupt processing, if any scheduler tracked batches were processed,
the scheduler's deferred worker thread is woken up. This can do more involved processing such as actually removing completed nodes from the
queue and freeing up the resources associated with them (internal memory allocations, DRM object references, context reference, etc.). The work handler also checks the in flight count and calls the submission code if a new slot has appeared.

When the scheduler's submit code is called, it scans the queued node list for the highest priority node that has no unmet dependencies. Note that the dependency calculation is complex as it must take inter-ring dependencies and potential preemptions into account. Note also that in the future this will be extended to include external dependencies such as the Android Native Sync file descriptors and/or the linux dma-buff synchronisation scheme.

If a suitable node is found then it is sent to execbuff_final() for submission to the hardware. The in flight count is then re-checked and
a new node popped from the list if appropriate.

Note that this patch does not implement pre-emptive scheduling. Only basic scheduling by re-ordering batch buffer submission is currently implemented. Pre-emption of actively executing batch buffers comes in the next patch series.

v2: Changed priority levels to +/-1023 due to feedback from Chris Wilson.

Removed redundant index from scheduler node.

Changed time stamps to use jiffies instead of raw monotonic. This provides lower resolution but improved compatibility with other i915 code.

Major re-write of completion tracking code due to struct fence conversion. The scheduler no longer has it's own private IRQ handler but just lets the existing request code handle completion events. Instead, the scheduler now hooks into the request notify code to be told when a request has completed.

Reduced driver mutex locking scope. Removal of scheduler nodes no longer grabs the mutex lock.

v3: Refactor of dependency generation to make the code more readable. Also added in read-read optimisation support - i.e., don't treat a shared read-only buffer as being a dependency.

Allowed the killing of queued nodes rather than only flying ones.

v4: Updated the commit message to better reflect the current state of the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array memory allocator function (kmalloc_array instead of kmalloc). Corrected the format of some comments. Wrapped some lines differently to keep the style checker happy.

Fixed a WARN_ON when killing nodes. The dependency removal code checks
that nodes being destroyed do not have any oustanding dependencies (which would imply they should not have been executed yet). In the case of nodes being destroyed, e.g. due to context banning, then this might well be the case - they have not been executed and do indeed have outstanding dependencies.

Re-instated the code to disble interrupts when not in use. The underlying problem causing broken IRQ reference counts seems to have been fixed now.

For: VIZ-1587

---
  drivers/gpu/drm/i915/Makefile         |   1 +
  drivers/gpu/drm/i915/i915_drv.h       |   4 +
  drivers/gpu/drm/i915/i915_gem.c       |   5 +
  drivers/gpu/drm/i915/i915_scheduler.c | 797
++++++++++++++++++++++++++++++++++
  drivers/gpu/drm/i915/i915_scheduler.h |  91 ++++
  5 files changed, 898 insertions(+)

create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h

drivers/gpu/drm/i915/Makefile +1 -0
index 15398c5..79cb38b 100644
@@ -10,6 +10,7 @@ ccflags-y := -Werror
  i915-y := i915_drv.o \
  	  i915_irq.o \
  	  i915_params.o \
+	  i915_scheduler.o \
            i915_suspend.o \
  	  i915_sysfs.o \
  	  intel_csr.o \
drivers/gpu/drm/i915/i915_drv.h +0 -0
index 10dffdd..38f423b 100644

drivers/gpu/drm/i915/i915_drv.h struct i915_execbuffer_params { · line 1699

Quoted 5 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
@@ -1695,6 +1695,8 @@ struct i915_execbuffer_params {
  	struct drm_i915_gem_request     *request;
  };
 
+struct i915_scheduler;
+

Rather add "i915_scheduler.h" include at the top and eliminate circular include dependencies. This is needed for the next comment.

drivers/gpu/drm/i915/i915_drv.h struct drm_i915_private { · line 1953

Quoted 9 lines
drivers/gpu/drm/i915/i915_drv.h +2 -0
  /* used in computing the new watermarks state */
  struct intel_wm_config {
  	unsigned int num_pipes_active;
@@ -1947,6 +1949,8 @@ struct drm_i915_private {
 
  	struct i915_runtime_pm pm;
 
+	struct i915_scheduler *scheduler;
+

As the scheduler is going to be enabled on all platforms to an extent, no point in making it a pointer. Just making it member like "pm" is the best, this also requires the above include change.

drivers/gpu/drm/i915/i915_gem.c line 41

Quoted 13 lines
drivers/gpu/drm/i915/i915_drv.h +0 -0
  	/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
  	struct {
  		int (*execbuf_submit)(struct i915_execbuffer_params *params,
drivers/gpu/drm/i915/i915_gem.c +1 -0
index cff3768..47aa85b 100644
@@ -37,6 +37,7 @@
  #include
  #include
  #include
+#include "i915_scheduler.h"
 

This should go before any <linux/...> not to mask missing includes from the header itself, so correct place would be right after #include "i915_trace.h"

Quoted 36 lines
drivers/gpu/drm/i915/i915_gem.c +4 -0
  #define RQ_BUG_ON(expr)
 
@@ -5242,6 +5243,10 @@ int i915_gem_init(struct drm_device *dev)
  	 */
  	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
 
+	ret = i915_scheduler_init(dev);
+	if (ret)
+		goto out_unlock;
+
  	ret = i915_gem_init_userptr(dev);
  	if (ret)
  		goto out_unlock;
drivers/gpu/drm/i915/i915_scheduler.c new file +17 -0
new file mode 100644
index 0000000..8cb9063
@@ -0,0 +1,797 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL

drivers/gpu/drm/i915/i915_scheduler.c line 27

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+#include "intel_drv.h"
+#include "i915_scheduler.h"

Again, this include should be the top one, should not require any includes before it.

drivers/gpu/drm/i915/i915_scheduler.c line 39

Quoted 12 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +12 -0
+
+static int         i915_scheduler_fly_node(struct i915_scheduler_queue_entry *node);
+static int         i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+						   struct i915_scheduler_queue_entry *remove);
+static int         i915_scheduler_submit(struct intel_engine_cs *ring,
+					 bool is_locked);
+static uint32_t    i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+					       struct intel_engine_cs *ring);
+static void        i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler);
+static int         i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+						struct i915_scheduler_queue_entry *target,
+						uint32_t bump);

Do not indent the function names like this, it becomes unmaintainable and messy very fast if somebody adds a new function with a more complex return type, which is a very likely thing to happen.

What I would do is move all the helper functions here at the top and order them so that the forward declarations are not needed, at least i915_scheduler_fly_node is not used before its definition.

drivers/gpu/drm/i915/i915_scheduler.c line 45

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+
+int i915_scheduler_init(struct drm_device *dev)
+{
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	int                     r;

Again, this indent is a no-go. I'll not mention it on further functions, assume it to be fixed for next revision.

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+
+	if (scheduler)
+		return 0;
+
+	scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
+	if (!scheduler)
+		return -ENOMEM;
+

drivers/gpu/drm/i915/i915_scheduler.c line 63

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	spin_lock_init(&scheduler->lock);
+
+	for (r = 0; r < I915_NUM_RINGS; r++)
+		INIT_LIST_HEAD(&scheduler->node_queue[r]);
+
+	/* Default tuning values: */
+	scheduler->priority_level_min     = -1023;
+	scheduler->priority_level_max     = 1023;
+	scheduler->priority_level_preempt = 900;
+	scheduler->min_flying             = 2;

This kind of indent is tolerable because it is a contained code block, but not needed either.

Quoted 22 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +22 -0
+
+	dev_priv->scheduler = scheduler;
+
+	return 0;
+}
+
+static void i915_generate_dependencies(struct i915_scheduler *scheduler,
+				       struct i915_scheduler_queue_entry *node,
+				       uint32_t ring)
+{
+	struct i915_scheduler_obj_entry *this, *that;
+	struct i915_scheduler_queue_entry  *test;
+	int                 i, j;
+	bool                found;
+
+	list_for_each_entry(test, &scheduler->node_queue[ring], link) {
+		if (I915_SQS_IS_COMPLETE(test))
+			continue;
+
+		/*
+		 * Batches on the same ring for the same
+		 * context must be kept in order.
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 92

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+		 */
+		found = (node->params.ctx == test->params.ctx) &&
+			(node->params.ring == test->params.ring);
+
+		if (found)
+			goto depends;
+

This is not needed.

Yeah, this code block has been re-worked a number of times with things being added and/or removed. I guess it had gotten a bit messy by now.

Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 97

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+		/*
+		 * Batches working on the same objects must
+		 * be kept in order.
+		 */
+		for (i = 0; (i < node->num_objs) && !found; i++) {

As the test is here already ---^

Quoted 9 lines

drivers/gpu/drm/i915/i915_scheduler.c line 104

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+			this = node->saved_objects + i;
+
+			for (j = 0; j < test->num_objs; j++) {
+				that = test->saved_objects + j;
+
+				if (this->obj != that->obj)
+					continue;

How about VMAs? There might be multiple mappings to an object, isn't it enough to depend on the required VMA instead of the whole object?

The object is what we get coming in from user land through the IOCTL. So why make things more complicated? If there are multiple VMAs referring to the same object then we can't just track an individual VMA as that would loose the dependency on all the other VMAs. Just because the object is mapped to someone else's address space doesn't mean that this batch buffer can't overwrite data they are reading.

Quoted 42 lines

drivers/gpu/drm/i915/i915_scheduler.c line 114

Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+
+				/* Only need to worry about writes */
+				if (this->read_only && that->read_only)
+					continue;
+
+				found = true;
+				break;
+			}
+		}
+

The following block is not needed.

drivers/gpu/drm/i915/i915_scheduler.c line 118

Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+		if (!found)
+			continue;
+
+depends:

Rather like this, in order to avoid a goto label;
if (found) {
...

drivers/gpu/drm/i915/i915_scheduler.c line 123

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+		node->dep_list[node->num_deps] = test;
+		node->num_deps++;
+	}
+}
+

Please add a brief kerneldoc above each function in the header, it's required. Adding it to non-trivial inline helper functions too will make reviewing much easier.

Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +11 -0
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
+{
+	struct drm_i915_private *dev_priv = qe->params.dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs  *ring = qe->params.ring;
+	struct i915_scheduler_queue_entry  *node;
+	struct i915_scheduler_queue_entry  *test;
+	unsigned long       flags;
+	bool                not_flying;
+	int                 i, r;
+	int                 incomplete = 0;
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 137

Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+
+	WARN_ON(!scheduler);
+

This kind of situations should have a be a BUG_ON, because scheduler being zero is literally going to cause an OOPS in the next dereference which is going to happen unconditionally. WARN + OOPS is kind of what BUG_ON should be used avoid. But this should be removed anyway after scheduler is made a data member of dev_priv.

The WARNs were originally BUGs but Daniel Vetter had the opposite demand. His view was the driver should never BUG under any circumstances. A WARN followed by an oops is better than a BUG because maybe it won't actually oops.

Quoted 29 lines

drivers/gpu/drm/i915/i915_scheduler.c line 138

Quoted 2 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +1 -0
+	if (1/*i915.scheduler_override & i915_so_direct_submit*/) {

I assume this is going to be addressed in a future commit. Could have been introduced in this patch, too.

drivers/gpu/drm/i915/i915_scheduler.c line 144

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+		int ret;
+
+		scheduler->flags[qe->params.ring->id] |= i915_sf_submitting;
+		ret = dev_priv->gt.execbuf_final(&qe->params);
+		scheduler->flags[qe->params.ring->id] &= ~i915_sf_submitting;
+

The kerneldoc should mention locking requirements of this function.

drivers/gpu/drm/i915/i915_scheduler.c line 157

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +13 -0
+		/*
+		 * Don't do any clean up on failure because the caller will
+		 * do it all anyway.
+		 */
+		if (ret)
+			return ret;
+
+		/* Free everything that is owned by the QE structure: */
+		kfree(qe->params.cliprects);
+		if (qe->params.dispatch_flags & I915_DISPATCH_SECURE)
+			i915_gem_execbuff_release_batch_obj(qe->params.batch_obj);
+
+		return 0;

Above piece of code looks like its own function, so it should probably be one.

Quoted 9 lines

drivers/gpu/drm/i915/i915_scheduler.c line 164

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	}
+
+	node = kmalloc(sizeof(*node), GFP_KERNEL);
+	if (!node)
+		return -ENOMEM;
+
+	*node = *qe;

Any reason we can't simply move ownership of qe? If not, I'd rather make a clone function

The qe pointer passed in is a reference to a stack local object in the execbuff code path. Thus ownership cannot be transferred. Doing it this way keeps the execbuff code nice and simple and all the dynamic memory management and list tracking is self contained within the scheduler.

Quoted 21 lines
Quoted 21 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +20 -0
+	INIT_LIST_HEAD(&node->link);
+	node->status = i915_sqs_queued;
+	node->stamp  = jiffies;
+	i915_gem_request_reference(node->params.request);
+
+	/* Need to determine the number of incomplete entries in the list as
+	 * that will be the maximum size of the dependency list.
+	 *
+	 * Note that the allocation must not be made with the spinlock acquired
+	 * as kmalloc can sleep. However, the unlock/relock is safe because no
+	 * new entries can be queued up during the unlock as the i915 driver
+	 * mutex is still held. Entries could be removed from the list but that
+	 * just means the dep_list will be over-allocated which is fine.
+	 */
+	spin_lock_irqsave(&scheduler->lock, flags);
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(test, &scheduler->node_queue[r], link) {
+			if (I915_SQS_IS_COMPLETE(test))
+				continue;
+
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 190

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+			incomplete++;
+		}
+	}
+
+	/* Temporarily unlock to allocate memory: */
+	spin_unlock_irqrestore(&scheduler->lock, flags);

I'd make the above piece of code a helper, these stats are to be counted for debugfs anyway, too?

The debugfs code counts all states, not just complete, and requires a scheduler stats object. Also, the above explanation about locking is not applicable in the debugfs case, only here. And it makes much more sense to me to keep the comment and the code all together in one very obvious and understandable block than splitting it up.

Quoted 57 lines
Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+	if (incomplete) {
+		node->dep_list = kmalloc_array(incomplete,
+					       sizeof(*node->dep_list),
+					       GFP_KERNEL);
+		if (!node->dep_list) {
+			kfree(node);
+			return -ENOMEM;
+		}
+	} else

drivers/gpu/drm/i915/i915_scheduler.c line 209

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+		node->dep_list = NULL;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+	node->num_deps = 0;
+
+	if (node->dep_list) {
+		for (r = 0; r < I915_NUM_RINGS; r++)
+			i915_generate_dependencies(scheduler, node, r);
+
+		WARN_ON(node->num_deps > incomplete);

Wouldn't this condition again lead to a crash? If so, should be BUG_ON to cause that crash as early as possible. WARN_ON is only good if there is a way of coping with the situation and no imminent system crash is bound to happen.

drivers/gpu/drm/i915/i915_scheduler.c line 216

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	}
+
+	if (node->priority > scheduler->priority_level_max)
+		node->priority = scheduler->priority_level_max;
+	else if (node->priority < scheduler->priority_level_min)
+		node->priority = scheduler->priority_level_min;
+

There is clamp_val macro in linux/kernel.h .

Quoted 23 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +23 -0
+	if ((node->priority > 0) && node->num_deps) {
+		i915_scheduler_priority_bump_clear(scheduler);
+
+		for (i = 0; i < node->num_deps; i++)
+			i915_scheduler_priority_bump(scheduler,
+					node->dep_list[i], node->priority);
+	}
+
+	list_add_tail(&node->link, &scheduler->node_queue[ring->id]);
+
+	not_flying = i915_scheduler_count_flying(scheduler, ring) <
+						 scheduler->min_flying;
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	if (not_flying)
+		i915_scheduler_submit(ring, true);
+
+	return 0;
+}
+
+static int i915_scheduler_fly_node(struct i915_scheduler_queue_entry *node)
+{
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 246

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	struct drm_i915_private *dev_priv = node->params.dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct intel_engine_cs  *ring;
+
+	WARN_ON(!scheduler);
+	WARN_ON(!node);
+	WARN_ON(node->status != i915_sqs_popped);

Other states had their I915_SQS_IS_* macro, why some don't?

The purpose of the macro is to allow the combining of individual states into classes. E.g. dead and complete can both be considered complete for the majority of cases. Only in certain situations do you need to know that it really was dead. Hence most places that don't really care just use the merged macros, whereas places like this that do care use the explicit enum value.

Quoted 164 lines
Quoted 52 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +51 -0
+
+	ring = node->params.ring;
+
+	/*
+	 * Add the node (which should currently be in state popped) to the
+	 * front of the queue. This ensure that flying nodes are always held
+	 * in hardware submission order.
+	 */
+	list_add(&node->link, &scheduler->node_queue[ring->id]);
+
+	node->status = i915_sqs_flying;
+
+	if (!(scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+		bool    success = true;
+
+		success = ring->irq_get(ring);
+		if (success)
+			scheduler->flags[ring->id] |= i915_sf_interrupts_enabled;
+		else
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+/*
+ * Nodes are considered valid dependencies if they are queued on any ring or
+ * if they are in flight on a different ring. In flight on the same ring is no
+ * longer interesting for non-premptive nodes as the ring serialises execution.
+ * For pre-empting nodes, all in flight dependencies are valid as they must not
+ * be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */
+static inline bool i915_scheduler_is_dependency_valid(
+			struct i915_scheduler_queue_entry *node, uint32_t idx)
+{
+	struct i915_scheduler_queue_entry *dep;
+
+	dep = node->dep_list[idx];
+	if (!dep)
+		return false;
+
+	if (I915_SQS_IS_QUEUED(dep))
+		return true;
+
+	if (I915_SQS_IS_FLYING(dep)) {
+		if (node->params.ring != dep->params.ring)
+			return true;
+	}
+

drivers/gpu/drm/i915/i915_scheduler.c line 307

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	return false;
+}
+
+static uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+					    struct intel_engine_cs *ring)
+{
+	struct i915_scheduler_queue_entry *node;
+	uint32_t                          flying = 0;
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link)

This loop keeps popping up, it could use a define similar to the ones in i915_drv.h ;

#define for_each_hpd_pin(__pin) \

Quoted 4 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +4 -0
+		if (I915_SQS_IS_FLYING(node))
+			flying++;
+
+	return flying;

drivers/gpu/drm/i915/i915_scheduler.c line 321

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+}
+
+/*
+ * Add a popped node back in to the queue. For example, because the ring was
+ * hung when execfinal() was called and thus the ring submission needs to be
+ * retried later.
+ */
+static void i915_scheduler_node_requeue(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);
	if (WARN_ON(!node))
		return;

Or rather no check at all, it's going to crash anyway even in the calling function if there's NULL, and it's internal function. It's relevant to check if the userspace.

Quoted 26 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +26 -0
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Seqno will be reassigned on relaunch */
+	node->params.request->seqno = 0;
+	node->status = i915_sqs_queued;
+}
+
+/*
+ * Give up on a node completely. For example, because it is causing the
+ * ring to hang or is using some resource that no longer exists.
+ */
+static void i915_scheduler_node_kill(struct i915_scheduler_queue_entry *node)
+{
+	WARN_ON(!node);
+	WARN_ON(I915_SQS_IS_COMPLETE(node));
+
+	node->status = i915_sqs_dead;
+}
+
+/*
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not.
+ */

drivers/gpu/drm/i915/i915_scheduler.c line 357

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
+{
+	struct drm_i915_private *dev_priv  = to_i915(req->ring->dev);
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	/* XXX: Need to map back from request to node */
+	struct i915_scheduler_queue_entry *node = NULL;
+	unsigned long       flags;
+
+	if (!node)
+		return false;

Not so sure if slicing the series down to an extent that functions are impossible to review, was a good idea. Idea is to slice things down, to reviewable pieces. It's hard to predict or keep looking forward the series what is going to come.

Quoted 35 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +35 -0
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	WARN_ON(!I915_SQS_IS_FLYING(node));
+
+	/* Node was in flight so mark it as complete. */
+	if (req->cancelled)
+		node->status = i915_sqs_dead;
+	else
+		node->status = i915_sqs_complete;
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	/*
+	 * XXX: If the in-flight list is now empty then new work should be
+	 * submitted. However, this function is called from interrupt context
+	 * and thus cannot acquire mutex locks and other such things that are
+	 * necessary for fresh submission.
+	 */
+
+	return true;
+}
+
+int i915_scheduler_remove(struct intel_engine_cs *ring)
+{
+	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry  *node, *node_next;
+	unsigned long       flags;
+	int                 flying = 0, queued = 0;
+	int                 ret = 0;
+	bool                do_submit;
+	uint32_t            min_seqno;
+	struct list_head    remove;
+
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 399

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+	if (list_empty(&scheduler->node_queue[ring->id]))
+		return 0;
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	/* /i915_scheduler_dump_locked(ring, "remove/pre");/ */
+

This should not be here at all.

Oops. Old debug code. Hadn't spotted it was still there.

Quoted 23 lines
Quoted 23 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +22 -0
+	/*
+	 * In the case where the system is idle, starting 'min_seqno' from a big
+	 * number will cause all nodes to be removed as they are now back to
+	 * being in-order. However, this will be a problem if the last one to
+	 * complete was actually out-of-order as the ring seqno value will be
+	 * lower than one or more completed buffers. Thus code looking for the
+	 * completion of said buffers will wait forever.
+	 * Instead, use the hardware seqno as the starting point. This means
+	 * that some buffers might be kept around even in a completely idle
+	 * system but it should guarantee that no-one ever gets confused when
+	 * waiting for buffer completion.
+	 */
+	min_seqno = ring->get_seqno(ring, true);
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (I915_SQS_IS_QUEUED(node))
+			queued++;
+		else if (I915_SQS_IS_FLYING(node))
+			flying++;
+		else if (I915_SQS_IS_COMPLETE(node))
+			continue;
+
Quoted 8 lines

drivers/gpu/drm/i915/i915_scheduler.c line 427

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +6 -0
+		if (node->params.request->seqno == 0)
+			continue;
+
+		if (!i915_seqno_passed(node->params.request->seqno, min_seqno))
+			min_seqno = node->params.request->seqno;
+	}

Couldn't these values be kept cached, instead of counting them at each function?

The 'queued' and flying totals could be kept cached but min_seqno is dependent upon the state of the hardware so needs to be recalculated. In which case calculating the totals here is trivial and avoids having extra code elsewhere to keep them up to date.

Quoted 251 lines
Quoted 46 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +45 -0
+
+	INIT_LIST_HEAD(&remove);
+	list_for_each_entry_safe(node, node_next, &scheduler->node_queue[ring->id], link) {
+		/*
+		 * Only remove completed nodes which have a lower seqno than
+		 * all pending nodes. While there is the possibility of the
+		 * ring's seqno counting backwards, all higher buffers must
+		 * be remembered so that the 'i915_seqno_passed()' test can
+		 * report that they have in fact passed.
+		 *
+		 * NB: This is not true for 'dead' nodes. The GPU reset causes
+		 * the software seqno to restart from its initial value. Thus
+		 * the dead nodes must be removed even though their seqno values
+		 * are potentially vastly greater than the current ring seqno.
+		 */
+		if (!I915_SQS_IS_COMPLETE(node))
+			continue;
+
+		if (node->status != i915_sqs_dead) {
+			if (i915_seqno_passed(node->params.request->seqno, min_seqno) &&
+			    (node->params.request->seqno != min_seqno))
+				continue;
+		}
+
+		list_del(&node->link);
+		list_add(&node->link, &remove);
+
+		/* Strip the dependency info while the mutex is still locked */
+		i915_scheduler_remove_dependent(scheduler, node);
+
+		continue;
+	}
+
+	/*
+	 * Release the interrupt reference count if there are no longer any
+	 * nodes to worry about.
+	 */
+	if (!flying && !queued &&
+	    (scheduler->flags[ring->id] & i915_sf_interrupts_enabled)) {
+		ring->irq_put(ring);
+		scheduler->flags[ring->id] &= ~i915_sf_interrupts_enabled;
+	}
+
+	/* Launch more packets now? */
+	do_submit = (queued > 0) && (flying < scheduler->min_flying);

drivers/gpu/drm/i915/i915_scheduler.c line 482

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	if (!do_submit && list_empty(&remove))
+		return ret;
+
+	mutex_lock(&ring->dev->struct_mutex);
+
+	if (do_submit)
+		ret = i915_scheduler_submit(ring, true);

Confusing to have this at a remove function. Function naming needs to be reconsidered or moved out from here.

Quoted 86 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +86 -0
+
+	while (!list_empty(&remove)) {
+		node = list_first_entry(&remove, typeof(*node), link);
+		list_del(&node->link);
+
+		/*
+		 * The batch buffer must be unpinned before it is unreferenced
+		 * otherwise the unpin fails with a missing vma!?
+		 */priority_bump_clear
+		if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
+			i915_gem_execbuff_release_batch_obj(node->params.batch_obj);
+
+		/* Free everything that is owned by the node: */
+		i915_gem_request_unreference(node->params.request);
+		kfree(node->params.cliprects);
+		kfree(node->dep_list);
+		kfree(node);
+	}
+
+	mutex_unlock(&ring->dev->struct_mutex);
+
+	return ret;
+}
+
+static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
+{
+	struct i915_scheduler_queue_entry *node;
+	int i;
+
+	/*
+	 * Ensure circular dependencies don't cause problems and that a bump
+	 * by object usage only bumps each using buffer once:
+	 */
+	for (i = 0; i < I915_NUM_RINGS; i++) {
+		list_for_each_entry(node, &scheduler->node_queue[i], link)
+			node->bumped = false;
+	}
+}
+
+static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *target,
+				uint32_t bump)
+{
+	uint32_t new_priority;
+	int      i, count;
+
+	if (target->priority >= scheduler->priority_level_max)
+		return 1;
+
+	if (target->bumped)
+		return 0;
+
+	new_priority = target->priority + bump;
+	if ((new_priority <= target->priority) ||
+	    (new_priority > scheduler->priority_level_max))
+		target->priority = scheduler->priority_level_max;
+	else
+		target->priority = new_priority;
+
+	count = 1;
+	target->bumped = true;
+
+	for (i = 0; i < target->num_deps; i++) {
+		if (!target->dep_list[i])
+			continue;
+
+		if (target->dep_list[i]->bumped)
+			continue;
+
+		count += i915_scheduler_priority_bump(scheduler,
+						      target->dep_list[i],
+						      bump);
+	}
+
+	return count;
+}
+
+static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *ring,
+				struct i915_scheduler_queue_entry **pop_node,
+				unsigned long *flags)
+{
+	struct drm_i915_private            *dev_priv = ring->dev->dev_private;
+	struct i915_scheduler              *scheduler = dev_priv->scheduler;
+	struct i915_scheduler_queue_entry  *best;
+	struct i915_scheduler_queue_entry  *node;
+	int     ret;

drivers/gpu/drm/i915/i915_scheduler.c line 578

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	int     i;
+	bool	any_queued;
+	bool	has_local, has_remote, only_remote;
+
+	*pop_node = NULL;
+	ret = -ENODATA;
+
+	any_queued = false;
+	only_remote = false;
+	best = NULL;

These should just be initialized in-place. But looking at the code forward.

Quoted 62 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +62 -0
+
+	list_for_each_entry(node, &scheduler->node_queue[ring->id], link) {
+		if (!I915_SQS_IS_QUEUED(node))
+			continue;
+		any_queued = true;
+
+		has_local  = false;
+		has_remote = false;
+		for (i = 0; i < node->num_deps; i++) {
+			if (!i915_scheduler_is_dependency_valid(node, i))
+				continue;
+
+			if (node->dep_list[i]->params.ring == node->params.ring)
+				has_local = true;
+			else
+				has_remote = true;
+		}
+
+		if (has_remote && !has_local)
+			only_remote = true;
+
+		if (!has_local && !has_remote) {
+			if (!best ||
+			    (node->priority > best->priority))
+				best = node;
+		}
+	}
+
+	if (best) {
+		list_del(&best->link);
+
+		INIT_LIST_HEAD(&best->link);
+		best->status  = i915_sqs_popped;
+
+		ret = 0;
+	} else {
+		/* Can only get here if:
+		 * (a) there are no buffers in the queue
+		 * (b) all queued buffers are dependent on other buffers
+		 *     e.g. on a buffer that is in flight on a different ring
+		 */
+		if (only_remote) {
+			/* The only dependent buffers are on another ring. */
+			ret = -EAGAIN;
+		} else if (any_queued) {
+			/* It seems that something has gone horribly wrong! */
+			DRM_ERROR("Broken dependency tracking on ring %d!\n",
+				  (int) ring->id);
+		}
+	}
+
+	/* i915_scheduler_dump_queue_pop(ring, best); */
+
+	*pop_node = best;
+	return ret;
+}
+
+static int i915_scheduler_submit(struct intel_engine_cs *ring, bool was_locked)
+{
+	struct drm_device   *dev = ring->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	struct i915_scheduler   *scheduler = dev_priv->scheduler;

drivers/gpu/drm/i915/i915_scheduler.c line 650

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	struct i915_scheduler_queue_entry  *node;
+	unsigned long       flags;
+	int                 ret = 0, count = 0;
+
+	if (!was_locked) {
+		ret = i915_mutex_lock_interruptible(dev);
+		if (ret)
+			return ret;
+	}
+

I don't really fancy this construct. Should be moved outside of this function for proper lockdep tracking.

Quoted 5 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +5 -0
+	WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+
+	spin_lock_irqsave(&scheduler->lock, flags);
+
+	/* First time around, complain if anything unexpected occurs: */

drivers/gpu/drm/i915/i915_scheduler.c line 665

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	ret = i915_scheduler_pop_from_queue_locked(ring, &node, &flags);
+	if (ret) {
+		spin_unlock_irqrestore(&scheduler->lock, flags);
+
+		if (!was_locked)
+			mutex_unlock(&dev->struct_mutex);
+
+		return ret;
+	}
+

Dropping the was_locked stuff, this should become a proper goto error label. e.g. out_unlock

Quoted 16 lines

drivers/gpu/drm/i915/i915_scheduler.c line 676

Quoted 11 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +11 -0
+	do {
+		WARN_ON(!node);
+		WARN_ON(node->params.ring != ring);
+		WARN_ON(node->status != i915_sqs_popped);
+		count++;
+
+		/*
+		 * The call to pop above will have removed the node from the
+		 * list. So add it back in and mark it as in flight.
+		 */
+		i915_scheduler_fly_node(node);

Why do we want to pull an object out of the list inside spin lock and push it back immediately in our critical code path? Seems like a waste for no obvious gain at this point. Why do not we rather just select an entry and modify it in-place, if it's going to stay in the same queue anyway.

The list order is significant. The element must be moved to the front to keep the submitted items in submission order. Doing it this way also keeps the code nicely partitioned and easier to understand/maintain. Plus, there is a plan to optimise the code by splitting the one single list into three separate ones - queued, flying, complete. If/when that happens, the element will have to be removed from one list and added to another.

Quoted 47 lines

drivers/gpu/drm/i915/i915_scheduler.c line 685

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+
+		scheduler->flags[ring->id] |= i915_sf_submitting;
+		spin_unlock_irqrestore(&scheduler->lock, flags);
+		ret = dev_priv->gt.execbuf_final(&node->params);
+		spin_lock_irqsave(&scheduler->lock, flags);
+		scheduler->flags[ring->id] &= ~i915_sf_submitting;
+
+		if (ret) {
+			int requeue = 1;

Multipurpose variable, not really a good idea. And as commented further, should not exist at all.

drivers/gpu/drm/i915/i915_scheduler.c line 698

Quoted 13 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +13 -0
+
+			/*
+			 * Oh dear! Either the node is broken or the ring is
+			 * busy. So need to kill the node or requeue it and try
+			 * again later as appropriate.
+			 */
+
+			switch (-ret) {
+			case ENODEV:
+			case ENOENT:
+				/* Fatal errors. Kill the node. */
+				requeue = -1;
+			break;

"break" indent is wrong.

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +7 -0
+
+			case EAGAIN:
+			case EBUSY:
+			case EIO:
+			case ENOMEM:
+			case ERESTARTSYS:
+			case EINTR:

drivers/gpu/drm/i915/i915_scheduler.c line 715

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+				/* Supposedly recoverable errors. */
+			break;
+
+			default:
+				/*
+				 * Assume the error is recoverable and hope
+				 * for the best.
+				 */
+				DRM_DEBUG_DRIVER("<%s> Got unexpected error from execfinal(): %d!\n",
+						 ring->name, ret);

There's MISSING_CASE macro, should use it.

Quoted 6 lines

drivers/gpu/drm/i915/i915_scheduler.c line 114

Quoted 3 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +3 -0
+			break;
+			}
+

Just move the code below this point to the switch, no point having a switch to categorize your options and then doing bunch of ifs to execute code that could be in switch.

One of the 'if' paths is to break out of the while loop. Can't do that from inside the switch.

Quoted 197 lines
Quoted 9 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +9 -0
+			/*
+			 * Check that the watchdog/reset code has not nuked
+			 * the node while we weren't looking:
+			 */
+			if (node->status == i915_sqs_dead)
+				requeue = 0;
+
+			if (requeue == 1) {
+				i915_scheduler_node_requeue(node);

drivers/gpu/drm/i915/i915_scheduler.c line 737

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+				/*
+				 * No point spinning if the ring is currently
+				 * unavailable so just give up and come back
+				 * later.
+				 */
+				break;
+			} else if (requeue == -1)
+				i915_scheduler_node_kill(node);
+		}
+

Ending here, this actual submission of a single node could go to its own helper function, these functions now become too long to follow, although they really are not doing anything complicated.

Quoted 8 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +8 -0
+		/* Keep launching until the sky is sufficiently full. */
+		if (i915_scheduler_count_flying(scheduler, ring) >=
+						scheduler->min_flying)
+			break;
+
+		ret = i915_scheduler_pop_from_queue_locked(ring, &node, &flags);
+	} while (ret == 0);
+

drivers/gpu/drm/i915/i915_scheduler.c line 755

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +10 -0
+	spin_unlock_irqrestore(&scheduler->lock, flags);
+
+	if (!was_locked)
+		mutex_unlock(&dev->struct_mutex);
+
+	/* Don't complain about not being able to submit extra entries */
+	if (ret == -ENODATA)
+		ret = 0;
+
+	return (ret < 0) ? ret : count;

This is a combined error and success path, keeping to the convention of kernel drivers is preferred;

	if (ret != -ENODATA)
		goto out_foo;

	return count;

out_unlock:

	spin_unlock(...);
out_foo:
	return ret;
Quoted 81 lines
drivers/gpu/drm/i915/i915_scheduler.c new file +42 -0
+}
+
+static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+				struct i915_scheduler_queue_entry *remove)
+{
+	struct i915_scheduler_queue_entry  *node;
+	int     i, r;
+	int     count = 0;
+
+	/*
+	 * Ensure that a node is not being removed which is still dependent
+	 * upon other (not completed) work. If that happens, it implies
+	 * something has gone very wrong with the dependency tracking! Note
+	 * that there is no need to worry if this node has been explicitly
+	 * killed for some reason - it might be being killed before it got
+	 * sent to the hardware.
+	 */
+	if (remove->status != i915_sqs_dead) {
+		for (i = 0; i < remove->num_deps; i++)
+			if ((remove->dep_list[i]) &&
+			    (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
+				count++;
+		WARN_ON(count);
+	}
+
+	/*
+	 * Remove this node from the dependency lists of any other node which
+	 * might be waiting on it.
+	 */
+	for (r = 0; r < I915_NUM_RINGS; r++) {
+		list_for_each_entry(node, &scheduler->node_queue[r], link) {
+			for (i = 0; i < node->num_deps; i++) {
+				if (node->dep_list[i] != remove)
+					continue;
+
+				node->dep_list[i] = NULL;
+			}
+		}
+	}
+
+	return 0;
+}
drivers/gpu/drm/i915/i915_scheduler.h new file +33 -0
new file mode 100644
index 0000000..00dc7f3
@@ -0,0 +1,91 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _I915_SCHEDULER_H_
+#define _I915_SCHEDULER_H_
+
+enum i915_scheduler_queue_status {
+	/* Limbo: */
+	i915_sqs_none = 0,
+	/* Not yet submitted to hardware: */
+	i915_sqs_queued,
+	/* Popped from queue, ready to fly: */

drivers/gpu/drm/i915/i915_scheduler.h line 43

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	i915_sqs_popped,
+	/* Sent to hardware for processing: */
+	i915_sqs_flying,
+	/* Finished processing on the hardware: */
+	i915_sqs_complete,
+	/* Killed by watchdog or catastrophic submission failure: */
+	i915_sqs_dead,
+	/* Limit value for use with arrays/loops */
+	i915_sqs_MAX
+};

These should be UPPERCASE_FOR_ENUM_VALUES . See i915_drv.h for samples.

drivers/gpu/drm/i915/i915_scheduler.h line 49

Quoted 6 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +6 -0
+
+#define I915_SQS_IS_QUEUED(node)	(((node)->status == i915_sqs_queued))
+#define I915_SQS_IS_FLYING(node)	(((node)->status == i915_sqs_flying))
+#define I915_SQS_IS_COMPLETE(node)	(((node)->status == i915_sqs_complete) || \
+					 ((node)->status == i915_sqs_dead))
+

Might be slightly confusing that name is IS_COMPLETE and there is actual COMPLETE value. Rather have the test like IS_DONE and then test for COMPLETE or DEAD, no confusion. Also, some states have their IS_* macro and others don't, is there going to be more?

Quoted 25 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +25 -0
+struct i915_scheduler_obj_entry {
+	struct drm_i915_gem_object          *obj;
+	bool                                read_only;
+};
+
+struct i915_scheduler_queue_entry {
+	struct i915_execbuffer_params       params;
+	/* -1023 = lowest priority, 0 = default, 1023 = highest */
+	int32_t                             priority;
+	struct i915_scheduler_obj_entry     *saved_objects;
+	int                                 num_objs;
+	bool                                bumped;
+	struct i915_scheduler_queue_entry   **dep_list;
+	int                                 num_deps;
+	enum i915_scheduler_queue_status    status;
+	unsigned long                       stamp;
+	struct list_head                    link;
+};
+
+struct i915_scheduler {
+	struct list_head    node_queue[I915_NUM_RINGS];
+	uint32_t            flags[I915_NUM_RINGS];
+	spinlock_t          lock;
+
+	/* Tuning parameters: */

drivers/gpu/drm/i915/i915_scheduler.h line 84

Quoted 10 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +10 -0
+	int32_t             priority_level_min;
+	int32_t             priority_level_max;
+	int32_t             priority_level_preempt;
+	uint32_t            min_flying;
+};
+
+/* Flag bits for i915_scheduler::flags */
+enum {
+	i915_sf_interrupts_enabled  = (1 << 0),
+	i915_sf_submitting          = (1 << 1),

Again, should be uppercase. Also, enums to the beginning of file.

Quoted 7 lines

drivers/gpu/drm/i915/i915_scheduler.h line 91

Quoted 7 lines
drivers/gpu/drm/i915/i915_scheduler.h new file +7 -0
+};
+
+int         i915_scheduler_init(struct drm_device *dev);
+int         i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+bool        i915_scheduler_notify_request(struct drm_i915_gem_request *req);
+
+#endif  /* _I915_SCHEDULER_H_ */

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx