Skip to contentSkip to conversation

Eliminating the execlist retired request queue

RFC In review Tvrtko Ursulin 3 participants 20 messages Apr 8, 2016 – Apr 11, 2016

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Eliminating the execlist retired request queue

From: Tvrtko Ursulin <[email protected]>

Retired request queue coupled with retired work handler is a scalability concern on some workloads of which one dramatic example is gem_close_race.

This series depend on i915_gem_object_pin_map series by Chris Wilson.

Brief outline of patches:

1/4) Allow lockless request unreference - needed for 4/4. 2/4) Pin the LRC until the following request is completed - needed for
GuC mode on its own and also enables 4/4.
3/4) Use i915_gem_object_pin_map in execlists to alleviate increase in
LRC pin/unpin this series adds.
4/4) Removes retired_req_queue.

Lightly tested so RFC for now. Did not spot any performance difference in my usual benchmarks but did not look very closely yet. There may be wins in CPU usage. TBD.

Easiest way to see the change is gem_close_race which finishes in 10-20 seconds with this series and otherwise takes ten minutes to half an hour.

TODO:

1) Patches 2 and 3 should probably swap order.
2) More benchmarking including CPU usage and power.
3) Need a better cover letter as well.

Chris Wilson (1):
drm/i915: Move releasing of the GEM request from free to retire/cancel

Tvrtko Ursulin (3):
drm/i915/guc: Keep the previous context pinned until the next one has
been completed
drm/i915: Use new i915_gem_object_pin_map for LRC drm/i915: Stop tracking execlists retired requests

 drivers/gpu/drm/i915/i915_drv.h         | 18 ++------
 drivers/gpu/drm/i915/i915_gem.c         | 63 +++++++++++++-------------
 drivers/gpu/drm/i915/intel_display.c    |  2 +-
 drivers/gpu/drm/i915/intel_lrc.c        | 78 ++++++++++++++++-----------------
 drivers/gpu/drm/i915/intel_lrc.h        |  2 +-
 drivers/gpu/drm/i915/intel_pm.c         |  2 +-
 drivers/gpu/drm/i915/intel_ringbuffer.h |  1 -
 7 files changed, 74 insertions(+), 92 deletions(-)
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

RFC 1/4 drm/i915: Move releasing of the GEM request from free to retire/cancel

From: Chris Wilson <[email protected]>

If we move the release of the GEM request (i.e. decoupling it from the various lists used for client and context tracking) after it is complete (either by the GPU retiring the request, or by the caller cancelling the request), we can remove the requirement that the final unreference of the GEM request need to be under the struct_mutex.

v2: Execlists as always is badly asymetric and year old patches still haven't landed to fix it up.

v3: Extracted, rebased and fixed for GuC. (Tvrtko Ursulin)

---
 drivers/gpu/drm/i915/i915_drv.h      | 14 ----------
 drivers/gpu/drm/i915/i915_gem.c      | 50 ++++++++++++++++++------------------
 drivers/gpu/drm/i915/intel_display.c |  2 +-
 drivers/gpu/drm/i915/intel_pm.c      |  2 +-
 4 files changed, 27 insertions(+), 41 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +0 -14
index da403f4fc679..fa199b7e51b5 100644
@@ -2352,23 +2352,9 @@ i915_gem_request_reference(struct drm_i915_gem_request *req)
 static inline void
 i915_gem_request_unreference(struct drm_i915_gem_request *req)
 {
-	WARN_ON(!mutex_is_locked(&req->engine->dev->struct_mutex));
 	kref_put(&req->ref, i915_gem_request_free);
 }
 
-static inline void
-i915_gem_request_unreference__unlocked(struct drm_i915_gem_request *req)
-{
-	struct drm_device *dev;
-
-	if (!req)
-		return;
-
-	dev = req->engine->dev;
-	if (kref_put_mutex(&req->ref, i915_gem_request_free, &dev->struct_mutex))
-		mutex_unlock(&dev->struct_mutex);
-}
-
 static inline void i915_gem_request_assign(struct drm_i915_gem_request **pdst,
 					   struct drm_i915_gem_request *src)
 {
drivers/gpu/drm/i915/i915_gem.c +25 -25

90 lines not on this page · raw message

drivers/gpu/drm/i915/intel_display.c +1 -1
index feb7028341b8..f2ce7310e2b6 100644
@@ -11362,7 +11362,7 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
 					    mmio_flip->crtc->reset_counter,
 					    false, NULL,
 					    &mmio_flip->i915->rps.mmioflips));
-		i915_gem_request_unreference__unlocked(mmio_flip->req);
+		i915_gem_request_unreference(mmio_flip->req);
 	}
 
 	/* For framebuffer backed by dmabuf, wait for fence */
drivers/gpu/drm/i915/intel_pm.c +1 -1
index 43b24a1f5ee6..346e353b4fe8 100644
@@ -7378,7 +7378,7 @@ static void __intel_rps_boost_work(struct work_struct *work)
 		gen6_rps_boost(to_i915(req->engine->dev), NULL,
 			       req->emitted_jiffies);
 
-	i915_gem_request_unreference__unlocked(req);
+	i915_gem_request_unreference(req);
 	kfree(boost);
 }
 
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Re: [RFC 1/4] drm/i915: Move releasing of the GEM request from free to retire/cancel

Quoted 8 lines · Tvrtko Ursulin

From: Chris Wilson <[email protected]>

If we move the release of the GEM request (i.e. decoupling it from the various lists used for client and context tracking) after it is complete (either by the GPU retiring the request, or by the caller cancelling the request), we can remove the requirement that the final unreference of the GEM request need to be under the struct_mutex.

Quoted 4 lines

v2: Execlists as always is badly asymetric and year old patches still haven't landed to fix it up.

v3: Extracted, rebased and fixed for GuC. (Tvrtko Ursulin)

After you mentioned the unbalanced, the patches I reordered to fix that are:

https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=83dcde26caa26f4113c3e441c3c96c504fd88e13
https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=9f386a21d3f28db763102b5c4f97a90bd0dcfd08
https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=9afd878e2c9f7825b99dc839c7b5deb553b62e32
https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=a842a2b0b7e90148966f35488209c969a9a9da54
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

Re: [RFC 1/4] drm/i915: Move releasing of the GEM request from free to retire/cancel

Quoted 14 lines · Chris Wilson
Quoted 8 lines · Tvrtko Ursulin

From: Chris Wilson <[email protected]>

If we move the release of the GEM request (i.e. decoupling it from the various lists used for client and context tracking) after it is complete (either by the GPU retiring the request, or by the caller cancelling the request), we can remove the requirement that the final unreference of the GEM request need to be under the struct_mutex.

Quoted 4 lines

v2: Execlists as always is badly asymetric and year old patches still haven't landed to fix it up.

v3: Extracted, rebased and fixed for GuC. (Tvrtko Ursulin)

Quoted 7 lines

After you mentioned the unbalanced, the patches I reordered to fix that are:

https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=83dcde26caa26f4113c3e441c3c96c504fd88e13
https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=9f386a21d3f28db763102b5c4f97a90bd0dcfd08
https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=9afd878e2c9f7825b99dc839c7b5deb553b62e32
https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=a842a2b0b7e90148966f35488209c969a9a9da54

Want to send these four as standalone straight away for review then?

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw

RFC 2/4 drm/i915/guc: Keep the previous context pinned until the next one has been completed

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

In GuC mode submitting requests is only putting them into the GuC queue with the actual submission to hardware following at some future point. This makes the per engine last context tracking insufficient for closing the premature context unpin race.

Instead we need to make requests track and pin the previous context on the same engine, and only unpin them when they themselves are retired. This will ensure contexts remain pinned in all cases until the are fully saved by the GPU.

v2: Only track contexts.

Cc Alex Dai <[email protected]>
Cc Dave Gordon <[email protected]>
Cc Nick Hoath <[email protected]>
Cc Chris Wilson <[email protected]>
---
 drivers/gpu/drm/i915/i915_drv.h  |  4 +++-
 drivers/gpu/drm/i915/i915_gem.c  |  5 +++++
 drivers/gpu/drm/i915/intel_lrc.c | 17 +++++++++++++++++
 3 files changed, 25 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +3 -1
index fa199b7e51b5..216b2297ae3e 100644
@@ -2283,6 +2283,9 @@ struct drm_i915_gem_request {
 	struct intel_context *ctx;
 	struct intel_ringbuffer *ringbuf;
 
+	/** Context preceding this one on the same engine. Can be NULL. */
+	struct intel_context *prev_ctx;
+
 	/** Batch buffer related to this request if any (used for
 	    error state dump only) */
 	struct drm_i915_gem_object *batch_obj;
@@ -2318,7 +2321,6 @@ struct drm_i915_gem_request {
 
 	/** Execlists no. of times this request has been sent to the ELSP */
 	int elsp_submitted;
-
 };
 
 struct drm_i915_gem_request * __must_check
drivers/gpu/drm/i915/i915_gem.c +5 -0
index c1df696f9002..118911ce7231 100644
@@ -1445,6 +1445,11 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
 	 */
 	request->ringbuf->last_retired_head = request->postfix;
 
+	if (request->prev_ctx) {
+		intel_lr_context_unpin(request->prev_ctx, request->engine);
+		request->prev_ctx = NULL;
+	}
+
 	__i915_gem_request_release(request);
 }
 
drivers/gpu/drm/i915/intel_lrc.c +17 -0
index 31445aa3429b..3d08dac0a9b0 100644
@@ -798,6 +798,23 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
 	if (intel_engine_stopped(engine))
 		return 0;
 
+	/*
+	 * Pin the previous context on this engine to ensure it is not
+	 * prematurely unpinned in GuC mode.
+	 * Previous context will be unpinned when this request is retired,
+	 * ensuring the GPU has switched out from the previous context and into
+	 * a new context at that point.
+	 */
+	if (engine->last_context) {
+		if (engine->last_context != request->i915->kernel_context) {
+			intel_lr_context_pin(engine->last_context, engine);
+			request->prev_ctx = engine->last_context;
+		}
+	}
+
+	/*
+	 * Track and pin the last context submitted on an engine.
+	 */
 	if (engine->last_context != request->ctx) {
 		if (engine->last_context)
 			intel_lr_context_unpin(engine->last_context, engine);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [RFC 2/4] drm/i915/guc: Keep the previous context pinned until the next one has been completed

Quoted 82 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

In GuC mode submitting requests is only putting them into the GuC queue with the actual submission to hardware following at some future point. This makes the per engine last context tracking insufficient for closing the premature context unpin race.

Instead we need to make requests track and pin the previous context on the same engine, and only unpin them when they themselves are retired. This will ensure contexts remain pinned in all cases until the are fully saved by the GPU.

v2: Only track contexts.

Cc Alex Dai <[email protected]>
Cc Dave Gordon <[email protected]>
Cc Nick Hoath <[email protected]>
Cc Chris Wilson <[email protected]>
---
 drivers/gpu/drm/i915/i915_drv.h  |  4 +++-
 drivers/gpu/drm/i915/i915_gem.c  |  5 +++++
 drivers/gpu/drm/i915/intel_lrc.c | 17 +++++++++++++++++
 3 files changed, 25 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +3 -1
index fa199b7e51b5..216b2297ae3e 100644
@@ -2283,6 +2283,9 @@ struct drm_i915_gem_request {
 	struct intel_context *ctx;
 	struct intel_ringbuffer *ringbuf;
 
+	/** Context preceding this one on the same engine. Can be NULL. */
+	struct intel_context *prev_ctx;
+
 	/** Batch buffer related to this request if any (used for
 	    error state dump only) */
 	struct drm_i915_gem_object *batch_obj;
@@ -2318,7 +2321,6 @@ struct drm_i915_gem_request {
 
 	/** Execlists no. of times this request has been sent to the ELSP */
 	int elsp_submitted;
-
 };
 
 struct drm_i915_gem_request * __must_check
drivers/gpu/drm/i915/i915_gem.c +5 -0
index c1df696f9002..118911ce7231 100644
@@ -1445,6 +1445,11 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
 	 */
 	request->ringbuf->last_retired_head = request->postfix;
 
+	if (request->prev_ctx) {
+		intel_lr_context_unpin(request->prev_ctx, request->engine);
+		request->prev_ctx = NULL;
+	}
+
 	__i915_gem_request_release(request);
 }
 
drivers/gpu/drm/i915/intel_lrc.c +10 -0
index 31445aa3429b..3d08dac0a9b0 100644
@@ -798,6 +798,23 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
 	if (intel_engine_stopped(engine))
 		return 0;
 
+	/*
+	 * Pin the previous context on this engine to ensure it is not
+	 * prematurely unpinned in GuC mode.
+	 * Previous context will be unpinned when this request is retired,
+	 * ensuring the GPU has switched out from the previous context and into
+	 * a new context at that point.
+	 */
+	if (engine->last_context) {
+		if (engine->last_context != request->i915->kernel_context) {
+			intel_lr_context_pin(engine->last_context, engine);

drivers/gpu/drm/i915/intel_lrc.c intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request) · line 820

Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +7 -0
+			request->prev_ctx = engine->last_context;
+		}
+	}
+
+	/*
+	 * Track and pin the last context submitted on an engine.
+	 */
 	if (engine->last_context != request->ctx) {
 		if (engine->last_context)
 			intel_lr_context_unpin(engine->last_context, engine);

We can use pinned_context to reduce the code complexity, not make it worse.

If you first apply the execlists default context fix, adding pinned_context is a reduction is code.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#6 lore raw

Re: [RFC 2/4] drm/i915/guc: Keep the previous context pinned until the next one has been completed

Quoted 97 lines · Chris Wilson
Quoted 82 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

In GuC mode submitting requests is only putting them into the GuC queue with the actual submission to hardware following at some future point. This makes the per engine last context tracking insufficient for closing the premature context unpin race.

Instead we need to make requests track and pin the previous context on the same engine, and only unpin them when they themselves are retired. This will ensure contexts remain pinned in all cases until the are fully saved by the GPU.

v2: Only track contexts.

Cc Alex Dai <[email protected]>
Cc Dave Gordon <[email protected]>
Cc Nick Hoath <[email protected]>
Cc Chris Wilson <[email protected]>
---
  drivers/gpu/drm/i915/i915_drv.h  |  4 +++-
  drivers/gpu/drm/i915/i915_gem.c  |  5 +++++
  drivers/gpu/drm/i915/intel_lrc.c | 17 +++++++++++++++++
  3 files changed, 25 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_drv.h +3 -1
index fa199b7e51b5..216b2297ae3e 100644
@@ -2283,6 +2283,9 @@ struct drm_i915_gem_request {
  	struct intel_context *ctx;
  	struct intel_ringbuffer *ringbuf;
 
+	/** Context preceding this one on the same engine. Can be NULL. */
+	struct intel_context *prev_ctx;
+
  	/** Batch buffer related to this request if any (used for
  	    error state dump only) */
  	struct drm_i915_gem_object *batch_obj;
@@ -2318,7 +2321,6 @@ struct drm_i915_gem_request {
 
  	/** Execlists no. of times this request has been sent to the ELSP */
  	int elsp_submitted;
-
  };
 
  struct drm_i915_gem_request * __must_check
drivers/gpu/drm/i915/i915_gem.c +5 -0
index c1df696f9002..118911ce7231 100644
@@ -1445,6 +1445,11 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
  	 */
  	request->ringbuf->last_retired_head = request->postfix;
 
+	if (request->prev_ctx) {
+		intel_lr_context_unpin(request->prev_ctx, request->engine);
+		request->prev_ctx = NULL;
+	}
+
  	__i915_gem_request_release(request);
  }
 
drivers/gpu/drm/i915/intel_lrc.c +10 -0
index 31445aa3429b..3d08dac0a9b0 100644
@@ -798,6 +798,23 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
  	if (intel_engine_stopped(engine))
  		return 0;
 
+	/*
+	 * Pin the previous context on this engine to ensure it is not
+	 * prematurely unpinned in GuC mode.
+	 * Previous context will be unpinned when this request is retired,
+	 * ensuring the GPU has switched out from the previous context and into
+	 * a new context at that point.
+	 */
+	if (engine->last_context) {
+		if (engine->last_context != request->i915->kernel_context) {
+			intel_lr_context_pin(engine->last_context, engine);

drivers/gpu/drm/i915/intel_lrc.c intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request) · line 820

Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +7 -0
+			request->prev_ctx = engine->last_context;
+		}
+	}
+
+	/*
+	 * Track and pin the last context submitted on an engine.
+	 */
  	if (engine->last_context != request->ctx) {
  		if (engine->last_context)
  			intel_lr_context_unpin(engine->last_context, engine);

We can use pinned_context to reduce the code complexity, not make it worse.

Quoted 2 lines

If you first apply the execlists default context fix, adding pinned_context is a reduction is code.

Okay I'll pull that patch in this series.

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#7 lore raw

RFC 3/4 drm/i915: Use new i915_gem_object_pin_map for LRC

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

We can use the new pin/lazy unpin API for more performance in the execlist submission paths.

---
 drivers/gpu/drm/i915/intel_lrc.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/intel_lrc.c +7 -8
index 3d08dac0a9b0..5e967943ce49 100644
@@ -1111,8 +1111,8 @@ static int intel_lr_context_do_pin(struct intel_context *ctx,
 	struct drm_i915_private *dev_priv = dev->dev_private;
 	struct drm_i915_gem_object *ctx_obj = ctx->engine[engine->id].state;
 	struct intel_ringbuffer *ringbuf = ctx->engine[engine->id].ringbuf;
-	struct page *lrc_state_page;
-	uint32_t *lrc_reg_state;
+	void *obj_addr;
+	u32 *lrc_reg_state;
 	int ret;
 
 	WARN_ON(!mutex_is_locked(&engine->dev->struct_mutex));
@@ -1122,11 +1122,11 @@ static int intel_lr_context_do_pin(struct intel_context *ctx,
 	if (ret)
 		return ret;
 
-	lrc_state_page = i915_gem_object_get_dirty_page(ctx_obj, LRC_STATE_PN);
-	if (WARN_ON(!lrc_state_page)) {
-		ret = -ENODEV;
+	obj_addr = i915_gem_object_pin_map(ctx_obj);
+	if (!obj_addr)
 		goto unpin_ctx_obj;
-	}
+
+	lrc_reg_state = obj_addr + LRC_STATE_PN * PAGE_SIZE;
 
 	ret = intel_pin_and_map_ringbuffer_obj(engine->dev, ringbuf);
 	if (ret)
@@ -1134,7 +1134,6 @@ static int intel_lr_context_do_pin(struct intel_context *ctx,
 
 	ctx->engine[engine->id].lrc_vma = i915_gem_obj_to_ggtt(ctx_obj);
 	intel_lr_context_descriptor_update(ctx, engine);
-	lrc_reg_state = kmap(lrc_state_page);
 	lrc_reg_state[CTX_RING_BUFFER_START+1] = ringbuf->vma->node.start;
 	ctx->engine[engine->id].lrc_reg_state = lrc_reg_state;
 	ctx_obj->dirty = true;
@@ -1177,7 +1176,7 @@ void intel_lr_context_unpin(struct intel_context *ctx,
 
 	WARN_ON(!mutex_is_locked(&ctx->i915->dev->struct_mutex));
 	if (--ctx->engine[engine->id].pin_count == 0) {
-		kunmap(kmap_to_page(ctx->engine[engine->id].lrc_reg_state));
+		i915_gem_object_unpin_map(ctx_obj);
 		intel_unpin_ringbuffer_obj(ctx->engine[engine->id].ringbuf);
 		i915_gem_object_ggtt_unpin(ctx_obj);
 		ctx->engine[engine->id].lrc_vma = NULL;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#8 lore raw

Re: [RFC 3/4] drm/i915: Use new i915_gem_object_pin_map for LRC

Quoted 4 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

We can use the new pin/lazy unpin API for more performance in the execlist submission paths.

The sticking point for me was the ringbuffer and Braswell having to re-ioremap it every context pin. I hadn't noticed the kmap of the context, but this does seem like a valid use for pin_map so I'm not complaining... (Just saying this half the solution ;)

Oh, and you don't have pin_map for your ringbuffer either yet. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#9 lore raw

Re: [RFC 3/4] drm/i915: Use new i915_gem_object_pin_map for LRC

Quoted 10 lines · Chris Wilson
Quoted 4 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

We can use the new pin/lazy unpin API for more performance in the execlist submission paths.

The sticking point for me was the ringbuffer and Braswell having to re-ioremap it every context pin. I hadn't noticed the kmap of the context, but this does seem like a valid use for pin_map so I'm not complaining... (Just saying this half the solution ;)

Pull in to the series "drm/i915: Move ioremap_wc tracking onto VMA" as well then?

Quoted 1 line

Oh, and you don't have pin_map for your ringbuffer either yet.

Confused, I did base this on top of "drm/i915: Refactor duplicate object vmap functions" and the half missing is the ioremap paths. Which you commented on above. What other ringbuffer is missing?

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#10 lore raw

Re: [RFC 3/4] drm/i915: Use new i915_gem_object_pin_map for LRC

Quoted 8 lines · Tvrtko Ursulin
Quoted 6 lines · Chris Wilson
Quoted 4 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

We can use the new pin/lazy unpin API for more performance in the execlist submission paths.

Quoted 7 lines
Quoted 4 lines

The sticking point for me was the ringbuffer and Braswell having to re-ioremap it every context pin. I hadn't noticed the kmap of the context, but this does seem like a valid use for pin_map so I'm not complaining... (Just saying this half the solution ;)

Pull in to the series "drm/i915: Move ioremap_wc tracking onto VMA" as well then?

That depends on "io-mapping: Specify mapping size for io_mapping_map_wc" and the vma->map_and_fenceable (though you can workaround that) iirc .

Quoted 5 lines
Quoted 1 line

Oh, and you don't have pin_map for your ringbuffer either yet.

Confused, I did base this on top of "drm/i915: Refactor duplicate object vmap functions" and the half missing is the ioremap paths. Which you commented on above. What other ringbuffer is missing?

No, I was just thinking it was still pending because I had a fixup in my
tree: drm/i915: Treat ringbuffer writes as write to normal memory

That and a patch to use WC vmaps for Braswell as well (that's still in preliminary form, but if you are interested
https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=3fa97e044f3b5a2c23e44ec659a8199f134d8fb5
it really helps with context creation stress tests - and not much else! ;)
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#11 lore raw

RFC 4/4 drm/i915: Stop tracking execlists retired requests

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

With the previous patch having extended the pinned lifetime of contexts by referencing the previous context from the current request until the latter is retired (completed by the GPU), we can now remove usage of execlist retired queue entirely.

This is because the above now guarantees that all execlist object access requirements are satisfied by this new tracking, and we can stop taking additional references and stop keeping request on the execlists retired queue.

The latter was a source of significant scalability issues in the driver causing performance hits on some tests. Most dramatical of which was igt/gem_close_race which had run time in tens of minutes which is now reduced to tens of seconds.

---
 drivers/gpu/drm/i915/i915_gem.c         | 10 +------
 drivers/gpu/drm/i915/intel_lrc.c        | 46 ++++++++++-----------------------
 drivers/gpu/drm/i915/intel_lrc.h        |  2 +-
 drivers/gpu/drm/i915/intel_ringbuffer.h |  1 -
 4 files changed, 16 insertions(+), 43 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +1 -9
index 118911ce7231..2f4bfe93b20c 100644
@@ -2901,13 +2901,7 @@ static void i915_gem_reset_engine_cleanup(struct drm_i915_private *dev_priv,
 		/* Ensure irq handler finishes or is cancelled. */
 		tasklet_kill(&engine->irq_tasklet);
 
-		spin_lock_bh(&engine->execlist_lock);
-		/* list_splice_tail_init checks for empty lists */
-		list_splice_tail_init(&engine->execlist_queue,
-				      &engine->execlist_retired_req_list);
-		spin_unlock_bh(&engine->execlist_lock);
-
-		intel_execlists_retire_requests(engine);
+		intel_execlists_cancel_requests(engine);
 	}
 
 	/*
@@ -3029,8 +3023,6 @@ i915_gem_retire_requests(struct drm_device *dev)
 			spin_lock_bh(&engine->execlist_lock);
 			idle &= list_empty(&engine->execlist_queue);
 			spin_unlock_bh(&engine->execlist_lock);
-
-			intel_execlists_retire_requests(engine);
 		}
 	}
 
drivers/gpu/drm/i915/intel_lrc.c +14 -32

105 lines not on this page · raw message

drivers/gpu/drm/i915/intel_lrc.h +1 -1
index 8de1ea536ad4..04cee14f2bb2 100644
@@ -119,6 +119,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
 			       struct drm_i915_gem_execbuffer2 *args,
 			       struct list_head *vmas);
 
-void intel_execlists_retire_requests(struct intel_engine_cs *engine);
+void intel_execlists_cancel_requests(struct intel_engine_cs *engine);
 
 #endif /* _INTEL_LRC_H_ */
drivers/gpu/drm/i915/intel_ringbuffer.h +0 -1
index 3a11705222fc..712c754bbd4e 100644
@@ -269,7 +269,6 @@ struct  intel_engine_cs {
 	struct tasklet_struct irq_tasklet;
 	spinlock_t execlist_lock; /* used inside tasklet, use spin_lock_bh */
 	struct list_head execlist_queue;
-	struct list_head execlist_retired_req_list;
 	unsigned int fw_domains;
 	unsigned int next_context_status_buffer;
 	unsigned int idle_lite_restore_wa;
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#12 lore raw

Re: [RFC 4/4] drm/i915: Stop tracking execlists retired requests

Quoted 15 lines · Tvrtko Ursulin
drivers/gpu/drm/i915/intel_lrc.c +0 -5
@@ -615,11 +613,6 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
 	struct drm_i915_gem_request *cursor;
 	int num_elements = 0;
 
-	if (request->ctx != request->i915->kernel_context)
-		intel_lr_context_pin(request->ctx, engine);
-
-	i915_gem_request_reference(request);
-
 	spin_lock_bh(&engine->execlist_lock);
 
 	list_for_each_entry(cursor, &engine->execlist_queue, execlist_link)
@@ -636,11 +629,12 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
 		if (request->ctx == tail_req->ctx) {
 			WARN(tail_req->elsp_submitted != 0,

drivers/gpu/drm/i915/intel_lrc.c static void execlists_context_queue(struct drm_i915_gem_request *request) · line 638

Quoted 10 lines
drivers/gpu/drm/i915/intel_lrc.c +3 -2
 				"More than 2 already-submitted reqs queued\n");
-			list_move_tail(&tail_req->execlist_link,
-				       &engine->execlist_retired_req_list);
+			list_del(&tail_req->execlist_link);
+			i915_gem_request_unreference(tail_req);
 		}
 	}
 
+	i915_gem_request_reference(request);
 	list_add_tail(&request->execlist_link, &engine->execlist_queue);

If you want to get truly radical, we do not need the ref on the request until it is submitted to hardware. (As the request cannot be retired until it has done so, it can leave the execlist_queue until we commit it to hw, or perform the cancel).

Lgtm,
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#13 lore raw

Re: [RFC 4/4] drm/i915: Stop tracking execlists retired requests

Quoted 24 lines · Chris Wilson
Quoted 23 lines · Tvrtko Ursulin
drivers/gpu/drm/i915/intel_lrc.c +2 -7
@@ -615,11 +613,6 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
  	struct drm_i915_gem_request *cursor;
  	int num_elements = 0;
 
-	if (request->ctx != request->i915->kernel_context)
-		intel_lr_context_pin(request->ctx, engine);
-
-	i915_gem_request_reference(request);
-
  	spin_lock_bh(&engine->execlist_lock);
 
  	list_for_each_entry(cursor, &engine->execlist_queue, execlist_link)
@@ -636,11 +629,12 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
  		if (request->ctx == tail_req->ctx) {
  			WARN(tail_req->elsp_submitted != 0,
  				"More than 2 already-submitted reqs queued\n");
-			list_move_tail(&tail_req->execlist_link,
-				       &engine->execlist_retired_req_list);
+			list_del(&tail_req->execlist_link);
+			i915_gem_request_unreference(tail_req);
  		}
  	}
 
Quoted 7 lines

drivers/gpu/drm/i915/intel_lrc.c static void execlists_context_queue(struct drm_i915_gem_request *request) · line 638

Quoted 2 lines
drivers/gpu/drm/i915/intel_lrc.c +1 -0
+	i915_gem_request_reference(request);
  	list_add_tail(&request->execlist_link, &engine->execlist_queue);

If you want to get truly radical, we do not need the ref on the request until it is submitted to hardware. (As the request cannot be retired until it has done so, it can leave the execlist_queue until we commit it to hw, or perform the cancel).

Don't know. It is simple and nice that reference is tied to presence on execlist_queue.

More importantly, the patch as presented has a flaw that it dereferences req->ctx from execlists_check_remove_request where the context pin may have disappeared already due context complete interrupts getting behind when coallescing.

I will need to cache ctx_id in the request I think. It is fine do to that since ctx id must be unique and stable for a request.

Maybe even I should pull in your patch which makes ctx ids stable and persistent aligned with context existence and not pin. Think I've seen something like that somewhere.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#14 lore raw

Re: [RFC 4/4] drm/i915: Stop tracking execlists retired requests

Quoted 42 lines · Tvrtko Ursulin
Quoted 24 lines · Chris Wilson
Quoted 23 lines · Tvrtko Ursulin
drivers/gpu/drm/i915/intel_lrc.c +2 -7
@@ -615,11 +613,6 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
 	struct drm_i915_gem_request *cursor;
 	int num_elements = 0;
 
-	if (request->ctx != request->i915->kernel_context)
-		intel_lr_context_pin(request->ctx, engine);
-
-	i915_gem_request_reference(request);
-
 	spin_lock_bh(&engine->execlist_lock);
 
 	list_for_each_entry(cursor, &engine->execlist_queue, execlist_link)
@@ -636,11 +629,12 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
 		if (request->ctx == tail_req->ctx) {
 			WARN(tail_req->elsp_submitted != 0,
 				"More than 2 already-submitted reqs queued\n");
-			list_move_tail(&tail_req->execlist_link,
-				       &engine->execlist_retired_req_list);
+			list_del(&tail_req->execlist_link);
+			i915_gem_request_unreference(tail_req);
 		}
 	}
 
Quoted 7 lines

drivers/gpu/drm/i915/intel_lrc.c static void execlists_context_queue(struct drm_i915_gem_request *request) · line 638

Quoted 2 lines
drivers/gpu/drm/i915/intel_lrc.c +1 -0
+	i915_gem_request_reference(request);
 	list_add_tail(&request->execlist_link, &engine->execlist_queue);

If you want to get truly radical, we do not need the ref on the request until it is submitted to hardware. (As the request cannot be retired until it has done so, it can leave the execlist_queue until we commit it to hw, or perform the cancel).

Don't know. It is simple and nice that reference is tied to presence on execlist_queue.

More importantly, the patch as presented has a flaw that it dereferences req->ctx from execlists_check_remove_request where the context pin may have disappeared already due context complete interrupts getting behind when coallescing.

Quoted 6 lines

I will need to cache ctx_id in the request I think. It is fine do to that since ctx id must be unique and stable for a request.

Maybe even I should pull in your patch which makes ctx ids stable and persistent aligned with context existence and not pin. Think I've seen something like that somewhere.

Yes, both OA, PASID and vGPU depend upon having stable ctx-id for the lifetime of the context. Dave expressed a concern that the GuC maybe interpretting it as the LRCA, but as far as I can see, lrc->ring_lcra and lrc->context_id are distinct and we never use the global context identifier itself (the closest is the low 32bits of lrc_desc which do not include our unique id).
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#15 lore raw

Re: [RFC 4/4] drm/i915: Stop tracking execlists retired requests

In reply to Tvrtko Ursulin
Quoted 37 lines · Tvrtko Ursulin
Quoted 24 lines · Chris Wilson
Quoted 23 lines · Tvrtko Ursulin
drivers/gpu/drm/i915/intel_lrc.c +2 -7
@@ -615,11 +613,6 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
 	struct drm_i915_gem_request *cursor;
 	int num_elements = 0;
 
-	if (request->ctx != request->i915->kernel_context)
-		intel_lr_context_pin(request->ctx, engine);
-
-	i915_gem_request_reference(request);
-
 	spin_lock_bh(&engine->execlist_lock);
 
 	list_for_each_entry(cursor, &engine->execlist_queue, execlist_link)
@@ -636,11 +629,12 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
 		if (request->ctx == tail_req->ctx) {
 			WARN(tail_req->elsp_submitted != 0,
 				"More than 2 already-submitted reqs queued\n");
-			list_move_tail(&tail_req->execlist_link,
-				       &engine->execlist_retired_req_list);
+			list_del(&tail_req->execlist_link);
+			i915_gem_request_unreference(tail_req);
 		}
 	}
 
Quoted 7 lines

drivers/gpu/drm/i915/intel_lrc.c static void execlists_context_queue(struct drm_i915_gem_request *request) · line 638

Quoted 2 lines
drivers/gpu/drm/i915/intel_lrc.c +1 -0
+	i915_gem_request_reference(request);
 	list_add_tail(&request->execlist_link, &engine->execlist_queue);

If you want to get truly radical, we do not need the ref on the request until it is submitted to hardware. (As the request cannot be retired until it has done so, it can leave the execlist_queue until we commit it to hw, or perform the cancel).

Don't know. It is simple and nice that reference is tied to presence on execlist_queue.

Quoted 7 lines

More importantly, the patch as presented has a flaw that it dereferences req->ctx from execlists_check_remove_request where the context pin may have disappeared already due context complete interrupts getting behind when coallescing.

I will need to cache ctx_id in the request I think. It is fine do to that since ctx id must be unique and stable for a request.

For comparison, tracking by requests rather than contexts:

https://cgit.freedesktop.org/~ickle/linux-2.6/commit/?h=tasklet&id=e23791ef88636aa6e3f850b61ab2c4e027af0e52
-Chris

Quoted 1 line
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#16 lore raw

✗ Fi.CI.BAT: failure for Eliminating the execlist retired request queue

In reply to Tvrtko Ursulin
== Series Details ==

== Series Details ==

Series: Eliminating the execlist retired request queue
URL : https://patchwork.freedesktop.org/series/5459/
State : failure

== Summary ==

  CC [M]  drivers/gpu/drm/i915/intel_dpll_mgr.o
  CC [M]  drivers/gpu/drm/i915/intel_fbc.o
  CC [M]  drivers/gpu/drm/i915/intel_fifo_underrun.o
  CC [M]  drivers/gpu/drm/i915/intel_frontbuffer.o
  CC [M]  drivers/gpu/drm/i915/intel_hotplug.o
  CC [M]  drivers/gpu/drm/i915/intel_modes.o
  CC [M]  drivers/gpu/drm/i915/intel_overlay.o
  CC [M]  drivers/gpu/drm/i915/intel_psr.o
  CC [M]  drivers/gpu/drm/i915/intel_sideband.o
  LD      drivers/usb/host/xhci-hcd.o
  LD      drivers/usb/host/built-in.o
  CC [M]  drivers/gpu/drm/i915/intel_sprite.o
  LD      drivers/usb/built-in.o
  CC [M]  drivers/gpu/drm/i915/intel_acpi.o
  CC [M]  drivers/gpu/drm/i915/intel_opregion.o
  CC [M]  drivers/gpu/drm/i915/dvo_ch7017.o
  CC [M]  drivers/gpu/drm/i915/intel_fbdev.o
  CC [M]  drivers/gpu/drm/i915/dvo_ch7xxx.o

cc1: some warnings being treated as errors
scripts/Makefile.build:291: recipe for target 'drivers/gpu/drm/i915/intel_lrc.o' failed make[4]: *** [drivers/gpu/drm/i915/intel_lrc.o] Error 1
make[4]: *** Waiting for unfinished jobs....
scripts/Makefile.build:440: recipe for target 'drivers/gpu/drm/i915' failed make[3]: *** [drivers/gpu/drm/i915] Error 2
scripts/Makefile.build:440: recipe for target 'drivers/gpu/drm' failed make[2]: *** [drivers/gpu/drm] Error 2
scripts/Makefile.build:440: recipe for target 'drivers/gpu' failed make[1]: *** [drivers/gpu] Error 2
Makefile:962: recipe for target 'drivers' failed
make: *** [drivers] Error 2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#17 lore raw
In reply to Tvrtko Ursulin
Quoted 6 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Retired request queue coupled with retired work handler is a scalability concern on some workloads of which one dramatic example is gem_close_race.

This series depend on i915_gem_object_pin_map series by Chris Wilson.

My biggest concern with this is that this touches code that has a known GPF and so imo cannot be tested until that *regression* has been fixed. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw
Quoted 10 lines · Chris Wilson
Quoted 6 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Retired request queue coupled with retired work handler is a scalability concern on some workloads of which one dramatic example is gem_close_race.

This series depend on i915_gem_object_pin_map series by Chris Wilson.

My biggest concern with this is that this touches code that has a known GPF and so imo cannot be tested until that *regression* has been fixed.

That's the ringbuffer one or something else?

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#19 lore raw
Quoted 8 lines · Tvrtko Ursulin
Quoted 6 lines · Chris Wilson
Quoted 5 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Retired request queue coupled with retired work handler is a scalability concern on some workloads of which one dramatic example is gem_close_race.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

This series depend on i915_gem_object_pin_map series by Chris Wilson.

My biggest concern with this is that this touches code that has a known GPF and so imo cannot be tested until that *regression* has been fixed.

That's the ringbuffer one or something else?

There's a GPF when we try to require a stale request because our state tracking is caput.

https://bugs.freedesktop.org/show_bug.cgi?id=93907

Skylake was also misrendering under the same load, but that's likely to be a different bug that I couldn't reproduce after fixing the above. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx