Skip to contentSkip to conversation

drm/i915/skl: Use correct use counters for force wakes

PATCH In review Tvrtko Ursulin 5 participants 10 messages Sep 25, 2014 – Sep 30, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915/skl: Use correct use counters for force wakes

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

---
 drivers/gpu/drm/i915/intel_lrc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/intel_lrc.c +2 -2
index 3048d78..4b33767 100644
@@ -301,7 +301,7 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 	 * Instead, we do the runtime_pm_get/put when creating/destroying requests.
 	 */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev_priv->dev) || IS_GEN9(dev_priv->dev)) {
 		if (dev_priv->uncore.fw_rendercount++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
 							      FORCEWAKE_RENDER);
@@ -326,7 +326,7 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 
 	/* Release Force Wakeup (see the big comment above). */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev_priv->dev) || IS_GEN9(dev_priv->dev)) {
 		if (--dev_priv->uncore.fw_rendercount == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
 							      FORCEWAKE_RENDER);
Signature · 1 line
2.1.0
#1 lore raw
Quoted 7 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

Is it worth a v2 to have gen >= 9 here?

Signature · 1 line
Damien
Quoted 29 lines
---
 drivers/gpu/drm/i915/intel_lrc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/intel_lrc.c +2 -2
index 3048d78..4b33767 100644
@@ -301,7 +301,7 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 	 * Instead, we do the runtime_pm_get/put when creating/destroying requests.
 	 */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev_priv->dev) || IS_GEN9(dev_priv->dev)) {
 		if (dev_priv->uncore.fw_rendercount++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
 							      FORCEWAKE_RENDER);
@@ -326,7 +326,7 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 
 	/* Release Force Wakeup (see the big comment above). */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev_priv->dev) || IS_GEN9(dev_priv->dev)) {
 		if (--dev_priv->uncore.fw_rendercount == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
 							      FORCEWAKE_RENDER);
Signature · 1 line
2.1.0
#2 lore raw
Quoted 10 lines · Damien Lespiau
Quoted 7 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

Is it worth a v2 to have gen >= 9 here?

I think we should have gen >= 8 here.

Shadowed ELSP's seems not to work on gen8. And the posting read will need fw anyways.

Assuming the shadowing works on skl and we can get rid of the posting read, we could run this part without taking forcewake.

-Mika

Quoted 36 lines
Signature · 1 line
Damien
Quoted 29 lines
---
 drivers/gpu/drm/i915/intel_lrc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/intel_lrc.c +2 -2
index 3048d78..4b33767 100644
@@ -301,7 +301,7 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 	 * Instead, we do the runtime_pm_get/put when creating/destroying requests.
 	 */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev_priv->dev) || IS_GEN9(dev_priv->dev)) {
 		if (dev_priv->uncore.fw_rendercount++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
 							      FORCEWAKE_RENDER);
@@ -326,7 +326,7 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 
 	/* Release Force Wakeup (see the big comment above). */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev_priv->dev) || IS_GEN9(dev_priv->dev)) {
 		if (--dev_priv->uncore.fw_rendercount == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
 							      FORCEWAKE_RENDER);
Signature · 1 line
2.1.0

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw
Quoted 9 lines · Mika Kuoppala
Quoted 7 lines · Damien Lespiau
Quoted 6 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Is it worth a v2 to have gen >= 9 here?

I think we should have gen >= 8 here.

But that would not match against the current implementation of GEN8 vs CHV read/write functions.

Quoted 5 lines

Shadowed ELSP's seems not to work on gen8. And the posting read will need fw anyways.

Assuming the shadowing works on skl and we can get rid of the posting read, we could run this part without taking forcewake.

I don't know what criteria would need to be satisfied to get rid of the posting read. On GEN9 only you are saying?

2nd part, how to test if shadowing works? Just remove force wakes and see what happens?

Thanks,

Tvrtko

#4 lore raw
Quoted 11 lines · Tvrtko Ursulin
Quoted 9 lines · Mika Kuoppala
Quoted 7 lines · Damien Lespiau
Quoted 6 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Is it worth a v2 to have gen >= 9 here?

I think we should have gen >= 8 here.

But that would not match against the current implementation of GEN8 vs CHV read/write functions.

The first problem here is that we have two sets of reference counts, one for FORCEWAKE_ALL (uncore.forcewake_count) and one for each forcewake engine on CHV/SKL. CHV has 2 fw engines, SKL 3.

So, in that regard, maybe the fw engine abstraction needs work to unify the reference counts.

The least intrusive change to make SKL not error out would be to have a separate code path for gen >= 9 with the 3 engines.

The thread leading to that patch is great and already has all the right questions. We just ignored them.

The most immediate one is, why are we waking up all the fw engines when writing a specific ring ELSP, but the whole thread is worth reading.

Quoted 11 lines
Quoted 5 lines

Shadowed ELSP's seems not to work on gen8. And the posting read will need fw anyways.

Assuming the shadowing works on skl and we can get rid of the posting read, we could run this part without taking forcewake.

I don't know what criteria would need to be satisfied to get rid of the posting read. On GEN9 only you are saying?

2nd part, how to test if shadowing works? Just remove force wakes and see what happens?

And that's a separate issue, can we use the shadow register mechanism to ensure the write lands and is that enough? Let's try to ask people internally about that.

HTH,

Signature · 1 line
Damien
#5 lore raw

PATCH 1/1 drm/i915/skl: Use correct use counters for force wakes

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

NOTE: Depending on feedback from hardware designers it may not be necessary to grab force wakes on Gen9 here. But for Gen8 it is needed due to a race between RC6 and ELSP writes.

v2: Added blitter force wake engine and made more future proof.
Added commit note.

---
 drivers/gpu/drm/i915/intel_lrc.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/intel_lrc.c +14 -3
index 3048d78..0792d7a 100644
@@ -276,7 +276,8 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 				 struct drm_i915_gem_object *ctx_obj0,
 				 struct drm_i915_gem_object *ctx_obj1)
 {
-	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct drm_device *dev = ring->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
 	uint64_t temp = 0;
 	uint32_t desc[4];
 	unsigned long flags;
@@ -301,13 +302,18 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 	 * Instead, we do the runtime_pm_get/put when creating/destroying requests.
 	 */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev) || INTEL_INFO(dev)->gen >= 9) {
 		if (dev_priv->uncore.fw_rendercount++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
 							      FORCEWAKE_RENDER);
 		if (dev_priv->uncore.fw_mediacount++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
 							      FORCEWAKE_MEDIA);
+		if (INTEL_INFO(dev)->gen >= 9) {
+			if (dev_priv->uncore.fw_blittercount++ == 0)
+				dev_priv->uncore.funcs.force_wake_get(dev_priv,
+							FORCEWAKE_BLITTER);
+		}
 	} else {
 		if (dev_priv->uncore.forcewake_count++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
@@ -326,13 +332,18 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 
 	/* Release Force Wakeup (see the big comment above). */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev) || INTEL_INFO(dev)->gen >= 9) {
 		if (--dev_priv->uncore.fw_rendercount == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
 							      FORCEWAKE_RENDER);
 		if (--dev_priv->uncore.fw_mediacount == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
 							      FORCEWAKE_MEDIA);
+		if (INTEL_INFO(dev)->gen >= 9) {
+			if (--dev_priv->uncore.fw_blittercount == 0)
+				dev_priv->uncore.funcs.force_wake_put(dev_priv,
+							FORCEWAKE_BLITTER);
+		}
 	} else {
 		if (--dev_priv->uncore.forcewake_count == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
Signature · 1 line
2.1.0
Reviewed-by on v1 1/1
#6 lore raw
Quoted 10 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

NOTE: Depending on feedback from hardware designers it may not be necessary to grab force wakes on Gen9 here. But for Gen8 it is needed due to a race between RC6 and ELSP writes.

Quoted 6 lines

v2: Added blitter force wake engine and made more future proof.
Added commit note.

---

This still has the issue of taking every fw engine, not looking at the ring we're queuing the work for. Also I'll add the note in a comment above the whole block. It does solve at least an error in current kernels so:

Signature · 1 line
Damien
Quoted 60 lines
 drivers/gpu/drm/i915/intel_lrc.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/intel_lrc.c +14 -3
index 3048d78..0792d7a 100644
@@ -276,7 +276,8 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 				 struct drm_i915_gem_object *ctx_obj0,
 				 struct drm_i915_gem_object *ctx_obj1)
 {
-	struct drm_i915_private *dev_priv = ring->dev->dev_private;
+	struct drm_device *dev = ring->dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
 	uint64_t temp = 0;
 	uint32_t desc[4];
 	unsigned long flags;
@@ -301,13 +302,18 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 	 * Instead, we do the runtime_pm_get/put when creating/destroying requests.
 	 */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev) || INTEL_INFO(dev)->gen >= 9) {
 		if (dev_priv->uncore.fw_rendercount++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
 							      FORCEWAKE_RENDER);
 		if (dev_priv->uncore.fw_mediacount++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
 							      FORCEWAKE_MEDIA);
+		if (INTEL_INFO(dev)->gen >= 9) {
+			if (dev_priv->uncore.fw_blittercount++ == 0)
+				dev_priv->uncore.funcs.force_wake_get(dev_priv,
+							FORCEWAKE_BLITTER);
+		}
 	} else {
 		if (dev_priv->uncore.forcewake_count++ == 0)
 			dev_priv->uncore.funcs.force_wake_get(dev_priv,
@@ -326,13 +332,18 @@ static void execlists_elsp_write(struct intel_engine_cs *ring,
 
 	/* Release Force Wakeup (see the big comment above). */
 	spin_lock_irqsave(&dev_priv->uncore.lock, flags);
-	if (IS_CHERRYVIEW(dev_priv->dev)) {
+	if (IS_CHERRYVIEW(dev) || INTEL_INFO(dev)->gen >= 9) {
 		if (--dev_priv->uncore.fw_rendercount == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
 							      FORCEWAKE_RENDER);
 		if (--dev_priv->uncore.fw_mediacount == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
 							      FORCEWAKE_MEDIA);
+		if (INTEL_INFO(dev)->gen >= 9) {
+			if (--dev_priv->uncore.fw_blittercount == 0)
+				dev_priv->uncore.funcs.force_wake_put(dev_priv,
+							FORCEWAKE_BLITTER);
+		}
 	} else {
 		if (--dev_priv->uncore.forcewake_count == 0)
 			dev_priv->uncore.funcs.force_wake_put(dev_priv,
Signature · 1 line
2.1.0
#7 lore raw
In reply to Tvrtko Ursulin
Quoted 6 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

Quoted 6 lines

NOTE: Depending on feedback from hardware designers it may not be necessary to grab force wakes on Gen9 here. But for Gen8 it is needed due to a race between RC6 and ELSP writes.

v2: Added blitter force wake engine and made more future proof.
Added commit note.

Speaking of futureproofing, what did you think of my patch to remove the duplicated counting logic?
-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
#8 lore raw
Quoted 11 lines · Chris Wilson
Quoted 10 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

NOTE: Depending on feedback from hardware designers it may not be necessary to grab force wakes on Gen9 here. But for Gen8 it is needed due to a race between RC6 and ELSP writes.

Quoted 5 lines
Quoted 2 lines

v2: Added blitter force wake engine and made more future proof.
Added commit note.

Speaking of futureproofing, what did you think of my patch to remove the duplicated counting logic?

[For reference it is called "[PATCH] drm/i915: Reduce duplicated forcewake logic".]

Disclaimer: I don't know this code that well - only had to dig into it a few days back when I hit this bug which resulted in my patch.

But from a glance your patch does make it look cleaner and indeed more future proof (much better separation between platform specific and generic). Does it need a proper review to move it forward?

Regards,

Tvrtko

#9 lore raw
Quoted 23 lines · Tvrtko Ursulin
Quoted 11 lines · Chris Wilson
Quoted 10 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Write and reads following the block changed use engine specific use counters and unless that is matched here force wake use counting goes bad. Same force wake is attempted to be taken twice which leads to at least time outs.

NOTE: Depending on feedback from hardware designers it may not be necessary to grab force wakes on Gen9 here. But for Gen8 it is needed due to a race between RC6 and ELSP writes.

Quoted 5 lines
Quoted 2 lines

v2: Added blitter force wake engine and made more future proof.
Added commit note.

Speaking of futureproofing, what did you think of my patch to remove the duplicated counting logic?

[For reference it is called "[PATCH] drm/i915: Reduce duplicated forcewake logic".]

Quoted 6 lines

Disclaimer: I don't know this code that well - only had to dig into it a few days back when I hit this bug which resulted in my patch.

But from a glance your patch does make it look cleaner and indeed more future proof (much better separation between platform specific and generic). Does it need a proper review to move it forward?

Yes. Volunteered?

And when you review pls reply with a blurb in case the existing commit message from Chris doesn't fully cover the bug you've stumbled over, so that lazy me can just copy-paste that while applying ;-)

Thanks, Daniel

Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch