Skip to contentSkip to conversation

drm/i915/ppgtt: Load address space after mi_set_context

PATCH In review Ben Widawsky 4 participants 5 messages Jun 12, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915/ppgtt: Load address space after mi_set_context

On GEN8 the PDPs are saved and restored with context, which means we must set them after the context switch has occurred. If we do not do this, we end up saving the new PDPs for the old context.

Example of a problem
LRI PDPs 1
MI_SET_CONTEXT bar
LRI_PDPs 2
MI_SET_CONTEXT foo // save PDPs 2 to bar's context

		  //  load foos PDPs

LRI PDPs 1
MI_SET_CONTEXT bar // save PDPs 1 to foo's context

It's all wacky. This should allow full PPGTT on Broadwell to work.

---
 drivers/gpu/drm/i915/i915_gem_context.c | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem_context.c +18 -6
index 3ffe308..b2434e0 100644
@@ -623,13 +623,12 @@ static int do_switch(struct intel_engine_cs *ring,
 	 */
 	from = ring->last_context;
 
-	if (USES_FULL_PPGTT(ring->dev)) {
-		ret = ppgtt->switch_mm(ppgtt, ring, false);
-		if (ret)
-			goto unpin_out;
-	}
-
 	if (ring != &dev_priv->ring[RCS]) {
+		if (USES_FULL_PPGTT(ring->dev)) {
+			ret = ppgtt->switch_mm(ppgtt, ring, false);
+			if (ret)
+				goto unpin_out;
+		}
 		if (from)
 			i915_gem_context_unreference(from);
 		goto done;
@@ -660,6 +659,19 @@ static int do_switch(struct intel_engine_cs *ring,
 	if (ret)
 		goto unpin_out;
 
+	if (USES_FULL_PPGTT(ring->dev)) {
+		ret = ppgtt->switch_mm(ppgtt, ring, false);
+		/* The hardware context switch is emitted, but we haven't
+		 * actually changed the state - so it's probably safe to bail
+		 * here. Still, let the user know something dangerous has
+		 * happened.
+		 */
+		if (ret) {
+			DRM_ERROR("Failed to change address space on context switch\n");
+			goto unpin_out;
+		}
+	}
+
 	for (i = 0; i < MAX_L3_SLICES; i++) {
 		if (!(to->remap_slice & (1<<i)))
 			continue;
Signature · 1 line
2.0.0
#1 lore raw
Quoted 9 lines · Ben Widawsky

On GEN8 the PDPs are saved and restored with context, which means we must set them after the context switch has occurred. If we do not do this, we end up saving the new PDPs for the old context.

Example of a problem
LRI PDPs 1
MI_SET_CONTEXT bar
LRI_PDPs 2
MI_SET_CONTEXT foo // save PDPs 2 to bar's context

Quoted 5 lines
		  //  load foos PDPs

LRI PDPs 1
MI_SET_CONTEXT bar // save PDPs 1 to foo's context

It's all wacky. This should allow full PPGTT on Broadwell to work.

Hmm. I had this impression too but now I'm not sure. The PDPs are listed as being in the execlist context. Do we save/restore that in ring buffer mode too? IIRC on ivb/hsw it got skipped.

And if the PDPs are really part of the context we shouldn't need to load them at all I think, except when we skip the restore (unitialized or default context). Or am I missing something here?

And what about ivb/hsw? Doesn't this reordering risk the context restore accessing the wrong PPGTT. That's assuming the context restore can already chase some pointers.

Quoted 56 lines
---
 drivers/gpu/drm/i915/i915_gem_context.c | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem_context.c +18 -6
index 3ffe308..b2434e0 100644
@@ -623,13 +623,12 @@ static int do_switch(struct intel_engine_cs *ring,
 	 */
 	from = ring->last_context;
 
-	if (USES_FULL_PPGTT(ring->dev)) {
-		ret = ppgtt->switch_mm(ppgtt, ring, false);
-		if (ret)
-			goto unpin_out;
-	}
-
 	if (ring != &dev_priv->ring[RCS]) {
+		if (USES_FULL_PPGTT(ring->dev)) {
+			ret = ppgtt->switch_mm(ppgtt, ring, false);
+			if (ret)
+				goto unpin_out;
+		}
 		if (from)
 			i915_gem_context_unreference(from);
 		goto done;
@@ -660,6 +659,19 @@ static int do_switch(struct intel_engine_cs *ring,
 	if (ret)
 		goto unpin_out;
 
+	if (USES_FULL_PPGTT(ring->dev)) {
+		ret = ppgtt->switch_mm(ppgtt, ring, false);
+		/* The hardware context switch is emitted, but we haven't
+		 * actually changed the state - so it's probably safe to bail
+		 * here. Still, let the user know something dangerous has
+		 * happened.
+		 */
+		if (ret) {
+			DRM_ERROR("Failed to change address space on context switch\n");
+			goto unpin_out;
+		}
+	}
+
 	for (i = 0; i < MAX_L3_SLICES; i++) {
 		if (!(to->remap_slice & (1<<i)))
 			continue;
Signature · 6 lines
2.0.0

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 2 lines
Ville Syrjälä
Intel OTC
#2 lore raw
Quoted 16 lines · Ville Syrjälä
Quoted 9 lines · Ben Widawsky

On GEN8 the PDPs are saved and restored with context, which means we must set them after the context switch has occurred. If we do not do this, we end up saving the new PDPs for the old context.

Example of a problem
LRI PDPs 1
MI_SET_CONTEXT bar
LRI_PDPs 2
MI_SET_CONTEXT foo // save PDPs 2 to bar's context

Quoted 5 lines
		  //  load foos PDPs

LRI PDPs 1
MI_SET_CONTEXT bar // save PDPs 1 to foo's context

It's all wacky. This should allow full PPGTT on Broadwell to work.

Quoted 7 lines

Hmm. I had this impression too but now I'm not sure. The PDPs are listed as being in the execlist context. Do we save/restore that in ring buffer mode too? IIRC on ivb/hsw it got skipped.

And if the PDPs are really part of the context we shouldn't need to load them at all I think, except when we skip the restore (unitialized or default context). Or am I missing something here?

Yes, I believe you are correct. However, I don't see any harm with the existing solution either, and I was hesitant to change stuff around, since at the time I just needed to make it work for the 64b series. But I think what you're saying is just adding an "&& !to->is_initialized" to the USES_FULL_PPGTT check, right? I can try that.

Quoted 4 lines

And what about ivb/hsw? Doesn't this reordering risk the context restore accessing the wrong PPGTT. That's assuming the context restore can already chase some pointers.

This is not meant for IVB/HSW. Empirically we've found it doesn't have the same behavior, as you said above. So that's definite an issue which requires a patch rev.

Quoted 61 lines
Quoted 57 lines
---
 drivers/gpu/drm/i915/i915_gem_context.c | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem_context.c +18 -6
index 3ffe308..b2434e0 100644
@@ -623,13 +623,12 @@ static int do_switch(struct intel_engine_cs *ring,
 	 */
 	from = ring->last_context;
 
-	if (USES_FULL_PPGTT(ring->dev)) {
-		ret = ppgtt->switch_mm(ppgtt, ring, false);
-		if (ret)
-			goto unpin_out;
-	}
-
 	if (ring != &dev_priv->ring[RCS]) {
+		if (USES_FULL_PPGTT(ring->dev)) {
+			ret = ppgtt->switch_mm(ppgtt, ring, false);
+			if (ret)
+				goto unpin_out;
+		}
 		if (from)
 			i915_gem_context_unreference(from);
 		goto done;
@@ -660,6 +659,19 @@ static int do_switch(struct intel_engine_cs *ring,
 	if (ret)
 		goto unpin_out;
 
+	if (USES_FULL_PPGTT(ring->dev)) {
+		ret = ppgtt->switch_mm(ppgtt, ring, false);
+		/* The hardware context switch is emitted, but we haven't
+		 * actually changed the state - so it's probably safe to bail
+		 * here. Still, let the user know something dangerous has
+		 * happened.
+		 */
+		if (ret) {
+			DRM_ERROR("Failed to change address space on context switch\n");
+			goto unpin_out;
+		}
+	}
+
 	for (i = 0; i < MAX_L3_SLICES; i++) {
 		if (!(to->remap_slice & (1<<i)))
 			continue;
Signature · 6 lines
2.0.0

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 2 lines
Ville Syrjälä
Intel OTC
Signature · 5 lines
Ben Widawsky, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw
Quoted 21 lines · Ben Widawsky
Quoted 20 lines · Ville Syrjälä
Quoted 9 lines · Ben Widawsky

On GEN8 the PDPs are saved and restored with context, which means we must set them after the context switch has occurred. If we do not do this, we end up saving the new PDPs for the old context.

Example of a problem
LRI PDPs 1
MI_SET_CONTEXT bar
LRI_PDPs 2
MI_SET_CONTEXT foo // save PDPs 2 to bar's context

Quoted 5 lines
		  //  load foos PDPs

LRI PDPs 1
MI_SET_CONTEXT bar // save PDPs 1 to foo's context

It's all wacky. This should allow full PPGTT on Broadwell to work.

Hmm. I had this impression too but now I'm not sure. The PDPs are listed as being in the execlist context. Do we save/restore that in ring buffer mode too? IIRC on ivb/hsw it got skipped.

Quoted 6 lines
Quoted 3 lines

And if the PDPs are really part of the context we shouldn't need to load them at all I think, except when we skip the restore (unitialized or default context). Or am I missing something here?

Yes, I believe you are correct. However, I don't see any harm with the existing solution either,

Yeah just a bit wasteful to load twice. Should still work fine.

Quoted 4 lines

and I was hesitant to change stuff around,
since at the time I just needed to make it work for the 64b series. But I think what you're saying is just adding an "&& !to->is_initialized" to the USES_FULL_PPGTT check, right? I can try that.

or '&& (hw_flags & MI_RESTORE_INHIBIT)' to catch both default and !initialized contexts.

Quoted 9 lines
Quoted 5 lines

And what about ivb/hsw? Doesn't this reordering risk the context restore accessing the wrong PPGTT. That's assuming the context restore can already chase some pointers.

This is not meant for IVB/HSW. Empirically we've found it doesn't have the same behavior, as you said above. So that's definite an issue which requires a patch rev.

OK

Quoted 65 lines
Quoted 62 lines
Quoted 58 lines
---
 drivers/gpu/drm/i915/i915_gem_context.c | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem_context.c +18 -6
index 3ffe308..b2434e0 100644
@@ -623,13 +623,12 @@ static int do_switch(struct intel_engine_cs *ring,
 	 */
 	from = ring->last_context;
 
-	if (USES_FULL_PPGTT(ring->dev)) {
-		ret = ppgtt->switch_mm(ppgtt, ring, false);
-		if (ret)
-			goto unpin_out;
-	}
-
 	if (ring != &dev_priv->ring[RCS]) {
+		if (USES_FULL_PPGTT(ring->dev)) {
+			ret = ppgtt->switch_mm(ppgtt, ring, false);
+			if (ret)
+				goto unpin_out;
+		}
 		if (from)
 			i915_gem_context_unreference(from);
 		goto done;
@@ -660,6 +659,19 @@ static int do_switch(struct intel_engine_cs *ring,
 	if (ret)
 		goto unpin_out;
 
+	if (USES_FULL_PPGTT(ring->dev)) {
+		ret = ppgtt->switch_mm(ppgtt, ring, false);
+		/* The hardware context switch is emitted, but we haven't
+		 * actually changed the state - so it's probably safe to bail
+		 * here. Still, let the user know something dangerous has
+		 * happened.
+		 */
+		if (ret) {
+			DRM_ERROR("Failed to change address space on context switch\n");
+			goto unpin_out;
+		}
+	}
+
 	for (i = 0; i < MAX_L3_SLICES; i++) {
 		if (!(to->remap_slice & (1<<i)))
 			continue;
Signature · 6 lines
2.0.0

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 2 lines
Ville Syrjälä
Intel OTC
Signature · 1 line
Ben Widawsky, Intel Open Source Technology Center
Signature · 2 lines
Ville Syrjälä
Intel OTC
#4 lore raw
In reply to Ben Widawsky
Quoted 3 lines · Ben Widawsky

This is not meant for IVB/HSW. Empirically we've found it doesn't have the same behavior, as you said above. So that's definite an issue which requires a patch rev.

Sounds like we might need a per-platform do_ctx_swithc function to encode these details? Maybe not given that execlist is the rage and that has a completely different submission path.
-Daniel

Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch