drm/i915: Also perform gpu reset under execlist mode.
Subsystems (2)
PATCH v2 1/1 drm/i915: Also perform gpu reset under execlist mode.
From: "Niu,Bing" <[email protected]>
It is found that i915 will not reset gpu under execlist mode when unload module. that will lead to some issues when unload/load module with different submission mode. e.g. from execlist mode to ring buffer mode via loading/unloading i915. Because HW is not in a reset state and registers are not clean under such condition.
--- drivers/gpu/drm/i915/i915_gem.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/gpu/drm/i915/i915_gem.c +8 -0
@@ -5115,6 +5115,14 @@ i915_gem_cleanup_ringbuffer(struct drm_device *dev)for_each_ring (ring, dev_priv, i) dev_priv->gt.cleanup_ring (ring); + +if (i915.enable_execlists) +/* +* Neither the BIOS, ourselves or any other kernel +* expects the system to be in execlists mode on startup, +* so we need to reset the GPU back to legacy mode. +*/ +intel_gpu_reset (dev); } static void
Signature · 6 lines
1.7.9.5 _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 9 lines · On Sat, Jul 04, 2015 at 12:27:34AM +0800, [email protected] wrote:
From: "Niu,Bing" <[email protected]>
It is found that i915 will not reset gpu under execlist mode when unload module. that will lead to some issues when unload/load module with different submission mode. e.g. from execlist mode to ring buffer mode via loading/unloading i915. Because HW is not in a reset state and registers are not clean under such condition.
Signed-off-by Niu,Bing <[email protected]>
I think we may end up doing the reset unconditionally in i915_driver_unload() because this argument holds for almost everything we setup. It's a bigger risk because of doing the gpu-reset on more machines, but module-unloading is a "developer feature"!
The only issue is making sure that the reset is ordered appropriately. -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 9 lines · Chris Wilson
Quoted 8 lines · On Sat, Jul 04, 2015 at 12:27:34AM +0800, [email protected] wrote:
From: "Niu,Bing" <[email protected]>
It is found that i915 will not reset gpu under execlist mode when unload module. that will lead to some issues when unload/load module with different submission mode. e.g. from execlist mode to ring buffer mode via loading/unloading i915. Because HW is not in a reset state and registers are not clean under such condition.
Quoted 7 lines
Quoted 1 line
Signed-off-by Niu,Bing <[email protected]>Reviewed-by Chris Wilson <[email protected]>I think we may end up doing the reset unconditionally in
i915_driver_unload()because this argument holds for almost everything we setup. It's a bigger risk because of doing the gpu-reset on more machines, but module-unloading is a "developer feature"!
And after that has been sorted, we should try reset on module load.
This way initial state would be identical to after reset/unload state. Now we have this situation that we don't know how much we are leaning on bios on state setup.
-Mika
Quoted 9 lines
The only issue is making sure that the reset is ordered appropriately. -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Hi Chris and Mika:
Thanks for the comments. I think that reset HW on module unload is
an good idea. For now I think we should choose a proper position in the module unload sequence to reset HW. As GPU reset is render engine reset plus ring imrs(They will become to alll F after full GPU reset), I think a proper position should be after render and interrupt shutdown path.
How about this place?
drivers/gpu/drm/i915/i915_dma.c +4 -1
@@ -1133,7 +1133,10 @@ int i915_driver_unload(struct drm_device *dev)pm_qos_remove_request (&dev_priv->pm_qos);i915_global_gtt_cleanup (dev); - +/* The only known way to stop the gpu from accessing the hw context is +* to reset it. Do this as the very last operation to avoid confusing +* other code, leading to spurious errors. */ +intel_gpu_reset (dev);intel_uncore_fini (dev);if (dev_priv->regs !=NULL )pci_iounmap (dev->pdev, dev_priv->regs);
drivers/gpu/drm/i915/i915_gem_context.c +0 -5
@@ -373,11 +373,6 @@ void i915_gem_context_fini(struct drm_device *dev)int i;if (dctx->legacy_hw_ctx.rcs_state) { -/* The only known way to stop the gpu from accessing the hw context is -* to reset it. Do this as the very last operation to avoid confusing -* other code, leading to spurious errors. */ -intel_gpu_reset (dev); -/* When default context is created and switched to,
base object refcount
* will be 2 (+1 from object creation and +1 fromdo_switch()).
* i915_gem_context_fini() will be called aftergpu_idle() has switched
Quoted 36 lines · Mika Kuoppala
Quoted 9 lines · Chris Wilson
Quoted 8 lines · On Sat, Jul 04, 2015 at 12:27:34AM +0800, [email protected] wrote:
From: "Niu,Bing" <[email protected]>
It is found that i915 will not reset gpu under execlist mode when unload module. that will lead to some issues when unload/load module with different submission mode. e.g. from execlist mode to ring buffer mode via loading/unloading i915. Because HW is not in a reset state and registers are not clean under such condition.
Quoted 7 lines
Quoted 1 line
Signed-off-by Niu,Bing <[email protected]>Reviewed-by Chris Wilson <[email protected]>I think we may end up doing the reset unconditionally in
i915_driver_unload()because this argument holds for almost everything we setup. It's a bigger risk because of doing the gpu-reset on more machines, but module-unloading is a "developer feature"!And after that has been sorted, we should try reset on module load.
This way initial state would be identical to after reset/unload state. Now we have this situation that we don't know how much we are leaning on bios on state setup.
-Mika
Quoted 9 lines
The only issue is making sure that the reset is ordered appropriately. -Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 5 lines
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 24 lines · Zhi Wang
Hi Chris and Mika:
Thanks for the comments. I think that reset HW on module unload
is an good idea. For now I think we should choose a proper position in the module unload sequence to reset HW. As GPU reset is render engine reset plus ring imrs(They will become to alll F after full GPU reset), I think a proper position should be after render and interrupt shutdown path.How about this place?
drivers/gpu/drm/i915/i915_dma.c +1
-1@@ -1133,7 +1133,10 @@ int i915_driver_unload(struct drm_device *dev)pm_qos_remove_request (&dev_priv->pm_qos);i915_global_gtt_cleanup (dev); - +/* The only known way to stop the gpu from accessing the hw context is
+ * to reset it. Do this as the very last operation to avoid
confusing
drivers/gpu/drm/i915/i915_dma.c int i915_driver_unload(struct drm_device *dev) · line 1139
Quoted 2 lines
drivers/gpu/drm/i915/i915_dma.c +2
-0+* other code, leading to spurious errors. */ +intel_gpu_reset (dev);
That feels right. The comment is out-of-place now and needs expansion to include other side effects for which the gpu reset is meritted.
But this is a riskier patch since we now start doing unconditional resets for gen3-gen5. Just requires more soak testing, but I would prefer it as (1) add execlists reset, (2) combine execlists reset + power context reset into a single unload reset. That way if we do get a regression in doing the unload reset we can revert back to execlists easily.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Hi Chris:
Thanks for the comments! I can understand that we're concerned
about regressions, so this is why I think put this reset in module unload path looks much safer. For safety, maybe we should only reset GPU perhaps only when GEN >= 6? That looks much easier and safer, also combine execlist reset and power context reset.
Or we just add this before i915_uncore_fini() inside i915_driver_unload()? This way looks much safer?
How about this one?
drivers/gpu/drm/i915/i915_dma.c +9 -0
@@ -1134,6 +1134,15 @@ int i915_driver_unload(struct drm_device *dev)i915_global_gtt_cleanup (dev); +/* +* Restore HW workload submission mode back to default mode when shutdown. +* It makes i915 module loading/unloading be able to switch between +* different workload submission mode on gen8+. And according to B-spec, +* the only way to reset HW workload submission mode to default mode is GPU reset. +*/ +if (i915.enable_execlists) +intel_gpu_reset (dev); +intel_uncore_fini (dev);if (dev_priv->regs !=NULL )pci_iounmap (dev->pdev, dev_priv->regs);
Quoted 31 lines · Chris Wilson
Quoted 24 lines · Zhi Wang
Hi Chris and Mika:
Thanks for the comments. I think that reset HW on module unload
is an good idea. For now I think we should choose a proper position in the module unload sequence to reset HW. As GPU reset is render engine reset plus ring imrs(They will become to alll F after full GPU reset), I think a proper position should be after render and interrupt shutdown path.How about this place?
drivers/gpu/drm/i915/i915_dma.c +1
-1@@ -1133,7 +1133,10 @@ int i915_driver_unload(struct drm_device *dev)pm_qos_remove_request (&dev_priv->pm_qos);i915_global_gtt_cleanup (dev); - +/* The only known way to stop the gpu from accessing the hw context is
+ * to reset it. Do this as the very last operation to avoid
confusingdrivers/gpu/drm/i915/i915_dma.c
Quoted 2 lines
drivers/gpu/drm/i915/i915_dma.c +2
-0+* other code, leading to spurious errors. */ +intel_gpu_reset (dev);That feels right. The comment is out-of-place now and needs expansion to include other side effects for which the gpu reset is meritted.
Quoted 8 lines
But this is a riskier patch since we now start doing unconditional resets for gen3-gen5. Just requires more soak testing, but I would prefer it as (1) add execlists reset, (2) combine execlists reset + power context reset into a single unload reset. That way if we do get a regression in doing the unload reset we can revert back to execlists easily.
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 11 lines · Zhi Wang
Hi Chris:
Thanks for the comments! I can understand that we're concerned
about regressions, so this is why I think put this reset in module unload path looks much safer. For safety, maybe we should only reset GPU perhaps only when GEN >= 6? That looks much easier and safer, also combine execlist reset and power context reset.Or we just add this before
i915_uncore_fini()insidei915_driver_unload()? This way looks much safer?How about this one?
No, if we are just targetting execlists, then disabling it in cleanup_ringbuffers as before is the cleanest (as that is the opposite stage to where we enable them).
The reset in i915_driver_unload() is preferred to replace all the resets required during unload. It is safe to move the context reset here as we do not disturb the GTT state between unpining the context and here. Making it conditional on gen>=5 is probably a good first step. -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Hi Chris:
How about this one? :)
drivers/gpu/drm/i915/i915_dma.c +15 -0
@@ -1134,6 +1134,21 @@ int i915_driver_unload(struct drm_device *dev)i915_global_gtt_cleanup (dev); +/* +* The only known way to stop the gpu from accessing the hw context in +* graphics memory space is to reset it. Do this as the very last +* operation to avoid confusing other code, leading to spurious errors. +* +* Besides, we also need to restore HW workload submission mode back +* to default mode when shutdown i915. +* +* It makes i915 module loading/unloading be able to switch between +* different workload submission mode on gen8+. And according to B-spec, +* the only way to reset HW workload submission mode to default mod is GPU reset. +*/ +if (INTEL_INFO (dev)->gen >=5 ) +intel_gpu_reset (dev); +intel_uncore_fini (dev);if (dev_priv->regs !=NULL )pci_iounmap (dev->pdev, dev_priv->regs);
drivers/gpu/drm/i915/i915_gem_context.c +0 -5
@@ -373,11 +373,6 @@ void i915_gem_context_fini(struct drm_device *dev)int i;if (dctx->legacy_hw_ctx.rcs_state) { -/* The only known way to stop the gpu from accessing the hw context is -* to reset it. Do this as the very last operation to avoid confusing -* other code, leading to spurious errors. */ -intel_gpu_reset (dev); -/* When default context is created and switched to,
base object refcount
* will be 2 (+1 from object creation and +1 fromdo_switch()).
* i915_gem_context_fini() will be called aftergpu_idle() has switched
Quoted 17 lines · Chris Wilson
Quoted 11 lines · Zhi Wang
Hi Chris:
Thanks for the comments! I can understand that we're concerned
about regressions, so this is why I think put this reset in module unload path looks much safer. For safety, maybe we should only reset GPU perhaps only when GEN >= 6? That looks much easier and safer, also combine execlist reset and power context reset.Or we just add this before
i915_uncore_fini()insidei915_driver_unload()? This way looks much safer?How about this one?
No, if we are just targetting execlists, then disabling it in
cleanup_ringbuffersas before is the cleanest (as that is the opposite stage to where we enable them).
Quoted 6 lines
The reset in
i915_driver_unload()is preferred to replace all the resets required during unload. It is safe to move the context reset here as we do not disturb the GTT state between unpining the context and here. Making it conditional on gen>=5 is probably a good first step. -Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 52 lines · Zhi Wang
Hi Chris:
How about this one? :)
drivers/gpu/drm/i915/i915_dma.c +2
-0@@ -1134,6 +1134,21 @@ int i915_driver_unload(struct drm_device *dev)i915_global_gtt_cleanup (dev); +/* +* The only known way to stop the gpu from accessing the hw context in
drivers/gpu/drm/i915/i915_dma.c +2
-0+* graphics memory space is to reset it. Do this as the very last +* operation to avoid confusing other code, leading to spurious errors.
drivers/gpu/drm/i915/i915_dma.c +6
-0+* +* Besides, we also need to restore HW workload submission mode back +* to default mode when shutdown i915. +* +* It makes i915 module loading/unloading be able to switch between +* different workload submission mode on gen8+. And according to B-spec,
+ * the only way to reset HW workload submission mode to
default mod is GPU reset.drivers/gpu/drm/i915/i915_dma.c +4
-0+*/ +if (INTEL_INFO (dev)->gen >=5 ) +intel_gpu_reset (dev); +intel_uncore_fini (dev);if (dev_priv->regs !=NULL )pci_iounmap (dev->pdev, dev_priv->regs);drivers/gpu/drm/i915/i915_gem_context.c +0
-1@@ -373,11 +373,6 @@ void i915_gem_context_fini(struct drm_device *dev)int i;if (dctx->legacy_hw_ctx.rcs_state) { -/* The only known way to stop the gpu from accessing the hw context is
- * to reset it. Do this as the very last operation
to avoid confusingdrivers/gpu/drm/i915/i915_gem_context.c +0
-3-* other code, leading to spurious errors. */ -intel_gpu_reset (dev); -/* When default context is created and switched to,
Quoted 5 lines
base object refcount
* will be 2 (+1 from object creation and +1 fromdo_switch()).
*i915_gem_context_fini()will be called aftergpu_idle()has switched
I'd be happy to r-b that. Having a verbose comment explaining why we put the shotgun to our head is definitely required, thanks. -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Thanks for the comments!:) I'm wondering that why the GEN>=5 is needed here. Is there any HW problems in GEN>=5 when driver wants to do GPU reset? I'm sorry that I don't know much about GEN<6...Just want to know more about the story:)
Quoted 54 lines · Chris Wilson
Quoted 53 lines · Zhi Wang
Hi Chris:
How about this one? :)
drivers/gpu/drm/i915/i915_dma.c +2
-0@@ -1134,6 +1134,21 @@ int i915_driver_unload(struct drm_device *dev)i915_global_gtt_cleanup (dev); +/* +* The only known way to stop the gpu from accessing the hw context in
drivers/gpu/drm/i915/i915_dma.c +2
-0+* graphics memory space is to reset it. Do this as the very last +* operation to avoid confusing other code, leading to spurious errors.
drivers/gpu/drm/i915/i915_dma.c +6
-0+* +* Besides, we also need to restore HW workload submission mode back +* to default mode when shutdown i915. +* +* It makes i915 module loading/unloading be able to switch between +* different workload submission mode on gen8+. And according to B-spec,
+ * the only way to reset HW workload submission mode to
default mod is GPU reset.drivers/gpu/drm/i915/i915_dma.c +4
-0+*/ +if (INTEL_INFO (dev)->gen >=5 ) +intel_gpu_reset (dev); +intel_uncore_fini (dev);if (dev_priv->regs !=NULL )pci_iounmap (dev->pdev, dev_priv->regs);drivers/gpu/drm/i915/i915_gem_context.c +0
-1@@ -373,11 +373,6 @@ void i915_gem_context_fini(struct drm_device *dev)int i;if (dctx->legacy_hw_ctx.rcs_state) { -/* The only known way to stop the gpu from accessing the hw context is
- * to reset it. Do this as the very last operation
to avoid confusingdrivers/gpu/drm/i915/i915_gem_context.c +0
-3-* other code, leading to spurious errors. */ -intel_gpu_reset (dev); -/* When default context is created and switched to, base object refcount
Quoted 9 lines
Quoted 4 lines
* will be 2 (+1 from object creation and +1 from
do_switch()).
*i915_gem_context_fini()will be called aftergpu_idle()has switchedI'd be happy to r-b that. Having a verbose comment explaining why we put the shotgun to our head is definitely required, thanks. -Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 4 lines · Zhi Wang
Thanks for the comments!:) I'm wondering that why the GEN>=5 is needed here. Is there any HW problems in GEN>=5 when driver wants to do GPU reset? I'm sorry that I don't know much about GEN<6...Just want to know more about the story:)
gen5 is about the earliest that I know the hw stated to get smart and doing automatic power saving by writing to pinned buffers. Before gen5 our reset hasn't been very reliable, and only recently have we had code that is meant to be able to reset the gpu on those gen - hence my reservation about doing so unconditionally.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Thanks! I had question about power context.
I was curious about HW context and once dumped the GEM context object on GEN6 before.
According to B-spec, Gen Graphics » BSpec » 3D-Media-GPGPU Engine » Registers in Render Engine » Render Logical Context Data » Register State Context [SNB+] » Register State Context [HSW] »
I thought the context should contain all the register regions in the list include the dark yellow marked registers as power context registers, but I saw the content of the context I got began with main context, LRI signature 110010bf. .. It's weird.
I don't know if the context saved by MI_SET_CONTEXT is just the power context in B-spec?If yes, why doest it looks like that?
Thanks,
Zhi.
Quoted 6 lines · Chris Wilson
Quoted 4 lines · Zhi Wang
Thanks for the comments!:) I'm wondering that why the GEN>=5 is needed here. Is there any HW problems in GEN>=5 when driver wants to do GPU reset? I'm sorry that I don't know much about GEN<6...Just want to know more about the story:)
Quoted 7 lines
gen5 is about the earliest that I know the hw stated to get smart and doing automatic power saving by writing to pinned buffers. Before gen5 our reset hasn't been very reliable, and only recently have we had code that is meant to be able to reset the gpu on those gen - hence my reservation about doing so unconditionally.
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx