Skip to contentSkip to conversation

drm/i915: Avoid GPU hang when coming out of P3 or P4

PATCH In review Peter Antoine 7 participants 18 messages Apr 27, 2015 – May 5, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915: Avoid GPU hang when coming out of P3 or P4

This patch fixed a timing issue that causes a GPU hang when a the system comes out of power saving.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=89600

---
 drivers/gpu/drm/i915/i915_drv.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +5 -5
index e70adfd..648866f 100644
@@ -712,6 +712,11 @@ static int i915_drm_resume(struct drm_device *dev)
 	intel_init_pch_refclk(dev);
 	drm_mode_config_reset(dev);
 
+	/* We need working interrupts for modeset enabling ... */
+	intel_runtime_pm_enable_interrupts(dev_priv);
+
+	intel_modeset_init_hw(dev);
+
 	mutex_lock(&dev->struct_mutex);
 	if (i915_gem_init_hw(dev)) {
 		DRM_ERROR("failed to re-initialize GPU, declaring wedged!\n");
@@ -719,11 +724,6 @@ static int i915_drm_resume(struct drm_device *dev)
 	}
 	mutex_unlock(&dev->struct_mutex);
 
-	/* We need working interrupts for modeset enabling ... */
-	intel_runtime_pm_enable_interrupts(dev_priv);
-
-	intel_modeset_init_hw(dev);
-
 	spin_lock_irq(&dev_priv->irq_lock);
 	if (dev_priv->display.hpd_irq_setup)
 		dev_priv->display.hpd_irq_setup(dev);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

Ignore this. I mean S3/S4 not P3/P4.

-----Original Message-----
From: Antoine, Peter
Sent: Monday, April 27, 2015 3:25 PM
To: [email protected]
Cc: S, Deepak; Weinehall, David; Tian, YeX; Antoine, Peter
Subject: [PATCH] drm/i915: Avoid GPU hang when coming out of P3 or P4

This patch fixed a timing issue that causes a GPU hang when a the system comes out of power saving.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=89600

---
 drivers/gpu/drm/i915/i915_drv.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +5 -0
@@ -712,6 +712,11 @@ static int i915_drm_resume(struct drm_device *dev)
 	intel_init_pch_refclk(dev);
 	drm_mode_config_reset(dev);
 
+	/* We need working interrupts for modeset enabling ... */
+	intel_runtime_pm_enable_interrupts(dev_priv);
+
+	intel_modeset_init_hw(dev);
+
 	mutex_lock(&dev->struct_mutex);
 	if (i915_gem_init_hw(dev)) {
 		DRM_ERROR("failed to re-initialize GPU, declaring wedged!\n"); @@ -719,11 +724,6 @@ static int i915_drm_resume(struct drm_device *dev)

}
mutex_unlock(&dev->struct_mutex);

- /* We need working interrupts for modeset enabling ... */
- intel_runtime_pm_enable_interrupts(dev_priv);
-
- intel_modeset_init_hw(dev);
-
spin_lock_irq(&dev_priv->irq_lock); if (dev_priv->display.hpd_irq_setup)
dev_priv->display.hpd_irq_setup(dev);
--
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#2 lore raw
In reply to Peter Antoine
Quoted 2 lines · Peter Antoine

This patch fixed a timing issue that causes a GPU hang when a the system comes out of power saving.

A few more details to explain the timing issue and why this is a fix and not just papering over the bug.

Quoted 11 lines

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=89600

---
 drivers/gpu/drm/i915/i915_drv.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +0 -0
index e70adfd..648866f 100644

drivers/gpu/drm/i915/i915_drv.c static int i915_drm_resume(struct drm_device *dev) · line 718

Quoted 7 lines
drivers/gpu/drm/i915/i915_drv.c +4 -0
@@ -712,6 +712,11 @@ static int i915_drm_resume(struct drm_device *dev)
 	intel_init_pch_refclk(dev);
 	drm_mode_config_reset(dev);
 
+	/* We need working interrupts for modeset enabling ... */
+	intel_runtime_pm_enable_interrupts(dev_priv);
+
+	intel_modeset_init_hw(dev);

Both? This makes resume do init_hw() in a different order to other pieces of the reset/resume puzzle. Why? Do we need to consider fixing the other pieces of code as well?
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH v2 1/1 drm/i915: Avoid GPU hang when coming out of S3 or S4

In reply to Peter Antoine

This patch fixed a timing issue that causes a GPU hang when a the system comes out of power saving.

During pm_resume, We are submitting batchbuffers before enabling Interrupts this is causing us to miss the context switch interrupt, and in consequence intel_execlists_handle_ctx_events is not triggered.

This patch is based on a patch from Deepak S <[email protected]> from another platform.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=89600

---
 drivers/gpu/drm/i915/i915_drv.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/i915_drv.c +5 -5
index e70adfd..648866f 100644
@@ -712,6 +712,11 @@ static int i915_drm_resume(struct drm_device *dev)
 	intel_init_pch_refclk(dev);
 	drm_mode_config_reset(dev);
 
+	/* We need working interrupts for modeset enabling ... */
+	intel_runtime_pm_enable_interrupts(dev_priv);
+
+	intel_modeset_init_hw(dev);
+
 	mutex_lock(&dev->struct_mutex);
 	if (i915_gem_init_hw(dev)) {
 		DRM_ERROR("failed to re-initialize GPU, declaring wedged!\n");
@@ -719,11 +724,6 @@ static int i915_drm_resume(struct drm_device *dev)
 	}
 	mutex_unlock(&dev->struct_mutex);
 
-	/* We need working interrupts for modeset enabling ... */
-	intel_runtime_pm_enable_interrupts(dev_priv);
-
-	intel_modeset_init_hw(dev);
-
 	spin_lock_irq(&dev_priv->irq_lock);
 	if (dev_priv->display.hpd_irq_setup)
 		dev_priv->display.hpd_irq_setup(dev);
Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 6 lines · Peter Antoine

This patch fixed a timing issue that causes a GPU hang when a the system comes out of power saving.

During pm_resume, We are submitting batchbuffers before enabling Interrupts this is causing us to miss the context switch interrupt, and in consequence intel_execlists_handle_ctx_events is not triggered.

Ah. Thanks. How about a code comment? :)

But that doesn't explain moving init_hw...
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Thanks Chirs for review, We moved "Init_hw" to initialize WA's before any BB submission.

Init_hw calls " init_clock_gating"

Thanks
Deepak

-----Original Message-----
From: Chris Wilson [mailto:[email protected]]
Sent: Tuesday, April 28, 2015 12:53 PM
To: Antoine, Peter
Cc: [email protected]; S, Deepak; Tian, YeX; Weinehall, David
Subject: Re: [Intel-gfx] [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 6 lines · Peter Antoine

This patch fixed a timing issue that causes a GPU hang when a the system comes out of power saving.

During pm_resume, We are submitting batchbuffers before enabling Interrupts this is causing us to miss the context switch interrupt, and in consequence intel_execlists_handle_ctx_events is not triggered.

Ah. Thanks. How about a code comment? :)

But that doesn't explain moving init_hw...
-Chris

--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 3 lines · Deepak

Thanks Chirs for review, We moved "Init_hw" to initialize WA's before any BB submission.

Init_hw calls " init_clock_gating"

But you appreciate the same issue exists on other paths? -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#7 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Yes agreed, we need to make changes in other paths :)

Quoted 7 lines · Chris Wilson
Quoted 3 lines · Deepak

Thanks Chirs for review, We moved "Init_hw" to initialize WA's before any BB submission.

Init_hw calls " init_clock_gating"

But you appreciate the same issue exists on other paths? -Chris

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#8 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

Peter.

-----Original Message-----
From: Deepak S [mailto:[email protected]]
Sent: Tuesday, April 28, 2015 9:56 AM
To: Chris Wilson; S, Deepak; Antoine, Peter; [email protected]; Tian, YeX; Weinehall, David
Subject: Re: [Intel-gfx] [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Yes agreed, we need to make changes in other paths :)

Quoted 7 lines · Chris Wilson
Quoted 3 lines · Deepak

Thanks Chirs for review, We moved "Init_hw" to initialize WA's before any BB submission.

Init_hw calls " init_clock_gating"

But you appreciate the same issue exists on other paths? -Chris

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#9 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 2 lines · Peter

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

You've identified an issue, but I think your patch is incomplete. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#10 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 5 lines · Chris Wilson
Quoted 2 lines · Peter

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

You've identified an issue, but I think your patch is incomplete.

I've tried my best to go through the remaining similar-looking code, but the rest seems fine (I might've missed something though).

The only thing I reacted on was that in intel_runtime_resume() the call to intel_init_pch_refclk() is conditional on IS_GEN6(), but none of the other invocations of intel_init_pch_refclk() are. The commit message doesn't seem to provide a sufficient explanation for why this is so.

Regards, David
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#11 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 7 lines · David Weinehall
Quoted 5 lines · Chris Wilson
Quoted 2 lines · Peter

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

You've identified an issue, but I think your patch is incomplete.

Quoted 7 lines

I've tried my best to go through the remaining similar-looking code, but the rest seems fine (I might've missed something though).

The only thing I reacted on was that in intel_runtime_resume() the call to intel_init_pch_refclk() is conditional on IS_GEN6(), but none of the other invocations of intel_init_pch_refclk() are. The commit message doesn't seem to provide a sufficient explanation for why this is so.

The explanation for moving init_hw() was that it setups clock_gating. If any in that path are required for enabling the rings, those should be move to init_render_ring() or the init_context.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#12 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 12 lines · Chris Wilson
Quoted 10 lines · David Weinehall
Quoted 5 lines · Chris Wilson
Quoted 2 lines · Peter

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

You've identified an issue, but I think your patch is incomplete.

I've tried my best to go through the remaining similar-looking code, but the rest seems fine (I might've missed something though).

Quoted 8 lines
Quoted 4 lines

The only thing I reacted on was that in intel_runtime_resume() the call to intel_init_pch_refclk() is conditional on IS_GEN6(), but none of the other invocations of intel_init_pch_refclk() are. The commit message doesn't seem to provide a sufficient explanation for why this is so.

The explanation for moving init_hw() was that it setups clock_gating. If any in that path are required for enabling the rings, those should be move to init_render_ring() or the init_context.

Yeah we've had this fun before. init_clock_gating is _not_ for GT workarounds, only for modeset workarounds. We might need to rename that hook to avoid getting bitten for eternity, but moving init_hw because of clock_gating to get the rings up and running is bogus.

As Chris said we need to identify which bits need to get moved to the ring init or w/a bb code and do that (and in a separate patch from enabling the interrupts early enough like this one here does).
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#13 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 27 lines · Daniel Vetter
Quoted 12 lines · Chris Wilson
Quoted 10 lines · David Weinehall
Quoted 5 lines · Chris Wilson
Quoted 2 lines · Peter

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

You've identified an issue, but I think your patch is incomplete.

I've tried my best to go through the remaining similar-looking code, but the rest seems fine (I might've missed something though).

Quoted 8 lines
Quoted 4 lines

The only thing I reacted on was that in intel_runtime_resume() the call to intel_init_pch_refclk() is conditional on IS_GEN6(), but none of the other invocations of intel_init_pch_refclk() are. The commit message doesn't seem to provide a sufficient explanation for why this is so.

The explanation for moving init_hw() was that it setups clock_gating. If any in that path are required for enabling the rings, those should be move to init_render_ring() or the init_context.

Yeah we've had this fun before. init_clock_gating is _not_ for GT workarounds, only for modeset workarounds. We might need to rename that hook to avoid getting bitten for eternity, but moving init_hw because of clock_gating to get the rings up and running is bogus.

Quoted 4 lines

As Chris said we need to identify which bits need to get moved to the ring init or w/a bb code and do that (and in a separate patch from enabling the interrupts early enough like this one here does).
-Daniel

Ok. More work is required.

But, we have two issues here. The open and accessible (to any user) ioctl that causes the driver (and in testing the kernel) to misbehave and cause the system to become unresponsive need or to reboot.This is covered by a simple de-reference check that protects the driver and the system.

Secondly, a nice to have, which is the hw-lock/context code that seems to have more issues with it and should be turned off when it is not required. This code has subtle connections that will need more work. The un-niceness of this code has been known about for a while and it would be good to turn off.

The first I would suggest is kind of important as is simply exploitable security hole, the second is just probably full of security holes.

Can we split this into two jobs, fix the actionable security hole, and get back to the nice to have later.

Peter.
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#14 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Ignore this response, replied to the wrong thread.

Quoted 47 lines · Peter Antoine
Quoted 27 lines · Daniel Vetter
Quoted 12 lines · Chris Wilson
Quoted 10 lines · David Weinehall
Quoted 5 lines · Chris Wilson
Quoted 2 lines · Peter

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

You've identified an issue, but I think your patch is incomplete.

I've tried my best to go through the remaining similar-looking code, but the rest seems fine (I might've missed something though).

Quoted 8 lines
Quoted 4 lines

The only thing I reacted on was that in intel_runtime_resume() the call to intel_init_pch_refclk() is conditional on IS_GEN6(), but none of the other invocations of intel_init_pch_refclk() are. The commit message doesn't seem to provide a sufficient explanation for why this is so.

The explanation for moving init_hw() was that it setups clock_gating. If any in that path are required for enabling the rings, those should be move to init_render_ring() or the init_context.

Yeah we've had this fun before. init_clock_gating is _not_ for GT workarounds, only for modeset workarounds. We might need to rename that hook to avoid getting bitten for eternity, but moving init_hw because of clock_gating to get the rings up and running is bogus.

Quoted 4 lines

As Chris said we need to identify which bits need to get moved to the ring init or w/a bb code and do that (and in a separate patch from enabling the interrupts early enough like this one here does).
-Daniel

Ok. More work is required.

But, we have two issues here. The open and accessible (to any user) ioctl that causes the driver (and in testing the kernel) to misbehave and cause the system to become unresponsive need or to reboot.This is covered by a simple de-reference check that protects the driver and the system.

Secondly, a nice to have, which is the hw-lock/context code that seems to have more issues with it and should be turned off when it is not required. This code has subtle connections that will need more work. The un-niceness of this code has been known about for a while and it would be good to turn off.

Quoted 7 lines

The first I would suggest is kind of important as is simply exploitable security hole, the second is just probably full of security holes.

Can we split this into two jobs, fix the actionable security hole, and get back to the nice to have later.

Peter.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#15 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

In reply to Daniel Vetter

I agree Daniel. We need two patches here, 1) moving the enabling of the interrupts early on. 2) split the WA initialization into GT & Display and move GT workarounds before ring init.

Thanks
Deepak

-----Original Message-----
From: Daniel Vetter [mailto:[email protected]] On Behalf Of Daniel Vetter
Sent: Monday, May 4, 2015 8:25 PM
To: Chris Wilson; Antoine, Peter; Deepak S; S, Deepak; [email protected]; Tian, YeX
Subject: Re: [Intel-gfx] [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

Quoted 17 lines · Chris Wilson
Quoted 7 lines · David Weinehall
Quoted 5 lines · Chris Wilson
Quoted 2 lines · Peter

So is the plan to push these patches and have follow-on work to cover the other paths? As this fixes the Bugzilla issue that has been raised.

You've identified an issue, but I think your patch is incomplete.

Quoted 7 lines

I've tried my best to go through the remaining similar-looking code, but the rest seems fine (I might've missed something though).

The only thing I reacted on was that in intel_runtime_resume() the call to intel_init_pch_refclk() is conditional on IS_GEN6(), but
none of the other invocations of intel_init_pch_refclk() are. The commit message doesn't seem to provide a sufficient explanation for why this is so.

Quoted 4 lines

The explanation for moving init_hw() was that it setups clock_gating. If any in that path are required for enabling the rings, those should be move to
init_render_ring() or the init_context.

Yeah we've had this fun before. init_clock_gating is _not_ for GT workarounds, only for modeset workarounds. We might need to rename that hook to avoid getting bitten for eternity, but moving init_hw because of clock_gating to get the rings up and running is bogus.

As Chris said we need to identify which bits need to get moved to the ring init or w/a bb code and do that (and in a separate patch from enabling the interrupts early enough like this one here does). -Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Tested-by on v2 1/1
#16 lore raw

Re: [PATCH v2] drm/i915: Avoid GPU hang when coming out of S3 or S4

In reply to Peter Antoine

Task id: 6271 -------------------------------------Summary-------------------------------------
Platform Delta drm-intel-nightly Series Applied
PNV 276/276 276/276
ILK -1 302/302 301/302

SNB                                  318/318              318/318
IVB                                  341/341              341/341
BYT                                  287/287              287/287
BDW                                  318/318              318/318

-------------------------------------Detailed-------------------------------------
Platform Test drm-intel-nightly Series Applied
*ILK igt@gem_fenced_exec_thrash@no-spare-fences-busy PASS(4) DMESG_WARN(1)PASS(1)
(dmesg patch applied)drm:i915_hangcheck_elapsed[i915]]*ERROR*Hangcheck_timer_elapsed...bsd_ring_idle@Hangcheck timer elapsed... bsd ring idle
Note: You need to pay more attention to line start with '*'
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Tested-by on v1 1/1
#17 lore raw
In reply to Peter Antoine

Task id: 6271 -------------------------------------Summary-------------------------------------
Platform Delta drm-intel-nightly Series Applied
PNV 276/276 276/276
ILK -1 302/302 301/302

SNB                                  318/318              318/318
IVB                                  341/341              341/341
BYT                                  287/287              287/287
BDW                                  318/318              318/318

-------------------------------------Detailed-------------------------------------
Platform Test drm-intel-nightly Series Applied
*ILK igt@gem_fenced_exec_thrash@no-spare-fences-busy PASS(4) DMESG_WARN(1)PASS(1)
(dmesg patch applied)drm:i915_hangcheck_elapsed[i915]]*ERROR*Hangcheck_timer_elapsed...bsd_ring_idle@Hangcheck timer elapsed... bsd ring idle
Note: You need to pay more attention to line start with '*'
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx