Skip to contentSkip to conversation

drm/i915/skl: Add support for the SAGV, fix underrun hangs

PATCH Superseded Lyude 4 participants 5 messages Aug 8, 2016 – Aug 9, 2016

Subsystems (3)

v10 of 12 Superseded Latest is v12 →

lore
Share & tools
lore

Lyude

author
#0 lore raw

PATCH v10 1/1 drm/i915/skl: Add support for the SAGV, fix underrun hangs

Since the watermark calculations for Skylake are still broken, we're apt to hitting underruns very easily under multi-monitor configurations. While it would be lovely if this was fixed, it's not. Another problem that's been coming from this however, is the mysterious issue of underruns causing full system hangs. An easy way to reproduce this with a skylake system:

- Get a laptop with a skylake GPU, and hook up two external monitors to it
- Move the cursor from the built-in LCD to one of the external displays as quickly as you can
- You'll get a few pipe underruns, and eventually the entire system will just freeze.

After doing a lot of investigation and reading through the bspec, I found the existence of the SAGV, which is responsible for adjusting the system agent voltage and clock frequencies depending on how much power we need. According to the bspec:

"The display engine access to system memory is blocked during the
adjustment time. SAGV defaults to enabled. Software must use the GT-driver pcode mailbox to disable SAGV when the display engine is not able to tolerate the blocking time."

The rest of the bspec goes on to explain that software can simply leave the SAGV enabled, and disable it when we use interlaced pipes/have more then one pipe active.

Sure enough, with this patchset the system hangs resulting from pipe underruns on Skylake have completely vanished on my T460s. Additionally,
the bspec mentions turning off the SAGV with more then one pipe enabled
as a workaround for display underruns. While this patch doesn't entirely fix that, it looks like it does improve the situation a little bit so it's likely this is going to be required to make watermarks on Skylake fully functional.

Changes since v9:
- Only enable/disable sagv on Skylake
Changes since v8:
- Add intel_state->modeset guard to the conditional for skl_enable_sagv()
Changes since v7:
- Remove GEN9_SAGV_LOW_FREQ, replace with GEN9_SAGV_IS_ENABLED (that's all we use it for anyway)
- Use GEN9_SAGV_IS_ENABLED instead of 0x1 for clarification
- Fix a styling error that snuck past me
Changes since v6:
- Protect skl_enable_sagv() with intel_state->modeset conditional in intel_atomic_commit_tail()
Changes since v5:
- Don't use is_power_of_2. Makes things confusing
- Don't use the old state to figure out whether or not to enable/disable the sagv, use the new one
- Split the loop in skl_disable_sagv into it's own function
- Move skl_sagv_enable/disable() calls into intel_atomic_commit_tail()
Changes since v4:
- Use is_power_of_2 against active_crtcs to check whether we have > 1 pipe enabled
- Fix skl_sagv_get_hw_state(): (temp & 0x1) indicates disabled, 0x0 enabled
- Call skl_sagv_enable/disable() from pre/post-plane updates
Changes since v3:
- Use time_before() to compare timeout to jiffies
Changes since v2:
- Really apply minor style nitpicks to patch this time
Changes since v1:
- Added comments about this probably being one of the requirements to fixing Skylake's watermark issues
- Minor style nitpicks from Matt Roper
- Disable these functions on Broxton, since it doesn't have an SAGV

squash! drm/i915/skl: Add support for the SAGV, fix underrun hangs

squash! drm/i915/skl: Add support for the SAGV, fix underrun hangs

---
 drivers/gpu/drm/i915/i915_drv.h      |   2 +
 drivers/gpu/drm/i915/i915_reg.h      |   4 ++
 drivers/gpu/drm/i915/intel_display.c |  12 ++++
 drivers/gpu/drm/i915/intel_drv.h     |   2 +
 drivers/gpu/drm/i915/intel_pm.c      | 112 +++++++++++++++++++++++++++++++++++
 5 files changed, 132 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index feec00f..eb449f6 100644
@@ -1948,6 +1948,8 @@ struct drm_i915_private {
 	struct i915_suspend_saved_registers regfile;
 	struct vlv_s0ix_state vlv_s0ix_state;
 
+	bool skl_sagv_enabled;
+
 	struct {
 		/*
 		 * Raw watermark latency values:
drivers/gpu/drm/i915/i915_reg.h +4 -0
index f38a5e2..f7e0bc2 100644
@@ -7170,6 +7170,10 @@ enum {
 #define   HSW_PCODE_DE_WRITE_FREQ_REQ		0x17
 #define   DISPLAY_IPS_CONTROL			0x19
 #define	  HSW_PCODE_DYNAMIC_DUTY_CYCLE_CONTROL	0x1A
+#define   GEN9_PCODE_SAGV_CONTROL		0x21
+#define     GEN9_SAGV_DISABLE			0x0
+#define     GEN9_SAGV_IS_DISABLED		0x1
+#define     GEN9_SAGV_DYNAMIC_FREQ              0x3
 #define GEN6_PCODE_DATA				_MMIO(0x138128)
 #define   GEN6_PCODE_FREQ_IA_RATIO_SHIFT	8
 #define   GEN6_PCODE_FREQ_RING_RATIO_SHIFT	16
drivers/gpu/drm/i915/intel_display.c +12 -0
index 9cbf543..8bff8a5 100644
@@ -13698,6 +13698,14 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state)
 		     intel_state->cdclk_pll_vco != dev_priv->cdclk_pll.vco))
 			dev_priv->display.modeset_commit_cdclk(state);
 
+		/*
+		 * SKL workaround: bspec recommends we disable the SAGV when we
+		 * have more then one pipe enabled
+		 */
+		if (IS_SKYLAKE(dev_priv) &&
+		    hweight32(intel_state->active_crtcs) > 1)
+			skl_disable_sagv(dev_priv);
+
 		intel_modeset_verify_disabled(dev);
 	}
 
@@ -13771,6 +13779,10 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state)
 		intel_modeset_verify_crtc(crtc, old_crtc_state, crtc->state);
 	}
 
+	if (IS_SKYLAKE(dev_priv) && intel_state->modeset &&
+	    hweight32(intel_state->active_crtcs) <= 1)
+		skl_enable_sagv(dev_priv);
+
 	drm_atomic_helper_commit_hw_done(state);
 
 	if (intel_state->modeset)
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 1ad2e2c5..221cbe0 100644
@@ -1700,6 +1700,8 @@ void ilk_wm_get_hw_state(struct drm_device *dev);
 void skl_wm_get_hw_state(struct drm_device *dev);
 void skl_ddb_get_hw_state(struct drm_i915_private *dev_priv,
 			  struct skl_ddb_allocation *ddb /* out */);
+int skl_enable_sagv(struct drm_i915_private *dev_priv);
+int skl_disable_sagv(struct drm_i915_private *dev_priv);
 uint32_t ilk_pipe_pixel_rate(const struct intel_crtc_state *pipe_config);
 bool ilk_disable_lp_wm(struct drm_device *dev);
 int sanitize_rc6_option(struct drm_i915_private *dev_priv, int enable_rc6);
drivers/gpu/drm/i915/intel_pm.c +112 -0

124 lines not on this page · raw message

Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author
#1 lore raw

PATCH v10 1/1 drm/i915/skl: Add support for the SAGV, fix underrun hangs

Since the watermark calculations for Skylake are still broken, we're apt to hitting underruns very easily under multi-monitor configurations. While it would be lovely if this was fixed, it's not. Another problem that's been coming from this however, is the mysterious issue of underruns causing full system hangs. An easy way to reproduce this with a skylake system:

- Get a laptop with a skylake GPU, and hook up two external monitors to it
- Move the cursor from the built-in LCD to one of the external displays as quickly as you can
- You'll get a few pipe underruns, and eventually the entire system will just freeze.

After doing a lot of investigation and reading through the bspec, I found the existence of the SAGV, which is responsible for adjusting the system agent voltage and clock frequencies depending on how much power we need. According to the bspec:

"The display engine access to system memory is blocked during the
adjustment time. SAGV defaults to enabled. Software must use the GT-driver pcode mailbox to disable SAGV when the display engine is not able to tolerate the blocking time."

The rest of the bspec goes on to explain that software can simply leave the SAGV enabled, and disable it when we use interlaced pipes/have more then one pipe active.

Sure enough, with this patchset the system hangs resulting from pipe underruns on Skylake have completely vanished on my T460s. Additionally,
the bspec mentions turning off the SAGV with more then one pipe enabled
as a workaround for display underruns. While this patch doesn't entirely fix that, it looks like it does improve the situation a little bit so it's likely this is going to be required to make watermarks on Skylake fully functional.

Changes since v9:
- Only enable/disable sagv on Skylake
Changes since v8:
- Add intel_state->modeset guard to the conditional for skl_enable_sagv()
Changes since v7:
- Remove GEN9_SAGV_LOW_FREQ, replace with GEN9_SAGV_IS_ENABLED (that's all we use it for anyway)
- Use GEN9_SAGV_IS_ENABLED instead of 0x1 for clarification
- Fix a styling error that snuck past me
Changes since v6:
- Protect skl_enable_sagv() with intel_state->modeset conditional in intel_atomic_commit_tail()
Changes since v5:
- Don't use is_power_of_2. Makes things confusing
- Don't use the old state to figure out whether or not to enable/disable the sagv, use the new one
- Split the loop in skl_disable_sagv into it's own function
- Move skl_sagv_enable/disable() calls into intel_atomic_commit_tail()
Changes since v4:
- Use is_power_of_2 against active_crtcs to check whether we have > 1 pipe enabled
- Fix skl_sagv_get_hw_state(): (temp & 0x1) indicates disabled, 0x0 enabled
- Call skl_sagv_enable/disable() from pre/post-plane updates
Changes since v3:
- Use time_before() to compare timeout to jiffies
Changes since v2:
- Really apply minor style nitpicks to patch this time
Changes since v1:
- Added comments about this probably being one of the requirements to fixing Skylake's watermark issues
- Minor style nitpicks from Matt Roper
- Disable these functions on Broxton, since it doesn't have an SAGV

squash! drm/i915/skl: Add support for the SAGV, fix underrun hangs

squash! drm/i915/skl: Add support for the SAGV, fix underrun hangs

---
 drivers/gpu/drm/i915/i915_drv.h      |   2 +
 drivers/gpu/drm/i915/i915_reg.h      |   4 ++
 drivers/gpu/drm/i915/intel_display.c |  12 ++++
 drivers/gpu/drm/i915/intel_drv.h     |   2 +
 drivers/gpu/drm/i915/intel_pm.c      | 112 +++++++++++++++++++++++++++++++++++
 5 files changed, 132 insertions(+)
drivers/gpu/drm/i915/i915_drv.h +2 -0
index feec00f..eb449f6 100644
@@ -1948,6 +1948,8 @@ struct drm_i915_private {
 	struct i915_suspend_saved_registers regfile;
 	struct vlv_s0ix_state vlv_s0ix_state;
 
+	bool skl_sagv_enabled;
+
 	struct {
 		/*
 		 * Raw watermark latency values:
drivers/gpu/drm/i915/i915_reg.h +4 -0
index f38a5e2..f7e0bc2 100644
@@ -7170,6 +7170,10 @@ enum {
 #define   HSW_PCODE_DE_WRITE_FREQ_REQ		0x17
 #define   DISPLAY_IPS_CONTROL			0x19
 #define	  HSW_PCODE_DYNAMIC_DUTY_CYCLE_CONTROL	0x1A
+#define   GEN9_PCODE_SAGV_CONTROL		0x21
+#define     GEN9_SAGV_DISABLE			0x0
+#define     GEN9_SAGV_IS_DISABLED		0x1
+#define     GEN9_SAGV_DYNAMIC_FREQ              0x3
 #define GEN6_PCODE_DATA				_MMIO(0x138128)
 #define   GEN6_PCODE_FREQ_IA_RATIO_SHIFT	8
 #define   GEN6_PCODE_FREQ_RING_RATIO_SHIFT	16
drivers/gpu/drm/i915/intel_display.c +12 -0
index 9cbf543..8bff8a5 100644
@@ -13698,6 +13698,14 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state)
 		     intel_state->cdclk_pll_vco != dev_priv->cdclk_pll.vco))
 			dev_priv->display.modeset_commit_cdclk(state);
 
+		/*
+		 * SKL workaround: bspec recommends we disable the SAGV when we
+		 * have more then one pipe enabled
+		 */
+		if (IS_SKYLAKE(dev_priv) &&
+		    hweight32(intel_state->active_crtcs) > 1)
+			skl_disable_sagv(dev_priv);
+
 		intel_modeset_verify_disabled(dev);
 	}
 
@@ -13771,6 +13779,10 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state)
 		intel_modeset_verify_crtc(crtc, old_crtc_state, crtc->state);
 	}
 
+	if (IS_SKYLAKE(dev_priv) && intel_state->modeset &&
+	    hweight32(intel_state->active_crtcs) <= 1)
+		skl_enable_sagv(dev_priv);
+
 	drm_atomic_helper_commit_hw_done(state);
 
 	if (intel_state->modeset)
drivers/gpu/drm/i915/intel_drv.h +2 -0
index 1ad2e2c5..221cbe0 100644
@@ -1700,6 +1700,8 @@ void ilk_wm_get_hw_state(struct drm_device *dev);
 void skl_wm_get_hw_state(struct drm_device *dev);
 void skl_ddb_get_hw_state(struct drm_i915_private *dev_priv,
 			  struct skl_ddb_allocation *ddb /* out */);
+int skl_enable_sagv(struct drm_i915_private *dev_priv);
+int skl_disable_sagv(struct drm_i915_private *dev_priv);
 uint32_t ilk_pipe_pixel_rate(const struct intel_crtc_state *pipe_config);
 bool ilk_disable_lp_wm(struct drm_device *dev);
 int sanitize_rc6_option(struct drm_i915_private *dev_priv, int enable_rc6);
drivers/gpu/drm/i915/intel_pm.c +112 -0

124 lines not on this page · raw message

Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#2 lore raw

Hey,

Op 08-08-16 om 23:03 schreef Lyude:

Quoted 28 lines

Since the watermark calculations for Skylake are still broken, we're apt to hitting underruns very easily under multi-monitor configurations. While it would be lovely if this was fixed, it's not. Another problem that's been coming from this however, is the mysterious issue of underruns causing full system hangs. An easy way to reproduce this with a skylake system:

- Get a laptop with a skylake GPU, and hook up two external monitors to it
- Move the cursor from the built-in LCD to one of the external displays as quickly as you can
- You'll get a few pipe underruns, and eventually the entire system will just freeze.

After doing a lot of investigation and reading through the bspec, I found the existence of the SAGV, which is responsible for adjusting the system agent voltage and clock frequencies depending on how much power we need. According to the bspec:

"The display engine access to system memory is blocked during the
adjustment time. SAGV defaults to enabled. Software must use the GT-driver pcode mailbox to disable SAGV when the display engine is not able to tolerate the blocking time."

The rest of the bspec goes on to explain that software can simply leave the SAGV enabled, and disable it when we use interlaced pipes/have more then one pipe active.

Quoted 7 lines

Sure enough, with this patchset the system hangs resulting from pipe underruns on Skylake have completely vanished on my T460s. Additionally,
the bspec mentions turning off the SAGV with more then one pipe enabled
as a workaround for display underruns. While this patch doesn't entirely fix that, it looks like it does improve the situation a little bit so it's likely this is going to be required to make watermarks on Skylake fully functional.

I think this patch goes with v9 6/6 and v8 2-5/6. If you're only updating a single patch it might be better to send it in reply to the original patch.

I'm testing the whole series on my prerelease skylake, and running into this:

[ 2794.933149] kms_cursor_legacy: starting subtest 2x-flip-vs-cursor-legacy
[ 2795.813970] [drm:skl_disable_sagv [i915]] *ERROR* Request to disable SAGV timed out

Value returned from skl_do_sagv_disable is always 0 for me, even when I bump the timeout to 15.

~Maarten

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw

Em Ter, 2016-08-09 às 14:44 +0200, Maarten Lankhorst escreveu:

Quoted 65 lines

Hey,

Op 08-08-16 om 23:03 schreef Lyude:

Quoted 41 lines

Since the watermark calculations for Skylake are still broken, we're apt
to hitting underruns very easily under multi-monitor configurations.
While it would be lovely if this was fixed, it's not. Another problem
that's been coming from this however, is the mysterious issue of underruns causing full system hangs. An easy way to reproduce this with
a skylake system:

- Get a laptop with a skylake GPU, and hook up two external monitors to
  it
- Move the cursor from the built-in LCD to one of the external displays
  as quickly as you can
- You'll get a few pipe underruns, and eventually the entire system will
  just freeze.

After doing a lot of investigation and reading through the bspec, I found the existence of the SAGV, which is responsible for adjusting the
system agent voltage and clock frequencies depending on how much power
we need. According to the bspec:

"The display engine access to system memory is blocked during the adjustment time. SAGV defaults to enabled. Software must use the GT-driver pcode mailbox to disable SAGV when the display engine is not
 able to tolerate the blocking time."

The rest of the bspec goes on to explain that software can simply leave
the SAGV enabled, and disable it when we use interlaced pipes/have more
then one pipe active.

Quoted 13 lines

Sure enough, with this patchset the system hangs resulting from pipe
underruns on Skylake have completely vanished on my T460s. Additionally,
the bspec mentions turning off the SAGV with more then one
pipe enabled
as a workaround for display underruns. While this patch doesn't entirely
fix that, it looks like it does improve the situation a little bit so
it's likely this is going to be required to make watermarks on Skylake
fully functional.

I think this patch goes with v9 6/6 and v8 2-5/6. If you're only updating a single patch it might be better to send it in reply to the original patch.

I'm testing the whole series on my prerelease skylake, and running into this:

Quoted 7 lines
[ 2794.933149] kms_cursor_legacy: starting subtest 2x-flip-vs-cursor-
legacy
[ 2795.813970] [drm:skl_disable_sagv [i915]] *ERROR* Request to

disable SAGV timed out

Value returned from skl_do_sagv_disable is always 0 for me, even when I bump the timeout to 15.

Yesterday I started testing this series, and I also noticed some visual
corruption: while browsing moderately-heavy websites on a maximized Firefox, I could see the desktop background sort of "blinking" in the screen (the background was not supposed to be visible). It looks like the problem was introduced by patch 4, but I can't be 100% sure since sometimes it's a little harder to reproduce it. Still, this is better than the current "X doesn't work" state that we have without the series.

Quoted 7 lines

~Maarten

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw

✓ Ro.CI.BAT: success for drm/i915/skl: Add support for the SAGV, fix underrun hangs (rev6)

In reply to Lyude
== Series Details ==

== Series Details ==

Series: drm/i915/skl: Add support for the SAGV, fix underrun hangs (rev6)
URL : https://patchwork.freedesktop.org/series/9736/
State : success

== Summary ==

Series 9736v6 drm/i915/skl: Add support for the SAGV, fix underrun hangs
http://patchwork.freedesktop.org/api/1.0/series/9736/revisions/6/mbox

Test kms_cursor_legacy:

        Subgroup basic-cursor-vs-flip-varying-size:
                fail       -> PASS       (ro-ilk1-i5-650)
        Subgroup basic-flip-vs-cursor-legacy:
                fail       -> PASS       (ro-byt-n2820)
fi-hsw-i7-4770k  total:107  pass:91   dwarn:0   dfail:0   fail:0   skip:15 
fi-kbl-qkkr      total:107  pass:82   dwarn:1   dfail:0   fail:0   skip:23 
fi-skl-i5-6260u  total:107  pass:98   dwarn:0   dfail:0   fail:0   skip:8  
fi-skl-i7-6700k  total:107  pass:84   dwarn:0   dfail:0   fail:0   skip:22 
fi-snb-i7-2600   total:107  pass:77   dwarn:0   dfail:0   fail:0   skip:29 
ro-bdw-i5-5250u  total:107  pass:97   dwarn:0   dfail:0   fail:0   skip:9  
ro-bdw-i7-5557U  total:107  pass:97   dwarn:0   dfail:0   fail:0   skip:9  
ro-bdw-i7-5600u  total:107  pass:79   dwarn:0   dfail:0   fail:0   skip:27 
ro-bsw-n3050     total:240  pass:194  dwarn:0   dfail:0   fail:4   skip:42 
ro-byt-n2820     total:240  pass:198  dwarn:0   dfail:0   fail:2   skip:40 
ro-hsw-i3-4010u  total:107  pass:87   dwarn:0   dfail:0   fail:0   skip:19 
ro-ilk1-i5-650   total:235  pass:174  dwarn:0   dfail:0   fail:1   skip:60 
ro-ivb-i7-3770   total:107  pass:80   dwarn:0   dfail:0   fail:0   skip:26 
ro-skl3-i5-6260u total:107  pass:98   dwarn:0   dfail:0   fail:0   skip:8  

Results at /archive/results/CI_IGT_test/RO_Patchwork_1769/

5df4316 drm-intel-nightly: 2016y-08m-08d-18h-35m-22s UTC integration manifest 4625cc9 drm/i915/skl: Add support for the SAGV, fix underrun hangs

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx