Skip to contentSkip to conversation

drm/i915: Avoid the gpu reset vs. modeset deadlock

PATCH In review Daniel Vetter 4 participants 4 messages Jul 20, 2017 – Aug 3, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915: Avoid the gpu reset vs. modeset deadlock

... using the biggest hammer we have. This is essentially a weaponized version of the timeout-based wedging Chris added in

commit 36703e79a982c8ce5a8e43833291f2719e92d0d1

Author: Chris Wilson <[email protected]>
Date:   Thu Jun 22 11:56:25 2017 +0100

drm/i915: Break modeset deadlocks on reset

Because defense-in-depth is good it's good to still have both. Also note that with the locking change we can now restrict this a lot (old gpus and special testing only), so this doesn't kill the TDR benefits on at least anything remotely modern.

And futuremore with a few tricks it should be possible to make a much more educated guess about whether an atomic commit is stuck waiting on the gpu (atomic_t counting the pending i915_sw_fence used by the atomic modeset code should do it), so we can improve this.

But for now just start with something that is guaranteed to recover faster, for much better CI througput.

This defacto reverts TDR on these platforms, but there's not really a single commit to specify as the sole offender.

v2: Add a debug message to explain what's going on. We can't DRM_ERROR because that spams CI. And the timeout based fallback still prints a DRM_ERROR, in case something goes wrong.

Fixes 4680816be336 ("drm/i915: Wait first for submission, before waiting for request completion")
Fixes 221fe7994554 ("drm/i915: Perform a direct reset of the GPU from the waiter")
Cc Chris Wilson <[email protected]>
Cc Mika Kuoppala <[email protected]>
Cc Joonas Lahtinen <[email protected]>
---
 drivers/gpu/drm/i915/intel_display.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/gpu/drm/i915/intel_display.c +6 -0
index 02b1f4966049..995522e40ec1 100644
@@ -3471,6 +3471,12 @@ void intel_prepare_reset(struct drm_i915_private *dev_priv)
 	    !gpu_reset_clobbers_display(dev_priv))
 		return;
 
+	/* We have a modeset vs reset deadlock, defensively unbreak it.
+	 *
+	 * FIXME: We can do a _lot_ better, this is just a first iteration.*/
+	i915_gem_set_wedged(dev_priv);
+	DRM_DEBUG_DRIVER("Wedging GPU to avoid deadlocks with pending modeset updates\n");
+
 	/*
 	 * Need mode_config.mutex so that we don't
 	 * trample ongoing ->detect() and whatnot.
Signature · 6 lines
2.13.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

✗ Fi.CI.BAT: failure for drm/i915: Avoid the gpu reset vs. modeset deadlock

== Series Details ==

== Series Details ==

Series: drm/i915: Avoid the gpu reset vs. modeset deadlock
URL : https://patchwork.freedesktop.org/series/27670/
State : failure

== Summary ==

Series 27670v1 drm/i915: Avoid the gpu reset vs. modeset deadlock
https://patchwork.freedesktop.org/api/1.0/series/27670/revisions/1/mbox/

Test gem_exec_fence:

        Subgroup await-hang-default:
                pass       -> FAIL       (fi-blb-e6850)
                pass       -> FAIL       (fi-pnv-d510)
Test gem_ringfill:
        Subgroup basic-default-hang:
                dmesg-warn -> PASS       (fi-blb-e6850) fdo#101600 +1
Test kms_cursor_legacy:
        Subgroup basic-busy-flip-before-cursor-atomic:
                fail       -> PASS       (fi-snb-2600) fdo#100215
Test kms_pipe_crc_basic:
        Subgroup hang-read-crc-pipe-b:
                dmesg-warn -> PASS       (fi-pnv-d510) fdo#101597
        Subgroup suspend-read-crc-pipe-b:
                pass       -> DMESG-WARN (fi-byt-j1900) fdo#101705 +1

fdo#101600 https://bugs.freedesktop.org/show_bug.cgi?id=101600
fdo#100215 https://bugs.freedesktop.org/show_bug.cgi?id=100215
fdo#101597 https://bugs.freedesktop.org/show_bug.cgi?id=101597
fdo#101705 https://bugs.freedesktop.org/show_bug.cgi?id=101705

fi-bdw-5557u     total:279  pass:268  dwarn:0   dfail:0   fail:0   skip:11  time:448s
fi-bdw-gvtdvm    total:279  pass:265  dwarn:0   dfail:0   fail:0   skip:14  time:431s
fi-blb-e6850     total:279  pass:224  dwarn:0   dfail:0   fail:1   skip:54  time:352s
fi-bsw-n3050     total:279  pass:243  dwarn:0   dfail:0   fail:0   skip:36  time:532s
fi-bxt-j4205     total:279  pass:260  dwarn:0   dfail:0   fail:0   skip:19  time:511s
fi-byt-j1900     total:279  pass:254  dwarn:1   dfail:0   fail:0   skip:24  time:497s
fi-byt-n2820     total:279  pass:251  dwarn:0   dfail:0   fail:0   skip:28  time:485s
fi-glk-2a        total:279  pass:260  dwarn:0   dfail:0   fail:0   skip:19  time:605s
fi-hsw-4770      total:279  pass:263  dwarn:0   dfail:0   fail:0   skip:16  time:438s
fi-hsw-4770r     total:279  pass:263  dwarn:0   dfail:0   fail:0   skip:16  time:413s
fi-ilk-650       total:279  pass:229  dwarn:0   dfail:0   fail:0   skip:50  time:421s
fi-ivb-3520m     total:279  pass:261  dwarn:0   dfail:0   fail:0   skip:18  time:503s
fi-ivb-3770      total:279  pass:261  dwarn:0   dfail:0   fail:0   skip:18  time:480s
fi-kbl-7500u     total:279  pass:261  dwarn:0   dfail:0   fail:0   skip:18  time:467s
fi-kbl-7560u     total:279  pass:269  dwarn:0   dfail:0   fail:0   skip:10  time:579s
fi-kbl-r         total:279  pass:261  dwarn:0   dfail:0   fail:0   skip:18  time:584s
fi-pnv-d510      total:279  pass:223  dwarn:0   dfail:0   fail:1   skip:55  time:562s
fi-skl-6260u     total:279  pass:269  dwarn:0   dfail:0   fail:0   skip:10  time:459s
fi-skl-6700hq    total:279  pass:262  dwarn:0   dfail:0   fail:0   skip:17  time:592s
fi-skl-6700k     total:279  pass:261  dwarn:0   dfail:0   fail:0   skip:18  time:471s
fi-skl-6770hq    total:279  pass:269  dwarn:0   dfail:0   fail:0   skip:10  time:477s
fi-skl-gvtdvm    total:279  pass:266  dwarn:0   dfail:0   fail:0   skip:13  time:437s
fi-skl-x1585l    total:279  pass:268  dwarn:0   dfail:0   fail:0   skip:11  time:471s
fi-snb-2520m     total:279  pass:251  dwarn:0   dfail:0   fail:0   skip:28  time:542s
fi-snb-2600      total:279  pass:250  dwarn:0   dfail:0   fail:0   skip:29  time:403s

07fe547642181ced02ac75bbf748cbb7ddd367b9 drm-tip: 2017y-07m-20d-18h-40m-36s UTC integration manifest 0b8dc77 drm/i915: Avoid the gpu reset vs. modeset deadlock

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_5257/
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Reviewed-by on v1 1/1
#2 lore raw
In reply to Daniel Vetter
Quoted 47 lines · Daniel Vetter

... using the biggest hammer we have. This is essentially a weaponized version of the timeout-based wedging Chris added in

commit 36703e79a982c8ce5a8e43833291f2719e92d0d1

Author: Chris Wilson <[email protected]>
Date:   Thu Jun 22 11:56:25 2017 +0100

drm/i915: Break modeset deadlocks on reset

Because defense-in-depth is good it's good to still have both. Also note that with the locking change we can now restrict this a lot (old gpus and special testing only), so this doesn't kill the TDR benefits on at least anything remotely modern.

And futuremore with a few tricks it should be possible to make a much more educated guess about whether an atomic commit is stuck waiting on the gpu (atomic_t counting the pending i915_sw_fence used by the atomic modeset code should do it), so we can improve this.

But for now just start with something that is guaranteed to recover faster, for much better CI througput.

This defacto reverts TDR on these platforms, but there's not really a single commit to specify as the sole offender.

v2: Add a debug message to explain what's going on. We can't DRM_ERROR because that spams CI. And the timeout based fallback still prints a DRM_ERROR, in case something goes wrong.

Fixes 4680816be336 ("drm/i915: Wait first for submission, before waiting for request completion")
Fixes 221fe7994554 ("drm/i915: Perform a direct reset of the GPU from the waiter")
Cc Chris Wilson <[email protected]>
Cc Mika Kuoppala <[email protected]>
Cc Joonas Lahtinen <[email protected]>
---
  drivers/gpu/drm/i915/intel_display.c | 6 ++++++
  1 file changed, 6 insertions(+)
drivers/gpu/drm/i915/intel_display.c +0 -0
index 02b1f4966049..995522e40ec1 100644
@@ -3471,6 +3471,12 @@ void intel_prepare_reset(struct drm_i915_private *dev_priv)
  	    !gpu_reset_clobbers_display(dev_priv))
  		return;

drivers/gpu/drm/i915/intel_display.c void intel_prepare_reset(struct drm_i915_private *dev_priv) · line 3482

Quoted 10 lines
drivers/gpu/drm/i915/intel_display.c +6 -0
 
+	/* We have a modeset vs reset deadlock, defensively unbreak it.
+	 *
+	 * FIXME: We can do a _lot_ better, this is just a first iteration.*/
+	i915_gem_set_wedged(dev_priv);
+	DRM_DEBUG_DRIVER("Wedging GPU to avoid deadlocks with pending modeset updates\n");
+
  	/*
  	 * Need mode_config.mutex so that we don't
  	 * trample ongoing ->detect() and whatnot.

Looks fine to me since it only affects very old platforms, and is a temporary measure.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Reviewed-by
#3 lore raw
Quoted 49 lines · Tvrtko Ursulin
Quoted 47 lines · Daniel Vetter

... using the biggest hammer we have. This is essentially a weaponized version of the timeout-based wedging Chris added in

commit 36703e79a982c8ce5a8e43833291f2719e92d0d1

Author: Chris Wilson <[email protected]>
Date:   Thu Jun 22 11:56:25 2017 +0100

drm/i915: Break modeset deadlocks on reset

Because defense-in-depth is good it's good to still have both. Also note that with the locking change we can now restrict this a lot (old gpus and special testing only), so this doesn't kill the TDR benefits on at least anything remotely modern.

And futuremore with a few tricks it should be possible to make a much more educated guess about whether an atomic commit is stuck waiting on the gpu (atomic_t counting the pending i915_sw_fence used by the atomic modeset code should do it), so we can improve this.

But for now just start with something that is guaranteed to recover faster, for much better CI througput.

This defacto reverts TDR on these platforms, but there's not really a single commit to specify as the sole offender.

v2: Add a debug message to explain what's going on. We can't DRM_ERROR because that spams CI. And the timeout based fallback still prints a DRM_ERROR, in case something goes wrong.

Fixes 4680816be336 ("drm/i915: Wait first for submission, before waiting for request completion")
Fixes 221fe7994554 ("drm/i915: Perform a direct reset of the GPU from the waiter")
Cc Chris Wilson <[email protected]>
Cc Mika Kuoppala <[email protected]>
Cc Joonas Lahtinen <[email protected]>
---
   drivers/gpu/drm/i915/intel_display.c | 6 ++++++
   1 file changed, 6 insertions(+)
drivers/gpu/drm/i915/intel_display.c +0 -0
index 02b1f4966049..995522e40ec1 100644
@@ -3471,6 +3471,12 @@ void intel_prepare_reset(struct drm_i915_private *dev_priv)
           !gpu_reset_clobbers_display(dev_priv))
               return;
 
Quoted 3 lines

drivers/gpu/drm/i915/intel_display.c void intel_prepare_reset(struct drm_i915_private *dev_priv) · line 3476

Quoted 3 lines
drivers/gpu/drm/i915/intel_display.c +3 -0
+     /* We have a modeset vs reset deadlock, defensively unbreak it.
+      *
+      * FIXME: We can do a _lot_ better, this is just a first iteration.*/

I think the standard is to move '*/' to a new line.

Quoted 8 lines

drivers/gpu/drm/i915/intel_display.c void intel_prepare_reset(struct drm_i915_private *dev_priv) · line 3482

Quoted 7 lines
drivers/gpu/drm/i915/intel_display.c +3 -0
+     i915_gem_set_wedged(dev_priv);
+     DRM_DEBUG_DRIVER("Wedging GPU to avoid deadlocks with pending modeset updates\n");
+
       /*
        * Need mode_config.mutex so that we don't
        * trample ongoing ->detect() and whatnot.
Quoted 5 lines

Looks fine to me since it only affects very old platforms, and is a temporary measure.

Also,

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx