Skip to contentSkip to conversation

drm/i915: Handle hanging during nonblocking modeset correctly.

PATCH In review Maarten Lankhorst 4 participants 15 messages Jan 26, 2017 – Jan 31, 2017

Subsystems (3)
lore
Share & tools
lore

Maarten Lankhorst

author maintainer
#0 lore raw

cover drm/i915: Handle hanging during nonblocking modeset correctly.

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

A nonblocking modeset with a hang on the old fb, followed by a blocking update was enough to trigger it.

For old platforms patch 2 is needed else kms_busy.extended-modeset-hang-oldfb-*
will put the gpu in a bad state when the 90s in swap_state is reached, for newer platforms patch 3 will fix this. The display doesn't need to be reset when there are no CS flips and no display reset.

If you have a newer platform, (gen4+) then the display reset can be simulated with the kms_busy.extended-modeset-hang-oldfb-with-reset-* test.

Maarten Lankhorst (3):
drm/atomic: Bump timeout for waiting for hw_done to 90s in swap_state drm/i915: Set a timeout when waiting for fence on the old fb drm/i915: Skip modeset locking when atomic pageflips are used.

 drivers/gpu/drm/drm_atomic_helper.c  | 12 ++++++++----
 drivers/gpu/drm/i915/intel_display.c | 14 +++++++++++++-
 2 files changed, 21 insertions(+), 5 deletions(-)
Signature · 6 lines
2.7.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author maintainer
#1 lore raw

PATCH 1/3 drm/atomic: Bump timeout for waiting for hw_done to 90s in swap_state

---
 drivers/gpu/drm/drm_atomic_helper.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
drivers/gpu/drm/drm_atomic_helper.c +8 -4
index b535f782b2c0..34338678db26 100644
@@ -1977,7 +1977,6 @@ void drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 				  bool stall)
 {
 	int i;
-	long ret;
 	struct drm_connector *connector;
 	struct drm_connector_state *old_conn_state, *new_conn_state;
 	struct drm_crtc *crtc;
@@ -1987,6 +1986,8 @@ void drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 	struct drm_crtc_commit *commit;
 
 	if (stall) {
+		long ret = 90*HZ;
+
 		for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) {
 			spin_lock(&crtc->commit_lock);
 			commit = list_first_entry_or_null(&crtc->commit_list,
@@ -1999,11 +2000,14 @@ void drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 				continue;
 
 			ret = wait_for_completion_timeout(&commit->hw_done,
-							  10*HZ);
-			if (ret == 0)
+							  ret);
+			drm_crtc_commit_put(commit);
+
+			if (ret == 0) {
 				DRM_ERROR("[CRTC:%d:%s] hw_done timed out\n",
 					  crtc->base.id, crtc->name);
-			drm_crtc_commit_put(commit);
+				break;
+			}
 		}
 	}
 
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#2 lore raw

PATCH 2/3 drm/i915: Set a timeout when waiting for fence on the old fb

In reply to Maarten Lankhorst
---
 drivers/gpu/drm/i915/intel_display.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/intel_display.c +1 -1
index 7c19b3c7a4ee..35d25e58a37e 100644
@@ -14797,7 +14797,7 @@ intel_prepare_plane_fb(struct drm_plane *plane,
 		if (needs_modeset(crtc_state)) {
 			ret = i915_sw_fence_await_reservation(&intel_state->commit_ready,
 							      old_obj->resv, NULL,
-							      false, 0,
+							      false, I915_RESET_TIMEOUT,
 							      GFP_KERNEL);
 			if (ret < 0)
 				return ret;
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#3 lore raw

PATCH 3/3 drm/i915: Skip modeset locking when atomic pageflips are used.

In reply to Maarten Lankhorst

With the atomic helper for pageflips there are no CS flips when that require resetting, so on most platforms we can completely skip the locking.

Because we may end up reverting the page_flip patch add an explicit function pointer check so that if someone reverts the page flip patch there will not be any issues if this is forgotten.

---
 drivers/gpu/drm/i915/intel_display.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/gpu/drm/i915/intel_display.c +12 -0
index 35d25e58a37e..bf5891fc1369 100644
@@ -3519,6 +3519,8 @@ static bool gpu_reset_clobbers_display(struct drm_i915_private *dev_priv)
 		INTEL_GEN(dev_priv) < 5 && !IS_G4X(dev_priv);
 }
 
+static const struct drm_crtc_funcs intel_crtc_funcs;
+
 void intel_prepare_reset(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = &dev_priv->drm;
@@ -3526,6 +3528,11 @@ void intel_prepare_reset(struct drm_i915_private *dev_priv)
 	struct drm_atomic_state *state;
 	int ret;
 
+	if (intel_crtc_funcs.page_flip == drm_atomic_helper_page_flip &&
+	    !i915.force_reset_modeset_test &&
+	    !gpu_reset_clobbers_display(dev_priv))
+		return;
+
 	/*
 	 * Need mode_config.mutex so that we don't
 	 * trample ongoing ->detect() and whatnot.
@@ -3574,6 +3581,11 @@ void intel_finish_reset(struct drm_i915_private *dev_priv)
 	struct drm_atomic_state *state = dev_priv->modeset_restore_state;
 	int ret;
 
+	if (intel_crtc_funcs.page_flip == drm_atomic_helper_page_flip &&
+	    !i915.force_reset_modeset_test &&
+	    !gpu_reset_clobbers_display(dev_priv))
+		return;
+
 	/*
 	 * Flips in the rings will be nuked by the reset,
 	 * so complete all pending flips so that user space
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#4 lore raw
In reply to Maarten Lankhorst
Quoted 6 lines · Maarten Lankhorst

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

A nonblocking modeset with a hang on the old fb, followed by a blocking update was enough to trigger it.

Quoted 7 lines

For old platforms patch 2 is needed else kms_busy.extended-modeset-hang-oldfb-*
will put the gpu in a bad state when the 90s in swap_state is reached, for newer platforms patch 3 will fix this. The display doesn't need to be reset when there are no CS flips and no display reset.

If you have a newer platform, (gen4+) then the display reset can be

g4x+

Quoted 18 lines

simulated with the kms_busy.extended-modeset-hang-oldfb-with-reset-* test.

Maarten Lankhorst (3):
drm/atomic: Bump timeout for waiting for hw_done to 90s in swap_state drm/i915: Set a timeout when waiting for fence on the old fb drm/i915: Skip modeset locking when atomic pageflips are used.

 drivers/gpu/drm/drm_atomic_helper.c  | 12 ++++++++----
 drivers/gpu/drm/i915/intel_display.c | 14 +++++++++++++-
 2 files changed, 21 insertions(+), 5 deletions(-)
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Signature · 6 lines
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw
In reply to Maarten Lankhorst
Quoted 2 lines · Maarten Lankhorst

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#6 lore raw
Quoted 10 lines · Chris Wilson
Quoted 2 lines · Maarten Lankhorst

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ... -Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#7 lore raw
Quoted 27 lines · Daniel Vetter
Quoted 10 lines · Chris Wilson
Quoted 2 lines · Maarten Lankhorst

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

Quoted 7 lines

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ...

The modeset lock may not just be waiting on our requests (even on pnv we can expect that there are already users celebrating that pnv+nouveau finally works ;) and that the display is not the only user/observer of those requests. Using the requests to break the modeset lock just feels like the wrong approach.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#8 lore raw
Quoted 36 lines · Chris Wilson
Quoted 27 lines · Daniel Vetter
Quoted 10 lines · Chris Wilson
Quoted 2 lines · Maarten Lankhorst

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

Quoted 7 lines

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ...

Quoted 5 lines

The modeset lock may not just be waiting on our requests (even on pnv we can expect that there are already users celebrating that pnv+nouveau finally works ;) and that the display is not the only user/observer of those requests. Using the requests to break the modeset lock just feels like the wrong approach.

It's a cycle, and we need to break it somewhere. Another option might be to break the cycle the same way we do it for gem locks: Wake up everyone and restart the modeset ioctl. Since the trouble only happens for synchronous modesets where we hold the locks while waiting for fences, we can also break out of that and restart. And I also don't think that would leak to other drivers, after all our gem locking restart dances also don't leak to other drivers - it's just our own driver's lock which are affected by these special wakupe semantics.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#9 lore raw
Quoted 43 lines · Daniel Vetter
Quoted 36 lines · Chris Wilson
Quoted 27 lines · Daniel Vetter
Quoted 10 lines · Chris Wilson
Quoted 2 lines · Maarten Lankhorst

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

Quoted 7 lines

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ...

Quoted 5 lines

The modeset lock may not just be waiting on our requests (even on pnv we can expect that there are already users celebrating that pnv+nouveau finally works ;) and that the display is not the only user/observer of those requests. Using the requests to break the modeset lock just feels like the wrong approach.

Quoted 8 lines

It's a cycle, and we need to break it somewhere. Another option might be to break the cycle the same way we do it for gem locks: Wake up everyone and restart the modeset ioctl. Since the trouble only happens for synchronous modesets where we hold the locks while waiting for fences, we can also break out of that and restart. And I also don't think that would leak to other drivers, after all our gem locking restart dances also don't leak to other drivers - it's just our own driver's lock which are affected by these special wakupe semantics.

It's a queue of nonblocking modesets that we need to worry about, afaik. Moving the wait for blocking modeset outside of modeset lock is easily achievable (and avoiding the other waits under both the modeset + struct_mutex I have at least an idea for). So the challenge is how to inject all-planes-off for gen3 and then allow the queue to continue again afterwards.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#10 lore raw
Quoted 53 lines · Chris Wilson
Quoted 43 lines · Daniel Vetter
Quoted 36 lines · Chris Wilson
Quoted 27 lines · Daniel Vetter
Quoted 10 lines · Chris Wilson
Quoted 2 lines · Maarten Lankhorst

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

Quoted 7 lines

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ...

Quoted 5 lines

The modeset lock may not just be waiting on our requests (even on pnv we can expect that there are already users celebrating that pnv+nouveau finally works ;) and that the display is not the only user/observer of those requests. Using the requests to break the modeset lock just feels like the wrong approach.

Quoted 8 lines

It's a cycle, and we need to break it somewhere. Another option might be to break the cycle the same way we do it for gem locks: Wake up everyone and restart the modeset ioctl. Since the trouble only happens for synchronous modesets where we hold the locks while waiting for fences, we can also break out of that and restart. And I also don't think that would leak to other drivers, after all our gem locking restart dances also don't leak to other drivers - it's just our own driver's lock which are affected by these special wakupe semantics.

Quoted 6 lines

It's a queue of nonblocking modesets that we need to worry about, afaik. Moving the wait for blocking modeset outside of modeset lock is easily achievable (and avoiding the other waits under both the modeset + struct_mutex I have at least an idea for). So the challenge is how to inject all-planes-off for gen3 and then allow the queue to continue again afterwards.

Hm right, I missed the nonblocking updates which don't take locks. But assuming we do the display reset for gpu resets as a full modeset (i.e. going through ->atomic_commit) it should still work out correctly:

Starting state: gpu is hung, nonblocking modeset waiting for some requests to complete.

1. hangcheck kicks in, fires off reset work.

2. We complete all requests with fence.error = -EIO and wake up any waiters. That means no re-queueing for older platforms, but oh well.

3. We grab all the display locks. Nothing happens yet.

4. We reset the chip, display dies.

5. We run ->atomic_commit to restore things. This will also force the nonblocking commit worker to complete before this display restore touches anything.

The only trouble I see is that the nonblocking worker can still touch the display block while we kill it, which isn't awesome. But we can fix that by waiting for all pending nonblocking commits in step 3 manually (without calling into atomic_commit), as long as we do step 2.

So completing everything with EIO unconditionally still seems like the simplest option that actually works for pre-g4x ...
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author maintainer
#11 lore raw

Op 30-01-17 om 09:17 schreef Daniel Vetter:

Quoted 55 lines · Chris Wilson
Quoted 48 lines · Daniel Vetter
Quoted 39 lines · Chris Wilson
Quoted 33 lines · Daniel Vetter
Quoted 25 lines · Chris Wilson
Quoted 9 lines · Maarten Lankhorst
Quoted 2 lines

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

Quoted 7 lines

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ...

Quoted 5 lines

The modeset lock may not just be waiting on our requests (even on pnv we can expect that there are already users celebrating that pnv+nouveau finally works ;) and that the display is not the only user/observer of those requests. Using the requests to break the modeset lock just feels like the wrong approach.

Quoted 8 lines

It's a cycle, and we need to break it somewhere. Another option might be to break the cycle the same way we do it for gem locks: Wake up everyone and restart the modeset ioctl. Since the trouble only happens for synchronous modesets where we hold the locks while waiting for fences, we can also break out of that and restart. And I also don't think that would leak to other drivers, after all our gem locking restart dances also don't leak to other drivers - it's just our own driver's lock which are affected by these special wakupe semantics.

Quoted 6 lines

It's a queue of nonblocking modesets that we need to worry about, afaik. Moving the wait for blocking modeset outside of modeset lock is easily achievable (and avoiding the other waits under both the modeset + struct_mutex I have at least an idea for). So the challenge is how to inject all-planes-off for gen3 and then allow the queue to continue again afterwards.

Quoted 6 lines

Hm right, I missed the nonblocking updates which don't take locks. But assuming we do the display reset for gpu resets as a full modeset (i.e. going through ->atomic_commit) it should still work out correctly:

Starting state: gpu is hung, nonblocking modeset waiting for some requests to complete.

Missing one evil detail here, else things would have moved forward..

A unrelated thread performs a blocking commit, and holds all locks until the nonblocking modeset completes.

Quoted 13 lines

1. hangcheck kicks in, fires off reset work.

2. We complete all requests with fence.error = -EIO and wake up any waiters. That means no re-queueing for older platforms, but oh well.

3. We grab all the display locks. Nothing happens yet.

4. We reset the chip, display dies.

5. We run ->atomic_commit to restore things. This will also force the nonblocking commit worker to complete before this display restore touches anything.

Quoted 8 lines

The only trouble I see is that the nonblocking worker can still touch the display block while we kill it, which isn't awesome. But we can fix that by waiting for all pending nonblocking commits in step 3 manually (without calling into atomic_commit), as long as we do step 2.

So completing everything with EIO unconditionally still seems like the simplest option that actually works for pre-g4x ...
-Daniel

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#12 lore raw
Quoted 60 lines · Maarten Lankhorst

Op 30-01-17 om 09:17 schreef Daniel Vetter:

Quoted 59 lines · Chris Wilson
Quoted 48 lines · Daniel Vetter
Quoted 39 lines · Chris Wilson
Quoted 33 lines · Daniel Vetter
Quoted 25 lines · Chris Wilson
Quoted 9 lines · Maarten Lankhorst
Quoted 2 lines

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

Quoted 7 lines

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ...

Quoted 5 lines

The modeset lock may not just be waiting on our requests (even on pnv we can expect that there are already users celebrating that pnv+nouveau finally works ;) and that the display is not the only user/observer of those requests. Using the requests to break the modeset lock just feels like the wrong approach.

Quoted 8 lines

It's a cycle, and we need to break it somewhere. Another option might be to break the cycle the same way we do it for gem locks: Wake up everyone and restart the modeset ioctl. Since the trouble only happens for synchronous modesets where we hold the locks while waiting for fences, we can also break out of that and restart. And I also don't think that would leak to other drivers, after all our gem locking restart dances also don't leak to other drivers - it's just our own driver's lock which are affected by these special wakupe semantics.

Quoted 6 lines

It's a queue of nonblocking modesets that we need to worry about, afaik. Moving the wait for blocking modeset outside of modeset lock is easily achievable (and avoiding the other waits under both the modeset + struct_mutex I have at least an idea for). So the challenge is how to inject all-planes-off for gen3 and then allow the queue to continue again afterwards.

Hm right, I missed the nonblocking updates which don't take locks. But assuming we do the display reset for gpu resets as a full modeset (i.e. going through ->atomic_commit) it should still work out correctly:

Quoted 5 lines
Quoted 2 lines

Starting state: gpu is hung, nonblocking modeset waiting for some requests to complete.

Missing one evil detail here, else things would have moved forward..

A unrelated thread performs a blocking commit, and holds all locks until the nonblocking modeset completes.

And where is the problem in that? If we first set all fences to -EIO, and then try to grab locks, that other thread will be able to complete. After all this scheme worked before we reworked the reset logic completely. -Daniel

Quoted 23 lines
Quoted 21 lines

1. hangcheck kicks in, fires off reset work.

2. We complete all requests with fence.error = -EIO and wake up any waiters. That means no re-queueing for older platforms, but oh well.

3. We grab all the display locks. Nothing happens yet.

4. We reset the chip, display dies.

5. We run ->atomic_commit to restore things. This will also force the nonblocking commit worker to complete before this display restore touches anything.

The only trouble I see is that the nonblocking worker can still touch the display block while we kill it, which isn't awesome. But we can fix that by waiting for all pending nonblocking commits in step 3 manually (without calling into atomic_commit), as long as we do step 2.

So completing everything with EIO unconditionally still seems like the simplest option that actually works for pre-g4x ...
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#13 lore raw

Hey,

Op 31-01-17 om 08:46 schreef Daniel Vetter:

Quoted 61 lines · Maarten Lankhorst
Quoted 60 lines

Op 30-01-17 om 09:17 schreef Daniel Vetter:

Quoted 59 lines · Chris Wilson
Quoted 48 lines · Daniel Vetter
Quoted 39 lines · Chris Wilson
Quoted 33 lines · Daniel Vetter
Quoted 25 lines · Chris Wilson
Quoted 9 lines · Maarten Lankhorst
Quoted 2 lines

When writing some testcases for nonblocking modesets. I found out that the infinite wait on the old fb was causing issues.

The crux of the issue here is the locked wait for old dependencies and the inability to inject the intel_prepare_reset disabling of all planes. There are a couple of locked waits on struct_mutex within the modeset locks for intel_overlay and if we happen to be using the display plane for the first time.

Quoted 12 lines

The first I suggested solving using fences to track dependencies and keep the order between atomic states. Cancelling the outstanding modesets, replacing with a disable and then on restore jumping to the final state look doable. It also requires avoiding the struct_mutex for disabling, which is quite easy. To avoid the wait under struct_mutex, we've talked about switching to mmio, but for starters we could move the wait from inside intel_overlay into the fence for the atomic operation. (But's that a little more surgery than we would like for intel_overlay I guess - dig out Ville's patches for overlay planes?) And to prevent the wait under struct_mutex for pin_to_display_plane, my plane is to move that to an async fenced operation that is then naturally waited upon by the atomic modeset.

A bit more a hack, but a different idea, and I think hack for gen234.0 is ok:

Quoted 7 lines

We complete all the requests before we start the hw reset with fence.error = -EIO. But we do this only when we need to get at the display locks. A slightly more elegant solution would be to trylock modeset locks, and if one of them fails (and only then) complete all requests with -EIO to get the concurrent modeset to proceed before we reset the hardware. That's essentially the logic we had before all the reworks, and it worked. But I didn't look at how scary that all would be to make it work again ...

Quoted 5 lines

The modeset lock may not just be waiting on our requests (even on pnv we can expect that there are already users celebrating that pnv+nouveau finally works ;) and that the display is not the only user/observer of those requests. Using the requests to break the modeset lock just feels like the wrong approach.

Quoted 8 lines

It's a cycle, and we need to break it somewhere. Another option might be to break the cycle the same way we do it for gem locks: Wake up everyone and restart the modeset ioctl. Since the trouble only happens for synchronous modesets where we hold the locks while waiting for fences, we can also break out of that and restart. And I also don't think that would leak to other drivers, after all our gem locking restart dances also don't leak to other drivers - it's just our own driver's lock which are affected by these special wakupe semantics.

Quoted 6 lines

It's a queue of nonblocking modesets that we need to worry about, afaik. Moving the wait for blocking modeset outside of modeset lock is easily achievable (and avoiding the other waits under both the modeset + struct_mutex I have at least an idea for). So the challenge is how to inject all-planes-off for gen3 and then allow the queue to continue again afterwards.

Hm right, I missed the nonblocking updates which don't take locks. But assuming we do the display reset for gpu resets as a full modeset (i.e. going through ->atomic_commit) it should still work out correctly:

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Starting state: gpu is hung, nonblocking modeset waiting for some requests to complete.

Missing one evil detail here, else things would have moved forward..

A unrelated thread performs a blocking commit, and holds all locks until the nonblocking modeset completes.

And where is the problem in that? If we first set all fences to -EIO, and then try to grab locks, that other thread will be able to complete. After all this scheme worked before we reworked the reset logic completely.

True, but we probably still want to cap the timeout (patch 2) to prevent a deadlock when a fence on another driver misbehaves.

And if we have a timeout, then things will move forward eventually even if we wait for locks, though it might still be a good idea to complete everything with -EIO first to make it happen faster. :)

~Maarten
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#14 lore raw

PATCH 1/1 drm/i915: Skip modeset locking when atomic pageflips are used.

In reply to Daniel Vetter

With the atomic helper for pageflips there are no CS flips when that require resetting, so on most platforms we can completely skip the locking.

Because we may end up reverting the page_flip patch add an explicit function pointer check so that if someone reverts the page flip patch there will not be any issues if this is forgotten.

Testcase: kms_busy.extended-modeset-hang-oldfb-*
---
This is a standalone patch to fix modeset hangs on g4x+.

The path for gen4 and lower is simulated in kms_busy.extended-modeset-hang-oldfb-with-reset and still fails.

 drivers/gpu/drm/i915/intel_display.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/gpu/drm/i915/intel_display.c +12 -0
index d8db1caec1b8..1dd480a6752a 100644
@@ -3529,6 +3529,8 @@ static bool gpu_reset_clobbers_display(struct drm_i915_private *dev_priv)
 		INTEL_GEN(dev_priv) < 5 && !IS_G4X(dev_priv);
 }
 
+static const struct drm_crtc_funcs intel_crtc_funcs;
+
 void intel_prepare_reset(struct drm_i915_private *dev_priv)
 {
 	struct drm_device *dev = &dev_priv->drm;
@@ -3536,6 +3538,11 @@ void intel_prepare_reset(struct drm_i915_private *dev_priv)
 	struct drm_atomic_state *state;
 	int ret;
 
+	if (intel_crtc_funcs.page_flip == drm_atomic_helper_page_flip &&
+	    !i915.force_reset_modeset_test &&
+	    !gpu_reset_clobbers_display(dev_priv))
+		return;
+
 	/*
 	 * Need mode_config.mutex so that we don't
 	 * trample ongoing ->detect() and whatnot.
@@ -3584,6 +3591,11 @@ void intel_finish_reset(struct drm_i915_private *dev_priv)
 	struct drm_atomic_state *state = dev_priv->modeset_restore_state;
 	int ret;
 
+	if (intel_crtc_funcs.page_flip == drm_atomic_helper_page_flip &&
+	    !i915.force_reset_modeset_test &&
+	    !gpu_reset_clobbers_display(dev_priv))
+		return;
+
 	/*
 	 * Flips in the rings will be nuked by the reset,
 	 * so complete all pending flips so that user space
Signature · 7 lines
2.7.4


_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel