Skip to contentSkip to conversation

drm/atomic: swap_state should stall on cleanup_done

PATCH NAK Rob Clark 2 participants 5 messages Oct 31, 2019 – Nov 1, 2019

Subsystems (2)
lore
Share & tools
lore

Rob Clark

author
#0 lore raw

PATCH 1/1 drm/atomic: swap_state should stall on cleanup_done

From: Rob Clark <[email protected]>

Stalling on cleanup_done ensures that any atomic state related to a nonblock commit no longer has dangling references to per-object state that can be freed.

Otherwise, if a !nonblock commit completes after a nonblock commit has swapped state (ie. the synchronous part of the nonblock commit comes before the !nonblock commit), but before the asynchronous part of the nonblock commit completes, what was the new per-object state in the nonblock commit can be freed.

This shows up with the new self-refresh helper, as _update_avg_times() dereferences the original old and new crtc_state.

Fixes d4da4e33341c ("drm: Measure Self Refresh Entry/Exit times to avoid thrashing")
Cc Sean Paul <[email protected]>

---
Other possibilities:
1) maybe block later before freeing atomic state?
2) refcount individual per-object state

 drivers/gpu/drm/drm_atomic_helper.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/drm_atomic_helper.c +3 -3
index 3ef2ac52ce94..a5d95429f91b 100644
@@ -2711,7 +2711,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}
@@ -2722,7 +2722,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}
@@ -2733,7 +2733,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}
Signature · 1 line
2.21.0
maintainer NAK
#1 lore raw

Op 31-10-2019 om 23:36 schreef Rob Clark:

Quoted 49 lines

From: Rob Clark <[email protected]>

Stalling on cleanup_done ensures that any atomic state related to a nonblock commit no longer has dangling references to per-object state that can be freed.

Otherwise, if a !nonblock commit completes after a nonblock commit has swapped state (ie. the synchronous part of the nonblock commit comes before the !nonblock commit), but before the asynchronous part of the nonblock commit completes, what was the new per-object state in the nonblock commit can be freed.

This shows up with the new self-refresh helper, as _update_avg_times() dereferences the original old and new crtc_state.

Fixes d4da4e33341c ("drm: Measure Self Refresh Entry/Exit times to avoid thrashing")
Cc Sean Paul <[email protected]>

---
Other possibilities:
1) maybe block later before freeing atomic state?
2) refcount individual per-object state

 drivers/gpu/drm/drm_atomic_helper.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/drm_atomic_helper.c +2 -2
index 3ef2ac52ce94..a5d95429f91b 100644
@@ -2711,7 +2711,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}
@@ -2722,7 +2722,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}

drivers/gpu/drm/drm_atomic_helper.c int drm_atomic_helper_swap_state(struct drm_atomic_state *state, · line 2739

Quoted 8 lines
drivers/gpu/drm/drm_atomic_helper.c +1 -1
@@ -2733,7 +2733,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}

Nack, hw_done means all new_crtc_state (from the old commit pov) dereferences are done.

Self refresh helpers should be fixed. :)

#4 lore raw
In reply to Rob Clark

Op 31-10-2019 om 23:36 schreef Rob Clark:

Quoted 58 lines

From: Rob Clark <[email protected]>

Stalling on cleanup_done ensures that any atomic state related to a nonblock commit no longer has dangling references to per-object state that can be freed.

Otherwise, if a !nonblock commit completes after a nonblock commit has swapped state (ie. the synchronous part of the nonblock commit comes before the !nonblock commit), but before the asynchronous part of the nonblock commit completes, what was the new per-object state in the nonblock commit can be freed.

This shows up with the new self-refresh helper, as _update_avg_times() dereferences the original old and new crtc_state.

Fixes d4da4e33341c ("drm: Measure Self Refresh Entry/Exit times to avoid thrashing")
Cc Sean Paul <[email protected]>

---
Other possibilities:
1) maybe block later before freeing atomic state?
2) refcount individual per-object state

 drivers/gpu/drm/drm_atomic_helper.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/drm_atomic_helper.c +3 -3
index 3ef2ac52ce94..a5d95429f91b 100644
@@ -2711,7 +2711,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}
@@ -2722,7 +2722,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}
@@ -2733,7 +2733,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
 			if (!commit)
 				continue;
 
-			ret = wait_for_completion_interruptible(&commit->hw_done);
+			ret = wait_for_completion_interruptible(&commit->cleanup_done);
 			if (ret)
 				return ret;
 		}
Quoted 1 line

From setup_commit():

* Completion of the hardware commit step must be signalled using
* drm_atomic_helper_commit_hw_done(). After this step the driver is not allowed
* to read or change any permanent software or hardware modeset state. The only
* exception is state protected by other means than &drm_modeset_lock locks.
* Only the free standing @state with pointers to the old state structures can
* be inspected, e.g. to clean up old buffers using
* drm_atomic_helper_cleanup_planes().

And the hw_done function says pretty much the same thing. :)

author
#2 lore raw
In reply to Maarten Lankhorst
Quoted 54 lines · Maarten Lankhorst

Op 31-10-2019 om 23:36 schreef Rob Clark:

Quoted 52 lines

From: Rob Clark <[email protected]>

Stalling on cleanup_done ensures that any atomic state related to a nonblock commit no longer has dangling references to per-object state that can be freed.

Otherwise, if a !nonblock commit completes after a nonblock commit has swapped state (ie. the synchronous part of the nonblock commit comes before the !nonblock commit), but before the asynchronous part of the nonblock commit completes, what was the new per-object state in the nonblock commit can be freed.

This shows up with the new self-refresh helper, as _update_avg_times() dereferences the original old and new crtc_state.

Fixes d4da4e33341c ("drm: Measure Self Refresh Entry/Exit times to avoid thrashing")
Cc Sean Paul <[email protected]>

---
Other possibilities:
1) maybe block later before freeing atomic state?
2) refcount individual per-object state

 drivers/gpu/drm/drm_atomic_helper.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/drm_atomic_helper.c +2 -2
index 3ef2ac52ce94..a5d95429f91b 100644
@@ -2711,7 +2711,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
                      if (!commit)
                              continue;
 
-                     ret = wait_for_completion_interruptible(&commit->hw_done);
+                     ret = wait_for_completion_interruptible(&commit->cleanup_done);
                      if (ret)
                              return ret;
              }
@@ -2722,7 +2722,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
                      if (!commit)
                              continue;
 
-                     ret = wait_for_completion_interruptible(&commit->hw_done);
+                     ret = wait_for_completion_interruptible(&commit->cleanup_done);
                      if (ret)
                              return ret;
              }
@@ -2733,7 +2733,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
                      if (!commit)
                              continue;
 
Quoted 8 lines

drivers/gpu/drm/drm_atomic_helper.c int drm_atomic_helper_swap_state(struct drm_atomic_state *state, · line 2717

Quoted 5 lines
drivers/gpu/drm/drm_atomic_helper.c +1 -1
-                     ret = wait_for_completion_interruptible(&commit->hw_done);
+                     ret = wait_for_completion_interruptible(&commit->cleanup_done);
                      if (ret)
                              return ret;
              }

Nack, hw_done means all new_crtc_state (from the old commit pov) dereferences are done.

hmm, it would be nice if the for_each_blah_in_state() iterators would splat on incorrect usage, then.. it tool a while to track down what was going wrong. And Sean claimed the self refresh helpers worked for him on rockchip/i915 (although I'm starting to suspect maybe he just didn't have enough debug options enabled to poison freed memory?)

Quoted 1 line

Self refresh helpers should be fixed. :)

Looks like what they need out of crtc_state is pretty minimal, maybe they could extract out crtc_state->self_refresh_active earlier..

BR,
-R

#3 lore raw

Op 01-11-2019 om 15:59 schreef Rob Clark:

Quoted 62 lines
Quoted 53 lines · Maarten Lankhorst

Op 31-10-2019 om 23:36 schreef Rob Clark:

Quoted 52 lines

From: Rob Clark <[email protected]>

Stalling on cleanup_done ensures that any atomic state related to a nonblock commit no longer has dangling references to per-object state that can be freed.

Otherwise, if a !nonblock commit completes after a nonblock commit has swapped state (ie. the synchronous part of the nonblock commit comes before the !nonblock commit), but before the asynchronous part of the nonblock commit completes, what was the new per-object state in the nonblock commit can be freed.

This shows up with the new self-refresh helper, as _update_avg_times() dereferences the original old and new crtc_state.

Fixes d4da4e33341c ("drm: Measure Self Refresh Entry/Exit times to avoid thrashing")
Cc Sean Paul <[email protected]>

---
Other possibilities:
1) maybe block later before freeing atomic state?
2) refcount individual per-object state

 drivers/gpu/drm/drm_atomic_helper.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/drm_atomic_helper.c +2 -2
index 3ef2ac52ce94..a5d95429f91b 100644
@@ -2711,7 +2711,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
                      if (!commit)
                              continue;
 
-                     ret = wait_for_completion_interruptible(&commit->hw_done);
+                     ret = wait_for_completion_interruptible(&commit->cleanup_done);
                      if (ret)
                              return ret;
              }
@@ -2722,7 +2722,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
                      if (!commit)
                              continue;
 
-                     ret = wait_for_completion_interruptible(&commit->hw_done);
+                     ret = wait_for_completion_interruptible(&commit->cleanup_done);
                      if (ret)
                              return ret;
              }
@@ -2733,7 +2733,7 @@ int drm_atomic_helper_swap_state(struct drm_atomic_state *state,
                      if (!commit)
                              continue;
 
Quoted 7 lines

drivers/gpu/drm/drm_atomic_helper.c int drm_atomic_helper_swap_state(struct drm_atomic_state *state, · line 2717

Quoted 5 lines
drivers/gpu/drm/drm_atomic_helper.c +1 -1
-                     ret = wait_for_completion_interruptible(&commit->hw_done);
+                     ret = wait_for_completion_interruptible(&commit->cleanup_done);
                      if (ret)
                              return ret;
              }

Nack, hw_done means all new_crtc_state (from the old commit pov) dereferences are done.

Quoted 5 lines

hmm, it would be nice if the for_each_blah_in_state() iterators would splat on incorrect usage, then.. it tool a while to track down what was going wrong. And Sean claimed the self refresh helpers worked for him on rockchip/i915 (although I'm starting to suspect maybe he just didn't have enough debug options enabled to poison freed memory?)

Could do a memset on the new arrays after hw_done?

Quoted 3 lines
Quoted 1 line

Self refresh helpers should be fixed. :)

Looks like what they need out of crtc_state is pretty minimal, maybe they could extract out crtc_state->self_refresh_active earlier..

Yeah, something like that would work. :)