locking: Fix __clear_task_blocked_on() warning from __ww_mutex_wound() path
Subsystems (2)
PATCH v2 1/1 locking: Fix __clear_task_blocked_on() warning from __ww_mutex_wound() path
The __clear_task_blocked_on() helper added a number of sanity checks ensuring we hold the mutex wait lock and that the task we are clearing blocked_on pointer (if set) matches the mutex.
However, there is an edge case in the _ww_mutex_wound() logic where we need to clear the blocked_on pointer for the task that owns the mutex, not the task that is waiting on the mutex.
For this case the sanity checks aren't valid, so handle this by allowing a NULL lock to skip the additional checks.
This was easier to miss, I realized, as the test-ww_mutex driver only exercises the wait-die class of ww_mutexes.
I've got a follow up patch to extend the test so that it will exercise both.
---
v2:
* Rewording of "lock" to "mutex" in commit and comment for clarity
* Rework __clear_task_blocked_on() to use READ_ONCE and WRITE_ONCE so we don't trip over the WARNING if two instances race, as suggested by K Prateek Nayak and Maarten Lankhorst
--- include/linux/sched.h | 23 +++++++++++++---------- kernel/locking/ww_mutex.h | 6 +++++- 2 files changed, 18 insertions(+), 11 deletions(-)
include/linux/sched.h +13 -10
@@ -2166,16 +2166,19 @@ static inline void set_task_blocked_on(struct task_struct *p, struct mutex *m)static inline void __clear_task_blocked_on (struct task_struct *p,struct mutex *m) { -WARN_ON_ONCE (!m); -/* Currently we serialize blocked_on under the mutex::wait_lock */ -lockdep_assert_held_once (&m->wait_lock); -/* -* There may be cases where we re-clear already cleared -* blocked_on relationships, but make sure we are not -* clearing the relationship with a different lock. -*/ -WARN_ON_ONCE (m && p->blocked_on && p->blocked_on != m); - p->blocked_on =NULL ; +if (m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); + +/* Currently we serialize blocked_on under the mutex::wait_lock */ +lockdep_assert_held_once (&m->wait_lock); +/* +* There may be cases where we re-clear already cleared +* blocked_on relationships, but make sure we are not +* clearing the relationship with a different lock. +*/ +WARN_ON_ONCE (blocked_on && blocked_on != m); + } +WRITE_ONCE (p->blocked_on,NULL ); }static inline void clear_task_blocked_on (struct task_struct *p,struct mutex *m)
kernel/locking/ww_mutex.h +5 -1
@@ -342,8 +342,12 @@ static bool __ww_mutex_wound(struct MUTEX *lock,* When waking up the task to wound, be sure to clear the * blocked_on pointer. Otherwise we can see circular * blocked_on relationships that can't resolve. +* +* NOTE: We pass NULL here instead of lock, because we +* are waking the mutex owner, who may be currently +* blocked on a different mutex. */ -__clear_task_blocked_on (owner, lock); +__clear_task_blocked_on (owner,NULL );wake_q_add (wake_q, owner); }return true;
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
K Prateek Nayak
reviewerHello John,
Quoted 14 lines · John Stultz
The
__clear_task_blocked_on()helper added a number of sanity checks ensuring we hold the mutex wait lock and that the task we are clearingblocked_onpointer (if set) matches the mutex.However, there is an edge case in the
_ww_mutex_wound()logic where we need to clear theblocked_onpointer for the task that owns the mutex, not the task that is waiting on the mutex.For this case the sanity checks aren't valid, so handle this by allowing a NULL lock to skip the additional checks.
This was easier to miss, I realized, as the test-ww_mutex driver only exercises the wait-die class of
ww_mutexes.
Quoted 8 lines
I've got a follow up patch to extend the test so that it will exercise both.
Fixes a4f0b6fef4b0 ("locking/mutex: Add p->blocked_on wrappers for correctness checks")Reported-by [email protected]Reported-by Maarten Lankhorst <[email protected]>Signed-off-by John Stultz <[email protected]>
I've been running this for a while and haven't seen any splats with syzbot's C reproducer.
include/linux/sched.h static inline void set_task_blocked_on(struct task_struct *p, struct mutex *m)
Quoted 33 lines
---
v2:
* Rewording of "lock" to "mutex" in commit and comment for clarity
* Rework__clear_task_blocked_on()to useREAD_ONCEandWRITE_ONCEso we don't trip over the WARNING if two instances race, as suggested by K Prateek Nayak and Maarten LankhorstCc Ingo Molnar <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Juri Lelli <[email protected]>Cc Vincent Guittot <[email protected]>Cc Dietmar Eggemann <[email protected]>Cc Valentin Schneider <[email protected]>Cc K Prateek Nayak <[email protected]>Cc Suleiman Souhlal <[email protected]>Cc Maarten Lankhorst <[email protected]>--- include/linux/sched.h | 23 +++++++++++++---------- kernel/locking/ww_mutex.h | 6 +++++- 2 files changed, 18 insertions(+), 11 deletions(-)include/linux/sched.h +0
-0@@ -2166,16 +2166,19 @@ static inline void set_task_blocked_on(struct task_struct *p, struct mutex *m)
Should we consider using WRITE_ONCE() in __set_task_blocked_on() and use a local copy of "blocked_on" there too?
I think a set_task_blocked_on() on a separate ww_mutex can still race with a wound on the ww_ctx which indiscriminately writes NULL to "owner->blocked_on" and can possibly lead to a splat for:
WARN_ON_ONCE(p->blocked_on && p->blocked_on != m);
^^^^^^^^^^^^^ ^^^^^^^^^^^^^
Sees p is blocked on "m" Turns NULL as a result
already. of a concurrent wound.
A READ_ONCE() and WRITE_ONCE() in __set_task_blocked_on() should help solve the splat in this very unlikely case too unless I'm mistaken.
Apart from that, this fix looks good. Feel free to include:
Signature · 2 lines
Thanks and Regards, Prateek
Quoted 27 lines
include/linux/sched.h +13
-10static inline void __clear_task_blocked_on (struct task_struct *p,struct mutex *m) { -WARN_ON_ONCE (!m); -/* Currently we serialize blocked_on under the mutex::wait_lock */ -lockdep_assert_held_once (&m->wait_lock); -/* -* There may be cases where we re-clear already cleared -* blocked_on relationships, but make sure we are not -* clearing the relationship with a different lock. -*/ -WARN_ON_ONCE (m && p->blocked_on && p->blocked_on != m); - p->blocked_on =NULL ; +if (m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); + +/* Currently we serialize blocked_on under the mutex::wait_lock */ +lockdep_assert_held_once (&m->wait_lock); +/* +* There may be cases where we re-clear already cleared +* blocked_on relationships, but make sure we are not +* clearing the relationship with a different lock. +*/ +WARN_ON_ONCE (blocked_on && blocked_on != m); + } +WRITE_ONCE (p->blocked_on,NULL ); }
PATCH v3 1/1 locking: Fix __clear_task_blocked_on() warning from __ww_mutex_wound() path
In reply to John StultzThe __clear_task_blocked_on() helper added a number of sanity checks ensuring we hold the mutex wait lock and that the task we are clearing blocked_on pointer (if set) matches the mutex.
However, there is an edge case in the _ww_mutex_wound() logic where we need to clear the blocked_on pointer for the task that owns the mutex, not the task that is waiting on the mutex.
For this case the sanity checks aren't valid, so handle this by allowing a NULL lock to skip the additional checks.
K Prateek Nayak and Maarten Lankhorst also pointed out that in this case where we don't hold the owner's mutex wait_lock, we need to be a bit more careful using READ_ONCE/WRITE_ONCE in both the __clear_task_blocked_on() and __set_task_blocked_on() implementations to avoid accidentally tripping WARN_ONs if two instances race. So do that here as well.
This issue was easier to miss, I realized, as the test-ww_mutex driver only exercises the wait-die class of ww_mutexes. I've sent a patch[1] to address this so the logic will be easier to test.
[1]: https://lore.kernel.org/lkml/[email protected]/
---
v2:
* Rewording of "lock" to "mutex" in commit and comment for clarity
* Rework __clear_task_blocked_on() to use READ_ONCE and WRITE_ONCE so we don't trip over the WARNING if two instances race, as suggested by K Prateek Nayak and Maarten Lankhorst
v3:
* Add READ_ONCE and WRITE_ONCE to __set_task_blocked_on(), to avoid tripping similar warnings as suggested by K Prateek Nayak
--- include/linux/sched.h | 29 +++++++++++++++++------------ kernel/locking/ww_mutex.h | 6 +++++- 2 files changed, 22 insertions(+), 13 deletions(-)
include/linux/sched.h +17 -12
@@ -2144,6 +2144,8 @@ static inline struct mutex *__get_task_blocked_on(struct task_struct *p)static inline void __set_task_blocked_on (struct task_struct *p,struct mutex *m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); +WARN_ON_ONCE (!m);/* The task should only be setting itself as blocked */ WARN_ON_ONCE (p != current);
@@ -2154,8 +2156,8 @@ static inline void __set_task_blocked_on(struct task_struct *p, struct mutex *m)* with a different mutex. Note, setting it to the same * lock repeatedly is ok. */ -WARN_ON_ONCE (p->blocked_on && p->blocked_on != m); - p->blocked_on = m; +WARN_ON_ONCE (blocked_on && blocked_on != m); +WRITE_ONCE (p->blocked_on, m); }static inline void set_task_blocked_on (struct task_struct *p,struct mutex *m)
@@ -2166,16 +2168,19 @@ static inline void set_task_blocked_on(struct task_struct *p, struct mutex *m)static inline void __clear_task_blocked_on (struct task_struct *p,struct mutex *m) { -WARN_ON_ONCE (!m); -/* Currently we serialize blocked_on under the mutex::wait_lock */ -lockdep_assert_held_once (&m->wait_lock); -/* -* There may be cases where we re-clear already cleared -* blocked_on relationships, but make sure we are not -* clearing the relationship with a different lock. -*/ -WARN_ON_ONCE (m && p->blocked_on && p->blocked_on != m); - p->blocked_on =NULL ; +if (m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); + +/* Currently we serialize blocked_on under the mutex::wait_lock */ +lockdep_assert_held_once (&m->wait_lock); +/* +* There may be cases where we re-clear already cleared +* blocked_on relationships, but make sure we are not +* clearing the relationship with a different lock. +*/ +WARN_ON_ONCE (blocked_on && blocked_on != m); + } +WRITE_ONCE (p->blocked_on,NULL ); }static inline void clear_task_blocked_on (struct task_struct *p,struct mutex *m)
kernel/locking/ww_mutex.h +5 -1
@@ -342,8 +342,12 @@ static bool __ww_mutex_wound(struct MUTEX *lock,* When waking up the task to wound, be sure to clear the * blocked_on pointer. Otherwise we can see circular * blocked_on relationships that can't resolve. +* +* NOTE: We pass NULL here instead of lock, because we +* are waking the mutex owner, who may be currently +* blocked on a different mutex. */ -__clear_task_blocked_on (owner, lock); +__clear_task_blocked_on (owner,NULL );wake_q_add (wake_q, owner); }return true;
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Den 2025-08-05 kl. 02:10, skrev John Stultz:
Quoted 135 lines
The
__clear_task_blocked_on()helper added a number of sanity checks ensuring we hold the mutex wait lock and that the task we are clearingblocked_onpointer (if set) matches the mutex.However, there is an edge case in the
_ww_mutex_wound()logic where we need to clear theblocked_onpointer for the task that owns the mutex, not the task that is waiting on the mutex.For this case the sanity checks aren't valid, so handle this by allowing a NULL lock to skip the additional checks.
K Prateek Nayak and Maarten Lankhorst also pointed out that in this case where we don't hold the owner's mutex
wait_lock, we need to be a bit more careful using READ_ONCE/WRITE_ONCE in both the__clear_task_blocked_on()and__set_task_blocked_on()implementations to avoid accidentally trippingWARN_ONsif two instances race. So do that here as well.This issue was easier to miss, I realized, as the test-ww_mutex driver only exercises the wait-die class of
ww_mutexes. I've sent a patch[1] to address this so the logic will be easier to test.[1]: https://lore.kernel.org/lkml/[email protected]/
Fixes a4f0b6fef4b0 ("locking/mutex: Add p->blocked_on wrappers for correctness checks")Reported-by [email protected]Reported-by Maarten Lankhorst <[email protected]>Signed-off-by John Stultz <[email protected]>Reviewed-by K Prateek Nayak <[email protected]>Tested-by K Prateek Nayak <[email protected]>---
v2:
* Rewording of "lock" to "mutex" in commit and comment for clarity
* Rework__clear_task_blocked_on()to useREAD_ONCEandWRITE_ONCEso we don't trip over the WARNING if two instances race, as suggested by K Prateek Nayak and Maarten Lankhorst
v3:
* AddREAD_ONCEandWRITE_ONCEto__set_task_blocked_on(), to avoid tripping similar warnings as suggested by K Prateek NayakCc Ingo Molnar <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Juri Lelli <[email protected]>Cc Vincent Guittot <[email protected]>Cc Dietmar Eggemann <[email protected]>Cc Valentin Schneider <[email protected]>Cc K Prateek Nayak <[email protected]>Cc Suleiman Souhlal <[email protected]>Cc Maarten Lankhorst <[email protected]>--- include/linux/sched.h | 29 +++++++++++++++++------------ kernel/locking/ww_mutex.h | 6 +++++- 2 files changed, 22 insertions(+), 13 deletions(-)include/linux/sched.h +17
-12@@ -2144,6 +2144,8 @@ static inline struct mutex *__get_task_blocked_on(struct task_struct *p)static inline void __set_task_blocked_on (struct task_struct *p,struct mutex *m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); +WARN_ON_ONCE (!m);/* The task should only be setting itself as blocked */ WARN_ON_ONCE (p != current);@@ -2154,8 +2156,8 @@ static inline void __set_task_blocked_on(struct task_struct *p, struct mutex *m)* with a different mutex. Note, setting it to the same * lock repeatedly is ok. */ -WARN_ON_ONCE (p->blocked_on && p->blocked_on != m); - p->blocked_on = m; +WARN_ON_ONCE (blocked_on && blocked_on != m); +WRITE_ONCE (p->blocked_on, m); }static inline void set_task_blocked_on (struct task_struct *p,struct mutex *m)@@ -2166,16 +2168,19 @@ static inline void set_task_blocked_on(struct task_struct *p, struct mutex *m)static inline void __clear_task_blocked_on (struct task_struct *p,struct mutex *m) { -WARN_ON_ONCE (!m); -/* Currently we serialize blocked_on under the mutex::wait_lock */ -lockdep_assert_held_once (&m->wait_lock); -/* -* There may be cases where we re-clear already cleared -* blocked_on relationships, but make sure we are not -* clearing the relationship with a different lock. -*/ -WARN_ON_ONCE (m && p->blocked_on && p->blocked_on != m); - p->blocked_on =NULL ; +if (m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); + +/* Currently we serialize blocked_on under the mutex::wait_lock */ +lockdep_assert_held_once (&m->wait_lock); +/* +* There may be cases where we re-clear already cleared +* blocked_on relationships, but make sure we are not +* clearing the relationship with a different lock. +*/ +WARN_ON_ONCE (blocked_on && blocked_on != m); + } +WRITE_ONCE (p->blocked_on,NULL ); }static inline void clear_task_blocked_on (struct task_struct *p,struct mutex *m)kernel/locking/ww_mutex.h +5
-1@@ -342,8 +342,12 @@ static bool __ww_mutex_wound(struct MUTEX *lock,* When waking up the task to wound, be sure to clear the * blocked_on pointer. Otherwise we can see circular * blocked_on relationships that can't resolve. +* +* NOTE: We pass NULL here instead of lock, because we +* are waking the mutex owner, who may be currently +* blocked on a different mutex. */ -__clear_task_blocked_on (owner, lock); +__clear_task_blocked_on (owner,NULL );wake_q_add (wake_q, owner); }return true;
Peter Zijlstra
maintainerQuoted 1 line · Maarten Lankhorst
Thanks both!
The following commit has been merged into the locking/urgent branch of tip:
The following commit has been merged into the locking/urgent branch of tip:
Commit-ID: 21924af67d69d7c9fdaf845be69043cfe75196a1 Gitweb: https://git.kernel.org/tip/21924af67d69d7c9fdaf845be69043cfe75196a1 Author: John Stultz <[email protected]> AuthorDate: Tue, 05 Aug 2025 00:10:02 Committer: Peter Zijlstra <[email protected]> CommitterDate: Wed, 13 Aug 2025 10:34:54 +02:00
locking: Fix __clear_task_blocked_on() warning from __ww_mutex_wound() path
The __clear_task_blocked_on() helper added a number of sanity checks ensuring we hold the mutex wait lock and that the task we are clearing blocked_on pointer (if set) matches the mutex.
However, there is an edge case in the _ww_mutex_wound() logic where we need to clear the blocked_on pointer for the task that owns the mutex, not the task that is waiting on the mutex.
For this case the sanity checks aren't valid, so handle this by allowing a NULL lock to skip the additional checks.
K Prateek Nayak and Maarten Lankhorst also pointed out that in this case where we don't hold the owner's mutex wait_lock, we need to be a bit more careful using READ_ONCE/WRITE_ONCE in both the __clear_task_blocked_on() and __set_task_blocked_on() implementations to avoid accidentally tripping WARN_ONs if two instances race. So do that here as well.
This issue was easier to miss, I realized, as the test-ww_mutex driver only exercises the wait-die class of ww_mutexes. I've sent a patch[1] to address this so the logic will be easier to test.
[1]: https://lore.kernel.org/lkml/[email protected]/
--- include/linux/sched.h | 29 +++++++++++++++++------------ kernel/locking/ww_mutex.h | 6 +++++- 2 files changed, 22 insertions(+), 13 deletions(-)
include/linux/sched.h +17 -12
@@ -2144,6 +2144,8 @@ static inline struct mutex *__get_task_blocked_on(struct task_struct *p)static inline void __set_task_blocked_on (struct task_struct *p,struct mutex *m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); +WARN_ON_ONCE (!m);/* The task should only be setting itself as blocked */ WARN_ON_ONCE (p != current);
@@ -2154,8 +2156,8 @@ static inline void __set_task_blocked_on(struct task_struct *p, struct mutex *m)* with a different mutex. Note, setting it to the same * lock repeatedly is ok. */ -WARN_ON_ONCE (p->blocked_on && p->blocked_on != m); - p->blocked_on = m; +WARN_ON_ONCE (blocked_on && blocked_on != m); +WRITE_ONCE (p->blocked_on, m); }static inline void set_task_blocked_on (struct task_struct *p,struct mutex *m)
@@ -2166,16 +2168,19 @@ static inline void set_task_blocked_on(struct task_struct *p, struct mutex *m)static inline void __clear_task_blocked_on (struct task_struct *p,struct mutex *m) { -WARN_ON_ONCE (!m); -/* Currently we serialize blocked_on under the mutex::wait_lock */ -lockdep_assert_held_once (&m->wait_lock); -/* -* There may be cases where we re-clear already cleared -* blocked_on relationships, but make sure we are not -* clearing the relationship with a different lock. -*/ -WARN_ON_ONCE (m && p->blocked_on && p->blocked_on != m); - p->blocked_on =NULL ; +if (m) { +struct mutex *blocked_on =READ_ONCE (p->blocked_on); + +/* Currently we serialize blocked_on under the mutex::wait_lock */ +lockdep_assert_held_once (&m->wait_lock); +/* +* There may be cases where we re-clear already cleared +* blocked_on relationships, but make sure we are not +* clearing the relationship with a different lock. +*/ +WARN_ON_ONCE (blocked_on && blocked_on != m); + } +WRITE_ONCE (p->blocked_on,NULL ); }static inline void clear_task_blocked_on (struct task_struct *p,struct mutex *m)
kernel/locking/ww_mutex.h +5 -1
@@ -342,8 +342,12 @@ static bool __ww_mutex_wound(struct MUTEX *lock,* When waking up the task to wound, be sure to clear the * blocked_on pointer. Otherwise we can see circular * blocked_on relationships that can't resolve. +* +* NOTE: We pass NULL here instead of lock, because we +* are waking the mutex owner, who may be currently +* blocked on a different mutex. */ -__clear_task_blocked_on (owner, lock); +__clear_task_blocked_on (owner,NULL );wake_q_add (wake_q, owner); }return true;