Skip to contentSkip to conversation

sched: do not call __put_task_struct() on rt if pi_blocked_on is set

PATCH Superseded Luis Claudio R. Goncalves 3 participants 5 messages Apr 10, 2025 Real-time Linux (PREEMPT_RT)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 sched: do not call __put_task_struct() on rt if pi_blocked_on is set

With PREEMPT_RT enabled, some of the calls to put_task_struct() coming from rt_mutex_adjust_prio_chain() could happen in preemptible context and with a mutex enqueued. That could lead to this sequence:

	rt_mutex_adjust_prio_chain()
	  put_task_struct()
	    __put_task_struct()
	      sched_ext_free()
	        spin_lock_irqsave()
	          rtlock_lock() --->  TRIGGERS
	                              lockdep_assert(!current->pi_blocked_on);

Fix that by unconditionally resorting to the deferred call to __put_task_struct().

v2: (Rostedt) remove the #ifdef from put_task_struct() and create
tsk_is_pi_blocked_on() in sched.h to make the change cleaner.

v3: (Sebastian and PeterZ) always call the RCU deferred __put_task_struct().

---
 include/linux/sched/task.h |   20 +++++---------------
 1 file changed, 5 insertions(+), 15 deletions(-)
include/linux/sched/task.h +5 -15
index 0f2aeb37bbb04..49847efe5559e 100644
@@ -134,22 +134,12 @@ static inline void put_task_struct(struct task_struct *t)
 		return;
 
 	/*
-	 * In !RT, it is always safe to call __put_task_struct().
-	 * Under RT, we can only call it in preemptible context.
-	 */
-	if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) {
-		static DEFINE_WAIT_OVERRIDE_MAP(put_task_map, LD_WAIT_SLEEP);
-
-		lock_map_acquire_try(&put_task_map);
-		__put_task_struct(t);
-		lock_map_release(&put_task_map);
-		return;
-	}
-
-	/*
-	 * under PREEMPT_RT, we can't call put_task_struct
+	 * In !RT, it is always safe to call __put_task_struct(),
+	 * but under PREEMPT_RT, we can't call put_task_struct
 	 * in atomic context because it will indirectly
-	 * acquire sleeping locks.
+	 * acquire sleeping locks. The same is true if the
+	 * current process has a mutex enqueued (blocked on
+	 * a PI chain).
 	 *
 	 * call_rcu() will schedule delayed_put_task_struct_rcu()
 	 * to be called in process context.
Signature · 1 line
2.49.0
#1 lore raw
Quoted 21 lines · Luis Claudio R. Goncalves
include/linux/sched/task.h +0 -14
@@ -134,22 +134,12 @@ static inline void put_task_struct(struct task_struct *t)
 		return;
 
 	/*
-	 * In !RT, it is always safe to call __put_task_struct().
-	 * Under RT, we can only call it in preemptible context.
-	 */
-	if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) {
-		static DEFINE_WAIT_OVERRIDE_MAP(put_task_map, LD_WAIT_SLEEP);
-
-		lock_map_acquire_try(&put_task_map);
-		__put_task_struct(t);
-		lock_map_release(&put_task_map);
-		return;
-	}
-
-	/*
-	 * under PREEMPT_RT, we can't call put_task_struct

include/linux/sched/task.h static inline void put_task_struct(struct task_struct *t) · line 145

Quoted 10 lines
include/linux/sched/task.h +5 -1
+	 * In !RT, it is always safe to call __put_task_struct(),
+	 * but under PREEMPT_RT, we can't call put_task_struct
 	 * in atomic context because it will indirectly
-	 * acquire sleeping locks.
+	 * acquire sleeping locks. The same is true if the
+	 * current process has a mutex enqueued (blocked on
+	 * a PI chain).
 	 *
 	 * call_rcu() will schedule delayed_put_task_struct_rcu()
 	 * to be called in process context.

Did you test it with lockdep with and without PREEMPT_RT? It would be nice to throw some testing on it.
This comment here "call_rcu will schedule bla in process context" is wrong. It will schedule the callback in softirq context. Unless RCU is configured to run the callbacks in rcuc/ thread which is the default for PREEMPT_RT. Also delayed_put_task_struct_rcu() does not exist, imho never did.

Sebastian

#2 lore raw
Quoted 27 lines · Sebastian Andrzej Siewior
Quoted 26 lines · Luis Claudio R. Goncalves
include/linux/sched/task.h +3 -15
@@ -134,22 +134,12 @@ static inline void put_task_struct(struct task_struct *t)
 		return;
 
 	/*
-	 * In !RT, it is always safe to call __put_task_struct().
-	 * Under RT, we can only call it in preemptible context.
-	 */
-	if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) {
-		static DEFINE_WAIT_OVERRIDE_MAP(put_task_map, LD_WAIT_SLEEP);
-
-		lock_map_acquire_try(&put_task_map);
-		__put_task_struct(t);
-		lock_map_release(&put_task_map);
-		return;
-	}
-
-	/*
-	 * under PREEMPT_RT, we can't call put_task_struct
+	 * In !RT, it is always safe to call __put_task_struct(),
+	 * but under PREEMPT_RT, we can't call put_task_struct
 	 * in atomic context because it will indirectly
-	 * acquire sleeping locks.
+	 * acquire sleeping locks. The same is true if the
Quoted 8 lines

include/linux/sched/task.h static inline void put_task_struct(struct task_struct *t) · line 145

Quoted 5 lines
include/linux/sched/task.h +2 -0
+	 * current process has a mutex enqueued (blocked on
+	 * a PI chain).
 	 *
 	 * call_rcu() will schedule delayed_put_task_struct_rcu()
 	 * to be called in process context.

Did you test it with lockdep with and without PREEMPT_RT? It would be nice to throw some testing on it.

I will re-run the full set of tests on both kernels.

Quoted 5 lines

This comment here "call_rcu will schedule bla in process context" is wrong. It will schedule the callback in softirq context. Unless RCU is configured to run the callbacks in rcuc/ thread which is the default for PREEMPT_RT. Also delayed_put_task_struct_rcu() does not exist, imho never did.

I kept the original comment about the call_rcu in process context, but didn't realize that wouldn't hold true for !RT. Would you prefer I adjust the comments (for RT vs non-RT and other possibilities) or remove them entirely?

And I completely missed delayed_put_task_struct_rcu() vs __put_task_struct_rcu_cb() in the original comment.

Thank you again for the review!
Luis

Quoted 3 lines

Sebastian

---end quoted text---

#3 lore raw
In reply to Luis Claudio R. Goncalves
Quoted 12 lines · Luis Claudio R. Goncalves

With PREEMPT_RT enabled, some of the calls to put_task_struct() coming from rt_mutex_adjust_prio_chain() could happen in preemptible context and with a mutex enqueued. That could lead to this sequence:

	rt_mutex_adjust_prio_chain()
	  put_task_struct()
	    __put_task_struct()
	      sched_ext_free()
	        spin_lock_irqsave()
	          rtlock_lock() --->  TRIGGERS
	                              lockdep_assert(!current->pi_blocked_on);
Quoted 7 lines

Fix that by unconditionally resorting to the deferred call to __put_task_struct().

v2: (Rostedt) remove the #ifdef from put_task_struct() and create
tsk_is_pi_blocked_on() in sched.h to make the change cleaner.

v3: (Sebastian and PeterZ) always call the RCU deferred __put_task_struct().

Changelog goes below the --- line.

Quoted 15 lines
---
 include/linux/sched/task.h |   20 +++++---------------
 1 file changed, 5 insertions(+), 15 deletions(-)
include/linux/sched/task.h +0 -1
index 0f2aeb37bbb04..49847efe5559e 100644
@@ -134,22 +134,12 @@ static inline void put_task_struct(struct task_struct *t)
 		return;
 
 	/*
-	 * In !RT, it is always safe to call __put_task_struct().

include/linux/sched/task.h static inline void put_task_struct(struct task_struct *t) · line 147

Quoted 10 lines
include/linux/sched/task.h +0 -10
-	 * Under RT, we can only call it in preemptible context.
-	 */
-	if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) {
-		static DEFINE_WAIT_OVERRIDE_MAP(put_task_map, LD_WAIT_SLEEP);
-
-		lock_map_acquire_try(&put_task_map);
-		__put_task_struct(t);
-		lock_map_release(&put_task_map);
-		return;
-	}

I don't think you've substantiated why the !PREEMPT_RT case needs to go.

Quoted 16 lines
include/linux/sched/task.h +5 -4
-
-	/*
-	 * under PREEMPT_RT, we can't call put_task_struct
+	 * In !RT, it is always safe to call __put_task_struct(),
+	 * but under PREEMPT_RT, we can't call put_task_struct
 	 * in atomic context because it will indirectly
-	 * acquire sleeping locks.
+	 * acquire sleeping locks. The same is true if the
+	 * current process has a mutex enqueued (blocked on
+	 * a PI chain).
 	 *
 	 * call_rcu() will schedule delayed_put_task_struct_rcu()
 	 * to be called in process context.
Signature · 1 line
2.49.0
#4 lore raw
Quoted 36 lines · Peter Zijlstra
Quoted 12 lines · Luis Claudio R. Goncalves

With PREEMPT_RT enabled, some of the calls to put_task_struct() coming from rt_mutex_adjust_prio_chain() could happen in preemptible context and with a mutex enqueued. That could lead to this sequence:

	rt_mutex_adjust_prio_chain()
	  put_task_struct()
	    __put_task_struct()
	      sched_ext_free()
	        spin_lock_irqsave()
	          rtlock_lock() --->  TRIGGERS
	                              lockdep_assert(!current->pi_blocked_on);
Quoted 7 lines

Fix that by unconditionally resorting to the deferred call to __put_task_struct().

v2: (Rostedt) remove the #ifdef from put_task_struct() and create
tsk_is_pi_blocked_on() in sched.h to make the change cleaner.

v3: (Sebastian and PeterZ) always call the RCU deferred __put_task_struct().

Changelog goes below the --- line.

Quoted 13 lines
---
 include/linux/sched/task.h |   20 +++++---------------
 1 file changed, 5 insertions(+), 15 deletions(-)
include/linux/sched/task.h +0 -0
index 0f2aeb37bbb04..49847efe5559e 100644
@@ -134,22 +134,12 @@ static inline void put_task_struct(struct task_struct *t)
 		return;
 
Quoted 14 lines

include/linux/sched/task.h static inline void put_task_struct(struct task_struct *t) · line 147

Quoted 12 lines
include/linux/sched/task.h +0 -11
 	/*
-	 * In !RT, it is always safe to call __put_task_struct().
-	 * Under RT, we can only call it in preemptible context.
-	 */
-	if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) {
-		static DEFINE_WAIT_OVERRIDE_MAP(put_task_map, LD_WAIT_SLEEP);
-
-		lock_map_acquire_try(&put_task_map);
-		__put_task_struct(t);
-		lock_map_release(&put_task_map);
-		return;
-	}

I don't think you've substantiated why the !PREEMPT_RT case needs to go.

That was my misunderstanding of "unconditionally call the deferred function". I see I took it too far and made the patch wrong.

I am testing v4 (closer to the original code with fixed comments) that is basically:

	if !RT
		__put_task_struct (original code)
	else
		call_rcu(__put_task_struct_rcu_cb)

With the corrected comments Sebastian pointed out.

As soon as the tests complete I will post v4.

Thanks,
Luis

Quoted 8 lines
Quoted 8 lines
include/linux/sched/task.h +3 -4
-
-	/*
-	 * under PREEMPT_RT, we can't call put_task_struct
+	 * In !RT, it is always safe to call __put_task_struct(),
+	 * but under PREEMPT_RT, we can't call put_task_struct
 	 * in atomic context because it will indirectly
-	 * acquire sleeping locks.
+	 * acquire sleeping locks. The same is true if the
Quoted 9 lines
Quoted 8 lines
include/linux/sched/task.h +2 -0
+	 * current process has a mutex enqueued (blocked on
+	 * a PI chain).
 	 *
 	 * call_rcu() will schedule delayed_put_task_struct_rcu()
 	 * to be called in process context.
Signature · 1 line
2.49.0

---end quoted text---