Skip to contentSkip to conversation

sched: set TIF_NEED_RESCHED before calling __trace_set_need_resched()

PATCH Superseded Sechang Lim 3 participants 3 messages Jun 25 Scheduler

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched: set TIF_NEED_RESCHED before calling __trace_set_need_resched()

set_tsk_need_resched() tests TIF_NEED_RESCHED, calls __trace_set_need_resched() if the flag is clear, then sets it via set_tsk_thread_flag(). A BPF raw_tp program attached to sched_set_need_resched executes synchronously inside __bpf_trace_run(). On return, __bpf_trace_run() drops the RCU lock with rcu_read_unlock_migrate(), which on the preempt-or-BH-disabled path calls set_need_resched_current() -> set_tsk_need_resched() again.

set_tsk_thread_flag() follows the tracepoint call, so every re-entrant frame sees TIF_NEED_RESCHED clear and calls __trace_set_need_resched() again:

  BUG: TASK stack guard page was hit at ffffc9001224ff98
  Oops: stack guard page: 0000 [#1] SMP KASAN PTI
  RIP: 0010:__bpf_trace_sched_set_need_resched_tp+0x1c/0x190
  Call Trace:
   trace_sched_set_need_resched_tp+0x110/0x130
   set_tsk_need_resched include/linux/sched.h:2076
   set_need_resched_current include/linux/sched.h:2094
   rcu_read_unlock_special+0x43a/0x440
   __rcu_read_unlock+0x9e/0x120
   rcu_read_unlock_migrate+0xa9/0x240
   __bpf_trace_run+0x131/0x180
   bpf_trace_run3+0x333/0x430
   __bpf_trace_sched_set_need_resched_tp+0x13a/0x190
   trace_sched_set_need_resched_tp+0x110/0x130
   set_tsk_need_resched include/linux/sched.h:2076
   ...

Replace the separate test_tsk_thread_flag() + set_tsk_thread_flag() pair with test_and_set_tsk_thread_flag().

Fixes adcc3bfa8806 ("sched: Adapt sched tracepoints for RV task model")
---
 include/linux/sched.h | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
include/linux/sched.h +2 -3
index ee06cba5c6f5..c9efd08dae92 100644
@@ -2071,10 +2071,9 @@ static inline int test_tsk_thread_flag(struct task_struct *tsk, int flag)
 
 static inline void set_tsk_need_resched(struct task_struct *tsk)
 {
-	if (tracepoint_enabled(sched_set_need_resched_tp) &&
-	    !test_tsk_thread_flag(tsk, TIF_NEED_RESCHED))
+	if (!test_and_set_tsk_thread_flag(tsk, TIF_NEED_RESCHED) &&
+	    tracepoint_enabled(sched_set_need_resched_tp))
 		__trace_set_need_resched(tsk, TIF_NEED_RESCHED);
-	set_tsk_thread_flag(tsk,TIF_NEED_RESCHED);
 }
 
 static inline void clear_tsk_need_resched(struct task_struct *tsk)
Signature · 1 line
2.43.0
maintainer
#1 lore raw
Quoted 43 lines · Sechang Lim

set_tsk_need_resched() tests TIF_NEED_RESCHED, calls __trace_set_need_resched() if the flag is clear, then sets it via set_tsk_thread_flag(). A BPF raw_tp program attached to sched_set_need_resched executes synchronously inside __bpf_trace_run(). On return, __bpf_trace_run() drops the RCU lock with rcu_read_unlock_migrate(), which on the preempt-or-BH-disabled path calls set_need_resched_current() -> set_tsk_need_resched() again.

set_tsk_thread_flag() follows the tracepoint call, so every re-entrant frame sees TIF_NEED_RESCHED clear and calls __trace_set_need_resched() again:

  BUG: TASK stack guard page was hit at ffffc9001224ff98
  Oops: stack guard page: 0000 [#1] SMP KASAN PTI
  RIP: 0010:__bpf_trace_sched_set_need_resched_tp+0x1c/0x190
  Call Trace:
   trace_sched_set_need_resched_tp+0x110/0x130
   set_tsk_need_resched include/linux/sched.h:2076
   set_need_resched_current include/linux/sched.h:2094
   rcu_read_unlock_special+0x43a/0x440
   __rcu_read_unlock+0x9e/0x120
   rcu_read_unlock_migrate+0xa9/0x240
   __bpf_trace_run+0x131/0x180
   bpf_trace_run3+0x333/0x430
   __bpf_trace_sched_set_need_resched_tp+0x13a/0x190
   trace_sched_set_need_resched_tp+0x110/0x130
   set_tsk_need_resched include/linux/sched.h:2076
   ...

Replace the separate test_tsk_thread_flag() + set_tsk_thread_flag() pair with test_and_set_tsk_thread_flag().

Fixes adcc3bfa8806 ("sched: Adapt sched tracepoints for RV task model")
---
 include/linux/sched.h | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
include/linux/sched.h +0 -0
index ee06cba5c6f5..c9efd08dae92 100644

include/linux/sched.h static inline int test_tsk_thread_flag(struct task_struct *tsk, int flag) · line 2077

Quoted 10 lines
include/linux/sched.h +2 -3
@@ -2071,10 +2071,9 @@ static inline int test_tsk_thread_flag(struct task_struct *tsk, int flag)
 
 static inline void set_tsk_need_resched(struct task_struct *tsk)
 {
-	if (tracepoint_enabled(sched_set_need_resched_tp) &&
-	    !test_tsk_thread_flag(tsk, TIF_NEED_RESCHED))
+	if (!test_and_set_tsk_thread_flag(tsk, TIF_NEED_RESCHED) &&
+	    tracepoint_enabled(sched_set_need_resched_tp))
 		__trace_set_need_resched(tsk, TIF_NEED_RESCHED);
-	set_tsk_thread_flag(tsk,TIF_NEED_RESCHED);
 }

__resched_curr() does the same, no?

Also, did you check if the RV model perhaps relies on the TIF bit not being set when the tracepoint is tripped?

#2 lore raw
Quoted 55 lines · Peter Zijlstra
Quoted 54 lines · Sechang Lim

set_tsk_need_resched() tests TIF_NEED_RESCHED, calls __trace_set_need_resched() if the flag is clear, then sets it via set_tsk_thread_flag().  A BPF raw_tp program attached to sched_set_need_resched executes synchronously inside __bpf_trace_run().
On return, __bpf_trace_run() drops the RCU lock with rcu_read_unlock_migrate(), which on the preempt-or-BH-disabled path calls set_need_resched_current() -> set_tsk_need_resched() again.

set_tsk_thread_flag() follows the tracepoint call, so every re- entrant
frame sees TIF_NEED_RESCHED clear and calls __trace_set_need_resched()
again:

  BUG: TASK stack guard page was hit at ffffc9001224ff98
  Oops: stack guard page: 0000 [#1] SMP KASAN PTI
  RIP: 0010:__bpf_trace_sched_set_need_resched_tp+0x1c/0x190
  Call Trace:
   trace_sched_set_need_resched_tp+0x110/0x130
   set_tsk_need_resched include/linux/sched.h:2076
   set_need_resched_current include/linux/sched.h:2094
   rcu_read_unlock_special+0x43a/0x440
   __rcu_read_unlock+0x9e/0x120
   rcu_read_unlock_migrate+0xa9/0x240
   __bpf_trace_run+0x131/0x180
   bpf_trace_run3+0x333/0x430
   __bpf_trace_sched_set_need_resched_tp+0x13a/0x190
   trace_sched_set_need_resched_tp+0x110/0x130
   set_tsk_need_resched include/linux/sched.h:2076

   ...

Replace the separate test_tsk_thread_flag() + set_tsk_thread_flag() pair
with test_and_set_tsk_thread_flag().

Fixes adcc3bfa8806 ("sched: Adapt sched tracepoints for RV task

model")

---
 include/linux/sched.h | 5 ++---

 1 file changed, 2 insertions(+), 3 deletions(-)
include/linux/sched.h +0 -0
index ee06cba5c6f5..c9efd08dae92 100644
@@ -2071,10 +2071,9 @@ static inline int
 test_tsk_thread_flag(struct task_struct *tsk, int flag)


 static inline void set_tsk_need_resched(struct task_struct *tsk) {

include/linux/sched.h +0 -2
-	if (tracepoint_enabled(sched_set_need_resched_tp) &&
-	    !test_tsk_thread_flag(tsk, TIF_NEED_RESCHED))
Quoted 8 lines
Quoted 6 lines
include/linux/sched.h +1 -0
+	if (!test_and_set_tsk_thread_flag(tsk, TIF_NEED_RESCHED)

&&

include/linux/sched.h +1 -0
+	    tracepoint_enabled(sched_set_need_resched_tp))

  __trace_set_need_resched(tsk, TIF_NEED_RESCHED);

include/linux/sched.h +0 -1
-	set_tsk_thread_flag(tsk,TIF_NEED_RESCHED);

 }

__resched_curr() does the same, no?

I think it wouldn't make the same re-entrant call but it's probably a good idea to make it consistent and have the flag set before the tracepoint every time.

Quoted 2 lines

Also, did you check if the RV model perhaps relies on the TIF bit not being set when the tracepoint is tripped?

It does not, we just check the tif passed to the tracepoint and not the one in the task, the assumption here is that it is set during that exact tracepoint (before or after doesn't matter, but it would matter if it was set on a previous event).
Depending on what handlers do, this may become a problem for RV as well.

Thanks,
Gabriele