tracing: Have osnoise_main() add a quiescent state for task rcu
Subsystems (2)
Steven Rostedt
maintainerPATCH 1/1 tracing: Have osnoise_main() add a quiescent state for task rcu
From: "Steven Rostedt (VMware)" <[email protected]>
ftracetest triggered:
INFO: rcu_tasks detected stalls on tasks: 00000000b92b832d: .. nvcsw: 1/1 holdout: 1 idle_cpu: -1/7 task:osnoise/7 state:R running task stack: 0 pid: 2133 ppid: 2 flags:0x00004000 Call Trace: ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? trace_hardirqs_on+0x2b/0xe0 ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? trace_clock_local+0xc/0x20 ? osnoise_main+0x10e/0x450 ? trace_softirq_entry_callback+0x50/0x50 ? kthread+0x153/0x170 ? __kthread_bind_mask+0x60/0x60 ? ret_from_fork+0x22/0x30
While running osnoise tracer with other tracers that rely on synchronize_rcu_tasks(), where that just hung.
The reason is that osnoise_main() never schedules out if the interval is less than 1, and this will cause synchronize_rcu_tasks() to never return.
--- kernel/trace/trace_osnoise.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/trace/trace_osnoise.c +4 -1
@@ -1216,8 +1216,11 @@ static int osnoise_main(void *data)* differently from hwlat_detector, the osnoise tracer can run * without a pause because preemption is on. */ -if (interval <1 ) +if (interval <1 ) { +/* Let synchronize_rcu_tasks() make progress */ +cond_resched_tasks_rcu_qs ();continue ; + }if (msleep_interruptible (interval))break ;
Signature · 1 line
2.29.2
Quoted 22 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
ftracetest triggered:
INFO: rcu_tasks detected stalls on tasks: 00000000b92b832d: .. nvcsw: 1/1 holdout: 1 idle_cpu: -1/7 task:osnoise/7 state:R running task stack: 0 pid: 2133 ppid: 2 flags:0x00004000 Call Trace: ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? trace_hardirqs_on+0x2b/0xe0 ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? trace_clock_local+0xc/0x20 ? osnoise_main+0x10e/0x450 ? trace_softirq_entry_callback+0x50/0x50 ? kthread+0x153/0x170 ? __kthread_bind_mask+0x60/0x60 ? ret_from_fork+0x22/0x30While running osnoise tracer with other tracers that rely on
synchronize_rcu_tasks(), where that just hung.
Quoted 6 lines
The reason is that
osnoise_main()never schedules out if the interval is less than 1, and this will causesynchronize_rcu_tasks()to never return.Fixes bce29ac9ce0bb ("trace: Add osnoise tracer")Signed-off-by Steven Rostedt (VMware) <[email protected]>
That looks like something I would do!
Quoted 24 lines
--- kernel/trace/trace_osnoise.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)kernel/trace/trace_osnoise.c +4
-1@@ -1216,8 +1216,11 @@ static int osnoise_main(void *data)* differently from hwlat_detector, the osnoise tracer can run * without a pause because preemption is on. */ -if (interval <1 ) +if (interval <1 ) { +/* Let synchronize_rcu_tasks() make progress */ +cond_resched_tasks_rcu_qs ();continue ; + }if (msleep_interruptible (interval))break ;Signature · 1 line
2.29.2
Quoted 19 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
ftracetest triggered:
INFO: rcu_tasks detected stalls on tasks: 00000000b92b832d: .. nvcsw: 1/1 holdout: 1 idle_cpu: -1/7 task:osnoise/7 state:R running task stack: 0 pid: 2133 ppid: 2 flags:0x00004000 Call Trace: ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? trace_hardirqs_on+0x2b/0xe0 ? asm_sysvec_apic_timer_interrupt+0x12/0x20 ? trace_clock_local+0xc/0x20 ? osnoise_main+0x10e/0x450 ? trace_softirq_entry_callback+0x50/0x50 ? kthread+0x153/0x170 ? __kthread_bind_mask+0x60/0x60 ? ret_from_fork+0x22/0x30
Quoted 5 lines
While running osnoise tracer with other tracers that rely on
synchronize_rcu_tasks(), where that just hung.The reason is that
osnoise_main()never schedules out if the interval is less than 1,
and there is no other task setting need resched (it has a regular cond_resched() on the run_osnoise() loop).
and this will cause synchronize_rcu_tasks() to never
Quoted 4 lines
return.
Fixes bce29ac9ce0bb ("trace: Add osnoise tracer")Signed-off-by Steven Rostedt (VMware) <[email protected]>
Thanks!
-- Daniel
Quoted 22 lines
--- kernel/trace/trace_osnoise.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)kernel/trace/trace_osnoise.c +4
-1@@ -1216,8 +1216,11 @@ static int osnoise_main(void *data)* differently from hwlat_detector, the osnoise tracer can run * without a pause because preemption is on. */ -if (interval <1 ) +if (interval <1 ) { +/* Let synchronize_rcu_tasks() make progress */ +cond_resched_tasks_rcu_qs ();continue ; + }if (msleep_interruptible (interval))break ;