cpuidle/rcu: Making arch_cpu_idle and rcu_idle_exit noinstr
Subsystems (10)
Jiri Olsa
reviewerPATCH 1/2 cpuidle/rcu: Making arch_cpu_idle and rcu_idle_exit noinstr
Making arch_cpu_idle and rcu_idle_exit noinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:
[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0
The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exit
The arch_cpu_idle and rcu_idle_exit are the only ones from above path that are traceble and cause this problem on my setup.
--- arch/x86/kernel/process.c | 2 +- kernel/rcu/tree.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/kernel/process.c +1 -1
@@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)/* * Called from the generic idle code. */ -void arch_cpu_idle (void ) +void noinstr arch_cpu_idle(void) {x86_idle (); }
kernel/rcu/tree.c +1 -1
@@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags;
Signature · 1 line
2.35.3
Jiri Olsa
reviewerPATCH 2/2 selftests/bpf: Remove filter for unsafe functions in kprobe_multi test
Removing filter for arch_cpu_idle and rcu_* functions.
Attaching to them was causing RCU warnings [1]:
[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0
With previous fix the functions causing that are no longer attachable.
[1] https://lore.kernel.org/bpf/CAEf4BzYtXWvBWzmadhLGqwf8_e2sruK6999th6c=b=O0WLkHOA@mail.gmail.com/
--- .../selftests/bpf/prog_tests/kprobe_multi_test.c | 10 ---------- 1 file changed, 10 deletions(-)
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +0 -10
@@ -354,16 +354,6 @@ static int get_syms(char ***symsp, size_t *cntp)continue ;if (sscanf (buf,"%ms$*[^\n]\n" , &name) !=1 )continue ; -/* -* We attach to almost all kernel functions and some of them -* will cause 'suspicious RCU usage' when fprobe is attached -* to them. Filter out the current culprits - arch_cpu_idle -* and rcu_* functions. -*/ -if (!strcmp (name,"arch_cpu_idle" )) -continue ; -if (!strncmp (name,"rcu_" ,4 )) -continue ; err =hashmap__add (map, name,NULL );if (err) {free (name);
Signature · 1 line
2.35.3
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exit
Quoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>
From an RCU viewpoint:
[ I considered asking for an instrumentation_on() in rcu_idle_exit(), but there is no point given that local_irq_restore() isn't something you instrument anyway. ]
Quoted 34 lines
--- arch/x86/kernel/process.c | 2 +- kernel/rcu/tree.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/process.c +1
-1@@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)/* * Called from the generic idle code. */ -void arch_cpu_idle (void ) +void noinstr arch_cpu_idle(void) {x86_idle (); }kernel/rcu/tree.c +1
-1@@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags;Signature · 1 line
2.35.3
Frederic Weisbecker
maintainerQuoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>
Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]
So local_irq_save() in the beginning of rcu_idle_exit() is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.
Also it calls into lockdep that might make use of RCU.
That's why rcu_idle_exit() is not noinstr yet. See this patch:
https://lore.kernel.org/lkml/[email protected]/
Thanks.
Quoted 35 lines
Quoted 35 lines
--- arch/x86/kernel/process.c | 2 +- kernel/rcu/tree.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/process.c +1
-1@@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)/* * Called from the generic idle code. */ -void arch_cpu_idle (void ) +void noinstr arch_cpu_idle(void) {x86_idle (); }kernel/rcu/tree.c +1
-1@@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags;Signature · 1 line
2.35.3
Paul E. McKenney
maintainerQuoted 40 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]
Quoted 8 lines
So
local_irq_save()in the beginning ofrcu_idle_exit()is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.Also it calls into lockdep that might make use of RCU.
That's why
rcu_idle_exit()is not noinstr yet. See this patch:
Ah, I should have looked at the context-tracking series again!
And I have to ask... How much debugging capability are we really losing by continuing to use the raw versions of local_irq_{save,restore}()?
Thanx, Paul
Quoted 37 lines
Thanks.
Quoted 35 lines
Quoted 35 lines
--- arch/x86/kernel/process.c | 2 +- kernel/rcu/tree.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/process.c +1
-1@@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)/* * Called from the generic idle code. */ -void arch_cpu_idle (void ) +void noinstr arch_cpu_idle(void) {x86_idle (); }kernel/rcu/tree.c +1
-1@@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags;Signature · 1 line
2.35.3
Quoted 40 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]
Quoted 8 lines
So
local_irq_save()in the beginning ofrcu_idle_exit()is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.Also it calls into lockdep that might make use of RCU.
That's why
rcu_idle_exit()is not noinstr yet. See this patch:
I see, could we mark it at least with notrace meanwhile?
jirka
Quoted 44 lines · Jiri Olsa
Quoted 43 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]So
local_irq_save()in the beginning ofrcu_idle_exit()is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.
Quoted 7 lines
Quoted 5 lines
Also it calls into lockdep that might make use of RCU.
That's why
rcu_idle_exit()is not noinstr yet. See this patch:I see, could we mark it at least with notrace meanwhile?
For the RCU part, how about as follows?
If this approach is reasonable, my guess would be that Frederic will pull it into his context-tracking series, perhaps using a revert of this patch to maintain sanity in the near term.
If this approach is unreasonable, well, that is Murphy for you!
For the x86 idle part, my feeling is still that the rcu_idle_enter() and rcu_idle_exit() need to be pushed deeper into the code. Perhaps an ongoing process as the idle loop continues to be dug deeper?
Thanx, Paul
------------------------------------------------------------------------
commit cd338be719a0a692e0d50e1a8438e1f6c7165d9c
Author: Paul E. McKenney <[email protected]> Date: Tue May 17 21:00:04 2022 -0700
rcu: Apply noinstr to rcu_idle_enter() and rcu_idle_exit()
This commit applies the "noinstr" tag to the rcu_idle_enter() and rcu_idle_exit() functions, which are invoked from portions of the idle loop that cannot be instrumented. These tags require reworking the rcu_eqs_enter() and rcu_eqs_exit() functions that these two functions invoke in order to cause them to use normal assertions rather than lockdep. In addition, within rcu_idle_exit(), the raw versions of local_irq_save() and local_irq_restore() are used, again to avoid issues with lockdep in uninstrumented code.
This patch is based in part on an earlier patch by Jiri Olsa, discussions with Peter Zijlstra and Frederic Weisbecker, earlier changes by Thomas Gleixner, and off-list discussions with Yonghong Song.
kernel/rcu/tree.c +7 -7
@@ -635,8 +635,8 @@ static noinstr void rcu_eqs_enter(bool user)return ; } -lockdep_assert_irqs_disabled ();instrumentation_begin (); +lockdep_assert_irqs_disabled ();trace_rcu_dyntick (TPS ("Start" ), rdp->dynticks_nesting,0 ,atomic_read (&rdp->dynticks));WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !user && !is_idle_task (current));rcu_preempt_deferred_qs (current);
@@ -663,9 +663,9 @@ static noinstr void rcu_eqs_enter(bool user)* If you add or remove a call to rcu_idle_enter(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_enter (void ) +void noinstr rcu_idle_enter(void) { -lockdep_assert_irqs_disabled (); +WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !raw_irqs_disabled ());rcu_eqs_enter (false); }EXPORT_SYMBOL_GPL (rcu_idle_enter);
@@ -865,7 +865,7 @@ static void noinstr rcu_eqs_exit(bool user)struct rcu_data *rdp;long oldval; -lockdep_assert_irqs_disabled (); +WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !raw_irqs_disabled ()); rdp =this_cpu_ptr (&rcu_data); oldval = rdp->dynticks_nesting;WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && oldval <0 );
@@ -900,13 +900,13 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags; -local_irq_save (flags); +raw_local_irq_save (flags);rcu_eqs_exit (false); -local_irq_restore (flags); +raw_local_irq_restore (flags); }EXPORT_SYMBOL_GPL (rcu_idle_exit);
Quoted 53 lines · Paul E. McKenney
Quoted 44 lines · Jiri Olsa
Quoted 43 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]So
local_irq_save()in the beginning ofrcu_idle_exit()is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.Quoted 7 lines
Quoted 5 lines
Also it calls into lockdep that might make use of RCU.
That's why
rcu_idle_exit()is not noinstr yet. See this patch:I see, could we mark it at least with notrace meanwhile?
Quoted 7 lines
For the RCU part, how about as follows?
If this approach is reasonable, my guess would be that Frederic will pull it into his context-tracking series, perhaps using a revert of this patch to maintain sanity in the near term.
If this approach is unreasonable, well, that is Murphy for you!
I checked and it works in my test ;-)
Quoted 4 lines
For the x86 idle part, my feeling is still that the
rcu_idle_enter()andrcu_idle_exit()need to be pushed deeper into the code. Perhaps an ongoing process as the idle loop continues to be dug deeper?
for arch_cpu_idle with noinstr I'm getting this W=1 warning:
vmlinux.o: warning: objtool: arch_cpu_idle()+0xb: call to {dynamic}() leaves .noinstr.text section
we could have it with notrace if that's a problem
thanks,
jirka
Quoted 83 lines
Thanx, Paul
------------------------------------------------------------------------
commit
cd338be719a0a692e0d50e1a8438e1f6c7165d9cAuthor: Paul E. McKenney <[email protected]> Date: Tue May 17 21:00:04 2022 -0700rcu: Apply noinstr to
rcu_idle_enter()andrcu_idle_exit()This commit applies the "noinstr" tag to the
rcu_idle_enter()andrcu_idle_exit()functions, which are invoked from portions of the idle loop that cannot be instrumented. These tags require reworking thercu_eqs_enter()andrcu_eqs_exit()functions that these two functions invoke in order to cause them to use normal assertions rather than lockdep. In addition, withinrcu_idle_exit(), the raw versions oflocal_irq_save()andlocal_irq_restore()are used, again to avoid issues with lockdep in uninstrumented code.This patch is based in part on an earlier patch by Jiri Olsa, discussions with Peter Zijlstra and Frederic Weisbecker, earlier changes by Thomas Gleixner, and off-list discussions with Yonghong Song.
Reported-by Jiri Olsa <[email protected]>Reported-by Alexei Starovoitov <[email protected]>Reported-by Andrii Nakryiko <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>Reviewed-by Yonghong Song <[email protected]>kernel/rcu/tree.c +7
-7@@ -635,8 +635,8 @@ static noinstr void rcu_eqs_enter(bool user)return ; } -lockdep_assert_irqs_disabled ();instrumentation_begin (); +lockdep_assert_irqs_disabled ();trace_rcu_dyntick (TPS ("Start" ), rdp->dynticks_nesting,0 ,atomic_read (&rdp->dynticks));WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !user && !is_idle_task (current));rcu_preempt_deferred_qs (current);@@ -663,9 +663,9 @@ static noinstr void rcu_eqs_enter(bool user)* If you add or remove a call to rcu_idle_enter(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_enter (void ) +void noinstr rcu_idle_enter(void) { -lockdep_assert_irqs_disabled (); +WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !raw_irqs_disabled ());rcu_eqs_enter (false); }EXPORT_SYMBOL_GPL (rcu_idle_enter);@@ -865,7 +865,7 @@ static void noinstr rcu_eqs_exit(bool user)struct rcu_data *rdp;long oldval; -lockdep_assert_irqs_disabled (); +WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !raw_irqs_disabled ()); rdp =this_cpu_ptr (&rcu_data); oldval = rdp->dynticks_nesting;WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && oldval <0 );@@ -900,13 +900,13 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags; -local_irq_save (flags); +raw_local_irq_save (flags);rcu_eqs_exit (false); -local_irq_restore (flags); +raw_local_irq_restore (flags); }EXPORT_SYMBOL_GPL (rcu_idle_exit);
Paul E. McKenney
maintainerQuoted 56 lines · Jiri Olsa
Quoted 55 lines · Paul E. McKenney
Quoted 44 lines · Jiri Olsa
Quoted 43 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]So
local_irq_save()in the beginning ofrcu_idle_exit()is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.Quoted 7 lines
Quoted 5 lines
Also it calls into lockdep that might make use of RCU.
That's why
rcu_idle_exit()is not noinstr yet. See this patch:I see, could we mark it at least with notrace meanwhile?
For the RCU part, how about as follows?
Quoted 7 lines
Quoted 5 lines
If this approach is reasonable, my guess would be that Frederic will pull it into his context-tracking series, perhaps using a revert of this patch to maintain sanity in the near term.
If this approach is unreasonable, well, that is Murphy for you!
I checked and it works in my test ;-)
Whew!!! One piece of the problem might be solved, then. ;-)
Quoted 9 lines
Quoted 3 lines
For the x86 idle part, my feeling is still that the
rcu_idle_enter()andrcu_idle_exit()need to be pushed deeper into the code. Perhaps an ongoing process as the idle loop continues to be dug deeper?for
arch_cpu_idlewith noinstr I'm getting this W=1 warning:vmlinux.o: warning: objtool: arch_cpu_idle()+0xb: call to {dynamic}() leaves .noinstr.text section
we could have it with notrace if that's a problem
I would be happy to queue the arch_cpu_idle() portion of your patch on -rcu, if that would move things forward. I suspect that additional x86_idle() surgery is required, but maybe I am just getting confused about what the x86_idle() function pointer can point to. But it looks to me like these need further help:
o static void amd_e400_idle(void)
Plus things it calls, like tick_broadcast_enter() and tick_broadcast_exit().
o static __cpuidle void mwait_idle(void)
So it might not be all that much additional work, even if I have avoided confusion about what the x86_idle() function pointer can point to. But I do not trust my ability to test this accurately.
Thoughts?
Thanx, Paul
Quoted 86 lines
thanks,
jirkaQuoted 83 lines
Thanx, Paul
------------------------------------------------------------------------
commit
cd338be719a0a692e0d50e1a8438e1f6c7165d9cAuthor: Paul E. McKenney <[email protected]> Date: Tue May 17 21:00:04 2022 -0700rcu: Apply noinstr to
rcu_idle_enter()andrcu_idle_exit()This commit applies the "noinstr" tag to the
rcu_idle_enter()andrcu_idle_exit()functions, which are invoked from portions of the idle loop that cannot be instrumented. These tags require reworking thercu_eqs_enter()andrcu_eqs_exit()functions that these two functions invoke in order to cause them to use normal assertions rather than lockdep. In addition, withinrcu_idle_exit(), the raw versions oflocal_irq_save()andlocal_irq_restore()are used, again to avoid issues with lockdep in uninstrumented code.This patch is based in part on an earlier patch by Jiri Olsa, discussions with Peter Zijlstra and Frederic Weisbecker, earlier changes by Thomas Gleixner, and off-list discussions with Yonghong Song.
Reported-by Jiri Olsa <[email protected]>Reported-by Alexei Starovoitov <[email protected]>Reported-by Andrii Nakryiko <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>Reviewed-by Yonghong Song <[email protected]>kernel/rcu/tree.c +7
-7@@ -635,8 +635,8 @@ static noinstr void rcu_eqs_enter(bool user)return ; } -lockdep_assert_irqs_disabled ();instrumentation_begin (); +lockdep_assert_irqs_disabled ();trace_rcu_dyntick (TPS ("Start" ), rdp->dynticks_nesting,0 ,atomic_read (&rdp->dynticks));WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !user && !is_idle_task (current));rcu_preempt_deferred_qs (current);@@ -663,9 +663,9 @@ static noinstr void rcu_eqs_enter(bool user)* If you add or remove a call to rcu_idle_enter(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_enter (void ) +void noinstr rcu_idle_enter(void) { -lockdep_assert_irqs_disabled (); +WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !raw_irqs_disabled ());rcu_eqs_enter (false); }EXPORT_SYMBOL_GPL (rcu_idle_enter);@@ -865,7 +865,7 @@ static void noinstr rcu_eqs_exit(bool user)struct rcu_data *rdp;long oldval; -lockdep_assert_irqs_disabled (); +WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !raw_irqs_disabled ()); rdp =this_cpu_ptr (&rcu_data); oldval = rdp->dynticks_nesting;WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && oldval <0 );@@ -900,13 +900,13 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags; -local_irq_save (flags); +raw_local_irq_save (flags);rcu_eqs_exit (false); -local_irq_restore (flags); +raw_local_irq_restore (flags); }EXPORT_SYMBOL_GPL (rcu_idle_exit);
Quoted 87 lines · Paul E. McKenney
Quoted 56 lines · Jiri Olsa
Quoted 55 lines · Paul E. McKenney
Quoted 44 lines · Jiri Olsa
Quoted 43 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]So
local_irq_save()in the beginning ofrcu_idle_exit()is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.Quoted 7 lines
Quoted 5 lines
Also it calls into lockdep that might make use of RCU.
That's why
rcu_idle_exit()is not noinstr yet. See this patch:I see, could we mark it at least with notrace meanwhile?
For the RCU part, how about as follows?
Quoted 7 lines
Quoted 5 lines
If this approach is reasonable, my guess would be that Frederic will pull it into his context-tracking series, perhaps using a revert of this patch to maintain sanity in the near term.
If this approach is unreasonable, well, that is Murphy for you!
I checked and it works in my test ;-)
Whew!!! One piece of the problem might be solved, then. ;-)
Quoted 9 lines
Quoted 3 lines
For the x86 idle part, my feeling is still that the
rcu_idle_enter()andrcu_idle_exit()need to be pushed deeper into the code. Perhaps an ongoing process as the idle loop continues to be dug deeper?for
arch_cpu_idlewith noinstr I'm getting this W=1 warning:vmlinux.o: warning: objtool: arch_cpu_idle()+0xb: call to {dynamic}() leaves .noinstr.text section
we could have it with notrace if that's a problem
I would be happy to queue the
arch_cpu_idle()portion of your patch on -rcu, if that would move things forward. I suspect that additionalx86_idle()surgery is required, but maybe I am just getting confused about what thex86_idle()function pointer can point to. But it looks to me like these need further help:o static void
amd_e400_idle(void)
Plus things it calls, liketick_broadcast_enter()andtick_broadcast_exit().
Quoted 5 lines
o static
__cpuidlevoidmwait_idle(void)So it might not be all that much additional work, even if I have avoided confusion about what the
x86_idle()function pointer can point to. But I do not trust my ability to test this accurately.
same here ;-) you're right, there will be other places based on x86_idle function pointer.. I'll check it, but perhaps we could address that when someone reports that
jirka
Paul E. McKenney
maintainerQuoted 94 lines · Jiri Olsa
Quoted 87 lines · Paul E. McKenney
Quoted 56 lines · Jiri Olsa
Quoted 55 lines · Paul E. McKenney
Quoted 44 lines · Jiri Olsa
Quoted 43 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 25 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitQuoted 4 lines
The
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>Quoted 7 lines
From an RCU viewpoint:
Reviewed-by Paul E. McKenney <[email protected]>[ I considered asking for an
instrumentation_on()inrcu_idle_exit(), but there is no point given thatlocal_irq_restore()isn't something you instrument anyway. ]So
local_irq_save()in the beginning ofrcu_idle_exit()is unsafe because it is instrumentable by the function (graph) tracers and the irqsoff tracer.Quoted 7 lines
Quoted 5 lines
Also it calls into lockdep that might make use of RCU.
That's why
rcu_idle_exit()is not noinstr yet. See this patch:I see, could we mark it at least with notrace meanwhile?
For the RCU part, how about as follows?
Quoted 7 lines
Quoted 5 lines
If this approach is reasonable, my guess would be that Frederic will pull it into his context-tracking series, perhaps using a revert of this patch to maintain sanity in the near term.
If this approach is unreasonable, well, that is Murphy for you!
I checked and it works in my test ;-)
Whew!!! One piece of the problem might be solved, then. ;-)
Quoted 9 lines
Quoted 3 lines
For the x86 idle part, my feeling is still that the
rcu_idle_enter()andrcu_idle_exit()need to be pushed deeper into the code. Perhaps an ongoing process as the idle loop continues to be dug deeper?for
arch_cpu_idlewith noinstr I'm getting this W=1 warning:vmlinux.o: warning: objtool: arch_cpu_idle()+0xb: call to {dynamic}() leaves .noinstr.text section
we could have it with notrace if that's a problem
I would be happy to queue the
arch_cpu_idle()portion of your patch on -rcu, if that would move things forward. I suspect that additionalx86_idle()surgery is required, but maybe I am just getting confused about what thex86_idle()function pointer can point to. But it looks to me like these need further help:o static void
amd_e400_idle(void)
Plus things it calls, liketick_broadcast_enter()andtick_broadcast_exit().Quoted 5 lines
o static
__cpuidlevoidmwait_idle(void)So it might not be all that much additional work, even if I have avoided confusion about what the
x86_idle()function pointer can point to. But I do not trust my ability to test this accurately.
Quoted 5 lines
same here ;-) you're right, there will be other places based on
x86_idlefunction pointer.. I'll check it, but perhaps we could address that when someone reports thatjirka
Any thoughts on the correct approach? One extreme would be to mark all sorts of things noinstr. Another extreme would be to enclose all sorts of things in RCU_NONIDLE(). Yet another extreme would be to push the rcu_idle_enter() and rcu_idle_exit() calls still deeper into the idle loop.
Or does Peter's recent series somehow cover all of this?
https://lore.kernel.org/all/[email protected]/
Thanx, Paul
kernel test robot
build botI love your patch! Perhaps something to improve:
Hi Jiri,
I love your patch! Perhaps something to improve:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Jiri-Olsa/cpuidle-rcu-Making-arch_cpu_idle-and-rcu_idle_exit-noinstr/20220516-043752 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master config: x86_64-randconfig-c002-20220516 (https://download.01.org/0day-ci/archive/20220517/[email protected]/config) compiler: gcc-11 (Debian 11.2.0-20) 11.2.0 reproduce (this is a W=1 build): # https://github.com/intel-lab-lkp/linux/commit/0b6fee32d730f621f2bfc4d8d9f0729814398415 git remote add linux-review https://github.com/intel-lab-lkp/linux git fetch --no-tags linux-review Jiri-Olsa/cpuidle-rcu-Making-arch_cpu_idle-and-rcu_idle_exit-noinstr/20220516-043752 git checkout 0b6fee32d730f621f2bfc4d8d9f0729814398415 # save the config file mkdir build_dir && cp config build_dir/.config make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
>> vmlinux.o: warning: objtool: arch_cpu_idle()+0x0: call to {dynamic}() leaves .noinstr.text section
>> vmlinux.o: warning: objtool: rcu_idle_exit()+0x16: call to trace_hardirqs_off() leaves .noinstr.text section vmlinux.o: warning: objtool: enter_from_user_mode()+0x18: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: syscall_enter_from_user_mode()+0x1d: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: syscall_enter_from_user_mode_prepare()+0x18: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_enter_from_user_mode()+0x18: call to __kcsan_check_access() leaves .noinstr.text section
Signature · 2 lines
0-DAY CI Kernel Test Service https://01.org/lkp
Quoted 39 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitThe
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>--- arch/x86/kernel/process.c | 2 +- kernel/rcu/tree.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/process.c +0
-0
arch/x86/kernel/process.c void arch_cpu_idle_dead(void) · line 723
Quoted 5 lines
arch/x86/kernel/process.c +1
-1@@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)/* * Called from the generic idle code. */ -void arch_cpu_idle (void ) +void noinstr arch_cpu_idle(void)
noinstr includes a lot of attributes:
#define noinstr \
noinline notrace __attribute((__section__(".noinstr.text"))) \
__no_kcsan __no_sanitize_address __no_profile__no_sanitize_coverage
should we use notrace here?
Quoted 16 lines
arch/x86/kernel/process.c +0
-0{x86_idle (); }kernel/rcu/tree.c +1
-1@@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags;
Quoted 50 lines · Yonghong Song
Quoted 39 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitThe
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>--- arch/x86/kernel/process.c | 2 +- kernel/rcu/tree.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/process.c +0
-0arch/x86/kernel/process.c void arch_cpu_idle_dead(void) · line 723
Quoted 5 lines
arch/x86/kernel/process.c +1
-1@@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)/* * Called from the generic idle code. */ -void arch_cpu_idle (void ) +void noinstr arch_cpu_idle(void)noinstr includes a lot of attributes:
Quoted 5 lines
#define noinstr \
noinline notrace __attribute((__section__(".noinstr.text"))) \
__no_kcsan__no_sanitize_address__no_profile__no_sanitize_coverageshould we use notrace here?
hm right, so notrace should be enough for our case (kprobe_multi) which is based on ftrace/fprobe jump
noinstr (among other things) adds the function also the kprobes blacklist, which will prevent standard kprobes to attach
ASAICS standard kprobes use rcu in probe path as well, like in opt_pre_handler function
so I think we should go with noinstr
jirka
Quoted 16 lines
Quoted 16 lines
arch/x86/kernel/process.c +0
-0{x86_idle (); }kernel/rcu/tree.c +1
-1@@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags;
Masami Hiramatsu (Google)
maintainerQuoted 60 lines · On Tue, 17 May 2022 09:49:33 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 50 lines · Yonghong Song
Quoted 39 lines · Jiri Olsa
Making
arch_cpu_idleandrcu_idle_exitnoinstr. Both functions run in rcu 'not watching' context and if there's tracer attached to them, which uses rcu (e.g. kprobe multi interface) it will hit RCU warning like:[ 3.017540] WARNING: suspicious RCU usage ... [ 3.018363] kprobe_multi_link_handler+0x68/0x1c0 [ 3.018364] ? kprobe_multi_link_handler+0x3e/0x1c0 [ 3.018366] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018367] ? arch_cpu_idle_dead+0x10/0x10 [ 3.018371] fprobe_handler.part.0+0xab/0x150 [ 3.018374] 0xffffffffa00080c8 [ 3.018393] ? arch_cpu_idle+0x5/0x10 [ 3.018398] arch_cpu_idle+0x5/0x10 [ 3.018399] default_idle_call+0x59/0x90 [ 3.018401] do_idle+0x1c3/0x1d0The call path is following:
default_idle_call
rcu_idle_enter
arch_cpu_idle
rcu_idle_exitThe
arch_cpu_idleandrcu_idle_exitare the only ones from above path that are traceble and cause this problem on my setup.Signed-off-by Jiri Olsa <[email protected]>--- arch/x86/kernel/process.c | 2 +- kernel/rcu/tree.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)arch/x86/kernel/process.c +0
-0arch/x86/kernel/process.c void arch_cpu_idle_dead(void) · line 723
Quoted 5 lines
arch/x86/kernel/process.c +1
-1@@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)/* * Called from the generic idle code. */ -void arch_cpu_idle (void ) +void noinstr arch_cpu_idle(void)noinstr includes a lot of attributes:
Quoted 5 lines
#define noinstr \
noinline notrace __attribute((__section__(".noinstr.text"))) \
__no_kcsan__no_sanitize_address__no_profile__no_sanitize_coverageshould we use notrace here?
hm right, so notrace should be enough for our case (
kprobe_multi) which is based on ftrace/fprobe jump
Quoted 7 lines
noinstr (among other things) adds the function also the kprobes blacklist, which will prevent standard kprobes to attach
ASAICS standard kprobes use rcu in probe path as well, like in
opt_pre_handlerfunctionso I think we should go with noinstr
Yes, I agree that noinstr is preferrable for these functions.
Thank you!
Quoted 19 lines
jirka
Quoted 16 lines
Quoted 16 lines
arch/x86/kernel/process.c +0
-0{x86_idle (); }kernel/rcu/tree.c +1
-1@@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)* If you add or remove a call to rcu_idle_exit(), be sure to test with * CONFIG_RCU_EQS_DEBUG=y. */ -void rcu_idle_exit (void ) +void noinstr rcu_idle_exit(void) {unsigned long flags;
Signature · 1 line
Masami Hiramatsu <[email protected]>
kernel test robot
build botI love your patch! Perhaps something to improve:
Hi Jiri,
I love your patch! Perhaps something to improve:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Jiri-Olsa/cpuidle-rcu-Making-arch_cpu_idle-and-rcu_idle_exit-noinstr/20220516-043752 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master config: x86_64-randconfig-a014-20220516 (https://download.01.org/0day-ci/archive/20220517/[email protected]/config) compiler: gcc-11 (Debian 11.2.0-20) 11.2.0 reproduce (this is a W=1 build): # https://github.com/intel-lab-lkp/linux/commit/0b6fee32d730f621f2bfc4d8d9f0729814398415 git remote add linux-review https://github.com/intel-lab-lkp/linux git fetch --no-tags linux-review Jiri-Olsa/cpuidle-rcu-Making-arch_cpu_idle-and-rcu_idle_exit-noinstr/20220516-043752 git checkout 0b6fee32d730f621f2bfc4d8d9f0729814398415 # save the config file mkdir build_dir && cp config build_dir/.config make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
vmlinux.o: warning: objtool: vmx_l1d_flush()+0x13: call to static_key_count() leaves .noinstr.text section vmlinux.o: warning: objtool: vmx_vcpu_enter_exit()+0x29: call to static_key_count() leaves .noinstr.text section vmlinux.o: warning: objtool: vmx_update_host_rsp()+0x3e: call to static_key_count() leaves .noinstr.text section vmlinux.o: warning: objtool: arch_cpu_idle()+0xb: call to {dynamic}() leaves .noinstr.text section
>> vmlinux.o: warning: objtool: rcu_idle_exit()+0x25: call to trace_hardirqs_on() leaves .noinstr.text section
vmlinux.o: warning: objtool: enter_from_user_mode()+0x1c: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: syscall_enter_from_user_mode()+0x21: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: syscall_enter_from_user_mode_prepare()+0x1c: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: exit_to_user_mode()+0x1b: call to static_key_count() leaves .noinstr.text section vmlinux.o: warning: objtool: syscall_exit_to_user_mode()+0x36: call to static_key_count() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_enter_from_user_mode()+0x1c: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_exit_to_user_mode()+0x22: call to static_key_count() leaves .noinstr.text section
Signature · 2 lines
0-DAY CI Kernel Test Service https://01.org/lkp
Hi Jiri,
Would you like to consider to add rcu_is_watching check in to solve this from the viewpoint of kprobe_multi_link_prog_run itself? And accounting of missed runs can be added as well to imporve observability.
Regards,
Ze
-----------------
From 29fd3cd713e65461325c2703cf5246a6fae5d4fe Mon Sep 17 00:00:00 2001
From: Ze Gao <[email protected]>
Date: Sat, 20 May 2023 17:32:05 +0800
Subject: [PATCH] bpf: kprobe_multi runs bpf progs only when rcu_is_watching
From the perspective of kprobe_multi_link_prog_run, any traceable functions can be attached while bpf progs need specical care and ought to be under rcu protection. To solve the likely rcu lockdep warns once for good, when (future) functions in idle path were attached accidentally, we better paying some cost to check at least in kernel-side, and return when rcu is not watching, which helps to avoid any unpredictable results.
--- kernel/trace/bpf_trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/bpf_trace.c +1 -1
@@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,struct bpf_run_ctx *old_run_ctx;int err; -if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 )) { +if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 || !rcu_is_watching ())) { err =0 ; goto out; }
Signature · 1 line
2.40.1
Re:
Quoted 18 lines · Ze Gao
Hi Jiri,
Would you like to consider to add
rcu_is_watchingcheck in to solve this from the viewpoint ofkprobe_multi_link_prog_runitself? And accounting of missed runs can be added as well to imporve observability.Regards,
Ze-----------------
From29fd3cd713e65461325c2703cf5246a6fae5d4feMon Sep 17 00:00:00 2001
From: Ze Gao <[email protected]>
Date: Sat, 20 May 2023 17:32:05 +0800
Subject: [PATCH] bpf:kprobe_multiruns bpf progs only whenrcu_is_watching
Quoted 7 lines
From the perspective of
kprobe_multi_link_prog_run, any traceable
functions can be attached while bpf progs need specical care and ought to be under rcu protection. To solve the likely rcu lockdep warns once for good, when (future) functions in idle path were attached accidentally, we better paying some cost to check at least in kernel-side, and return when rcu is not watching, which helps to avoid any unpredictable results.
kprobe_multi/fprobe share the same set of attachments with fentry. Currently, fentry does not filter with !rcu_is_watching, maybe because this is an extreme corner case. Not sure whether it is worthwhile or not.
Maybe if you can give a concrete example (e.g., attachment point) with current code base to show what the issue you encountered and it will make it easier to judge whether adding !rcu_is_watching() is necessary or not.
Quoted 19 lines
Signed-off-by Ze Gao <[email protected]>--- kernel/trace/bpf_trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/trace/bpf_trace.c +1
-1@@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,struct bpf_run_ctx *old_run_ctx;int err; -if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 )) { +if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 || !rcu_is_watching ())) { err =0 ; goto out; }
Re:
Quoted 4 lines
kprobe_multi/fprobe share the same set of attachments with fentry. Currently, fentry does not filter with !rcu_is_watching, maybe because this is an extreme corner case. Not sure whether it is worthwhile or not.
Agreed, it's rare, especially after Peter's patches which push narrow down rcu eqs regions
in the idle path and reduce the chance of any traceable functions happening in between.
However, from RCU's perspective, we ought to check if rcu_is_watching theoretically
when there's a chance our code will run in the idle path and also we need rcu to be alive,
And also we cannot simply make assumptions for any future changes in the idle path.
You know, just like what was hit in the thread.
Quoted 4 lines
Maybe if you can give a concrete example (e.g., attachment point) with current code base to show what the issue you encountered and it will make it easier to judge whether adding !rcu_is_watching() is necessary or not.
I can reproduce likely warnings on v6.1.18 where arch_cpu_idle is traceable but not on the latest version
so far. But as I state above, in theory we need it. So here is a gentle ping :) .
Re:
Quoted 23 lines · Ze Gao
Quoted 4 lines
kprobe_multi/fprobe share the same set of attachments with fentry. Currently, fentry does not filter with !rcu_is_watching, maybe because this is an extreme corner case. Not sure whether it is worthwhile or not.
Agreed, it's rare, especially after Peter's patches which push narrow down rcu eqs regions
in the idle path and reduce the chance of any traceable functions happening in between.However, from RCU's perspective, we ought to check if
rcu_is_watchingtheoretically
when there's a chance our code will run in the idle path and also we need rcu to be alive,
And also we cannot simply make assumptions for any future changes in the idle path.
You know, just like what was hit in the thread.Quoted 4 lines
Maybe if you can give a concrete example (e.g., attachment point) with current code base to show what the issue you encountered and it will make it easier to judge whether adding !rcu_is_watching() is necessary or not.
Quoted 4 lines
I can reproduce likely warnings on v6.1.18 where
arch_cpu_idleis traceable but not on the latest version
so far. But as I state above, in theory we need it. So here is a gentle ping :) .
hum, this change [1] added rcu_is_watching check to ftrace_test_recursion_trylock, which we use in fprobe_handler and is coming to fprobe_exit_handler in [2]
I might be missing something, but it seems like we don't need another rcu_is_watching call on kprobe_multi level
jirka
[1] d099dbfd3306 cpuidle: tracing: Warn about !rcu_is_watching()
[2] https://lore.kernel.org/bpf/[email protected]/
Masami Hiramatsu (Google)
maintainerRe:
Quoted 29 lines · On Sun, 21 May 2023 22:26:37 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 23 lines · Ze Gao
Quoted 4 lines
kprobe_multi/fprobe share the same set of attachments with fentry. Currently, fentry does not filter with !rcu_is_watching, maybe because this is an extreme corner case. Not sure whether it is worthwhile or not.
Agreed, it's rare, especially after Peter's patches which push narrow down rcu eqs regions
in the idle path and reduce the chance of any traceable functions happening in between.However, from RCU's perspective, we ought to check if
rcu_is_watchingtheoretically
when there's a chance our code will run in the idle path and also we need rcu to be alive,
And also we cannot simply make assumptions for any future changes in the idle path.
You know, just like what was hit in the thread.Quoted 4 lines
Maybe if you can give a concrete example (e.g., attachment point) with current code base to show what the issue you encountered and it will make it easier to judge whether adding !rcu_is_watching() is necessary or not.
Quoted 4 lines
I can reproduce likely warnings on v6.1.18 where
arch_cpu_idleis traceable but not on the latest version
so far. But as I state above, in theory we need it. So here is a gentle ping :) .
Quoted 5 lines
hum, this change [1] added
rcu_is_watchingcheck toftrace_test_recursion_trylock, which we use infprobe_handlerand is coming tofprobe_exit_handlerin [2]I might be missing something, but it seems like we don't need another
rcu_is_watchingcall onkprobe_multilevel
Good point! OK, then it seems we don't need it. The rethook continues to use the rcu_is_watching() because it is also used from kprobes, but the kprobe_multi doesn't need it.
Thank you,
Quoted 6 lines
jirka
[1]
d099dbfd3306cpuidle: tracing: Warn about !rcu_is_watching()
[2] https://lore.kernel.org/bpf/[email protected]/
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Re:
In reply to Jiri OlsaOops, I missed that. Thanks for pointing that out, which I thought is conditional use of rcu_is_watching before.
One last point, I think we should double check on this
"fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs the same check for fentry as well.
Regards,
Ze
Re:
Quoted 7 lines · Ze Gao
Oops, I missed that. Thanks for pointing that out, which I thought is conditional use of
rcu_is_watchingbefore.One last point, I think we should double check on this
"fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs the same check for fentry as well.
I would suggest that we address rcu_is_watching issue for fentry only if we do have a reproducible case to show something goes wrong...
Quoted 3 lines
Regards,
Ze
Masami Hiramatsu (Google)
maintainerRe:
In reply to Ze GaoQuoted 7 lines · On Mon, 22 May 2023 10:07:42 +0800 Ze Gao <[email protected]> wrote:
Oops, I missed that. Thanks for pointing that out, which I thought is conditional use of
rcu_is_watchingbefore.One last point, I think we should double check on this
"fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs the same check for fentry as well.
rcu_is_watching() comment says;
* if the current CPU is not in its idle loop or is in an interrupt or
* NMI handler, return true.
Thus it returns *fault* if the current CPU is in the idle loop and not any interrupt(including NMI) context. This means if any tracable function is called from idle loop, it can be !rcu_is_watching(). I meant, this is 'context' based check, thus fentry can not filter out that some commonly used functions is called from that context but it can be detected.
Thank you,
Quoted 3 lines
Regards,
Ze
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Paul E. McKenney
maintainerRe:
Quoted 13 lines · Masami Hiramatsu
Quoted 7 lines · On Mon, 22 May 2023 10:07:42 +0800 Ze Gao <[email protected]> wrote:
Oops, I missed that. Thanks for pointing that out, which I thought is conditional use of
rcu_is_watchingbefore.One last point, I think we should double check on this
"fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs the same check for fentry as well.
rcu_is_watching()comment says;
Quoted 8 lines
* if the current CPU is not in its idle loop or is in an interrupt or
* NMI handler, return true.Thus it returns *fault* if the current CPU is in the idle loop and not any interrupt(including NMI) context. This means if any tracable function is called from idle loop, it can be !rcu_is_watching(). I meant, this is 'context' based check, thus fentry can not filter out that some commonly used functions is called from that context but it can be detected.
It really does return false (rather than faulting?) if the current CPU is deep within the idle loop.
In addition, the recent x86/entry rework (thank you Peter and Thomas!) mean that the "idle loop" is quite restricted, as can be seen by the invocations of ct_cpuidle_enter() and ct_cpuidle_exit(). For example, in default_idle_call(), these are immediately before and after the call to arch_cpu_idle().
Would the following help? Or am I missing your point?
Thanx, Paul
------------------------------------------------------------------------
kernel/rcu/tree.c +8 -4
@@ -679,10 +679,14 @@ static void rcu_disable_urgency_upon_qs(struct rcu_data *rdp)/** * rcu_is_watching - see if RCU thinks that the current CPU is not idle * -* Return true if RCU is watching the running CPU, which means that this -* CPU can safely enter RCU read-side critical sections. In other words, -* if the current CPU is not in its idle loop or is in an interrupt or -* NMI handler, return true. +* Return @true if RCU is watching the running CPU and @false otherwise. +* An @true return means that this CPU can safely enter RCU read-side +* critical sections. +* +* More specifically, if the current CPU is not deep within its idle +* loop, return @true. Note that rcu_is_watching() will return @true if +* invoked from an interrupt or NMI handler, even if that interrupt or +* NMI interrupted the CPU while it was deep within its idle loop. * * Make notrace because it can be called by the internal functions of * ftrace, and making this notrace removes unnecessary recursion calls.
Masami Hiramatsu (Google)
maintainerRe:
Quoted 23 lines · On Mon, 22 May 2023 23:59:28 -0700 "Paul E. McKenney" <[email protected]>…
Quoted 13 lines · Masami Hiramatsu
Quoted 7 lines · On Mon, 22 May 2023 10:07:42 +0800 Ze Gao <[email protected]> wrote:
Oops, I missed that. Thanks for pointing that out, which I thought is conditional use of
rcu_is_watchingbefore.One last point, I think we should double check on this
"fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs the same check for fentry as well.
rcu_is_watching()comment says;Quoted 8 lines
* if the current CPU is not in its idle loop or is in an interrupt or
* NMI handler, return true.Thus it returns *fault* if the current CPU is in the idle loop and not any interrupt(including NMI) context. This means if any tracable function is called from idle loop, it can be !rcu_is_watching(). I meant, this is 'context' based check, thus fentry can not filter out that some commonly used functions is called from that context but it can be detected.
Quoted 8 lines
It really does return false (rather than faulting?) if the current CPU is deep within the idle loop.
In addition, the recent x86/entry rework (thank you Peter and Thomas!) mean that the "idle loop" is quite restricted, as can be seen by the invocations of
ct_cpuidle_enter()andct_cpuidle_exit(). For example, indefault_idle_call(), these are immediately before and after the call toarch_cpu_idle().
Thanks! I also found that the default_idle_call() is enough small and it seems not happening on fentry because there are no commonly used functions on that path.
Quoted 2 lines
Would the following help? Or am I missing your point?
Yes, thank you for the update!
Quoted 28 lines
Thanx, Paul
------------------------------------------------------------------------
kernel/rcu/tree.c +8
-4@@ -679,10 +679,14 @@ static void rcu_disable_urgency_upon_qs(struct rcu_data *rdp)/** * rcu_is_watching - see if RCU thinks that the current CPU is not idle * -* Return true if RCU is watching the running CPU, which means that this -* CPU can safely enter RCU read-side critical sections. In other words, -* if the current CPU is not in its idle loop or is in an interrupt or -* NMI handler, return true. +* Return @true if RCU is watching the running CPU and @false otherwise. +* An @true return means that this CPU can safely enter RCU read-side +* critical sections. +* +* More specifically, if the current CPU is not deep within its idle +* loop, return @true. Note that rcu_is_watching() will return @true if +* invoked from an interrupt or NMI handler, even if that interrupt or +* NMI interrupted the CPU while it was deep within its idle loop. * * Make notrace because it can be called by the internal functions of * ftrace, and making this notrace removes unnecessary recursion calls.
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Steven Rostedt
maintainerRe: kprobes and rcu_is_watching()
In reply to Ze Gao[ Added a subject, as I always want to delete these emails as spam! ]
Quoted 8 lines · On Mon, 22 May 2023 10:07:42 +0800 Ze Gao <[email protected]> wrote:
Oops, I missed that. Thanks for pointing that out, which I thought is conditional use of
rcu_is_watchingbefore.One last point, I think we should double check on this
"fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs the same check for fentry as well.
Note that trace_test_and_set_recursion() (which is used by ftrace_test_recursion_trylock()) checks for rcu_is_watching() and returns false if it isn't (and the trylock will fail).
-- Steve
Re: kprobes and rcu_is_watching()
Thanks Steven, I think we've come to a consensus on this.
The question here is whether bpf tracing fentry i.e., __bpf_prog_enter{_sleepable}
needs to check rcu_is_watching as well before using rcu related calls. And Yonghong suggested making a change when there is indeed some bad case occurring since it's rare the tracee is in the idle path.
Regards,
Ze
Quoted 20 lines · Steven Rostedt
[ Added a subject, as I always want to delete these emails as spam! ]
Quoted 8 lines · On Mon, 22 May 2023 10:07:42 +0800 Ze Gao <[email protected]> wrote:
Oops, I missed that. Thanks for pointing that out, which I thought is conditional use of
rcu_is_watchingbefore.One last point, I think we should double check on this
"fentry does not filter with !rcu_is_watching"
as quoted from Yonghong and argue whether it needs the same check for fentry as well.Note that
trace_test_and_set_recursion()(which is used byftrace_test_recursion_trylock()) checks forrcu_is_watching()and returns false if it isn't (and the trylock will fail).-- Steve
Re:
In reply to Ze GaoQuoted 5 lines · Ze Gao
Hi Jiri,
Would you like to consider to add
rcu_is_watchingcheck in to solve this from the viewpoint ofkprobe_multi_link_prog_run
I think this was discussed in here:
https://lore.kernel.org/bpf/[email protected]/
and was considered a bug, there's fix mentioned later in the thread
there's also this recent patchset:
https://lore.kernel.org/bpf/[email protected]/
that solves related problems
Quoted 2 lines
itself? And accounting of missed runs can be added as well to imporve observability.
right, we count fprobe->nmissed but it's not exposed, we should allow to get 'missed' stats from both fprobe and kprobe_multi later, which is missing now, will check
thanks,
jirka
Quoted 40 lines
Regards,
Ze-----------------
From29fd3cd713e65461325c2703cf5246a6fae5d4feMon Sep 17 00:00:00 2001
From: Ze Gao <[email protected]>
Date: Sat, 20 May 2023 17:32:05 +0800
Subject: [PATCH] bpf:kprobe_multiruns bpf progs only whenrcu_is_watchingFrom the perspective of
kprobe_multi_link_prog_run, any traceable functions can be attached while bpf progs need specical care and ought to be under rcu protection. To solve the likely rcu lockdep warns once for good, when (future) functions in idle path were attached accidentally, we better paying some cost to check at least in kernel-side, and return when rcu is not watching, which helps to avoid any unpredictable results.Signed-off-by Ze Gao <[email protected]>--- kernel/trace/bpf_trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/trace/bpf_trace.c +1
-1@@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,struct bpf_run_ctx *old_run_ctx;int err; -if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 )) { +if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 || !rcu_is_watching ())) { err =0 ; goto out; }Signature · 1 line
2.40.1
Masami Hiramatsu (Google)
maintainerRe:
Quoted 10 lines · On Sun, 21 May 2023 10:08:46 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 5 lines · Ze Gao
Hi Jiri,
Would you like to consider to add
rcu_is_watchingcheck in to solve this from the viewpoint ofkprobe_multi_link_prog_runI think this was discussed in here:
https://lore.kernel.org/bpf/[email protected]/
Quoted 6 lines
and was considered a bug, there's fix mentioned later in the thread
there's also this recent patchset:
https://lore.kernel.org/bpf/[email protected]/that solves related problems
I think this rcu_is_watching() is a bit different issue. This rcu_is_watching() check is required if the kprobe_multi_link_prog_run() uses any RCU API. E.g. rethook_try_get() is also checks rcu_is_watching() because it uses call_rcu().
Thank you,
Quoted 51 lines
Quoted 3 lines
itself? And accounting of missed runs can be added as well to imporve observability.
right, we count
fprobe->nmissedbut it's not exposed, we should allow to get 'missed' stats from both fprobe andkprobe_multilater, which is missing now, will checkthanks,
jirkaQuoted 40 lines
Regards,
Ze-----------------
From29fd3cd713e65461325c2703cf5246a6fae5d4feMon Sep 17 00:00:00 2001
From: Ze Gao <[email protected]>
Date: Sat, 20 May 2023 17:32:05 +0800
Subject: [PATCH] bpf:kprobe_multiruns bpf progs only whenrcu_is_watchingFrom the perspective of
kprobe_multi_link_prog_run, any traceable functions can be attached while bpf progs need specical care and ought to be under rcu protection. To solve the likely rcu lockdep warns once for good, when (future) functions in idle path were attached accidentally, we better paying some cost to check at least in kernel-side, and return when rcu is not watching, which helps to avoid any unpredictable results.Signed-off-by Ze Gao <[email protected]>--- kernel/trace/bpf_trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/trace/bpf_trace.c +1
-1@@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,struct bpf_run_ctx *old_run_ctx;int err; -if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 )) { +if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 || !rcu_is_watching ())) { err =0 ; goto out; }Signature · 1 line
2.40.1
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Re:
Quoted 19 lines · Masami Hiramatsu
Quoted 15 lines · On Sun, 21 May 2023 10:08:46 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 5 lines · Ze Gao
Hi Jiri,
Would you like to consider to add
rcu_is_watchingcheck in to solve this from the viewpoint ofkprobe_multi_link_prog_runI think this was discussed in here:
https://lore.kernel.org/bpf/[email protected]/and was considered a bug, there's fix mentioned later in the thread
there's also this recent patchset:
https://lore.kernel.org/bpf/[email protected]/
Quoted 6 lines
Quoted 1 line
that solves related problems
I think this
rcu_is_watching()is a bit different issue. Thisrcu_is_watching()check is required if thekprobe_multi_link_prog_run()uses any RCU API. E.g.rethook_try_get()is also checksrcu_is_watching()because it usescall_rcu().
Yes, that's my point!
Regards,
Ze
Quoted 56 lines
Quoted 44 lines
Quoted 4 lines
itself? And accounting of missed runs can be added as well to imporve observability.
right, we count
fprobe->nmissedbut it's not exposed, we should allow to get 'missed' stats from both fprobe andkprobe_multilater, which is missing now, will checkthanks,
jirkaQuoted 32 lines
Regards,
Ze-----------------
From29fd3cd713e65461325c2703cf5246a6fae5d4feMon Sep 17 00:00:00 2001
From: Ze Gao <[email protected]>
Date: Sat, 20 May 2023 17:32:05 +0800
Subject: [PATCH] bpf:kprobe_multiruns bpf progs only whenrcu_is_watchingFrom the perspective of
kprobe_multi_link_prog_run, any traceable functions can be attached while bpf progs need specical care and ought to be under rcu protection. To solve the likely rcu lockdep warns once for good, when (future) functions in idle path were attached accidentally, we better paying some cost to check at least in kernel-side, and return when rcu is not watching, which helps to avoid any unpredictable results.Signed-off-by Ze Gao <[email protected]>--- kernel/trace/bpf_trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/trace/bpf_trace.c +0
-0@@ -2622,7 +2622,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,struct bpf_run_ctx *old_run_ctx;int err;Quoted 8 lines
Quoted 8 lines
kernel/trace/bpf_trace.c +1
-2-if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 )) { +if (unlikely (__this_cpu_inc_return (bpf_prog_active) !=1 || !rcu_is_watching ())) { err =0 ; goto out; } --2.40.1
--
Masami Hiramatsu (Google) <[email protected]>