bpf: Remove migrate_disable in kprobe_multi_link_prog_run
Subsystems (4)
PATCH v2 1/1 bpf: Remove migrate_disable in kprobe_multi_link_prog_run
bpf program should run under migration disabled, kprobe_multi_link_prog_run called all the way from graph tracer, which disables preemption in function_graph_enter_regs, as Jiri and Yonghong suggested, there is no need to use migrate_disable. As a result, some overhead maybe will be reduced.
--- kernel/trace/bpf_trace.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
Change list:
v1 -> v2:
- s/called the way/called all the way/.(Jiri)
v1: https://lore.kernel.org/bpf/[email protected]
kernel/trace/bpf_trace.c +7 -2
@@ -2734,14 +2734,19 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, goto out; } -migrate_disable (); +/* +* bpf program should run under migration disabled, kprobe_multi_link_prog_run +* called all the way from graph tracer, which disables preemption in +* function_graph_enter_regs, so there is no need to use migrate_disable. +* Accessing the above percpu data bpf_prog_active is also safe for the same +* reason. +*/ rcu_read_lock (); regs =ftrace_partial_regs (fregs,bpf_kprobe_multi_pt_regs_ptr ()); old_run_ctx =bpf_set_run_ctx (&run_ctx.session_ctx.run_ctx); err =bpf_prog_run (link->link.prog, regs);bpf_reset_run_ctx (old_run_ctx);rcu_read_unlock (); -migrate_enable (); out:__this_cpu_dec (bpf_prog_active);
Signature · 1 line
2.48.1
kernel/trace/bpf_trace.c kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
Quoted 25 lines · Tao Chen
bpf program should run under migration disabled,
kprobe_multi_link_prog_runcalled all the way from graph tracer, which disables preemption infunction_graph_enter_regs, as Jiri and Yonghong suggested, there is no need to usemigrate_disable. As a result, some overhead maybe will be reduced.Fixes 0dcac2725406 ("bpf: Add multi kprobe link")Acked-by Yonghong Song <[email protected]>Acked-by Jiri Olsa <[email protected]>Signed-off-by Tao Chen <[email protected]>--- kernel/trace/bpf_trace.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Change list:
v1 -> v2:
- s/called the way/called all the way/.(Jiri)
v1: https://lore.kernel.org/bpf/[email protected]kernel/trace/bpf_trace.c +0
-0@@ -2734,14 +2734,19 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
even though bpf_prog_run() eventually calls cant_migrate(), we should add it before that __this_cpu_inc_return() call as well, because that one is relying on that non-migration independently from bpf_prog_run()
kernel/trace/bpf_trace.c kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, · line 2743
Quoted 11 lines
kernel/trace/bpf_trace.c +7
-1goto out; } -migrate_disable (); +/* +* bpf program should run under migration disabled, kprobe_multi_link_prog_run +* called all the way from graph tracer, which disables preemption in +* function_graph_enter_regs, so there is no need to use migrate_disable. +* Accessing the above percpu data bpf_prog_active is also safe for the same +* reason. +*/
let's shorten this a bit to something like:
/* graph tracer framework ensures we won't migrate */ cant_migrate();
all the other stuff in the comment can become outdated way too easily and/or is sort of general BPF implementation knowledge
pw-bot: cr
Quoted 13 lines
kernel/trace/bpf_trace.c +0
-2rcu_read_lock (); regs =ftrace_partial_regs (fregs,bpf_kprobe_multi_pt_regs_ptr ()); old_run_ctx =bpf_set_run_ctx (&run_ctx.session_ctx.run_ctx); err =bpf_prog_run (link->link.prog, regs);bpf_reset_run_ctx (old_run_ctx);rcu_read_unlock (); -migrate_enable (); out:__this_cpu_dec (bpf_prog_active); --2.48.1
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Tao Chen
bpf program should run under migration disabled,
kprobe_multi_link_prog_runcalled all the way from graph tracer, which disables preemption infunction_graph_enter_regs, as Jiri and Yonghong suggested, there is no need to usemigrate_disable. As a result, some overhead maybe will be reduced.Fixes 0dcac2725406 ("bpf: Add multi kprobe link")Acked-by Yonghong Song <[email protected]>Acked-by Jiri Olsa <[email protected]>Signed-off-by Tao Chen <[email protected]>--- kernel/trace/bpf_trace.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Change list:
v1 -> v2:
- s/called the way/called all the way/.(Jiri)
v1: https://lore.kernel.org/bpf/[email protected]
Quoted 10 lines
kernel/trace/bpf_trace.c kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
Quoted 5 lines
kernel/trace/bpf_trace.c +0
-0@@ -2734,14 +2734,19 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,even though
bpf_prog_run()eventually callscant_migrate(), we should add it before that__this_cpu_inc_return()call as well, because that one is relying on that non-migration independently frombpf_prog_run()
Hi Andrii,
There is __this_cpu_preempt_check in __this_cpu_inc_return, and the judgment criteria are similar to cant_migrate, and I'm not sure if it is enough.
Quoted 12 lines
kernel/trace/bpf_trace.c kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, · line 2743
Quoted 11 lines
kernel/trace/bpf_trace.c +7
-1goto out; } -migrate_disable (); +/* +* bpf program should run under migration disabled, kprobe_multi_link_prog_run +* called all the way from graph tracer, which disables preemption in +* function_graph_enter_regs, so there is no need to use migrate_disable. +* Accessing the above percpu data bpf_prog_active is also safe for the same +* reason. +*/
Quoted 3 lines
let's shorten this a bit to something like:
/* graph tracer framework ensures we won't migrate */
will change it in v3.
Quoted 21 lines
cant_migrate();all the other stuff in the comment can become outdated way too easily and/or is sort of general BPF implementation knowledge
pw-bot: cr
Quoted 13 lines
kernel/trace/bpf_trace.c +0
-2rcu_read_lock (); regs =ftrace_partial_regs (fregs,bpf_kprobe_multi_pt_regs_ptr ()); old_run_ctx =bpf_set_run_ctx (&run_ctx.session_ctx.run_ctx); err =bpf_prog_run (link->link.prog, regs);bpf_reset_run_ctx (old_run_ctx);rcu_read_unlock (); -migrate_enable (); out:__this_cpu_dec (bpf_prog_active); --2.48.1
Signature · 2 lines
Best Regards Tao Chen
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Tao Chen
bpf program should run under migration disabled,
kprobe_multi_link_prog_runcalled all the way from graph tracer, which disables preemption infunction_graph_enter_regs, as Jiri and Yonghong suggested, there is no need to usemigrate_disable. As a result, some overhead maybe will be reduced.Fixes 0dcac2725406 ("bpf: Add multi kprobe link")Acked-by Yonghong Song <[email protected]>Acked-by Jiri Olsa <[email protected]>Signed-off-by Tao Chen <[email protected]>--- kernel/trace/bpf_trace.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Change list:
v1 -> v2:
- s/called the way/called all the way/.(Jiri)
v1: https://lore.kernel.org/bpf/[email protected]
Quoted 10 lines
kernel/trace/bpf_trace.c kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
Quoted 5 lines
kernel/trace/bpf_trace.c +0
-0@@ -2734,14 +2734,19 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,even though
bpf_prog_run()eventually callscant_migrate(), we should add it before that__this_cpu_inc_return()call as well, because that one is relying on that non-migration independently frombpf_prog_run()
maybe cant_sleep() is better like trace_call_bpf, cant_sleep dose not check migration_disabled again, which is done in __this_cpu_preempt_check. I will add it in v3.
Quoted 36 lines
kernel/trace/bpf_trace.c kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, · line 2743
Quoted 11 lines
kernel/trace/bpf_trace.c +7
-1goto out; } -migrate_disable (); +/* +* bpf program should run under migration disabled, kprobe_multi_link_prog_run +* called all the way from graph tracer, which disables preemption in +* function_graph_enter_regs, so there is no need to use migrate_disable. +* Accessing the above percpu data bpf_prog_active is also safe for the same +* reason. +*/ let's shorten this a bit to something like:
/* graph tracer framework ensures we won't migrate */
cant_migrate();all the other stuff in the comment can become outdated way too easily and/or is sort of general BPF implementation knowledge
pw-bot: cr
Quoted 13 lines
kernel/trace/bpf_trace.c +0
-2rcu_read_lock (); regs =ftrace_partial_regs (fregs,bpf_kprobe_multi_pt_regs_ptr ()); old_run_ctx =bpf_set_run_ctx (&run_ctx.session_ctx.run_ctx); err =bpf_prog_run (link->link.prog, regs);bpf_reset_run_ctx (old_run_ctx);rcu_read_unlock (); -migrate_enable (); out:__this_cpu_dec (bpf_prog_active); --2.48.1
Signature · 2 lines
Best Regards Tao Chen