bpf: Add support to attach return prog in kprobe multi
Subsystems (7)
Jiri Olsa
reviewercover bpf: Add support to attach return prog in kprobe multi
hi,
adding support to attach both entry and return bpf program on single kprobe multi link.
Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.
At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.
In addition it's possible to control the execution of the return probe with the return value of the entry bpf program. If the entry program returns 0 the return probe is installed and executed, otherwise it's skip.
I'm still working on the tetragon change, so I'll be sending non-RFC version once that's ready, meanwhile any ideas and feedback on the approach would be great.
The change in bpftrace [1] using the new interface shows speed increase with tracing perf bench messaging:
# perf bench sched messaging -l 100000
having system wide bpftrace:
# bpftrace -e 'kprobe:ksys_write { }, kretprobe:ksys_write { }'without bpftrace:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes run
Total time: 119.595 [sec]
Performance counter stats for 'perf bench sched messaging -l 100000':
102,419,967,282 cycles:u
5,652,444,107,001 cycles:k
5,782,645,019,612 cycles
22,187,151,206 instructions:u # 0.22 insn per cycle
2,979,040,498,455 instructions:k # 0.53 insn per cycle 119.671169829 seconds time elapsed
94.959198000 seconds user
1815.371616000 seconds syswith current bpftrace:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes run
Total time: 221.153 [sec]
Performance counter stats for 'perf bench sched messaging -l 100000':
125,292,164,504 cycles:u
10,315,020,393,735 cycles:k
10,501,379,274,042 cycles
22,187,583,545 instructions:u # 0.18 insn per cycle
4,856,893,111,303 instructions:k # 0.47 insn per cycle 221.229234283 seconds time elapsed
103.792498000 seconds user
3432.643302000 seconds syswith bpftrace using the new interface:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes run
Total time: 157.825 [sec]
Performance counter stats for 'perf bench sched messaging -l 100000':
102,423,112,279 cycles:u
7,450,856,354,744 cycles:k
7,584,769,726,693 cycles
22,187,270,661 instructions:u # 0.22 insn per cycle
3,985,522,383,425 instructions:k # 0.53 insn per cycle 157.900787760 seconds time elapsed
97.953898000 seconds user
2425.314753000 seconds systhanks,
jirka
[1] https://github.com/bpftrace/bpftrace/pull/2984
---
Jiri Olsa (4):
fprobe: Add entry/exit callbacks types
bpf: Add return prog to kprobe multi
libbpf: Add return_prog_fd to kprobe multi opts selftests/bpf: Add kprobe multi return prog test
include/linux/fprobe.h | 18 ++++++++++------ include/uapi/linux/bpf.h | 4 +++- kernel/trace/bpf_trace.c | 50 ++++++++++++++++++++++++++++++++----------- tools/include/uapi/linux/bpf.h | 4 +++- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.c | 5 +++++ tools/lib/bpf/libbpf.h | 6 +++++- tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 53 +++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 226 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c
Jiri Olsa
reviewerRFC PATCH 1/4 fprobe: Add entry/exit callbacks types
We are going to store callbacks in following change, so this will ease up the code.
--- include/linux/fprobe.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
include/linux/fprobe.h +12 -6
@@ -7,6 +7,16 @@#include <linux/ftrace.h> #include <linux/rethook.h> +struct fprobe ; + +typedef int (*fprobe_entry_cb )(struct fprobe *fp,unsigned long entry_ip, +unsigned long ret_ip,struct pt_regs *regs, +void *entry_data); + +typedef void (*fprobe_exit_cb )(struct fprobe *fp,unsigned long entry_ip, +unsigned long ret_ip,struct pt_regs *regs, +void *entry_data); +/** * struct fprobe - ftrace based probe. * @ops: The ftrace_ops.
@@ -34,12 +44,8 @@ struct fprobe {size_t entry_data_size;int nr_maxactive; -int (*entry_handler)(struct fprobe *fp,unsigned long entry_ip, -unsigned long ret_ip,struct pt_regs *regs, -void *entry_data); -void (*exit_handler)(struct fprobe *fp,unsigned long entry_ip, -unsigned long ret_ip,struct pt_regs *regs, -void *entry_data); +fprobe_entry_cb entry_handler; +fprobe_exit_cb exit_handler; };/* This fprobe is soft-disabled. */
Signature · 1 line
2.43.0
Jiri Olsa
reviewerRFC PATCH 2/4 bpf: Add return prog to kprobe multi
In reply to Jiri OlsaAdding support to attach both entry and return bpf program on single kprobe multi link.
Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.
At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.
In addition it's possible to control the execution of the return probe with the return value of the entry bpf program. If the entry program returns 0 the return probe is installed and executed, otherwise it's skip.
--- include/uapi/linux/bpf.h | 4 ++- kernel/trace/bpf_trace.c | 50 ++++++++++++++++++++++++++-------- tools/include/uapi/linux/bpf.h | 4 ++- 3 files changed, 44 insertions(+), 14 deletions(-)
include/uapi/linux/bpf.h +3 -1
@@ -1243,7 +1243,8 @@ enum bpf_perf_event_type {* BPF_TRACE_KPROBE_MULTI attach type to create return probe. */ enum { -BPF_F_KPROBE_MULTI_RETURN = (1U <<0 ) +BPF_F_KPROBE_MULTI_RETURN = (1U <<0 ), +BPF_F_KPROBE_MULTI_RETURN_PROG = (1U <<1 ), };/* link_create.uprobe_multi.flags used in LINK_CREATE command for
@@ -1690,6 +1691,7 @@ union bpf_attr {__aligned_u64 syms;__aligned_u64 addrs;__aligned_u64 cookies; +__u32 return_prog_fd; } kprobe_multi;struct {/* this is overlaid with the target_btf_id above. */
kernel/trace/bpf_trace.c +38 -12
132 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +3 -1
@@ -1243,7 +1243,8 @@ enum bpf_perf_event_type {* BPF_TRACE_KPROBE_MULTI attach type to create return probe. */ enum { -BPF_F_KPROBE_MULTI_RETURN = (1U <<0 ) +BPF_F_KPROBE_MULTI_RETURN = (1U <<0 ), +BPF_F_KPROBE_MULTI_RETURN_PROG = (1U <<1 ), };/* link_create.uprobe_multi.flags used in LINK_CREATE command for
@@ -1690,6 +1691,7 @@ union bpf_attr {__aligned_u64 syms;__aligned_u64 addrs;__aligned_u64 cookies; +__u32 return_prog_fd; } kprobe_multi;struct {/* this is overlaid with the target_btf_id above. */
Signature · 1 line
2.43.0
Jiri Olsa
reviewerRFC PATCH 3/4 libbpf: Add return_prog_fd to kprobe multi opts
In reply to Jiri OlsaAdding support to specify return prog file descriptor in struct bpf_kprobe_multi_opts.
When the return_prog bool is set true the return_prog_fd is used as bpf program to be executed for the return probe.
--- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.c | 5 +++++ tools/lib/bpf/libbpf.h | 6 +++++- 4 files changed, 12 insertions(+), 1 deletion(-)
tools/lib/bpf/bpf.c +1 -0
@@ -771,6 +771,7 @@ int bpf_link_create(int prog_fd, int target_fd, attr.link_create.kprobe_multi.syms =ptr_to_u64 (OPTS_GET (opts, kprobe_multi.syms,0 )); attr.link_create.kprobe_multi.addrs =ptr_to_u64 (OPTS_GET (opts, kprobe_multi.addrs,0 )); attr.link_create.kprobe_multi.cookies =ptr_to_u64 (OPTS_GET (opts, kprobe_multi.cookies,0 )); + attr.link_create.kprobe_multi.return_prog_fd =OPTS_GET (opts, kprobe_multi.return_prog_fd,0 );if (!OPTS_ZEROED (opts, kprobe_multi))return libbpf_err (-EINVAL );break ;
tools/lib/bpf/bpf.h +1 -0
@@ -399,6 +399,7 @@ struct bpf_link_create_opts {const char **syms;const unsigned long *addrs;const __u64 *cookies; +__u32 return_prog_fd; } kprobe_multi;struct {__u32 flags;
tools/lib/bpf/libbpf.c +5 -0
@@ -11073,6 +11073,11 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, lopts.kprobe_multi.cnt = cnt; lopts.kprobe_multi.flags = retprobe ?BPF_F_KPROBE_MULTI_RETURN :0 ; +if (OPTS_GET (opts, return_prog, false)) { + lopts.kprobe_multi.return_prog_fd =OPTS_GET (opts, return_prog_fd,0 ); + lopts.kprobe_multi.flags |=BPF_F_KPROBE_MULTI_RETURN_PROG ; + } + link =calloc (1 ,sizeof (*link));if (!link) { err = -ENOMEM ;
tools/lib/bpf/libbpf.h +5 -1
@@ -539,10 +539,14 @@ struct bpf_kprobe_multi_opts {size_t cnt;/* create return kprobes */ bool retprobe; +/* attach return program (specified by return_prog_fd file descriptor) */ +bool return_prog; +/* return program fd */ +int return_prog_fd; size_t :0; }; -#define bpf_kprobe_multi_opts__last_field retprobe +#define bpf_kprobe_multi_opts__last_field return_prog_fd LIBBPF_API struct bpf_link * bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
Signature · 1 line
2.43.0
Jiri Olsa
reviewerRFC PATCH 4/4 selftests/bpf: Add kprobe multi return prog test
In reply to Jiri OlsaAdding test for kprobe multi return program.
--- .../bpf/prog_tests/kprobe_multi_test.c | 53 +++++++++ .../bpf/progs/kprobe_multi_return_prog.c | 105 ++++++++++++++++++ 2 files changed, 158 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +53 -0
@@ -4,6 +4,7 @@#include "trace_helpers.h" #include "kprobe_multi_empty.skel.h" #include "kprobe_multi_override.skel.h" +#include "kprobe_multi_return_prog.skel.h" #include "bpf/libbpf_internal.h" #include "bpf/hashmap.h"
@@ -326,6 +327,56 @@ static void test_attach_api_fails(void)kprobe_multi__destroy (skel); } +static void test_attach_return_prog (void ) +{ +struct kprobe_multi_return_prog *skel =NULL ; +LIBBPF_OPTS (bpf_kprobe_multi_opts, opts); +LIBBPF_OPTS (bpf_test_run_opts, topts); +struct bpf_link *link =NULL ; +int err, prog_fd; + + skel =kprobe_multi_return_prog__open_and_load (); +if (!ASSERT_OK_PTR (skel,"fentry_raw_skel_load" )) + goto cleanup; + + prog_fd =bpf_program__fd (skel->progs.test_kretprobe); + opts.return_prog = true; + opts.return_prog_fd = prog_fd; + + skel->bss->pid =getpid (); + link =bpf_program__attach_kprobe_multi_opts (skel->progs.test_kprobe, +"bpf_fentry_test*" , &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_kprobe_multi_opts2" )) + goto cleanup; + + prog_fd =bpf_program__fd (skel->progs.trigger); + err =bpf_prog_test_run_opts (prog_fd, &topts); +ASSERT_OK (err,"test_run" ); +ASSERT_EQ (topts.retval,0 ,"test_run" ); + +ASSERT_EQ (skel->bss->kprobe_test1_result,1 ,"kprobe_test1_result" ); +ASSERT_EQ (skel->bss->kprobe_test2_result,1 ,"kprobe_test2_result" ); +ASSERT_EQ (skel->bss->kprobe_test3_result,1 ,"kprobe_test3_result" ); +ASSERT_EQ (skel->bss->kprobe_test4_result,1 ,"kprobe_test4_result" ); +ASSERT_EQ (skel->bss->kprobe_test5_result,1 ,"kprobe_test5_result" ); +ASSERT_EQ (skel->bss->kprobe_test6_result,1 ,"kprobe_test6_result" ); +ASSERT_EQ (skel->bss->kprobe_test7_result,1 ,"kprobe_test7_result" ); +ASSERT_EQ (skel->bss->kprobe_test8_result,1 ,"kprobe_test8_result" ); + +ASSERT_EQ (skel->bss->kretprobe_test1_result,0 ,"kretprobe_test1_result" ); +ASSERT_EQ (skel->bss->kretprobe_test2_result,1 ,"kretprobe_test2_result" ); +ASSERT_EQ (skel->bss->kretprobe_test3_result,0 ,"kretprobe_test3_result" ); +ASSERT_EQ (skel->bss->kretprobe_test4_result,1 ,"kretprobe_test4_result" ); +ASSERT_EQ (skel->bss->kretprobe_test5_result,0 ,"kretprobe_test5_result" ); +ASSERT_EQ (skel->bss->kretprobe_test6_result,1 ,"kretprobe_test6_result" ); +ASSERT_EQ (skel->bss->kretprobe_test7_result,0 ,"kretprobe_test7_result" ); +ASSERT_EQ (skel->bss->kretprobe_test8_result,1 ,"kretprobe_test8_result" ); + +cleanup: +bpf_link__destroy (link); +kprobe_multi_return_prog__destroy (skel); +} +static size_t symbol_hash (long key,void *ctx__maybe_unused ) {return str_hash ((const char *) key);
@@ -538,4 +589,6 @@ void test_kprobe_multi_test(void)test_attach_api_fails ();if (test__start_subtest ("attach_override" ))test_attach_override (); +if (test__start_subtest ("return_prog" )) +test_attach_return_prog (); }
tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c new file +105 -0
105 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH RFC bpf-next 2/4] bpf: Add return prog to kprobe multi
In reply to Jiri OlsaQuoted 6 lines · Jiri Olsa
In addition it's possible to control the execution of the return probe with the return value of the entry bpf program. If the entry program returns 0 the return probe is installed and executed, otherwise it's skip.
Let's not sneak in a big change in behavior like this.
kernel/trace/bpf_trace.c kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip, · line 2836
Quoted 8 lines
kernel/trace/bpf_trace.c +2
-2@@ -2828,8 +2832,8 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip,struct bpf_kprobe_multi_link *link; link = container_of(fp, struct bpf_kprobe_multi_link, fp); -kprobe_multi_link_prog_run (link,get_entry_ip (fentry_ip), regs); -return 0 ; +return kprobe_multi_link_prog_run (link, link->link.prog, +get_entry_ip (fentry_ip), regs);
This is big.
What motivates this change?
Quoted 14 lines · Jiri Olsa
hi,
adding support to attach both entry and return bpf program on single kprobe multi link.Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.
At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.
Quoted 5 lines
In addition it's possible to control the execution of the return probe with the return value of the entry bpf program. If the entry program returns 0 the return probe is installed and executed, otherwise it's skip.
In general, I think this is a very useful ability to have a combined entry/return program.
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf. You just declare SEC("kprobe.wrap/...") (or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).
Another frequently requested feature and a very common use case is to measure duration of the function, so if we have a custom type, we can have a field to record entry timestamp and let user calculate duration, if necessary. Without this users have to pay a bunch of extra overhead to record timestamp and put it into hashmap (keyed by thread id) or thread local storage (even if they have no other use for thread local storage).
Also, consider that a similar concept is applicable to uprobes and we should do that as well, in similar fashion. And the above approach works for both kprobe/kretprobe and uprobe/uretprobe cases, because they have the same pt_regs data structure as a context (even if for exit probes most of the values of pt_regs are not that meaningful).
So anyways, great feature, but let's discuss end-to-end usability story before we finalize the implementation?
Quoted 40 lines
I'm still working on the tetragon change, so I'll be sending non-RFC version once that's ready, meanwhile any ideas and feedback on the approach would be great.
The change in bpftrace [1] using the new interface shows speed increase with tracing perf bench messaging:
# perf bench sched messaging -l 100000having system wide bpftrace:
# bpftrace -e 'kprobe:ksys_write { }, kretprobe:ksys_write { }'without bpftrace:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes runTotal time: 119.595 [sec]Performance counter stats for 'perf bench sched messaging -l 100000': 102,419,967,282 cycles:u 5,652,444,107,001 cycles:k 5,782,645,019,612 cycles 22,187,151,206 instructions:u # 0.22 insn per cycle 2,979,040,498,455 instructions:k # 0.53 insn per cycle 119.671169829 seconds time elapsed 94.959198000 seconds user 1815.371616000 seconds syswith current bpftrace:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes run
Quoted 5 lines
Total time: 221.153 [sec]Performance counter stats for 'perf bench sched messaging -l 100000':
125,292,164,504 cycles:u
btw, why +25% in user space?... this looks weird
Quoted 55 lines
10,315,020,393,735 cycles:k 10,501,379,274,042 cycles 22,187,583,545 instructions:u # 0.18 insn per cycle 4,856,893,111,303 instructions:k # 0.47 insn per cycle221.229234283 seconds time elapsed 103.792498000 seconds user 3432.643302000 seconds syswith bpftrace using the new interface:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes runTotal time: 157.825 [sec]Performance counter stats for 'perf bench sched messaging -l 100000': 102,423,112,279 cycles:u 7,450,856,354,744 cycles:k 7,584,769,726,693 cycles 22,187,270,661 instructions:u # 0.22 insn per cycle 3,985,522,383,425 instructions:k # 0.53 insn per cycle 157.900787760 seconds time elapsed 97.953898000 seconds user 2425.314753000 seconds systhanks,
jirka[1] https://github.com/bpftrace/bpftrace/pull/2984
---
Jiri Olsa (4):
fprobe: Add entry/exit callbacks types
bpf: Add return prog to kprobe multi
libbpf: Addreturn_prog_fdto kprobe multi opts selftests/bpf: Add kprobe multi return prog testinclude/linux/fprobe.h | 18 ++++++++++------ include/uapi/linux/bpf.h | 4 +++- kernel/trace/bpf_trace.c | 50 ++++++++++++++++++++++++++++++++----------- tools/include/uapi/linux/bpf.h | 4 +++- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.c | 5 +++++ tools/lib/bpf/libbpf.h | 6 +++++- tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 53 +++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 226 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c
Re: [PATCH RFC bpf-next 2/4] bpf: Add return prog to kprobe multi
In reply to Alexei StarovoitovQuoted 13 lines · Alexei Starovoitov
Quoted 6 lines · Jiri Olsa
In addition it's possible to control the execution of the return probe with the return value of the entry bpf program. If the entry program returns 0 the return probe is installed and executed, otherwise it's skip.
Let's not sneak in a big change in behavior like this.
Quoted 3 lines
(quoted diff) +0
-0@@ -2828,8 +2832,8 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip, struct bpf_kprobe_multi_link *link;
Quoted 8 lines
kernel/trace/bpf_trace.c kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip, · line 2836
Quoted 5 lines
kernel/trace/bpf_trace.c +2
-2link = container_of(fp, struct bpf_kprobe_multi_link, fp); -kprobe_multi_link_prog_run (link,get_entry_ip (fentry_ip), regs); -return 0 ; +return kprobe_multi_link_prog_run (link, link->link.prog, +get_entry_ip (fentry_ip), regs);This is big.
What motivates this change?
hm, this makes the difference only when the new interface is used, so I thought about it as part of the new interface.. but right, if existing programs are attached with new interface and un-aware of this, that might cause issues
in tetragon we use logic where the entry probe does some filtering and related return probe either gathers other data or quit based on that filter result
with this change we can eliminate return probe being triggered completely when the entry filter does not pass
jirka
Quoted 15 lines · Andrii Nakryiko
Quoted 14 lines · Jiri Olsa
hi,
adding support to attach both entry and return bpf program on single kprobe multi link.Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.
At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.
Quoted 8 lines
Quoted 5 lines
In addition it's possible to control the execution of the return probe with the return value of the entry bpf program. If the entry program returns 0 the return probe is installed and executed, otherwise it's skip.
In general, I think this is a very useful ability to have a combined entry/return program.
great, thanks
Quoted 9 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 6 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
Quoted 4 lines
Another frequently requested feature and a very common use case is to measure duration of the function, so if we have a custom type, we can have a field to record entry timestamp and let user calculate
you mean bpf program context right?
Quoted 4 lines
duration, if necessary. Without this users have to pay a bunch of extra overhead to record timestamp and put it into hashmap (keyed by thread id) or thread local storage (even if they have no other use for thread local storage).
sounds good
Quoted 6 lines
Also, consider that a similar concept is applicable to uprobes and we should do that as well, in similar fashion. And the above approach works for both kprobe/kretprobe and uprobe/uretprobe cases, because they have the same
pt_regsdata structure as a context (even if for exit probes most of the values ofpt_regsare not that meaningful).
ok, that seems useful for uprobes as well
btw I have another unrelated change in stash that that let you choose the bpf_prog_active or prog->active re-entry check before program is executed in krobe_multi link.. I also added extra flag, and it still seems like good idea to me, thoughts? ;-)
Quoted 3 lines
So anyways, great feature, but let's discuss end-to-end usability story before we finalize the implementation?
yep, it does say RFC in the subject ;-)
Quoted 42 lines
Quoted 42 lines
I'm still working on the tetragon change, so I'll be sending non-RFC version once that's ready, meanwhile any ideas and feedback on the approach would be great.
The change in bpftrace [1] using the new interface shows speed increase with tracing perf bench messaging:
# perf bench sched messaging -l 100000having system wide bpftrace:
# bpftrace -e 'kprobe:ksys_write { }, kretprobe:ksys_write { }'without bpftrace:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes runTotal time: 119.595 [sec]Performance counter stats for 'perf bench sched messaging -l 100000': 102,419,967,282 cycles:u 5,652,444,107,001 cycles:k 5,782,645,019,612 cycles 22,187,151,206 instructions:u # 0.22 insn per cycle 2,979,040,498,455 instructions:k # 0.53 insn per cycle 119.671169829 seconds time elapsed 94.959198000 seconds user 1815.371616000 seconds syswith current bpftrace:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes run
Quoted 7 lines
Quoted 5 lines
Total time: 221.153 [sec]Performance counter stats for 'perf bench sched messaging -l 100000':
125,292,164,504 cycles:ubtw, why +25% in user space?... this looks weird
hmm right.. maybe there's some bpftrace timer code that got triggered/executed more times because it takes long to finish the workload? I'll check
thanks,
jirka
Quoted 57 lines
Quoted 57 lines
10,315,020,393,735 cycles:k 10,501,379,274,042 cycles 22,187,583,545 instructions:u # 0.18 insn per cycle 4,856,893,111,303 instructions:k # 0.47 insn per cycle221.229234283 seconds time elapsed 103.792498000 seconds user 3432.643302000 seconds syswith bpftrace using the new interface:
# Running 'sched/messaging' benchmark:
# 20 sender and receiver processes per group
# 10 groups == 400 processes runTotal time: 157.825 [sec]Performance counter stats for 'perf bench sched messaging -l 100000': 102,423,112,279 cycles:u 7,450,856,354,744 cycles:k 7,584,769,726,693 cycles 22,187,270,661 instructions:u # 0.22 insn per cycle 3,985,522,383,425 instructions:k # 0.53 insn per cycle 157.900787760 seconds time elapsed 97.953898000 seconds user 2425.314753000 seconds systhanks,
jirka[1] https://github.com/bpftrace/bpftrace/pull/2984
---
Jiri Olsa (4):
fprobe: Add entry/exit callbacks types
bpf: Add return prog to kprobe multi
libbpf: Addreturn_prog_fdto kprobe multi opts selftests/bpf: Add kprobe multi return prog testinclude/linux/fprobe.h | 18 ++++++++++------ include/uapi/linux/bpf.h | 4 +++- kernel/trace/bpf_trace.c | 50 ++++++++++++++++++++++++++++++++----------- tools/include/uapi/linux/bpf.h | 4 +++- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.c | 5 +++++ tools/lib/bpf/libbpf.h | 6 +++++- tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 53 +++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 226 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_return_prog.c
Quoted 33 lines · Jiri Olsa
Quoted 15 lines · Andrii Nakryiko
Quoted 14 lines · Jiri Olsa
hi,
adding support to attach both entry and return bpf program on single kprobe multi link.Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.
At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.
Quoted 8 lines
Quoted 5 lines
In addition it's possible to control the execution of the return probe with the return value of the entry bpf program. If the entry program returns 0 the return probe is installed and executed, otherwise it's skip.
In general, I think this is a very useful ability to have a combined entry/return program.
great, thanks
Quoted 5 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
Quoted 8 lines
Quoted 4 lines
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate attach_type and link type, just like multi-kprobe and multi-uprobe are still kprobe programs.
Quoted 11 lines
Quoted 7 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
Quoted 7 lines
Quoted 5 lines
Another frequently requested feature and a very common use case is to measure duration of the function, so if we have a custom type, we can have a field to record entry timestamp and let user calculate
you mean bpf program context right?
Yes, when I was writing that I was imagining a new context type with new fields.
But thinking about this a bit more, I like a slightly different approach now. Instead of having a new context type, kernel capturing timestamps, etc, what if we introduce a concept of "a session". This paired kprobe+kretprobe (and same for uprobe+uretprobe) share a common session. So, say, if we are tracing kernel function abc() and we have our kwrapper (let's call it that for now, kwrapper and uwrapper?) program attached to abc, we have something like this:
1) abc() is called
2) we intercept it at its entry, initialize "session"
3) call our kwrapper program in entry mode
4) abc() proceeds, until return
5) our kwrapper program is called in exit mode
6) at this point the session has ended
If abc() is called again or in parallel on another CPU, each such abc() invocation (with kwrapper prog) has its own session state.
Now, it all sounds scary, but a session could be really just a 8 byte value that has to last for the duration of a single kprobe+kretprobe execution.
Imagine we introduce just one new kfunc that would be usable from kwrapper and uwrapper programs:
u64 *bpf_get_session_cookie(void);
It returns a pointer to an 8-byte slot which initially is set to zero by kernel (before kprobe/entry part). And then the program can put anything into it. Either from entry (kprobe/uprobe) or exit (kretprobe/uretprobe) executions. Whatever value was stored in entry mode will still be there in exit mode as well.
This gives a lot of flexibility without requiring the kernel to do anything expensive.
E.g., it's easy to detect if kwrapper program is an entry call or exit call:
u64 *cookie = bpf_get_session_cookie();
if (*cookie == 0) {
/* entry mode */
*cookie = 1;
} else {
/* exit mode */
}
Or if we wanted to measure function duration/latency:
u64 *cookie = bpf_get_session_cookie();
if (*cookie == 0) {
*cookie = bpf_get_ktime_ns();
} else {
u64 duration = bpf_get_ktime_ns() - *cookie;
/* output duration somehow */
}
Yes, I realize special-casing zero might be a bit inconvenient, but I think simplicity trumps a potential for zero to be a valid value (and there are always ways to work around zero as a meaningful value).
Now, in more complicated cases 8 bytes of temporary session state isn't enough, just like BPF cookie being 8 byte (read-only) value might not be enough. But the solution is the same as with the BPF cookie. You just use those 8 bytes as a key into ARRAY/HASHMAP/whatnot storage. It's simple and fast enough for pretty much any case.
Now, how do we implement this storage session? I recently looked into uprobe/uretprobe implementation, and I know that for uretprobes there is a dynamically allocated uretprobe_instance (or whatever the name) allocated for each runtime invocation of uretprobe. We can use that. We'll need to allocate this instance a bit earlier, before uprobe part of uwrapper BPF program has to be executed, initialize session cookie to zero, and store pointer to this memory in bpf_run_ctx (just like we do for BPF cookie).
I bet there is something similar in the kretprobe case, where we can carve out 8 bytes and pass it to both entry and exit parts of kwrapper program.
So anyways, might need some internal refactoring, but seems very doable and will give a lot of power and flexibility for tracing use cases.
It would be cool to also have a similar fentry/fexit combo with "BPF session cookie", but that's something we can think about separately, probably.
Quoted 9 lines
Quoted 5 lines
duration, if necessary. Without this users have to pay a bunch of extra overhead to record timestamp and put it into hashmap (keyed by thread id) or thread local storage (even if they have no other use for thread local storage).
sounds good
Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Also, consider that a similar concept is applicable to uprobes and we should do that as well, in similar fashion. And the above approach works for both kprobe/kretprobe and uprobe/uretprobe cases, because they have the same
pt_regsdata structure as a context (even if for exit probes most of the values ofpt_regsare not that meaningful).ok, that seems useful for uprobes as well
yes, absolutely
Quoted 6 lines
btw I have another unrelated change in stash that that let you choose the
bpf_prog_activeorprog->activere-entry check before program is executed inkrobe_multilink.. I also added extra flag, and it still seems like good idea to me, thoughts? ;-)
no specific opinion on this from my side, I guess
Quoted 8 lines
Quoted 3 lines
So anyways, great feature, but let's discuss end-to-end usability story before we finalize the implementation?
yep, it does say RFC in the subject ;-)
Quoted 2 lines
[...]
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
Quoted 8 lines
Quoted 4 lines
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.
ok, having new attach type on top of kprobe_multi link makes sense
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).
Quoted 8 lines
Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
ok, we should probably do that.. given this enhancement will be available on latest kernel anyway, we could use global subprogs as well
the related bpftrace might be bit more challenging.. will have to generate program calling entry or return program now, but seems doable of course
Quoted 71 lines
Quoted 8 lines
Quoted 6 lines
Another frequently requested feature and a very common use case is to measure duration of the function, so if we have a custom type, we can have a field to record entry timestamp and let user calculate
you mean bpf program context right?
Yes, when I was writing that I was imagining a new context type with new fields.
But thinking about this a bit more, I like a slightly different approach now. Instead of having a new context type, kernel capturing timestamps, etc, what if we introduce a concept of "a session". This paired kprobe+kretprobe (and same for uprobe+uretprobe) share a common session. So, say, if we are tracing kernel function
abc()and we have our kwrapper (let's call it that for now, kwrapper and uwrapper?) program attached to abc, we have something like this:1)
abc()is called
2) we intercept it at its entry, initialize "session"
3) call our kwrapper program in entry mode
4)abc()proceeds, until return
5) our kwrapper program is called in exit mode
6) at this point the session has endedIf
abc()is called again or in parallel on another CPU, each suchabc()invocation (with kwrapper prog) has its own session state.Now, it all sounds scary, but a session could be really just a 8 byte value that has to last for the duration of a single kprobe+kretprobe execution.
Imagine we introduce just one new kfunc that would be usable from kwrapper and uwrapper programs:
u64
*bpf_get_session_cookie(void);It returns a pointer to an 8-byte slot which initially is set to zero by kernel (before kprobe/entry part). And then the program can put anything into it. Either from entry (kprobe/uprobe) or exit (kretprobe/uretprobe) executions. Whatever value was stored in entry mode will still be there in exit mode as well.
This gives a lot of flexibility without requiring the kernel to do anything expensive.
E.g., it's easy to detect if kwrapper program is an entry call or exit call:
u64 *cookie =
bpf_get_session_cookie();
if (*cookie == 0) {
/* entry mode */
*cookie = 1;
} else {
/* exit mode */
}Or if we wanted to measure function duration/latency:
u64 *cookie =
bpf_get_session_cookie();
if (*cookie == 0) {
*cookie =bpf_get_ktime_ns();
} else {
u64 duration =bpf_get_ktime_ns()- *cookie;
/* output duration somehow */
}Yes, I realize special-casing zero might be a bit inconvenient, but I think simplicity trumps a potential for zero to be a valid value (and there are always ways to work around zero as a meaningful value).
Quoted 5 lines
Now, in more complicated cases 8 bytes of temporary session state isn't enough, just like BPF cookie being 8 byte (read-only) value might not be enough. But the solution is the same as with the BPF cookie. You just use those 8 bytes as a key into ARRAY/HASHMAP/whatnot storage. It's simple and fast enough for pretty much any case.
I was recently asked for a way to have function arguments available in the return kprobe as it is in fexit programs (which was not an option to use, because we don't have fast multi attach for it)
using the hash map to store arguments and storing its key in the session data might be solution for this
Quoted 10 lines
Now, how do we implement this storage session? I recently looked into uprobe/uretprobe implementation, and I know that for uretprobes there is a dynamically allocated
uretprobe_instance(or whatever the name) allocated for each runtime invocation of uretprobe. We can use that. We'll need to allocate this instance a bit earlier, before uprobe part of uwrapper BPF program has to be executed, initialize session cookie to zero, and store pointer to this memory inbpf_run_ctx(just like we do for BPF cookie).
I think you refer to the return_instance and it seems like we could use it, I'll check on that
Quoted 4 lines
I bet there is something similar in the kretprobe case, where we can carve out 8 bytes and pass it to both entry and exit parts of kwrapper program.
for kprobes.. both kprobe and kprobe_multi/fprobe use rethook to invoke return probes, so I guess we could use it and store that shared data in there
btw Masami is in process of removing rethook from kprobe_multi/fprobe, as part of migrating fprobe on top of ftrace [0]
but instead the rethook I think there'll be some sort of shadow stack/data area accessible from both entry and return probes, that we could use
jirka
[0] https://lore.kernel.org/bpf/170723204881.502590.11906735097521170661.stgit@devnote2/
Quoted 48 lines
So anyways, might need some internal refactoring, but seems very doable and will give a lot of power and flexibility for tracing use cases.
It would be cool to also have a similar fentry/fexit combo with "BPF session cookie", but that's something we can think about separately, probably.
Quoted 9 lines
Quoted 5 lines
duration, if necessary. Without this users have to pay a bunch of extra overhead to record timestamp and put it into hashmap (keyed by thread id) or thread local storage (even if they have no other use for thread local storage).
sounds good
Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Also, consider that a similar concept is applicable to uprobes and we should do that as well, in similar fashion. And the above approach works for both kprobe/kretprobe and uprobe/uretprobe cases, because they have the same
pt_regsdata structure as a context (even if for exit probes most of the values ofpt_regsare not that meaningful).ok, that seems useful for uprobes as well
yes, absolutely
Quoted 6 lines
btw I have another unrelated change in stash that that let you choose the
bpf_prog_activeorprog->activere-entry check before program is executed inkrobe_multilink.. I also added extra flag, and it still seems like good idea to me, thoughts? ;-)no specific opinion on this from my side, I guess
Quoted 8 lines
Quoted 3 lines
So anyways, great feature, but let's discuss end-to-end usability story before we finalize the implementation?
yep, it does say RFC in the subject ;-)
Quoted 2 lines
[...]
Re: [PATCH RFC bpf-next 1/4] fprobe: Add entry/exit callbacks types
In reply to Jiri OlsaHi,
Quoted 3 lines · On Wed, 7 Feb 2024 16:35:47 +0100 Jiri Olsa <[email protected]> wrote:
We are going to store callbacks in following change, so this will ease up the code.
Yeah, this looks good to me.
Hmm, can I pick this in my for-next tree?
Thank you,
Quoted 44 lines
Signed-off-by Jiri Olsa <[email protected]>--- include/linux/fprobe.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)include/linux/fprobe.h +12
-6@@ -7,6 +7,16 @@#include <linux/ftrace.h> #include <linux/rethook.h> +struct fprobe ; + +typedef int (*fprobe_entry_cb )(struct fprobe *fp,unsigned long entry_ip, +unsigned long ret_ip,struct pt_regs *regs, +void *entry_data); + +typedef void (*fprobe_exit_cb )(struct fprobe *fp,unsigned long entry_ip, +unsigned long ret_ip,struct pt_regs *regs, +void *entry_data); +/** * struct fprobe - ftrace based probe. * @ops: The ftrace_ops. @@ -34,12 +44,8 @@ struct fprobe {size_t entry_data_size;int nr_maxactive; -int (*entry_handler)(struct fprobe *fp,unsigned long entry_ip, -unsigned long ret_ip,struct pt_regs *regs, -void *entry_data); -void (*exit_handler)(struct fprobe *fp,unsigned long entry_ip, -unsigned long ret_ip,struct pt_regs *regs, -void *entry_data); +fprobe_entry_cb entry_handler; +fprobe_exit_cb exit_handler; };/* This fprobe is soft-disabled. */ Signature · 1 line
2.43.0
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Quoted 44 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
Quoted 8 lines
Quoted 4 lines
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes senseQuoted 9 lines
Quoted 9 lines
Quoted 8 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 8 lines
Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
Quoted 7 lines
ok, we should probably do that.. given this enhancement will be available on latest kernel anyway, we could use global subprogs as well
the related bpftrace might be bit more challenging.. will have to generate program calling entry or return program now, but seems doable of course
So you want users to still have separate kprobe and kretprobe in bpftrace, but combine them into this kwrapper transparently? It does seem doable, but hopefully we'll be able to write kwrapper programs in bpftrace directly as well.
Quoted 78 lines
Quoted 72 lines
Quoted 9 lines
Quoted 7 lines
Another frequently requested feature and a very common use case is to measure duration of the function, so if we have a custom type, we can have a field to record entry timestamp and let user calculate
you mean bpf program context right?
Yes, when I was writing that I was imagining a new context type with new fields.
But thinking about this a bit more, I like a slightly different approach now. Instead of having a new context type, kernel capturing timestamps, etc, what if we introduce a concept of "a session". This paired kprobe+kretprobe (and same for uprobe+uretprobe) share a common session. So, say, if we are tracing kernel function
abc()and we have our kwrapper (let's call it that for now, kwrapper and uwrapper?) program attached to abc, we have something like this:1)
abc()is called
2) we intercept it at its entry, initialize "session"
3) call our kwrapper program in entry mode
4)abc()proceeds, until return
5) our kwrapper program is called in exit mode
6) at this point the session has endedIf
abc()is called again or in parallel on another CPU, each suchabc()invocation (with kwrapper prog) has its own session state.Now, it all sounds scary, but a session could be really just a 8 byte value that has to last for the duration of a single kprobe+kretprobe execution.
Imagine we introduce just one new kfunc that would be usable from kwrapper and uwrapper programs:
u64
*bpf_get_session_cookie(void);It returns a pointer to an 8-byte slot which initially is set to zero by kernel (before kprobe/entry part). And then the program can put anything into it. Either from entry (kprobe/uprobe) or exit (kretprobe/uretprobe) executions. Whatever value was stored in entry mode will still be there in exit mode as well.
This gives a lot of flexibility without requiring the kernel to do anything expensive.
E.g., it's easy to detect if kwrapper program is an entry call or exit call:
u64 *cookie =
bpf_get_session_cookie();
if (*cookie == 0) {
/* entry mode */
*cookie = 1;
} else {
/* exit mode */
}Or if we wanted to measure function duration/latency:
u64 *cookie =
bpf_get_session_cookie();
if (*cookie == 0) {
*cookie =bpf_get_ktime_ns();
} else {
u64 duration =bpf_get_ktime_ns()- *cookie;
/* output duration somehow */
}Yes, I realize special-casing zero might be a bit inconvenient, but I think simplicity trumps a potential for zero to be a valid value (and there are always ways to work around zero as a meaningful value).
Quoted 5 lines
Now, in more complicated cases 8 bytes of temporary session state isn't enough, just like BPF cookie being 8 byte (read-only) value might not be enough. But the solution is the same as with the BPF cookie. You just use those 8 bytes as a key into ARRAY/HASHMAP/whatnot storage. It's simple and fast enough for pretty much any case.
Quoted 6 lines
I was recently asked for a way to have function arguments available in the return kprobe as it is in fexit programs (which was not an option to use, because we don't have fast multi attach for it)
using the hash map to store arguments and storing its key in the session data might be solution for this
if you are ok using hashmap keyed by tid, you can do it today without any kernel changes. With session cookie you'll be able to utilize faster ARRAY map (by building a simple ID allocator to get a free slot in ARRAY map).
Quoted 12 lines
Quoted 11 lines
Now, how do we implement this storage session? I recently looked into uprobe/uretprobe implementation, and I know that for uretprobes there is a dynamically allocated
uretprobe_instance(or whatever the name) allocated for each runtime invocation of uretprobe. We can use that. We'll need to allocate this instance a bit earlier, before uprobe part of uwrapper BPF program has to be executed, initialize session cookie to zero, and store pointer to this memory inbpf_run_ctx(just like we do for BPF cookie).
Quoted 3 lines
I think you refer to the
return_instanceand it seems like we could use it, I'll check on that
yep, probably
Quoted 9 lines
Quoted 4 lines
I bet there is something similar in the kretprobe case, where we can carve out 8 bytes and pass it to both entry and exit parts of kwrapper program.
for kprobes.. both kprobe and kprobe_multi/fprobe use rethook to invoke return probes, so I guess we could use it and store that shared data in there
Quoted 5 lines
btw Masami is in process of removing rethook from kprobe_multi/fprobe, as part of migrating fprobe on top of ftrace [0]
but instead the rethook I think there'll be some sort of shadow stack/data area accessible from both entry and return probes, that we could use
ok, cool. We also need to be careful to not share session cookie between unrelated programs. E.g., if two independent kwrapper programs are attached to the same function, they should each have their own cookie. Otherwise it's not clear how to build anything reliable on top of that, tbh. This might be a problem, though, right?
Quoted 54 lines
jirka
[0] https://lore.kernel.org/bpf/170723204881.502590.11906735097521170661.stgit@devnote2/
Quoted 48 lines
So anyways, might need some internal refactoring, but seems very doable and will give a lot of power and flexibility for tracing use cases.
It would be cool to also have a similar fentry/fexit combo with "BPF session cookie", but that's something we can think about separately, probably.
Quoted 9 lines
Quoted 5 lines
duration, if necessary. Without this users have to pay a bunch of extra overhead to record timestamp and put it into hashmap (keyed by thread id) or thread local storage (even if they have no other use for thread local storage).
sounds good
Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Also, consider that a similar concept is applicable to uprobes and we should do that as well, in similar fashion. And the above approach works for both kprobe/kretprobe and uprobe/uretprobe cases, because they have the same
pt_regsdata structure as a context (even if for exit probes most of the values ofpt_regsare not that meaningful).ok, that seems useful for uprobes as well
yes, absolutely
Quoted 6 lines
btw I have another unrelated change in stash that that let you choose the
bpf_prog_activeorprog->activere-entry check before program is executed inkrobe_multilink.. I also added extra flag, and it still seems like good idea to me, thoughts? ;-)no specific opinion on this from my side, I guess
Quoted 8 lines
Quoted 3 lines
So anyways, great feature, but let's discuss end-to-end usability story before we finalize the implementation?
yep, it does say RFC in the subject ;-)
Quoted 2 lines
[...]
Quoted 49 lines · Andrii Nakryiko
Quoted 48 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
Quoted 8 lines
Quoted 4 lines
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes senseQuoted 9 lines
Quoted 9 lines
Quoted 8 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 8 lines
Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
ok, we should probably do that.. given this enhancement will be available on latest kernel anyway, we could use global subprogs as well
Quoted 6 lines
Quoted 3 lines
the related bpftrace might be bit more challenging.. will have to generate program calling entry or return program now, but seems doable of course
So you want users to still have separate kprobe and kretprobe in bpftrace, but combine them into this kwrapper transparently? It does
no I meant I'd need to generate the wrapper program for the new interface.. which is extra compared to current bpftrace changes
Quoted 2 lines
seem doable, but hopefully we'll be able to write kwrapper programs in bpftrace directly as well.
yes, it should be fine
SNIP
Quoted 15 lines
Quoted 15 lines
Quoted 10 lines
Yes, I realize special-casing zero might be a bit inconvenient, but I think simplicity trumps a potential for zero to be a valid value (and there are always ways to work around zero as a meaningful value).
Now, in more complicated cases 8 bytes of temporary session state isn't enough, just like BPF cookie being 8 byte (read-only) value might not be enough. But the solution is the same as with the BPF cookie. You just use those 8 bytes as a key into ARRAY/HASHMAP/whatnot storage. It's simple and fast enough for pretty much any case.
I was recently asked for a way to have function arguments available in the return kprobe as it is in fexit programs (which was not an option to use, because we don't have fast multi attach for it)
Quoted 7 lines
Quoted 2 lines
using the hash map to store arguments and storing its key in the session data might be solution for this
if you are ok using hashmap keyed by tid, you can do it today without any kernel changes. With session cookie you'll be able to utilize faster ARRAY map (by building a simple ID allocator to get a free slot in ARRAY map).
ok
SNIP
Quoted 11 lines
Quoted 11 lines
Quoted 3 lines
I bet there is something similar in the kretprobe case, where we can carve out 8 bytes and pass it to both entry and exit parts of kwrapper program.
for kprobes.. both kprobe and kprobe_multi/fprobe use rethook to invoke return probes, so I guess we could use it and store that shared data in there
btw Masami is in process of removing rethook from kprobe_multi/fprobe, as part of migrating fprobe on top of ftrace [0]
Quoted 8 lines
Quoted 2 lines
but instead the rethook I think there'll be some sort of shadow stack/data area accessible from both entry and return probes, that we could use
ok, cool. We also need to be careful to not share session cookie between unrelated programs. E.g., if two independent kwrapper programs are attached to the same function, they should each have their own cookie. Otherwise it's not clear how to build anything reliable on top of that, tbh. This might be a problem, though, right?
IIRC it's tracer specific data, the shadow stack data should be unique for tracer and its called function, but I'll double check on that
jirka
Quoted 108 lines · Jiri Olsa
Quoted 49 lines · Andrii Nakryiko
Quoted 48 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
Quoted 8 lines
Quoted 4 lines
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes senseQuoted 9 lines
Quoted 9 lines
Quoted 8 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 8 lines
Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
ok, we should probably do that.. given this enhancement will be available on latest kernel anyway, we could use global subprogs as well
Quoted 6 lines
Quoted 3 lines
the related bpftrace might be bit more challenging.. will have to generate program calling entry or return program now, but seems doable of course
So you want users to still have separate kprobe and kretprobe in bpftrace, but combine them into this kwrapper transparently? It does
no I meant I'd need to generate the wrapper program for the new interface.. which is extra compared to current bpftrace changes
Quoted 2 lines
seem doable, but hopefully we'll be able to write kwrapper programs in bpftrace directly as well.
yes, it should be fine
SNIP
Quoted 15 lines
Quoted 15 lines
Quoted 10 lines
Yes, I realize special-casing zero might be a bit inconvenient, but I think simplicity trumps a potential for zero to be a valid value (and there are always ways to work around zero as a meaningful value).
Now, in more complicated cases 8 bytes of temporary session state isn't enough, just like BPF cookie being 8 byte (read-only) value might not be enough. But the solution is the same as with the BPF cookie. You just use those 8 bytes as a key into ARRAY/HASHMAP/whatnot storage. It's simple and fast enough for pretty much any case.
I was recently asked for a way to have function arguments available in the return kprobe as it is in fexit programs (which was not an option to use, because we don't have fast multi attach for it)
Quoted 7 lines
Quoted 2 lines
using the hash map to store arguments and storing its key in the session data might be solution for this
if you are ok using hashmap keyed by tid, you can do it today without any kernel changes. With session cookie you'll be able to utilize faster ARRAY map (by building a simple ID allocator to get a free slot in ARRAY map).
ok
SNIP
Quoted 14 lines
Quoted 13 lines
Quoted 3 lines
I bet there is something similar in the kretprobe case, where we can carve out 8 bytes and pass it to both entry and exit parts of kwrapper program.
for kprobes.. both kprobe and kprobe_multi/fprobe use rethook to invoke return probes, so I guess we could use it and store that shared data in there
btw Masami is in process of removing rethook from kprobe_multi/fprobe, as part of migrating fprobe on top of ftrace [0]
but instead the rethook I think there'll be some sort of shadow stack/data area accessible from both entry and return probes, that we could use
Quoted 8 lines
Quoted 5 lines
ok, cool. We also need to be careful to not share session cookie between unrelated programs. E.g., if two independent kwrapper programs are attached to the same function, they should each have their own cookie. Otherwise it's not clear how to build anything reliable on top of that, tbh. This might be a problem, though, right?
IIRC it's tracer specific data, the shadow stack data should be unique for tracer and its called function, but I'll double check on that
Masami,
we recently discussed the possibility to store data between entry/return probe, IIUC your current patchset [0] allows that, but it seems to be shared across all the tracers for the given function (__ftrace_return_to_handler).. is the plan to make the shadow stack per tracer and function? /cc Steven
thanks,
jirka
[0] https://lore.kernel.org/bpf/170723204881.502590.11906735097521170661.stgit@devnote2/
Re: [PATCH RFC bpf-next 1/4] fprobe: Add entry/exit callbacks types
In reply to Masami Hiramatsu (Google)Quoted 9 lines · Masami Hiramatsu
Hi,
Quoted 3 lines · On Wed, 7 Feb 2024 16:35:47 +0100 Jiri Olsa <[email protected]> wrote:
We are going to store callbacks in following change, so this will ease up the code.
Quoted 5 lines
Yeah, this looks good to me.
Hmm, can I pick this in my for-next tree?
I don't mind you picking that up, I'll have to send new version of the rest of the patchset, but I think I'll still need these types
jirka
Quoted 51 lines
Thank you,
Quoted 44 lines
Signed-off-by Jiri Olsa <[email protected]>--- include/linux/fprobe.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)include/linux/fprobe.h +12
-6@@ -7,6 +7,16 @@#include <linux/ftrace.h> #include <linux/rethook.h> +struct fprobe ; + +typedef int (*fprobe_entry_cb )(struct fprobe *fp,unsigned long entry_ip, +unsigned long ret_ip,struct pt_regs *regs, +void *entry_data); + +typedef void (*fprobe_exit_cb )(struct fprobe *fp,unsigned long entry_ip, +unsigned long ret_ip,struct pt_regs *regs, +void *entry_data); +/** * struct fprobe - ftrace based probe. * @ops: The ftrace_ops. @@ -34,12 +44,8 @@ struct fprobe {size_t entry_data_size;int nr_maxactive; -int (*entry_handler)(struct fprobe *fp,unsigned long entry_ip, -unsigned long ret_ip,struct pt_regs *regs, -void *entry_data); -void (*exit_handler)(struct fprobe *fp,unsigned long entry_ip, -unsigned long ret_ip,struct pt_regs *regs, -void *entry_data); +fprobe_entry_cb entry_handler; +fprobe_exit_cb exit_handler; };/* This fprobe is soft-disabled. */ Signature · 1 line
2.43.0Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Steven Rostedt
maintainerQuoted 5 lines · On Wed, 14 Feb 2024 21:55:03 +0100 Jiri Olsa <[email protected]> wrote:
Masami,
we recently discussed the possibility to store data between entry/return probe, IIUC your current patchset [0] allows that, but it seems to be shared across all the tracers for the given function (__ftrace_return_to_handler).. is the plan to make the shadow stack per tracer and function? /cc Steven
The shadow stack is per task, but it saves unique data per tracer per function.
It allows up to 16 different tracers attached to function graph tracing at a time, as there is a limited shadow stack size, which all the attached tracers use. Now you can create your own shadow stack per user (bpf program?) and have a single registered function graph user that will demux the data coming in and out of the function.
-- Steve
Quoted 10 lines · Steven Rostedt
Quoted 5 lines · On Wed, 14 Feb 2024 21:55:03 +0100 Jiri Olsa <[email protected]> wrote:
Masami,
we recently discussed the possibility to store data between entry/return probe, IIUC your current patchset [0] allows that, but it seems to be shared across all the tracers for the given function (__ftrace_return_to_handler).. is the plan to make the shadow stack per tracer and function? /cc Steven
Quoted 7 lines
The shadow stack is per task, but it saves unique data per tracer per function.
It allows up to 16 different tracers attached to function graph tracing at a time, as there is a limited shadow stack size, which all the attached tracers use. Now you can create your own shadow stack per user (bpf program?) and have a single registered function graph user that will demux the data coming in and out of the function.
ok, I guess that's fgraph_reserve_data/fgraph_retrieve_data api
we'd like to share data between bpf programs executed from entry and exit handlers, that should do it
thanks,
jirka
Quoted 54 lines · Jiri Olsa
Quoted 53 lines · Andrii Nakryiko
Quoted 51 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
Quoted 8 lines
Quoted 4 lines
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes senseQuoted 9 lines
Quoted 9 lines
Quoted 8 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 8 lines
Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
ok, we should probably do that.. given this enhancement will be available on latest kernel anyway, we could use global subprogs as well
the related bpftrace might be bit more challenging.. will have to generate program calling entry or return program now, but seems doable of course
Quoted 5 lines
Quoted 2 lines
So you want users to still have separate kprobe and kretprobe in bpftrace, but combine them into this kwrapper transparently? It does
no I meant I'd need to generate the wrapper program for the new interface.. which is extra compared to current bpftrace changes
If you end up introducing this new kwrapper program type in libbpf, I think that it'll make sense to have something similar in bpftrace, too. Allowing users to write separate kprobe and kretprobe programs and transparently combining them into kwrapper doesn't seem to bring much value to me.
Jirka, if you need help with implementing bpftrace support for this, let me know. I'm very interested in having this capability there.
Viktor
Quoted 60 lines
Quoted 3 lines
seem doable, but hopefully we'll be able to write kwrapper programs in bpftrace directly as well.
yes, it should be fine
SNIP
Quoted 15 lines
Quoted 15 lines
Quoted 10 lines
Yes, I realize special-casing zero might be a bit inconvenient, but I think simplicity trumps a potential for zero to be a valid value (and there are always ways to work around zero as a meaningful value).
Now, in more complicated cases 8 bytes of temporary session state isn't enough, just like BPF cookie being 8 byte (read-only) value might not be enough. But the solution is the same as with the BPF cookie. You just use those 8 bytes as a key into ARRAY/HASHMAP/whatnot storage. It's simple and fast enough for pretty much any case.
I was recently asked for a way to have function arguments available in the return kprobe as it is in fexit programs (which was not an option to use, because we don't have fast multi attach for it)
Quoted 7 lines
Quoted 2 lines
using the hash map to store arguments and storing its key in the session data might be solution for this
if you are ok using hashmap keyed by tid, you can do it today without any kernel changes. With session cookie you'll be able to utilize faster ARRAY map (by building a simple ID allocator to get a free slot in ARRAY map).
ok
SNIP
Quoted 11 lines
Quoted 11 lines
Quoted 3 lines
I bet there is something similar in the kretprobe case, where we can carve out 8 bytes and pass it to both entry and exit parts of kwrapper program.
for kprobes.. both kprobe and kprobe_multi/fprobe use rethook to invoke return probes, so I guess we could use it and store that shared data in there
btw Masami is in process of removing rethook from kprobe_multi/fprobe, as part of migrating fprobe on top of ftrace [0]
Quoted 8 lines
Quoted 2 lines
but instead the rethook I think there'll be some sort of shadow stack/data area accessible from both entry and return probes, that we could use
ok, cool. We also need to be careful to not share session cookie between unrelated programs. E.g., if two independent kwrapper programs are attached to the same function, they should each have their own cookie. Otherwise it's not clear how to build anything reliable on top of that, tbh. This might be a problem, though, right?
IIRC it's tracer specific data, the shadow stack data should be unique for tracer and its called function, but I'll double check on that
jirka
Quoted 58 lines · Viktor Malik
Quoted 57 lines · Jiri Olsa
Quoted 55 lines · Andrii Nakryiko
Quoted 44 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
Quoted 8 lines
Quoted 4 lines
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes senseQuoted 9 lines
Quoted 9 lines
Quoted 8 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 8 lines
Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
Quoted 7 lines
ok, we should probably do that.. given this enhancement will be available on latest kernel anyway, we could use global subprogs as well
the related bpftrace might be bit more challenging.. will have to generate program calling entry or return program now, but seems doable of course
So you want users to still have separate kprobe and kretprobe in bpftrace, but combine them into this kwrapper transparently? It does
Quoted 8 lines
Quoted 2 lines
no I meant I'd need to generate the wrapper program for the new interface.. which is extra compared to current bpftrace changes
If you end up introducing this new kwrapper program type in libbpf, I think that it'll make sense to have something similar in bpftrace, too. Allowing users to write separate kprobe and kretprobe programs and transparently combining them into kwrapper doesn't seem to bring much value to me.
I kind of liked the idea of not introducing new probe type and silently making things faster ;-) but having wrapper probe type makes also sense
Quoted 3 lines
Jirka, if you need help with implementing bpftrace support for this, let me know. I'm very interested in having this capability there.
that'd be great, I'll send new version of kernel changes soon
thanks,
jirka
Quoted 17 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 12 lines
Quoted 8 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 7 lines
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes sense
hi,
I have further question in here.. ;-)
so I implemented the new behaviour on top of new attach_type, but I keep thinking that it's the 'same/similar logic' as if I added extra flag to attr.link_create.kprobe_multi.flags, which results in much simpler change
is following logic breaking backward compatibility in any way?
- having new flag in attr.link_create.kprobe_multi.flags
- that force the attach of the bpf program to entry/exit function probes
- and the kprobe entry program return value now controls invocation on the exit probe
so the new flag changes the semantics of the entry program return value, which seems fine to me, because it depends on the new flag.. do I miss any other problem with that?
thanks,
jirka
Quoted 193 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 8 lines
Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
ok, we should probably do that.. given this enhancement will be available on latest kernel anyway, we could use global subprogs as well
the related bpftrace might be bit more challenging.. will have to generate program calling entry or return program now, but seems doable of course
Quoted 71 lines
Quoted 8 lines
Quoted 6 lines
Another frequently requested feature and a very common use case is to measure duration of the function, so if we have a custom type, we can have a field to record entry timestamp and let user calculate
you mean bpf program context right?
Yes, when I was writing that I was imagining a new context type with new fields.
But thinking about this a bit more, I like a slightly different approach now. Instead of having a new context type, kernel capturing timestamps, etc, what if we introduce a concept of "a session". This paired kprobe+kretprobe (and same for uprobe+uretprobe) share a common session. So, say, if we are tracing kernel function
abc()and we have our kwrapper (let's call it that for now, kwrapper and uwrapper?) program attached to abc, we have something like this:1)
abc()is called
2) we intercept it at its entry, initialize "session"
3) call our kwrapper program in entry mode
4)abc()proceeds, until return
5) our kwrapper program is called in exit mode
6) at this point the session has endedIf
abc()is called again or in parallel on another CPU, each suchabc()invocation (with kwrapper prog) has its own session state.Now, it all sounds scary, but a session could be really just a 8 byte value that has to last for the duration of a single kprobe+kretprobe execution.
Imagine we introduce just one new kfunc that would be usable from kwrapper and uwrapper programs:
u64
*bpf_get_session_cookie(void);It returns a pointer to an 8-byte slot which initially is set to zero by kernel (before kprobe/entry part). And then the program can put anything into it. Either from entry (kprobe/uprobe) or exit (kretprobe/uretprobe) executions. Whatever value was stored in entry mode will still be there in exit mode as well.
This gives a lot of flexibility without requiring the kernel to do anything expensive.
E.g., it's easy to detect if kwrapper program is an entry call or exit call:
u64 *cookie =
bpf_get_session_cookie();
if (*cookie == 0) {
/* entry mode */
*cookie = 1;
} else {
/* exit mode */
}Or if we wanted to measure function duration/latency:
u64 *cookie =
bpf_get_session_cookie();
if (*cookie == 0) {
*cookie =bpf_get_ktime_ns();
} else {
u64 duration =bpf_get_ktime_ns()- *cookie;
/* output duration somehow */
}Yes, I realize special-casing zero might be a bit inconvenient, but I think simplicity trumps a potential for zero to be a valid value (and there are always ways to work around zero as a meaningful value).
Quoted 5 lines
Now, in more complicated cases 8 bytes of temporary session state isn't enough, just like BPF cookie being 8 byte (read-only) value might not be enough. But the solution is the same as with the BPF cookie. You just use those 8 bytes as a key into ARRAY/HASHMAP/whatnot storage. It's simple and fast enough for pretty much any case.
I was recently asked for a way to have function arguments available in the return kprobe as it is in fexit programs (which was not an option to use, because we don't have fast multi attach for it)
using the hash map to store arguments and storing its key in the session data might be solution for this
Quoted 10 lines
Now, how do we implement this storage session? I recently looked into uprobe/uretprobe implementation, and I know that for uretprobes there is a dynamically allocated
uretprobe_instance(or whatever the name) allocated for each runtime invocation of uretprobe. We can use that. We'll need to allocate this instance a bit earlier, before uprobe part of uwrapper BPF program has to be executed, initialize session cookie to zero, and store pointer to this memory inbpf_run_ctx(just like we do for BPF cookie).I think you refer to the
return_instanceand it seems like we could use it, I'll check on thatQuoted 4 lines
I bet there is something similar in the kretprobe case, where we can carve out 8 bytes and pass it to both entry and exit parts of kwrapper program.
for kprobes.. both kprobe and kprobe_multi/fprobe use rethook to invoke return probes, so I guess we could use it and store that shared data in there
btw Masami is in process of removing rethook from kprobe_multi/fprobe, as part of migrating fprobe on top of ftrace [0]
but instead the rethook I think there'll be some sort of shadow stack/data area accessible from both entry and return probes, that we could use
jirka
[0] https://lore.kernel.org/bpf/170723204881.502590.11906735097521170661.stgit@devnote2/
Quoted 48 lines
So anyways, might need some internal refactoring, but seems very doable and will give a lot of power and flexibility for tracing use cases.
It would be cool to also have a similar fentry/fexit combo with "BPF session cookie", but that's something we can think about separately, probably.
Quoted 9 lines
Quoted 5 lines
duration, if necessary. Without this users have to pay a bunch of extra overhead to record timestamp and put it into hashmap (keyed by thread id) or thread local storage (even if they have no other use for thread local storage).
sounds good
Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Also, consider that a similar concept is applicable to uprobes and we should do that as well, in similar fashion. And the above approach works for both kprobe/kretprobe and uprobe/uretprobe cases, because they have the same
pt_regsdata structure as a context (even if for exit probes most of the values ofpt_regsare not that meaningful).ok, that seems useful for uprobes as well
yes, absolutely
Quoted 6 lines
btw I have another unrelated change in stash that that let you choose the
bpf_prog_activeorprog->activere-entry check before program is executed inkrobe_multilink.. I also added extra flag, and it still seems like good idea to me, thoughts? ;-)no specific opinion on this from my side, I guess
Quoted 8 lines
Quoted 3 lines
So anyways, great feature, but let's discuss end-to-end usability story before we finalize the implementation?
yep, it does say RFC in the subject ;-)
Quoted 2 lines
[...]
Quoted 27 lines · Jiri Olsa
Quoted 17 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 12 lines
Quoted 8 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 7 lines
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes sense
Quoted 8 lines
hi,
I have further question in here.. ;-)so I implemented the new behaviour on top of new
attach_type, but I keep thinking that it's the 'same/similar logic' as if I added extra flag to attr.link_create.kprobe_multi.flags, which results in much simpler changeis following logic breaking backward compatibility in any way?
Even if it doesn't break compatibility, isn't new attach_type better to explicitly show that semantics is completely different (it's two BPF program executions, before and after, for each specified kprobe), isn't that different enough? We can also have a different set of helpers for this kwrapper/uwrapper program type (probably could do it through flags as well, but attach_type again seems a better fit here..)?
What makes it harder if you are doing it as a new attach_type?
Quoted 32 lines
- having new flag in attr.link_create.kprobe_multi.flags
- that force the attach of the bpf program to entry/exit function probes
- and the kprobe entry program return value now controls invocation on the exit probeso the new flag changes the semantics of the entry program return value, which seems fine to me, because it depends on the new flag.. do I miss any other problem with that?
thanks,
jirkaQuoted 19 lines
Quoted 18 lines
Quoted 10 lines
Quoted 9 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
[...]
Quoted 46 lines · Andrii Nakryiko
Quoted 27 lines · Jiri Olsa
Quoted 17 lines · Jiri Olsa
On Mon, Feb 12, 2024 at 08:06:06PM -0800, Andrii Nakryiko wrote:
SNIP
Quoted 13 lines
Quoted 12 lines
Quoted 8 lines
But the way you implement it with extra flag and extra fd parameter makes it harder to have a nice high-level support in libbpf (and presumably other BPF loader libraries) for this.
When I was thinking about doing something like this, I was considering adding a new program type, actually. That way it's possible to define this "let's skip return probe" protocol without backwards compatibility concerns. It's easier to use it declaratively in libbpf.
ok, that seems cleaner.. but we need to use current kprobe programs, so not sure at the moment how would that fit in.. did you mean new link type?
Quoted 7 lines
Quoted 5 lines
It's kind of a less important detail, actually. New program type would allow us to have an entirely different context type, but I think we can make do with the existing kprobe program type. We can have a separate
attach_typeand link type, just like multi-kprobe and multi-uprobe are still kprobe programs.ok, having new attach type on top of
kprobe_multilink makes senseQuoted 8 lines
hi,
I have further question in here.. ;-)so I implemented the new behaviour on top of new
attach_type, but I keep thinking that it's the 'same/similar logic' as if I added extra flag to attr.link_create.kprobe_multi.flags, which results in much simpler changeis following logic breaking backward compatibility in any way?
Even if it doesn't break compatibility, isn't new
attach_typebetter to explicitly show that semantics is completely different (it's two BPF program executions, before and after, for each specified kprobe), isn't that different enough? We can also have a different set of helpers for this kwrapper/uwrapper program type (probably could do it through flags as well, butattach_typeagain seems a better fit here..)?
Quoted 1 line
What makes it harder if you are doing it as a new
attach_type?
I just looked at the RFC patches you sent, and I guess it looks fine to me with the flag as well, given kretprobe.multi is also specified through flags.
Quoted 35 lines
Quoted 33 lines
- having new flag in attr.link_create.kprobe_multi.flags
- that force the attach of the bpf program to entry/exit function probes
- and the kprobe entry program return value now controls invocation on the exit probeso the new flag changes the semantics of the entry program return value, which seems fine to me, because it depends on the new flag.. do I miss any other problem with that?
thanks,
jirkaQuoted 19 lines
Quoted 18 lines
Quoted 10 lines
Quoted 9 lines
You just declare
SEC("kprobe.wrap/...")(or whatever the name, something to designate that it's both entry and exit probe) as one program and in the code there would be some way to determine whether we are in entry mode or exit mode (helper or field in the custom context type, the latter being faster and more usable, but it's probably not critical).Quoted 3 lines
hum, so the single program would be for both entry and exit probe, I'll need to check how bad it'd be for us, but it'd probably mean just one extra tail call, so it's likely ok
I guess, I don't know what you are doing there :) I'd recommend looking at utilizing BPF global subprogs instead of tail calls, if your kernel allows for that, as that's a saner way to scale BPF verification.
[...]