bpf: Add tracing_multi link support for bpf progs
v2 cover bpf: Add tracing_multi link support for bpf progs
Similar to the tracing_multi link support for kernel functions [1], add support for bpf progs.
With this support, I'll be able to enhance the function-graph feature of bpfsnoop [2]. It will trace all bpf progs, including their subprogs, to draw a full function call graph including bpf prog call sites.
The attachment could speed-up when there are hundreds of targets. For example, on high CPU core count servers, it probably runs 200 tc-bpf progs as Kubernetes CNI, and each tc-bpf prog might have 4 subprogs. Hence, there will be 1000 targets to trace for bpfsnoop's function-graph feature.
With tracing_multi link support for bpf progs, bpfsnoop will be able to attach to all targets quickly.
When attaching to 1000 targets, fentry.multi is much faster than fentry on a 16c16g QEMU VM on an AMD EPYC 9745 server.
./bench tracing-multi-attach-progs
Setting up benchmark 'tracing-multi-attach-progs'...
tracing-multi-attach-progs: prepared 1000 identical BPF program targets tracing-multi-attach-progs: fentry created and attached 1000 programs/links in 338.171ms tracing-multi-attach-progs: fentry.multi created one program and attached one 1000-target link in 47.345ms tracing-multi-attach-progs: fentry.multi creation/attachment speedup is 7.14x
Unlike the tracing_multi link support for kernel functions, the support for bpf progs does not rely on the ftrace CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS. Hence, the support for bpf progs will apply to all archs that have trampoline support by design.
Links:
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://github.com/bpfsnoop/bpfsnoop
Changes:
v1 -> v2:
* Drop the factoring update_fentry_multi() helper.
* Add bpf_arch_text_poke_batch() to x86_64 JIT backend.
* Add bpf_text_poke_batch() helper.
* Move trampoline_lock_all(), trampoline_unlock_all(), and bpf_trampoline_multi_attach_init() out of the CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard.
* Factor out bpf_trampoline_multi_prog_attach/detach() to get rid of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS.
* Shrink CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard scope for bpf_tracing_multi_attach().
* Explain justification in cover letter. (Andrii)
* Check '!funcs[i]' before libbpf_find_prog_btf_id() in libbpf. (Sashiko)
* Add BPF_F_TRACING_MULTI_TGT_PROGS in link info UAPI. (Sashiko)
* Verify '\0' before skipping '_' when parsing prog name from symbol name in bpftool. (Sashiko)
* Expose libbpf's libbpf_find_prog_btf_id() in attach failure tests. (Sashiko)
* Select the target prog according to the 'first' value. (Sashiko)
* Add qsort() in link info test's setup_tmulti_bpf_prog_targets(). (Sashiko)
* Drop '((u64)tgt_prog->aux->id << 32) | ids[i]' in link fdinfo. (bot+bpf-ci)
* Use different cookie values for different targets in cookie tests. (bot+bpf-ci)
* v1: https://lore.kernel.org/bpf/[email protected]/
Leon Hwang (21):
bpf: Initialize ftrace_managed in bpf_trampoline_get()
bpf: Drop unnecessary ftrace_location() in
register/unregister/modify_fentry_multi()
bpf, x86: Factor out bpf_arch_text_poke_prepare_insns() helper on
x86_64
bpf, x86: Factor out bpf_arch_text_poke_check_ip() helper on x86_64 bpf, x86: Introduce bpf_arch_text_poke_batch() on x86_64
bpf: Add bpf_text_poke_batch() helper in trampoline
bpf: Check tgt_prog before is_tracing_multi_id() in verifier
bpf: Move helpers out of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard in
trampoline
bpf: Add bpf_trampoline_multi_prog_attach/detach()
bpf: Shrink CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard scope for
bpf_tracing_multi_attach()
bpf: Add tracing_multi link support for bpf progs
libbpf: Add tracing_multi link support for bpf progs
bpf: Add tracing_multi link fdinfo support for bpf progs
bpf: Add tracing_multi link info support for bpf progs selftests/bpf: Add tracing_multi bpf prog attach test selftests/bpf: Add tracing_multi bpf prog attach failure tests selftests/bpf: Add tracing_multi bpf prog cookie test selftests/bpf: Add tracing_multi bpf prog rollback test selftests/bpf: Add tracing_multi bpf prog link info test selftests/bpf: Test tailcall with fentry.multi selftests/bpf: Benchmark tracing_multi bpf progs attachment
arch/x86/net/bpf_jit_comp.c | 100 +++++- include/linux/bpf.h | 38 +- include/uapi/linux/bpf.h | 9 +- kernel/bpf/core.c | 5 + kernel/bpf/trampoline.c | 339 ++++++++++++++++-- kernel/bpf/verifier.c | 22 +- kernel/trace/bpf_trace.c | 243 ++++++++++--- tools/bpf/bpftool/link.c | 63 +++- tools/include/uapi/linux/bpf.h | 9 +- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/bpf.h | 2 + tools/lib/bpf/libbpf.c | 43 ++- tools/lib/bpf/libbpf.h | 4 +- tools/lib/bpf/libbpf_internal.h | 1 + tools/testing/selftests/bpf/Makefile | 1 + tools/testing/selftests/bpf/bench.c | 5 + .../benchs/bench_tracing_multi_attach_progs.c | 193 ++++++++++ .../selftests/bpf/prog_tests/fill_link_info.c | 169 ++++++++- .../selftests/bpf/prog_tests/tailcalls.c | 64 ++++ .../selftests/bpf/prog_tests/tracing_multi.c | 254 +++++++++++++ .../bpf/progs/tracing_multi_attach_progs.c | 24 ++ .../selftests/bpf/progs/tracing_multi_bpf.c | 61 ++++ 22 files changed, 1495 insertions(+), 155 deletions(-) create mode 100644 tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_attach_progs.c create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_bpf.c
Signature · 1 line
2.55.0
PATCH v2 1/21 bpf: Initialize ftrace_managed in bpf_trampoline_get()
A trampoline managed by ftrace is determined by its target, that said ftrace_managed is already known when the trampoline is initialized. so, set 'ftrace_managed = tr->ip != 0' in bpf_trampoline_get() instead of relying on the first regular or tracing_multi attachment to set it.
This makes the backend selection independent of attachment order and keeps the registration callbacks focused on installing the trampoline.
No functional change intended.
--- kernel/bpf/trampoline.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)
kernel/bpf/trampoline.c +2 -9
@@ -475,18 +475,11 @@ static int register_fentry(struct bpf_trampoline *tr, struct bpf_tramp_image *im void *data __maybe_unused) {void *new_addr = im->image; -void *ip = tr->func.addr; -unsigned long faddr;int ret; - faddr =ftrace_location ((unsigned long )ip); -if (faddr) { +if (tr->func.ftrace_managed) {if (!tr->fops)return -ENOTSUPP ; - tr->func.ftrace_managed = true; - } - -if (tr->func.ftrace_managed) { ret =direct_ops_add (tr, new_addr); }else { ret =bpf_trampoline_update_fentry (tr,0 ,NULL , new_addr);
@@ -1286,6 +1279,7 @@ struct bpf_trampoline *bpf_trampoline_get(u64 key,memcpy (&tr->func.model, &tgt_info->fmodel,sizeof (tgt_info->fmodel)); tr->func.addr = (void *)tgt_info->tgt_addr; + tr->func.ftrace_managed = tr->ip !=0 ; out:trampoline_unlock (tr);return tr;
@@ -1649,7 +1643,6 @@ static int register_fentry_multi(struct bpf_trampoline *tr, struct bpf_tramp_imaif (bpf_trampoline_use_jmp (tr->flags)) addr =ftrace_jmp_set (addr); - tr->func.ftrace_managed = true;ftrace_hash_add (data->reg, data->entry, ip, addr); tr->cur_image = im;return 0 ;
Signature · 1 line
2.55.0
Re: [PATCH bpf-next v2 01/21] bpf: Initialize ftrace_managed in bpf_trampoline_get()
Quoted 11 lines · Leon Hwang
A trampoline managed by ftrace is determined by its target, that said
ftrace_managedis already known when the trampoline is initialized. so, set 'ftrace_managed=tr->ip!= 0' inbpf_trampoline_get()instead of relying on the first regular ortracing_multiattachment to set it.This makes the backend selection independent of attachment order and keeps the registration callbacks focused on installing the trampoline.
No functional change intended.
Signed-off-by Leon Hwang <[email protected]>
jirka
Quoted 47 lines
--- kernel/bpf/trampoline.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)kernel/bpf/trampoline.c +2
-9@@ -475,18 +475,11 @@ static int register_fentry(struct bpf_trampoline *tr, struct bpf_tramp_image *im void *data __maybe_unused) {void *new_addr = im->image; -void *ip = tr->func.addr; -unsigned long faddr;int ret; - faddr =ftrace_location ((unsigned long )ip); -if (faddr) { +if (tr->func.ftrace_managed) {if (!tr->fops)return -ENOTSUPP ; - tr->func.ftrace_managed = true; - } - -if (tr->func.ftrace_managed) { ret =direct_ops_add (tr, new_addr); }else { ret =bpf_trampoline_update_fentry (tr,0 ,NULL , new_addr);@@ -1286,6 +1279,7 @@ struct bpf_trampoline *bpf_trampoline_get(u64 key,memcpy (&tr->func.model, &tgt_info->fmodel,sizeof (tgt_info->fmodel)); tr->func.addr = (void *)tgt_info->tgt_addr; + tr->func.ftrace_managed = tr->ip !=0 ; out:trampoline_unlock (tr);return tr;@@ -1649,7 +1643,6 @@ static int register_fentry_multi(struct bpf_trampoline *tr, struct bpf_tramp_imaif (bpf_trampoline_use_jmp (tr->flags)) addr =ftrace_jmp_set (addr); - tr->func.ftrace_managed = true;ftrace_hash_add (data->reg, data->entry, ip, addr); tr->cur_image = im;return 0 ;Signature · 1 line
2.55.0
PATCH v2 2/21 bpf: Drop unnecessary ftrace_location() in register/unregister/modify_fentry_multi()
In reply to Leon HwangSince 'tr->ip' is already a 'ftrace_location()'-ed IP in bpf_trampoline_lookup(), it is unnecessary to call ftrace_location() again in register/unregister/modify_fentry_multi().
No functional changes intended.
--- kernel/bpf/trampoline.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
kernel/bpf/trampoline.c +3 -5
@@ -1629,8 +1629,8 @@ static void ftrace_hash_add(struct ftrace_hash *hash, struct ftrace_func_entry *static int register_fentry_multi (struct bpf_trampoline *tr,struct bpf_tramp_image *im,void *ptr) {unsigned long addr = (unsigned long ) im->image; -unsigned long ip =ftrace_location (tr->ip);struct bpf_tracing_multi_data *data = ptr; +unsigned long ip = tr->ip;/* * Different BTF ids get different trampolines, but they can still
@@ -1651,13 +1651,12 @@ static int register_fentry_multi(struct bpf_trampoline *tr, struct bpf_tramp_imastatic int unregister_fentry_multi (struct bpf_trampoline *tr,u32 orig_flags,void *ptr) {unsigned long addr = (unsigned long ) tr->cur_image->image; -unsigned long ip =ftrace_location (tr->ip);struct bpf_tracing_multi_data *data = ptr;if (bpf_trampoline_use_jmp (tr->flags)) addr =ftrace_jmp_set (addr); -ftrace_hash_add (data->unreg, data->entry, ip, addr); +ftrace_hash_add (data->unreg, data->entry, tr->ip, addr); tr->cur_image =NULL ;return 0 ; }
@@ -1666,13 +1665,12 @@ static int modify_fentry_multi(struct bpf_trampoline *tr, u32 orig_flags, struct bool lock_direct_mutex, void *ptr) { unsigned long addr = (unsigned long) im->image; -unsigned long ip =ftrace_location (tr->ip);struct bpf_tracing_multi_data *data = ptr;if (bpf_trampoline_use_jmp (tr->flags)) addr =ftrace_jmp_set (addr); -ftrace_hash_add (data->modify, data->entry, ip, addr); +ftrace_hash_add (data->modify, data->entry, tr->ip, addr); tr->cur_image = im;return 0 ; }
Signature · 1 line
2.55.0
Re: [PATCH bpf-next v2 02/21] bpf: Drop unnecessary ftrace_location() in register/unregister/modify_fentry_multi()
Quoted 7 lines · Leon Hwang
Since '
tr->ip' is already a 'ftrace_location()'-ed IP inbpf_trampoline_lookup(), it is unnecessary to callftrace_location()again in register/unregister/modify_fentry_multi().No functional changes intended.
Signed-off-by Leon Hwang <[email protected]>
jirka
Quoted 51 lines
--- kernel/bpf/trampoline.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)kernel/bpf/trampoline.c +3
-5@@ -1629,8 +1629,8 @@ static void ftrace_hash_add(struct ftrace_hash *hash, struct ftrace_func_entry *static int register_fentry_multi (struct bpf_trampoline *tr,struct bpf_tramp_image *im,void *ptr) {unsigned long addr = (unsigned long ) im->image; -unsigned long ip =ftrace_location (tr->ip);struct bpf_tracing_multi_data *data = ptr; +unsigned long ip = tr->ip;/* * Different BTF ids get different trampolines, but they can still @@ -1651,13 +1651,12 @@ static int register_fentry_multi(struct bpf_trampoline *tr, struct bpf_tramp_imastatic int unregister_fentry_multi (struct bpf_trampoline *tr,u32 orig_flags,void *ptr) {unsigned long addr = (unsigned long ) tr->cur_image->image; -unsigned long ip =ftrace_location (tr->ip);struct bpf_tracing_multi_data *data = ptr;if (bpf_trampoline_use_jmp (tr->flags)) addr =ftrace_jmp_set (addr); -ftrace_hash_add (data->unreg, data->entry, ip, addr); +ftrace_hash_add (data->unreg, data->entry, tr->ip, addr); tr->cur_image =NULL ;return 0 ; }@@ -1666,13 +1665,12 @@ static int modify_fentry_multi(struct bpf_trampoline *tr, u32 orig_flags, struct bool lock_direct_mutex, void *ptr) { unsigned long addr = (unsigned long) im->image; -unsigned long ip =ftrace_location (tr->ip);struct bpf_tracing_multi_data *data = ptr;if (bpf_trampoline_use_jmp (tr->flags)) addr =ftrace_jmp_set (addr); -ftrace_hash_add (data->modify, data->entry, ip, addr); +ftrace_hash_add (data->modify, data->entry, tr->ip, addr); tr->cur_image = im;return 0 ; }Signature · 1 line
2.55.0
PATCH v2 3/21 bpf, x86: Factor out bpf_arch_text_poke_prepare_insns() helper on x86_64
In reply to Leon HwangIn the following commit, it also needs to prepare both old_insn and new_insn when add text-poke batch support.
So, factor out bpf_arch_text_poke_prepare_insns() helper for text-poke batch support.
No functional changes intended.
--- arch/x86/net/bpf_jit_comp.c | 38 ++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-)
arch/x86/net/bpf_jit_comp.c +25 -13
@@ -646,13 +646,12 @@ static int emit_jump(u8 **pprog, void *func, void *ip)return emit_patch (pprog, func, ip,0xE9 ); } -static int __bpf_arch_text_poke (void *ip,enum bpf_text_poke_type old_t, -enum bpf_text_poke_type new_t, -void *old_addr,void *new_addr) +static int bpf_arch_text_poke_prepare_insns (void *ip,enum bpf_text_poke_type old_t, +enum bpf_text_poke_type new_t, +void *old_addr,void *new_addr, +u8 *old_insn,u8 *new_insn) {const u8 *nop_insn = x86_nops[5 ]; -u8 old_insn[X86_PATCH_SIZE ]; -u8 new_insn[X86_PATCH_SIZE ];u8 *prog;int ret;
@@ -676,18 +675,31 @@ static int __bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,return ret; } - ret = -EBUSY ; -mutex_lock (&text_mutex); +return 0 ; +} + +static int __bpf_arch_text_poke (void *ip,enum bpf_text_poke_type old_t, +enum bpf_text_poke_type new_t, +void *old_addr,void *new_addr) +{ +u8 old_insn[X86_PATCH_SIZE ]; +u8 new_insn[X86_PATCH_SIZE ]; +int ret; + + ret =bpf_arch_text_poke_prepare_insns (ip, old_t, new_t, old_addr, new_addr, old_insn, + new_insn); +if (ret) +return ret; + +guard (mutex)(&text_mutex); +if (memcmp (ip, old_insn,X86_PATCH_SIZE )) - goto out; - ret =1 ; +return -EBUSY ;if (memcmp (ip, new_insn,X86_PATCH_SIZE )) {smp_text_poke_single (ip, new_insn,X86_PATCH_SIZE ,NULL ); - ret =0 ; +return 0 ; } -out: -mutex_unlock (&text_mutex); -return ret; +return 1 ; } int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
Signature · 1 line
2.55.0
PATCH v2 4/21 bpf, x86: Factor out bpf_arch_text_poke_check_ip() helper on x86_64
In reply to Leon HwangIn the following commit, it also needs to check the target IP and adds ENDBR_INSN_SIZE when add text-poke batch support.
So, factor out bpf_arch_text_poke_check_ip() helper for text-poke batch support.
No functional changes intended.
--- arch/x86/net/bpf_jit_comp.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)
arch/x86/net/bpf_jit_comp.c +17 -4
@@ -702,10 +702,10 @@ static int __bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,return 1 ; } -int bpf_arch_text_poke (void *ip,enum bpf_text_poke_type old_t, -enum bpf_text_poke_type new_t,void *old_addr, -void *new_addr) +static int bpf_arch_text_poke_check_ip (void **ipp) { +void *ip = *ipp; +if (!is_kernel_text ((long )ip) && !is_bpf_text_address ((long )ip))/* BPF poking in modules is not supported */
@@ -716,7 +716,20 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,* with an ENDBR instruction. */ if (is_endbr (ip)) - ip +=ENDBR_INSN_SIZE ; + *ipp +=ENDBR_INSN_SIZE ; + +return 0 ; +} + +int bpf_arch_text_poke (void *ip,enum bpf_text_poke_type old_t, +enum bpf_text_poke_type new_t,void *old_addr, +void *new_addr) +{ +int ret; + + ret =bpf_arch_text_poke_check_ip (&ip); +if (ret) +return ret;return __bpf_arch_text_poke (ip, old_t, new_t, old_addr, new_addr); }
Signature · 1 line
2.55.0
PATCH v2 5/21 bpf, x86: Introduce bpf_arch_text_poke_batch() on x86_64
In reply to Leon HwangIntroduce bpf_arch_text_poke_batch() to patch multiple targets in batch, which utilizes smp_text_poke_batch_add/finish() on x86_64.
Thereafter, it will able to text-poke multiple bpf progs at once via bpf_arch_text_poke_batch().
--- arch/x86/net/bpf_jit_comp.c | 41 +++++++++++++++++++++++++++++++++++++ include/linux/bpf.h | 9 ++++++++ kernel/bpf/core.c | 5 +++++ 3 files changed, 55 insertions(+)
arch/x86/net/bpf_jit_comp.c +41 -0
@@ -734,6 +734,47 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,return __bpf_arch_text_poke (ip, old_t, new_t, old_addr, new_addr); } +int bpf_arch_text_poke_batch (struct bpf_text_poke **pokes,u32 cnt) +{ +u8 old_insn[X86_PATCH_SIZE ]; +u8 new_insn[X86_PATCH_SIZE ]; +int err; +u32 i; + +for (i =0 ; i < cnt; i++) { + err =bpf_arch_text_poke_check_ip (&pokes[i]->ip); +if (err) +return err; + } + +guard (mutex)(&text_mutex); + +for (i =0 ; i < cnt; i++) { +struct bpf_text_poke *poke = pokes[i]; + + err =bpf_arch_text_poke_prepare_insns (poke->ip, poke->old_t, poke->new_t, + poke->old_addr, poke->new_addr, old_insn, + new_insn); +if (err) +return err; +if (memcmp (poke->ip, old_insn,X86_PATCH_SIZE )) +return -EBUSY ; + } + +for (i =0 ; i < cnt; i++) { +struct bpf_text_poke *poke = pokes[i]; + +bpf_arch_text_poke_prepare_insns (poke->ip, poke->old_t, poke->new_t, + poke->old_addr, poke->new_addr, old_insn, + new_insn); +if (memcmp (poke->ip, new_insn,X86_PATCH_SIZE )) +smp_text_poke_batch_add (poke->ip, new_insn,X86_PATCH_SIZE ,NULL ); + } + +smp_text_poke_batch_finish (); +return 0 ; +} +#define EMIT_LFENCE () EMIT3(0x0F, 0xAE, 0xE8)static void __emit_indirect_jump (u8 **pprog,int reg,bool ereg)
include/linux/bpf.h +9 -0
@@ -4231,6 +4231,15 @@ enum bpf_text_poke_type { BPF_MOD_JUMP, }; +struct bpf_text_poke { +void *ip; +void *old_addr; +void *new_addr; +enum bpf_text_poke_type old_t; +enum bpf_text_poke_type new_t; +}; + +int bpf_arch_text_poke_batch (struct bpf_text_poke **pokes,u32 cnt);int bpf_arch_text_poke (void *ip,enum bpf_text_poke_type old_t,enum bpf_text_poke_type new_t,void *old_addr,void *new_addr);
kernel/bpf/core.c +5 -0
@@ -3484,6 +3484,11 @@ int __weak skb_copy_bits(const struct sk_buff *skb, int offset, void *to,return -EFAULT ; } +int __weak bpf_arch_text_poke_batch (struct bpf_text_poke **pokes,u32 cnt) +{ +return -EOPNOTSUPP ; +} +int __weak bpf_arch_text_poke (void *ip,enum bpf_text_poke_type old_t,enum bpf_text_poke_type new_t,void *old_addr,void *new_addr)
Signature · 1 line
2.55.0
PATCH v2 6/21 bpf: Add bpf_text_poke_batch() helper in trampoline
In reply to Leon HwangBefore calling bpf_arch_text_poke_batch(), check the duplicated target pokes by sorting the pokes.
Sorting pokes is needed for smp_text_poke_batch_add() on x86_64 for better performance, see arch/x86/kernel/alternative.c::smp_text_poke_batch_add().
If the arch does not implement bpf_arch_text_poke_batch(), fallback to text-poke each target via bpf_arch_text_poke().
--- kernel/bpf/trampoline.c | 51 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+)
kernel/bpf/trampoline.c +51 -0
@@ -13,6 +13,7 @@#include <linux/bpf_verifier.h> #include <linux/bpf_lsm.h> #include <linux/delay.h> +#include <linux/sort.h> /* dummy _ops. The verifier will operate on target program's ops. */ const struct bpf_verifier_ops bpf_extension_verifier_ops = {
@@ -1863,6 +1864,56 @@ void bpf_trampoline_multi_detach(struct bpf_prog *prog,CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS && CONFIG_BPF_SYSCALL */ +static int bpf_text_poke_cmp (const void *a,const void *b) +{ +const struct bpf_text_poke *pa = *(struct bpf_text_poke *const *)a; +const struct bpf_text_poke *pb = *(struct bpf_text_poke *const *)b; +unsigned long ipa = (unsigned long ) pa->ip; +unsigned long ipb = (unsigned long ) pb->ip; + +return (ipa > ipb) - (ipa < ipb); +} + +static int bpf_text_poke_batch (struct bpf_text_poke **pokes,u32 cnt) +{ +int i, err; + +/* +* Sort pokes for two purposes: +* 1. Check duplicated poke target IPs. +* 2. Sorted poke target IPs would be friendly for poking them on x86, +* see arch/x86/kernel/alternative.c::smp_text_poke_batch_add(). +*/ +sort_nonatomic (pokes, cnt,sizeof (*pokes), bpf_text_poke_cmp,NULL ); + +for (i =1 ; i < cnt; i++) +if (pokes[i -1 ]->ip == pokes[i]->ip) +return -EINVAL ; + + err =bpf_arch_text_poke_batch (pokes, cnt); +if (err != -EOPNOTSUPP ) +return err; + +for (i =0 ; i < cnt; i++) { +struct bpf_text_poke *p = pokes[i]; + + err =bpf_arch_text_poke (p->ip, p->old_t, p->new_t, p->old_addr, p->new_addr); +if (err) + goto rollback; + } + +return 0 ; + +rollback: +for (i--; i >=0 ; i--) { +struct bpf_text_poke *p = pokes[i]; + +WARN_ON_ONCE (bpf_arch_text_poke (p->ip, p->new_t, p->old_t, p->new_addr, + p->old_addr)); + } +return err; +} +static int __init init_trampolines (void ) {int i;
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v2 06/21] bpf: Add bpf_text_poke_batch() helper in trampoline
TL;DR: The new static bpf_text_poke_batch() has no caller in this patch, so it may trigger -Wunused-function and, with CONFIG_WERROR=y, break the build and bisection; consider squashing it or…
Quoted 1 line
bpf: Add
bpf_text_poke_batch()helper in trampoline
TL;DR: The new static bpf_text_poke_batch() has no caller in this patch, so it may trigger -Wunused-function and, with CONFIG_WERROR=y, break the build and bisection; consider squashing it or marking __maybe_unused.
Quoted 4 lines
kernel/bpf/trampoline.c +0
-0
[ ... ]
Quoted 6 lines
kernel/bpf/trampoline.c +2
-0@@ -1863,6 +1864,56 @@ void bpf_trampoline_multi_detach(struct bpf_prog *prog,CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS && CONFIG_BPF_SYSCALL */ +static int bpf_text_poke_cmp (const void *a,const void *b) +{
[ ... ]
kernel/bpf/trampoline.c void bpf_trampoline_multi_detach(struct bpf_prog *prog, · line 1878
Quoted 2 lines
kernel/bpf/trampoline.c +2
-0+static int bpf_text_poke_batch (struct bpf_text_poke **pokes,u32 cnt) +{
bpf_text_poke_batch() is static and nothing calls it in this commit. Will this trigger a -Wunused-function warning for any config that builds trampoline.o?
kernel/bpf/trampoline.c: warning: 'bpf_text_poke_batch' defined but not used [-Wunused-function]
With CONFIG_WERROR=y that becomes a build failure, which would break bisection at this point in the series.
The first callers, bpf_trampoline_multi_prog_attach() and bpf_trampoline_multi_prog_detach(), only arrive later in the series in 59e80b53cc6b ("bpf: Add bpf_trampoline_multi_prog_attach/detach()"). That commit also wraps these helpers in #ifdef CONFIG_BPF_SYSCALL so they are not unused in !CONFIG_BPF_SYSCALL builds.
Could this helper be squashed into the commit that adds its first user? If it stays separate, could it be marked __maybe_unused until that user arrives?
kernel/bpf/trampoline.c void bpf_trampoline_multi_detach(struct bpf_prog *prog, · line 1882
Quoted 4 lines
kernel/bpf/trampoline.c +4
-0+int i, err; + +/* +* Sort pokes for two purposes:
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
PATCH v2 7/21 bpf: Check tgt_prog before is_tracing_multi_id() in verifier
In reply to Leon HwangWhen adding tracing_multi link support for bpf progs, it must handle tgt_prog first in bpf_check_attach_target() to avoid the case: the btf_id is a valid BTF ID of tgt_prog, and also is "bpf_multi_func_btf_id[0]".
--- kernel/bpf/verifier.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)
kernel/bpf/verifier.c +12 -10
@@ -22284,22 +22284,24 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,return -EOPNOTSUPP ; } -/* -* *.multi programs don't need an address during program -* verification, we just take the module ref if needed. -*/ -if (is_tracing_multi_id (prog, btf_id)) { +if (tgt_prog) { +if (subprog ==0 ) + addr = (long )tgt_prog->bpf_func; +else + addr = (long )tgt_prog->aux->func[subprog]->bpf_func; + }else if (is_tracing_multi_id (prog, btf_id)) { +/* +* *.multi programs don't need an address during program +* verification, we just take the module ref if needed. +* *.multi programs don't have tgt_prog during program +* verification. +*/ if (btf_is_module (btf)) { mod =btf_try_get_module (btf);if (!mod)return -ENOENT ; } addr =0 ; - }else if (tgt_prog) { -if (subprog ==0 ) - addr = (long ) tgt_prog->bpf_func; -else - addr = (long ) tgt_prog->aux->func[subprog]->bpf_func; }else {if (btf_is_module (btf)) { mod =btf_try_get_module (btf);
Signature · 1 line
2.55.0
PATCH v2 8/21 bpf: Move helpers out of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard in trampoline
In reply to Leon HwangThe following helpers are not relied on theCONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard.
* trampoline_lock_all()
* trampoline_unlock_all()
* bpf_trampoline_multi_attach_init()
Move them out of the CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard in order to add tracing_multi link support for bpf progs.
Keep the CONFIG_BPF_SYSCALL guard to avoid compiling warning when CONFIG_BPF_SYSCALL is disabled.
--- kernel/bpf/trampoline.c | 48 ++++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 22 deletions(-)
kernel/bpf/trampoline.c +26 -22
@@ -60,6 +60,32 @@ static void trampoline_unlock(struct bpf_trampoline *tr)mutex_unlock (select_trampoline_lock (tr)); } +#ifdef CONFIG_BPF_SYSCALL + +static void trampoline_lock_all (void ) +{ +int i; + +for (i =0 ; i <TRAMPOLINE_LOCKS_TABLE_SIZE ; i++) +mutex_lock (&trampoline_locks[i].mutex); +} + +static void trampoline_unlock_all (void ) +{ +int i; + +for (i =0 ; i <TRAMPOLINE_LOCKS_TABLE_SIZE ; i++) +mutex_unlock (&trampoline_locks[i].mutex); +} + +static void bpf_trampoline_multi_attach_init (struct bpf_trampoline *tr) +{ + tr->multi_attach.old_image = tr->cur_image; + tr->multi_attach.old_flags = tr->flags; +} + +#endif /* CONFIG_BPF_SYSCALL */ +struct bpf_trampoline_ops {int (*register_fentry)(struct bpf_trampoline *tr,struct bpf_tramp_image *im,void *data);int (*unregister_fentry)(struct bpf_trampoline *tr,u32 orig_flags,void *data);
@@ -1574,22 +1600,6 @@ int __weak arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,defined (CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS ) && \ defined(CONFIG_BPF_SYSCALL) -static void trampoline_lock_all (void ) -{ -int i; - -for (i =0 ; i <TRAMPOLINE_LOCKS_TABLE_SIZE ; i++) -mutex_lock (&trampoline_locks[i].mutex); -} - -static void trampoline_unlock_all (void ) -{ -int i; - -for (i =0 ; i <TRAMPOLINE_LOCKS_TABLE_SIZE ; i++) -mutex_unlock (&trampoline_locks[i].mutex); -} -static void remove_tracing_multi_data (struct bpf_tracing_multi_data *data) {ftrace_hash_remove (data->reg);
@@ -1682,12 +1692,6 @@ static const struct bpf_trampoline_ops trampoline_multi_ops = { .modify_fentry = modify_fentry_multi, }; -static void bpf_trampoline_multi_attach_init (struct bpf_trampoline *tr) -{ - tr->multi_attach.old_image = tr->cur_image; - tr->multi_attach.old_flags = tr->flags; -} -static void bpf_trampoline_multi_attach_free (struct bpf_trampoline *tr) {/*
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v2 08/21] bpf: Move helpers out of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard in trampoline
TL;DR: Moved trampoline helpers may trigger unused-function warnings (build failure with CONFIG_WERROR) on non-x86_64 configs with CONFIG_BPF_SYSCALL until a later patch adds callers; only this…
Quoted 1 line
bpf: Move helpers out of
CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPSguard in trampoline
TL;DR: Moved trampoline helpers may trigger unused-function warnings (build failure with CONFIG_WERROR) on non-x86_64 configs with CONFIG_BPF_SYSCALL until a later patch adds callers; only this commit is affected.
Quoted 24 lines
kernel/bpf/trampoline.c +16
-0@@ -60,6 +60,32 @@ static void trampoline_unlock(struct bpf_trampoline *tr)mutex_unlock (select_trampoline_lock (tr)); } +#ifdef CONFIG_BPF_SYSCALL + +static void trampoline_lock_all (void ) +{ +int i; + +for (i =0 ; i <TRAMPOLINE_LOCKS_TABLE_SIZE ; i++) +mutex_lock (&trampoline_locks[i].mutex); +} + +static void trampoline_unlock_all (void ) +{ +int i; + +for (i =0 ; i <TRAMPOLINE_LOCKS_TABLE_SIZE ; i++) +mutex_unlock (&trampoline_locks[i].mutex);
kernel/bpf/trampoline.c static void trampoline_unlock(struct bpf_trampoline *tr) · line 88
Quoted 10 lines
kernel/bpf/trampoline.c +10
-0+} + +static void bpf_trampoline_multi_attach_init (struct bpf_trampoline *tr) +{ + tr->multi_attach.old_image = tr->cur_image; + tr->multi_attach.old_flags = tr->flags; +} + +#endif /* CONFIG_BPF_SYSCALL */ +
Can this add a new "defined but not used" warning on configs that have CONFIG_BPF_SYSCALL but not CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS?
These three helpers are static and are now guarded only by CONFIG_BPF_SYSCALL, but their only callers at this point in the series, bpf_trampoline_multi_attach() and bpf_trampoline_multi_detach(), are still inside:
#if defined(CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS) && \
defined(CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS) && \
defined(CONFIG_BPF_SYSCALL)trampoline.o is built for CONFIG_BPF_JIT, and only x86_64 selects HAVE_SINGLE_FTRACE_DIRECT_OPS. On arm64, riscv, s390, x86_32, or x86_64 without DYNAMIC_FTRACE_WITH_DIRECT_CALLS, a CONFIG_BPF_JIT=y, CONFIG_BPF_SYSCALL=y build would define trampoline_lock_all(), trampoline_unlock_all() and bpf_trampoline_multi_attach_init() with no users, and -Wunused-function would warn for each of them. With CONFIG_WERROR=y that becomes a build failure.
The commit message says the CONFIG_BPF_SYSCALL guard is kept "to avoid compiling warning when CONFIG_BPF_SYSCALL is disabled", but this seems to trade that warning for the same kind of warning when CONFIG_BPF_SYSCALL is enabled.
The helpers only get callers outside the old guard in a later patch of the series, commit 59e80b53cc6b ("bpf: Add bpf_trampoline_multi_prog_attach/detach()"), which adds users under #ifdef CONFIG_BPF_SYSCALL. So the final state looks
fine, but this commit on its own does not build warning-free on those configs.
Would it make sense to fold this move into that later patch, or mark the helpers __maybe_unused for the interim, so each commit in the series builds cleanly?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
PATCH v2 9/21 bpf: Add bpf_trampoline_multi_prog_attach/detach()
In reply to Leon HwangThe existing tracing_multi helpers update ftrace-managed kernel function targets through the ftrace direct-call API. BPF prog targets are not ftrace-managed and require text pokes at their entry points.
Add bpf_trampoline_multi_prog_attach/detach() helpers that prepare each target trampoline, apply the text pokes in batch, and then commit the staged trampoline images. Roll back the added programs and trampoline images when attach fails.
Set BPF_TRAMP_F_TAIL_CALL_CTX for tail-call-reachable target bpf progs so the generated trampolines preserve the tail_call_cnt.
--- include/linux/bpf.h | 45 ++++++--- kernel/bpf/trampoline.c | 216 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 246 insertions(+), 15 deletions(-)
include/linux/bpf.h +30 -15
@@ -1496,9 +1496,10 @@ struct bpf_trampoline {struct bpf_tramp_image *cur_image;/* Images not freed yet, cur_image and older ones still in use */ struct list_head images; -/* Used as temporary old image storage for multi_attach */ +/* Used as temporary images storage for multi_attach */ struct {struct bpf_tramp_image *old_image; +struct bpf_tramp_image *new_image;u32 old_flags; } multi_attach; };
@@ -1616,6 +1617,9 @@ int bpf_trampoline_multi_attach(struct bpf_prog *prog, u32 *ids, struct bpf_tracing_multi_link *link);void bpf_trampoline_multi_detach (struct bpf_prog *prog,struct bpf_tracing_multi_link *link); +int bpf_trampoline_multi_prog_attach (struct bpf_prog *prog,u32 *ids,u64 *keys, +struct bpf_tracing_multi_link *link); +void bpf_trampoline_multi_prog_detach (struct bpf_tracing_multi_link *link);void bpf_trampoline_set_flags (struct bpf_trampoline *tr,u32 flags);/*
@@ -1739,6 +1743,14 @@ static inline void bpf_trampoline_multi_detach(struct bpf_prog *prog, struct bpf_tracing_multi_link *link) { } + +static inline int bpf_trampoline_multi_prog_attach (struct bpf_prog *prog,u32 *ids,u64 *keys, +struct bpf_tracing_multi_link *link) +{ +return -EOPNOTSUPP ; +} + +static inline void bpf_trampoline_multi_prog_detach (struct bpf_tracing_multi_link *link) {}static inline void bpf_trampoline_set_flags (struct bpf_trampoline *tr,u32 flags) {}#endif
@@ -2089,10 +2101,25 @@ struct bpf_tracing_link {struct bpf_prog *tgt_prog; }; +enum bpf_text_poke_type { +BPF_MOD_NOP , +BPF_MOD_CALL , +BPF_MOD_JUMP , +}; + +struct bpf_text_poke { +void *ip; +void *old_addr; +void *new_addr; +enum bpf_text_poke_type old_t; +enum bpf_text_poke_type new_t; +}; +struct bpf_tracing_multi_node {struct bpf_tramp_node node;struct bpf_trampoline *trampoline;struct ftrace_func_entry entry; +struct bpf_text_poke poke; };struct bpf_tracing_multi_data {
@@ -2107,6 +2134,8 @@ struct bpf_tracing_multi_link {struct bpf_tracing_multi_data data;u64 *cookies;struct bpf_tramp_node *fexits; +struct bpf_text_poke **pokes; +struct bpf_prog **tgt_progs;int nodes_cnt; struct bpf_tracing_multi_node nodes[] __counted_by(nodes_cnt); };
@@ -4225,20 +4254,6 @@ static inline u32 bpf_xdp_sock_convert_ctx_access(enum bpf_access_type type, }#endif /* CONFIG_INET */ -enum bpf_text_poke_type { -BPF_MOD_NOP , -BPF_MOD_CALL , -BPF_MOD_JUMP , -}; - -struct bpf_text_poke { -void *ip; -void *old_addr; -void *new_addr; -enum bpf_text_poke_type old_t; -enum bpf_text_poke_type new_t; -}; -int bpf_arch_text_poke_batch (struct bpf_text_poke **pokes,u32 cnt); int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t, enum bpf_text_poke_type new_t, void *old_addr,
kernel/bpf/trampoline.c +216 -0
246 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v2 09/21] bpf: Add bpf_trampoline_multi_prog_attach/detach()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Critical] bpf: Use-After-Free of trampoline images when text poke batch fails in detach --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] bpf: Use-After-Free of trampoline images when text poke batch fails in detach --
commit 357aa776c5f19ab7b39da8fb3843fb0141fc3b7f
Author: Leon Hwang <[email protected]>
bpf: Add bpf_trampoline_multi_prog_attach/detach()
This commit introduces multi-attach and detach helpers for BPF trampolines. These helpers prepare target trampolines, apply text pokes in batch, and commit the staged trampoline images.
Quoted 3 lines
kernel/bpf/trampoline.c +0
-0
[ ... ]
kernel/bpf/trampoline.c static int bpf_text_poke_batch(struct bpf_text_poke **pokes, u32 cnt) · line 1997
Quoted 7 lines
kernel/bpf/trampoline.c +7
-0+static void bpf_trampoline_multi_prog_rollback (struct bpf_trampoline *tr) +{ +if (tr->multi_attach.new_image) +bpf_tramp_image_put (tr->multi_attach.new_image); + + tr->flags = tr->multi_attach.old_flags; + tr->multi_attach.old_image =NULL ;
[Severity: Critical]
If __bpf_trampoline_unlink_prog() fails and calls bpf_trampoline_multi_prog_rollback(), old_image is cleared for the node.
Does this guarantee that a subsequent bpf_text_poke_batch() will fail due to a NOP expectation mismatch?
Quoted 3 lines
kernel/bpf/trampoline.c +3
-0+ tr->multi_attach.new_image =NULL ; + tr->multi_attach.old_flags =0 ; +}
[ ... ]
Quoted 11 lines
kernel/bpf/trampoline.c +11
-0+void bpf_trampoline_multi_prog_detach (struct bpf_tracing_multi_link *link) +{ +struct bpf_tracing_multi_node *mnode; +int i, err; + +trampoline_lock_all (); + +for_each_mnode (mnode, link) { +bpf_trampoline_multi_attach_init (mnode->trampoline); + err =__bpf_trampoline_unlink_prog (&mnode->node, mnode->trampoline,NULL , + &trampoline_multi_prog_ops,NULL );
kernel/bpf/trampoline.c static int bpf_text_poke_batch(struct bpf_text_poke **pokes, u32 cnt) · line 2121
Quoted 10 lines
kernel/bpf/trampoline.c +10
-0+if (WARN_ONCE (err,"__bpf_trampoline_unlink_prog failed: %d\n" , err)) +bpf_trampoline_multi_prog_rollback (mnode->trampoline); + } + +for_each_mnode (mnode, link) { + link->pokes[i] = &mnode->poke; +bpf_text_poke_init (link->pokes[i], mnode->trampoline); + } + +WARN_ON_ONCE (bpf_text_poke_batch (link->pokes, link->nodes_cnt));
[Severity: Critical]
If bpf_text_poke_batch() fails here (for example, due to a hardware or memory error during patching, or a NOP mismatch from a prior unlink failure), the failure is wrapped in WARN_ON_ONCE but otherwise ignored.
The code then proceeds to unconditionally call bpf_trampoline_multi_prog_commit() for all nodes, which frees old_image via bpf_tramp_image_put().
Since the kernel text remains unmodified after a failed poke batch, it will continue to jump to the freed old_image. Can this lead to a use-after-free on traced paths?
Quoted 9 lines
kernel/bpf/trampoline.c +9
-0+ +for_each_mnode (mnode, link) +bpf_trampoline_multi_prog_commit (mnode->trampoline); + +trampoline_unlock_all (); + +for_each_mnode (mnode, link) +bpf_trampoline_put (mnode->trampoline); +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
[email protected]
AI reviewRe: [PATCH bpf-next v2 09/21] bpf: Add bpf_trampoline_multi_prog_attach/detach()
In reply to Leon HwangTL;DR: bpf_trampoline_multi_prog_detach() may, if one unlink fails (e.g. -ENOMEM), still commit and free old images after a failed batch poke, so other targets could run freed trampoline memory.
Quoted 1 line
bpf: Add bpf_trampoline_multi_prog_attach/detach()
TL;DR: bpf_trampoline_multi_prog_detach() may, if one unlink fails (e.g. -ENOMEM), still commit and free old images after a failed batch poke, so other targets could run freed trampoline memory.
Quoted 3 lines
kernel/bpf/trampoline.c +0
-0
[ ... ]
Quoted 24 lines
kernel/bpf/trampoline.c +23
-0@@ -1918,6 +1923,217 @@ static int bpf_text_poke_batch(struct bpf_text_poke **pokes, u32 cnt) +static void bpf_trampoline_multi_prog_commit (struct bpf_trampoline *tr) +{ +struct bpf_tramp_image *old_image = tr->multi_attach.old_image; + + tr->cur_image = tr->multi_attach.new_image; + tr->multi_attach.old_image =NULL ; + tr->multi_attach.new_image =NULL ; + tr->multi_attach.old_flags =0 ; + +if (old_image) +bpf_tramp_image_put (old_image); +} + +static void bpf_trampoline_multi_prog_rollback (struct bpf_trampoline *tr) +{ +if (tr->multi_attach.new_image) +bpf_tramp_image_put (tr->multi_attach.new_image); + + tr->flags = tr->multi_attach.old_flags; + tr->multi_attach.old_image =NULL ; + tr->multi_attach.new_image =NULL ; + tr->multi_attach.old_flags =0 ; +}
[ ... ]
Quoted 4 lines
kernel/bpf/trampoline.c +4
-0+void bpf_trampoline_multi_prog_detach (struct bpf_tracing_multi_link *link) +{ +struct bpf_tracing_multi_node *mnode; +int i, err;
kernel/bpf/trampoline.c static int bpf_text_poke_batch(struct bpf_text_poke **pokes, u32 cnt) · line 2114
Quoted 10 lines
kernel/bpf/trampoline.c +10
-0+ +trampoline_lock_all (); + +for_each_mnode (mnode, link) { +bpf_trampoline_multi_attach_init (mnode->trampoline); + err =__bpf_trampoline_unlink_prog (&mnode->node, mnode->trampoline,NULL , + &trampoline_multi_prog_ops,NULL ); +if (WARN_ONCE (err,"__bpf_trampoline_unlink_prog failed: %d\n" , err)) +bpf_trampoline_multi_prog_rollback (mnode->trampoline); + }
If __bpf_trampoline_unlink_prog() fails for one trampoline here, can this leave other targets calling freed trampoline images?
The failure is reachable. bpf_trampoline_update() returns an error when bpf_trampoline_get_progs() or bpf_tramp_image_alloc() fails (-ENOMEM), or when bpf_jit_charge_modmem() returns -EPERM because bpf_jit_limit is exceeded and the task closing the link fd is not bpf_capable(). The comment in bpf_trampoline_multi_attach_free() already treats this failure as reachable on the ftrace detach path.
Say the unlink fails for trampoline B:
1. bpf_trampoline_multi_attach_init() has set old_image = cur_image, which is still live in the text, and new_image = NULL.
2. bpf_trampoline_multi_prog_rollback() then clears old_image to NULL as well.
3. bpf_text_poke_init() therefore builds a poke for B with old_addr and new_addr both NULL (BPF_MOD_NOP to BPF_MOD_NOP), while B's entry still holds a call or jump to its live image.
4. On x86, bpf_arch_text_poke_batch() checks every poke before patching anything:
if (memcmp(poke->ip, old_insn, X86_PATCH_SIZE))
return -EBUSY;B's check fails, so no text is patched for any target in the link. The generic fallback rolls back the pokes it already applied, with the same result.
kernel/bpf/trampoline.c static int bpf_text_poke_batch(struct bpf_text_poke **pokes, u32 cnt) · line 2124
Quoted 10 lines
kernel/bpf/trampoline.c +10
-0+ +for_each_mnode (mnode, link) { + link->pokes[i] = &mnode->poke; +bpf_text_poke_init (link->pokes[i], mnode->trampoline); + } + +WARN_ON_ONCE (bpf_text_poke_batch (link->pokes, link->nodes_cnt)); + +for_each_mnode (mnode, link) +bpf_trampoline_multi_prog_commit (mnode->trampoline);
The batch error only triggers WARN_ON_ONCE(), and bpf_trampoline_multi_prog_commit() still runs for every node.
For every other target A in the link, old_A is passed to bpf_tramp_image_put() and freed after a tasks-rcu grace period. A's entry still calls old_A, because it was never repointed, so the next run of target A executes freed trampoline memory. Meanwhile new_A becomes cur_image without ever having been installed.
For B itself, commit sets cur_image to NULL while the text still calls the old image. That image is never put, so it leaks along with its trampoline reference. If another prog (for example a plain fentry) is still attached to B and is later detached, bpf_trampoline_update() reaches total == 0 and calls unregister_fentry(), which dereferences tr->cur_image->image while cur_image is NULL. Any new attach to B instead goes through register_fentry(), expects a nop at the entry, and fails with -EBUSY.
The ftrace-based bpf_trampoline_multi_detach() avoids this. On unlink failure it leaves cur_image unchanged, and bpf_trampoline_multi_attach_free() skips freeing old_image when old_image == cur_image. Could the prog detach path get the same treatment?
For example, on unlink failure the trampoline could be kept on its current image (new_image = old_image, so the poke is an identity poke and commit neither puts nor drops cur_image), or the node could be left out of the batch. It also seems that images should not be committed when bpf_text_poke_batch() fails.
This path is not called by this commit. It is wired up later in the series by b56b0344ff12 ("bpf: Add tracing_multi link support for bpf progs"), where bpf_tracing_multi_link_release() calls bpf_trampoline_multi_prog_detach(). The function is unchanged at the end of the series, so no later commit fixes this.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
Re: [PATCH bpf-next v2 09/21] bpf: Add bpf_trampoline_multi_prog_attach/detach()
In reply to Leon HwangOn Mon, Oct 12, 2026 at 11:40:11PM +0800, Leon Hwang wrote:
SNIP
Quoted 13 lines
include/linux/bpf.h +9
-0@@ -2089,10 +2101,25 @@ struct bpf_tracing_link {struct bpf_prog *tgt_prog; }; +enum bpf_text_poke_type { +BPF_MOD_NOP , +BPF_MOD_CALL , +BPF_MOD_JUMP , +}; + +struct bpf_text_poke { +void *ip; +void *old_addr;
include/linux/bpf.h struct bpf_tracing_link { · line 2122
Quoted 10 lines
include/linux/bpf.h +6
-0+void *new_addr; +enum bpf_text_poke_type old_t; +enum bpf_text_poke_type new_t; +}; +struct bpf_tracing_multi_node {struct bpf_tramp_node node;struct bpf_trampoline *trampoline;struct ftrace_func_entry entry; +struct bpf_text_poke poke;
could we have union of entry and poke ?
jirka
PATCH v2 10/21 bpf: Shrink CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard scope for bpf_tracing_multi_attach()
In reply to Leon HwangWhen adding tracing_multi link support for bpf progs, it will use the same UAPI of the tracing_multi link for kernel functions in bpf_tracing_multi_attach().
However, the tracing_multi link support for bpf progs does not require CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS. So, shrink its guard scope to add new code in bpf_tracing_multi_attach() for the tracing_multi link support for bpf progs.
To simplify the change, validate CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS only here, because the execution path in trampoline of the tracing_multi link support for kernel functions is already guarded with the CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS and CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS.
Meanwhile, provide a stub of bpf_trampoline_multi_detach() out of the CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS and CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard.
--- kernel/bpf/trampoline.c | 5 +++++ kernel/trace/bpf_trace.c | 15 +++------------ 2 files changed, 8 insertions(+), 12 deletions(-)
kernel/bpf/trampoline.c +5 -0
@@ -1867,6 +1867,11 @@ void bpf_trampoline_multi_detach(struct bpf_prog *prog,#undef for_each_mnode_cnt#undef for_each_mnode +#else + +void bpf_trampoline_multi_detach (struct bpf_prog *prog, +struct bpf_tracing_multi_link *link) {} +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS && CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS && CONFIG_BPF_SYSCALL */
kernel/trace/bpf_trace.c +3 -12
@@ -3679,9 +3679,6 @@ __bpf_kfunc int bpf_copy_from_user_task_str_dynptr(const struct bpf_dynptr *dptr__bpf_kfunc_end_defs (); -#if defined(CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS ) && \ - defined(CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS ) -static void bpf_tracing_multi_link_release (struct bpf_link *link) { struct bpf_tracing_multi_link *tr_link =
@@ -3858,6 +3855,9 @@ int bpf_tracing_multi_attach(struct bpf_prog *prog, const union bpf_attr *attr)if (attr->link_create.flags || attr->link_create.target_fd)return -EINVAL ; +if (!IS_ENABLED (CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS )) +return -EOPNOTSUPP ; + ids =kvmalloc_objs (*ids, cnt);if (!ids)return -ENOMEM ;
@@ -3924,12 +3924,3 @@ int bpf_tracing_multi_attach(struct bpf_prog *prog, const union bpf_attr *attr)kvfree (link);return err; } - -#else - -int bpf_tracing_multi_attach (struct bpf_prog *prog,const union bpf_attr *attr) -{ -return -EOPNOTSUPP ; -} - -#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS && CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS */
Signature · 1 line
2.55.0
PATCH v2 11/21 bpf: Add tracing_multi link support for bpf progs
In reply to Leon HwangSimilar to the tracing_multi link support for kernel functions, add support for bpf progs by reusing the 'ids' fields as BTF ids, and adding fds array to indicate the target bpf prog. Then, for each pair of fd+id, they will be the attach targets of tracing_multi link.
When attaching bpf prog, it attaches to the targets via bpf_trampoline_multi_prog_attach(), that would work on all trampoline-supported archs.
Since tracing_multi prog cannot recognize its targets at load time, reject attaching to any other tracing prog to avoid two-levels nested tracing
attachment: e.g. fentry <- fentry.multi <- another fentry.
--- include/uapi/linux/bpf.h | 2 + kernel/trace/bpf_trace.c | 177 ++++++++++++++++++++++++++++----- tools/include/uapi/linux/bpf.h | 2 + 3 files changed, 156 insertions(+), 25 deletions(-)
include/uapi/linux/bpf.h +2 -0
@@ -1927,6 +1927,8 @@ union bpf_attr {__aligned_u64 ids;__aligned_u64 cookies;__u32 cnt; + __u32 :32 ; +__aligned_u64 fds; } tracing_multi; }; } link_create;
kernel/trace/bpf_trace.c +152 -25
273 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +2 -0
@@ -1927,6 +1927,8 @@ union bpf_attr {__aligned_u64 ids;__aligned_u64 cookies;__u32 cnt; + __u32 :32 ; +__aligned_u64 fds; } tracing_multi; }; } link_create;
Signature · 1 line
2.55.0
[email protected]
AI reviewTL;DR: The new tracing_multi fds field is not tail-checked, so older 7.2/7.3 kernels may silently treat program BTF ids as vmlinux ids, attaching to unrelated kernel functions. Suggests a…
Quoted 1 line
bpf: Add
tracing_multilink support for bpf progs
TL;DR: The new tracing_multi fds field is not tail-checked, so older 7.2/7.3 kernels may silently treat program BTF ids as vmlinux ids, attaching to unrelated kernel functions. Suggests a link_create flag to gate it.
include/uapi/linux/bpf.h union bpf_attr { · line 1934
Quoted 13 lines
include/uapi/linux/bpf.h +2
-0@@ -1927,6 +1927,8 @@ union bpf_attr {__aligned_u64 ids;__aligned_u64 cookies;__u32 cnt; + __u32 :32 ; +__aligned_u64 fds; } tracing_multi; }; } link_create;
Can a kernel that already has tracing_multi but not fds reject a request that sets fds?
The generic tail-zero check only covers bytes after BPF_LINK_CREATE_LAST_FIELD:
#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.path_fd
uprobe_multi.path_fd ends at byte 64 of union bpf_attr, while the new tracing_multi.fds sits at bytes 40-47 (overlapping uprobe_multi.cookies). So CHECK_ATTR(BPF_LINK_CREATE) in link_create() never looks at it.
The tracing_multi link shipped in 7.2 (c1d32dea5d46 "bpf: Add support for tracing multi link"), and there bpf_tracing_multi_attach() reads only ids, cookies and cnt:
uids = u64_to_user_ptr(attr->link_create.tracing_multi.ids);
cnt = attr->link_create.tracing_multi.cnt;
...
if (attr->link_create.flags || attr->link_create.target_fd)
return -EINVAL;
...
err = bpf_trampoline_multi_attach(prog, ids, link);bpf_trampoline_multi_attach() then resolves each id with bpf_check_attach_btf_id_multi(prog->aux->attach_btf, prog, ids[i], ...), which looks the id up in the vmlinux (or module) BTF that the fentry.multi program was loaded against. If the type is a FUNC with an ftrace location, the program is attached there.
The libbpf change later in this series (85b62f93fd2d "libbpf: Add tracing_multi link support for bpf progs") fills ids[] from libbpf_find_prog_btf_id(funcs[i], fds[i], ...) and sets lopts.tracing_multi.fds = fds, with no feature probe. A new libbpf running on a 7.2 or 7.3 kernel would therefore send target-program BTF ids, and the older kernel would treat them as vmlinux BTF ids. Depending on what those ids name in vmlinux BTF, that gives either:
- a misleading -EINVAL or -ENOENT that cannot be told apart from bad input, or
- a successful link whose program runs on unrelated kernel functions instead of the requested BPF program functions.
No later commit in the series changes this. ef1ca3de15ec only adds a flag to the bpf_link_info output struct, and older kernels already reject a non-zero attr->link_create.flags in bpf_tracing_multi_attach().
Could the program-target mode be keyed off a new link_create.flags bit (for example a BPF_F_TRACING_MULTI_PROGS style flag), so that kernels without fds support fail with -EINVAL instead of reinterpreting ids?
Related to that, the new padding is not checked for zero:
include/uapi/linux/bpf.h
Quoted 1 line
include/uapi/linux/bpf.h +1
-0+ __u32 :32 ;
so it cannot later become an input field either.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
On Mon, Oct 12, 2026 at 11:40:13PM +0800, Leon Hwang wrote:
SNIP
Quoted 6 lines
kernel/trace/bpf_trace.c +1
-1@@ -3787,64 +3796,129 @@ static const struct bpf_link_ops bpf_tracing_multi_link_lops = {#endif }; -static int ids_cmp_r (const void *pa,const void *pb,const void *priv__maybe_unused ) +struct tracing_multi_sort_data {
kernel/trace/bpf_trace.c static const struct bpf_link_ops bpf_tracing_multi_link_lops = { · line 3793
Quoted 10 lines
kernel/trace/bpf_trace.c +7
-2+u64 *keys; +struct bpf_prog **progs; +u32 *ids; +u64 *cookies; +}; + +static int keys_cmp_r (const void *pa,const void *pb,const void *priv__maybe_unused ) { -u32 a = *(u32 *) pa; -u32 b = *(u32 *) pb;
I think you could just s/32/64/ and keep the rest in here and have simpler diff
Quoted 18 lines
kernel/trace/bpf_trace.c +8
-5+const u64 *key_a = pa, *key_b = pb; -return (a > b) - (a < b); +return (*key_a > *key_b) - (*key_a < *key_b); } -static void ids_swap_r (void *a,void *b,int size__maybe_unused , -const void *priv__maybe_unused ) +static void keys_swap_r (void *a,void *b,int size__maybe_unused , +const void *priv__maybe_unused ) { +const struct tracing_multi_sort_data *data = priv; +struct bpf_prog **prog_a, **prog_b;u64 *cookie_a, *cookie_b, *cookies; -u32 *id_a = a, *id_b = b, *ids; -void **data = (void **) priv; +u64 *key_a = a, *key_b = b, *keys; +u32 *id_a, *id_b, *ids;
kernel/trace/bpf_trace.c static const struct bpf_link_ops bpf_tracing_multi_link_lops = { · line 3828
Quoted 10 lines
kernel/trace/bpf_trace.c +7
-2+ + keys = data->keys; + ids = data->ids; + cookies = data->cookies; + id_a = ids + (key_a - keys); + id_b = ids + (key_b - keys); - ids = data[0 ]; - cookies = data[1 ]; +if (data->progs) {
nit, let's unify with the rest of the code and have progs variable for data->progs ?
SNIP
Quoted 15 lines
kernel/trace/bpf_trace.c +10
-1- err =check_dup_ids (ids, cookies, cnt); + keys =kvmalloc_objs (*keys, cnt); +if (!keys) { + err = -ENOMEM ; + goto error; + } + +for (int i =0 ; i < cnt; i++) + keys[i] =bpf_trampoline_compute_key (progs ? progs[i] :NULL , btf, ids[i]); + + err =check_dup_keys (keys, progs, ids, cookies, cnt);if (err) goto error;
kernel/trace/bpf_trace.c int bpf_tracing_multi_attach(struct bpf_prog *prog, const union bpf_attr *attr) · line 4029
Quoted 9 lines
kernel/trace/bpf_trace.c +4
-1@@ -3908,19 +4022,32 @@ int bpf_tracing_multi_attach(struct bpf_prog *prog, const union bpf_attr *attr) link->nodes_cnt = cnt; link->cookies = cookies; link->fexits = fexits; + link->tgt_progs = progs; + link->pokes = pokes; - err =bpf_trampoline_multi_attach (prog, ids, link); +if (progs) + err =bpf_trampoline_multi_prog_attach (prog, ids, keys, link);
you need keys computed ahead for dups check, so let's the above bpf_trampoline_compute_key calls be the single place we compute keys and pass keys to bpf_trampoline_multi_attach below as well?
kernel/trace/bpf_trace.c int bpf_tracing_multi_attach(struct bpf_prog *prog, const union bpf_attr *attr) · line 4031
Quoted 2 lines
kernel/trace/bpf_trace.c +2
-0+else + err =bpf_trampoline_multi_attach (prog, ids, link);
nit, let's add empty line in here?
kernel/trace/bpf_trace.c int bpf_tracing_multi_attach(struct bpf_prog *prog, const union bpf_attr *attr) · line 4040
Quoted 9 lines
kernel/trace/bpf_trace.c +3
-0kvfree (ids); +kvfree (keys); +kvfree (fds);if (err) { +bpf_put_progs (progs, cnt);bpf_link_cleanup (&link_primer);return err; }return bpf_link_settle (&link_primer);
thanks,
jirka
PATCH v2 12/21 libbpf: Add tracing_multi link support for bpf progs
In reply to Leon HwangBy design, a tracing_multi link does not support both kernel functions and bpf progs at the same time.
In libbpf, the bpf_tracing_multi_opts options provides two new fields for tracing bpf progs:
* 'fds': the fd array of the target bpf progs. One fd must be provided multiple times for multiple targets.
* 'funcs': the function name array of the target bpf progs.
The 'cnt' field indicates the size of both 'fds' and 'funcs' arrays.
The 'funcs' will be translated to 'ids' by 'libbpf_find_prog_btf_id()'.
At last, 'fds'+'ids' will be passed to kernel.
--- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/bpf.h | 2 ++ tools/lib/bpf/libbpf.c | 41 +++++++++++++++++++++++++++++++++++++---- tools/lib/bpf/libbpf.h | 4 +++- 4 files changed, 43 insertions(+), 5 deletions(-)
tools/lib/bpf/bpf.c +1 -0
@@ -852,6 +852,7 @@ int bpf_link_create(int prog_fd, int target_fd, attr.link_create.tracing_multi.ids =ptr_to_u64 (OPTS_GET (opts, tracing_multi.ids,0 )); attr.link_create.tracing_multi.cookies =ptr_to_u64 (OPTS_GET (opts, tracing_multi.cookies,0 )); attr.link_create.tracing_multi.cnt =OPTS_GET (opts, tracing_multi.cnt,0 ); + attr.link_create.tracing_multi.fds =ptr_to_u64 (OPTS_GET (opts, tracing_multi.fds,0 ));if (!OPTS_ZEROED (opts, tracing_multi))return libbpf_err (-EINVAL );break ;
tools/lib/bpf/bpf.h +2 -0
@@ -474,6 +474,8 @@ struct bpf_link_create_opts {const __u32 *ids;const __u64 *cookies;__u32 cnt; + __u32 :32 ; +const int *fds; } tracing_multi; }; size_t :0;
tools/lib/bpf/libbpf.c +37 -4
70 lines not on this page · raw message
tools/lib/bpf/libbpf.h +3 -1
@@ -746,10 +746,12 @@ struct bpf_tracing_multi_opts {const __u32 *ids;const __u64 *cookies;size_t cnt; +const int *fds; +const char **funcs; size_t :0; }; -#define bpf_tracing_multi_opts__last_field cnt +#define bpf_tracing_multi_opts__last_field funcs LIBBPF_API struct bpf_link * bpf_program__attach_tracing_multi(const struct bpf_program *prog, const char *pattern,
Signature · 1 line
2.55.0
PATCH v2 13/21 bpf: Add tracing_multi link fdinfo support for bpf progs
In reply to Leon HwangShow the target bpf prog info in the fdinfo.
$ cat /proc/self/fdinfo/13 pos: 0 flags: 02000000 mnt_id: 18 ino: 1045 link_type: tracing_multi link_id: 9 prog_tag: 9a7f9e3ab6f138ab prog_id: 25 attach_type: 61 cnt: 2 prog-id btf-id cookie func 21 4 16706819 target_1 22 5 16706819 target_2
Different from the fdinfo of link whose targets are kernel functions, here shows 'prog-id' instead of 'obj-id' to describe the ID of the target bpf progs. And, 'func' shows the concrete function name of the target bpf progs, because the function name can be retrieved by bpftool via 'prog-id'
+ 'btf-id'.
--- kernel/trace/bpf_trace.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-)
kernel/trace/bpf_trace.c +24 -5
@@ -3761,26 +3761,45 @@ static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link, }#ifdef CONFIG_PROC_FS +static const char *bpf_prog_func_name (struct bpf_prog *prog,u32 btf_id) +{ +const struct btf *btf = prog->aux->btf; +const struct btf_type *t; + + t =btf_type_by_id (btf, btf_id); +if (!t || !btf_type_is_func (t)) +return "" ; + +return btf_name_by_offset (btf, t->name_off); +} +static void bpf_tracing_multi_show_fdinfo (const struct bpf_link *link,struct seq_file *seq) {struct bpf_tracing_multi_link *tr_link = container_of(link, struct bpf_tracing_multi_link, link);bool has_cookies = !!tr_link->cookies; +bool has_progs = !!tr_link->tgt_progs;seq_printf (seq,"attach_type:\t%u\n" , tr_link->link.attach_type);seq_printf (seq,"cnt:\t%u\n" , tr_link->nodes_cnt); -seq_printf (seq,"%s\t %s\t %s\t %s\n" ,"obj-id" ,"btf-id" ,"cookie" ,"func" ); +seq_printf (seq,"%s\t %s\t %s\t %s\n" , has_progs ?"prog-id" :"obj-id" ,"btf-id" ,"cookie" , +"func" );for (int i =0 ; i < tr_link->nodes_cnt; i++) {struct bpf_tracing_multi_node *mnode = &tr_link->nodes[i]; +u64 cookie = has_cookies ? tr_link->cookies[i] :0 ;u32 btf_id, obj_id;bpf_trampoline_unpack_key (mnode->trampoline->key, &obj_id, &btf_id); -seq_printf (seq,"%u\t %u\t %llu\t %pS\n" , - obj_id, btf_id, - has_cookies ? tr_link->cookies[i] :0 , - (void *) mnode->trampoline->ip); +if (has_progs) +seq_printf (seq,"%u\t %u\t %llu\t %s\n" , + obj_id, btf_id, cookie, +bpf_prog_func_name (tr_link->tgt_progs[i], btf_id)); +else +seq_printf (seq,"%u\t %u\t %llu\t %pS\n" , + obj_id, btf_id, cookie, + (void *) mnode->trampoline->ip);cond_resched (); }
Signature · 1 line
2.55.0
PATCH v2 14/21 bpf: Add tracing_multi link info support for bpf progs
In reply to Leon HwangShow bpf prog info in the link info.
$ bpftool link 9: tracing_multi prog 25 attach_type trace_fsession_multi btf_obj_id 1 count 2 prog_id func_btf_id addr cookie func [module] 21 4 ffffffffc080064c 16706819 target_1 [bpf] 22 5 ffffffffc08006d4 16706819 target_2 [bpf] pids test_progs(98)
--- include/uapi/linux/bpf.h | 7 +++- kernel/trace/bpf_trace.c | 26 ++++++++++---- tools/bpf/bpftool/link.c | 63 +++++++++++++++++++++++++++------- tools/include/uapi/linux/bpf.h | 7 +++- 4 files changed, 82 insertions(+), 21 deletions(-)
include/uapi/linux/bpf.h +6 -1
@@ -6891,6 +6891,10 @@ struct bpf_btf_info {__u32 kernel_btf; } __attribute__((aligned(8))); +enum { +BPF_F_TRACING_MULTI_TGT_PROGS = (1U <<0 ), +}; +struct bpf_link_info {__u32 type;__u32 id;
@@ -6976,10 +6980,11 @@ struct bpf_link_info {__u32 attach_type;__u32 count;/* in/out: tracing_multi target count */ __u32 btf_obj_id; - __u32 :32 ; +__u32 flags;__aligned_u64 ids;__aligned_u64 addrs;__aligned_u64 cookies; +__aligned_u64 func_btf_ids; } tracing_multi;struct {__u32 type;/* enum bpf_perf_event_type */
kernel/trace/bpf_trace.c +19 -7
@@ -3709,15 +3709,16 @@ static void bpf_tracing_multi_link_dealloc(struct bpf_link *link)static int bpf_tracing_multi_link_fill_link_info (const struct bpf_link *link,struct bpf_link_info *info) { +u32 __user *fids =u64_to_user_ptr (info->tracing_multi.func_btf_ids);u64 __user *ucookies =u64_to_user_ptr (info->tracing_multi.cookies);u64 __user *uaddrs =u64_to_user_ptr (info->tracing_multi.addrs);u32 __user *uids =u64_to_user_ptr (info->tracing_multi.ids);struct bpf_tracing_multi_link *tr_link;u32 ucount = info->tracing_multi.count; -bool has_cookies, show_addrs; +bool has_cookies, show_addrs, has_progs;int err =0 ; -if ((uids || ucookies || uaddrs) && !ucount) +if ((uids || ucookies || uaddrs || fids) && !ucount)return -EINVAL ; tr_link = container_of(link, struct bpf_tracing_multi_link, link);
@@ -3725,9 +3726,13 @@ static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link, info->tracing_multi.attach_type = tr_link->link.attach_type; info->tracing_multi.count = tr_link->nodes_cnt; info->tracing_multi.btf_obj_id =btf_obj_id (tr_link->link.prog->aux->attach_btf); + has_progs = !!tr_link->tgt_progs; + info->tracing_multi.flags = has_progs ?BPF_F_TRACING_MULTI_TGT_PROGS :0 ; -if (!uids && !ucookies && !uaddrs) +if (!uids && !ucookies && !uaddrs && !fids)return 0 ; +if (has_progs != !!fids) +return -EINVAL ;if (ucount < tr_link->nodes_cnt) err = -ENOSPC ;
@@ -3739,20 +3744,27 @@ static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link,for (int i =0 ; i < ucount; i++) {struct bpf_tracing_multi_node *mnode = &tr_link->nodes[i]; +u32 prog_id, btf_id;u64 addr, cookie; -u32 id; -bpf_trampoline_unpack_key (mnode->trampoline->key,NULL , &id); +bpf_trampoline_unpack_key (mnode->trampoline->key, &prog_id, &btf_id); - addr = show_addrs ? mnode->trampoline->ip :0 ; +if (!show_addrs) + addr =0 ; +else if (has_progs) + addr = (u64 ) mnode->trampoline->func.addr; +else + addr = mnode->trampoline->ip; cookie = has_cookies ? tr_link->cookies[i] :0 ; -if (uids &&put_user (id, uids + i)) +if (uids &&put_user (has_progs ? prog_id : btf_id, uids + i))return -EFAULT ;if (uaddrs &&put_user (addr, uaddrs + i))return -EFAULT ;if (ucookies &&put_user (cookie, ucookies + i))return -EFAULT ; +if (has_progs &&put_user ((u32 ) mnode->trampoline->key, fids + i)) +return -EFAULT ;cond_resched (); }
tools/bpf/bpftool/link.c +51 -12
140 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +6 -1
@@ -6817,6 +6817,10 @@ struct sk_reuseport_md {#define BPF_TAG_SIZE 8 +enum { +BPF_F_TRACING_MULTI_TGT_PROGS = (1U <<0 ), +}; +struct bpf_prog_info {__u32 type;__u32 id;
@@ -6976,10 +6980,11 @@ struct bpf_link_info {__u32 attach_type;__u32 count;/* in/out: tracing_multi target count */ __u32 btf_obj_id; - __u32 :32 ; +__u32 flags;__aligned_u64 ids;__aligned_u64 addrs;__aligned_u64 cookies; +__aligned_u64 func_btf_ids; } tracing_multi;struct {__u32 type;/* enum bpf_perf_event_type */
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v2 14/21] bpf: Add tracing_multi link info support for bpf progs
TL;DR: The new -EINVAL in tracing_multi fill_link_info may break existing userspace, e.g. bpftool link show, when querying links with bpf prog targets without func_btf_ids; it may stop listing later…
Quoted 1 line
bpf: Add
tracing_multilink info support for bpf progs
TL;DR: The new -EINVAL in tracing_multi fill_link_info may break existing userspace, e.g. bpftool link show, when querying links with bpf prog targets without func_btf_ids; it may stop listing later links.
Quoted 4 lines
kernel/trace/bpf_trace.c +0
-0
[ ... ]
Quoted 5 lines
kernel/trace/bpf_trace.c +1
-0@@ -3725,9 +3726,13 @@ static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link, info->tracing_multi.attach_type = tr_link->link.attach_type; info->tracing_multi.count = tr_link->nodes_cnt; info->tracing_multi.btf_obj_id =btf_obj_id (tr_link->link.prog->aux->attach_btf); + has_progs = !!tr_link->tgt_progs;
kernel/trace/bpf_trace.c static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link, · line 3738
Quoted 10 lines
kernel/trace/bpf_trace.c +4
-1+ info->tracing_multi.flags = has_progs ?BPF_F_TRACING_MULTI_TGT_PROGS :0 ; -if (!uids && !ucookies && !uaddrs) +if (!uids && !ucookies && !uaddrs && !fids)return 0 ; +if (has_progs != !!fids) +return -EINVAL ;if (ucount < tr_link->nodes_cnt) err = -ENOSPC ;
Does this new -EINVAL break existing userspace that queries a tracing_multi link with bpf prog targets?
A caller written before this change passes ids, addrs and cookies, but has no way to supply func_btf_ids. For example, the bpftool already in the tree does a two-step query in do_show_link():
if (info.type == BPF_LINK_TYPE_TRACING_MULTI && !info.tracing_multi.ids) {
...
info.tracing_multi.ids = ptr_to_u64(ids);
info.tracing_multi.addrs = ptr_to_u64(addrs);
info.tracing_multi.cookies = ptr_to_u64(cookies);
goto again;On the second call, for a link with prog targets, has_progs is true and fids is NULL, so this check returns -EINVAL:
if (has_progs != !!fids)
return -EINVAL;bpf_link_get_info_by_fd() returns before copy_to_user() when fill_link_info() fails, so the caller also never sees the new flags value that would explain the failure.
do_show_link() then prints "can't get link info: Invalid argument", and do_show() does:
err = do_show_link(fd);
if (err)
break;so "bpftool link show" stops at the first such link and does not list the links after it. Any other tool that only wants addrs or cookies for this link type would hit the same error.
The reverse direction also fails: a link without prog targets and a non-NULL func_btf_ids now returns -EINVAL for a request that would have succeeded before.
These queries worked before this commit. The check looks like it is needed only because ids now changes meaning, and carries prog IDs when has_progs is set:
put_user(has_progs ? prog_id : btf_id, uids + i)
Would it work to keep ids as BTF IDs and return the prog IDs in a separate new array, or to fill only the arrays the caller asked for, the way ids, addrs and cookies are already optional one by one?
The commit message does not mention this behavior, and no later patch in the series changes it.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
[email protected]
AI reviewRe: [PATCH bpf-next v2 14/21] bpf: Add tracing_multi link info support for bpf progs
In reply to Leon HwangThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] bpftool: buffer over-read of uninitialized heap memory in sym_name_trim_prefix() - [Medium] bpf…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] bpftool: buffer over-read of uninitialized heap memory in sym_name_trim_prefix()
- [Medium] bpf: strict fids check in bpf_tracing_multi_link_fill_link_info() breaks UAPI --
commit 69edd76aa1ef54260b1403707834fa758ec91952
Author: Leon Hwang <[email protected]>
bpf: Add tracing_multi link info support for bpf progs
This commit adds support for showing BPF program information in tracing_multi links via bpftool.
Quoted 4 lines
kernel/trace/bpf_trace.c +0
-0
[ ... ]
kernel/trace/bpf_trace.c static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link, · line 3735
Quoted 12 lines
kernel/trace/bpf_trace.c +5
-1@@ -3725,9 +3726,13 @@ static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link, info->tracing_multi.attach_type = tr_link->link.attach_type; info->tracing_multi.count = tr_link->nodes_cnt; info->tracing_multi.btf_obj_id =btf_obj_id (tr_link->link.prog->aux->attach_btf); + has_progs = !!tr_link->tgt_progs; + info->tracing_multi.flags = has_progs ?BPF_F_TRACING_MULTI_TGT_PROGS :0 ; -if (!uids && !ucookies && !uaddrs) +if (!uids && !ucookies && !uaddrs && !fids)return 0 ; +if (has_progs != !!fids) +return -EINVAL ;
[Severity: Medium]
Does this strict equality check break forward compatibility for generic userspace tools?
If a tool unconditionally pre-allocates and provides a func_btf_ids array while querying an older, non-BPF-targeted tracing_multi link (where has_progs is false), bpf_obj_get_info_by_fd() will fail with -EINVAL.
Would it be safer to gracefully ignore or zero-fill the fids buffer if the target does not have programs, rather than rejecting the query entirely?
Quoted 4 lines
tools/bpf/bpftool/link.c +0
-0
[ ... ]
Quoted 9 lines
tools/bpf/bpftool/link.c +5
-0@@ -422,12 +422,31 @@ show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr)jsonw_end_array (json_wtr); } +#define BPF_PROG_PREFIX "bpf_prog_" +#define BPF_PROG_PREFIX_LEN (sizeof(BPF_PROG_PREFIX) - 1) + +static const char * +sym_name_trim_prefix (struct kernel_sym *sym,bool is_prog)
tools/bpf/bpftool/link.c show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr) · line 439
Quoted 10 lines
tools/bpf/bpftool/link.c +10
-0+{ +int prefix_len; + +if (!is_prog) +return sym->name; + +/* Ref kernel/bpf/core.c::bpf_prog_ksym_set_name() */ + prefix_len =BPF_PROG_PREFIX_LEN ; + prefix_len +=BPF_TAG_SIZE *2 ; +if (sym->name[prefix_len] !='\0' )
[Severity: Medium]
Can this result in an out-of-bounds read if the kernel symbol name is shorter than the prefix length?
In bpftool, sym->name is allocated using libbpf_reallocarray() and populated via sscanf(), which null-terminates the string but leaves the remainder of the buffer uninitialized.
If a target address resolves to a short kernel symbol (for example, due to a TOCTOU race where a BPF program is unloaded and its vmalloc address is reused), sym->name[prefix_len] evaluates uninitialized heap memory. If that byte happens to be non-zero, it skips it and returns a pointer into the uninitialized region, potentially corrupting CLI or JSON output.
Should we verify that strlen(sym->name) is sufficiently large before checking the character at prefix_len?
Quoted 3 lines
tools/bpf/bpftool/link.c +3
-0+ prefix_len +=1 ;/* skip the '_' */ +return sym->name + prefix_len; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
Re: [PATCH bpf-next v2 14/21] bpf: Add tracing_multi link info support for bpf progs
In reply to Leon HwangQuoted 10 lines · Leon Hwang
Show bpf prog info in the link info.
$ bpftool link 9: tracing_multi prog 25 attach_type trace_fsession_multi btf_obj_id 1 count 2 prog_id func_btf_id addr cookie func [module] 21 4 ffffffffc080064c 16706819 target_1 [bpf] 22 5 ffffffffc08006d4 16706819 target_2 [bpf] pids test_progs(98)
Quoted 5 lines
Signed-off-by Leon Hwang <[email protected]>--- include/uapi/linux/bpf.h | 7 +++- kernel/trace/bpf_trace.c | 26 ++++++++++---- tools/bpf/bpftool/link.c | 63 +++++++++++++++++++++++++++-------
I think you should split the bpftool change into separate patch
jirka
kernel test robot
build botRe: [PATCH bpf-next v2 14/21] bpf: Add tracing_multi link info support for bpf progs
In reply to Leon Hwangkernel test robot noticed the following build warnings:
Hi Leon,
kernel test robot noticed the following build warnings:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Leon-Hwang/bpf-Initialize-ftrace_managed-in-bpf_trampoline_get/20261012-234003 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20261012154024.29423-15-leon.hwang%40linux.dev patch subject: [PATCH bpf-next v2 14/21] bpf: Add tracing_multi link info support for bpf progs config: i386-randconfig-1011-20261013 (https://download.01.org/0day-ci/archive/20261015/[email protected]/config) compiler: gcc-14 (Debian 14.2.0-19) 14.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261015/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
kernel/trace/bpf_trace.c: In function 'bpf_tracing_multi_link_fill_link_info':
>> kernel/trace/bpf_trace.c:3755:32: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
3755 | addr = (u64) mnode->trampoline->func.addr;
| ^vim +3755 kernel/trace/bpf_trace.c
3708
3709 static int bpf_tracing_multi_link_fill_link_info(const struct bpf_link *link,
3710 struct bpf_link_info *info)
3711 {
3712 u32 __user *fids = u64_to_user_ptr(info->tracing_multi.func_btf_ids);
3713 u64 __user *ucookies = u64_to_user_ptr(info->tracing_multi.cookies);
3714 u64 __user *uaddrs = u64_to_user_ptr(info->tracing_multi.addrs);
3715 u32 __user *uids = u64_to_user_ptr(info->tracing_multi.ids);
3716 struct bpf_tracing_multi_link *tr_link;
3717 u32 ucount = info->tracing_multi.count;
3718 bool has_cookies, show_addrs, has_progs;
3719 int err = 0;
3720
3721 if ((uids || ucookies || uaddrs || fids) && !ucount)
3722 return -EINVAL;
3723
3724 tr_link = container_of(link, struct bpf_tracing_multi_link, link);
3725
3726 info->tracing_multi.attach_type = tr_link->link.attach_type;
3727 info->tracing_multi.count = tr_link->nodes_cnt;
3728 info->tracing_multi.btf_obj_id = btf_obj_id(tr_link->link.prog->aux->attach_btf);
3729 has_progs = !!tr_link->tgt_progs;
3730 info->tracing_multi.flags = has_progs ? BPF_F_TRACING_MULTI_TGT_PROGS : 0;
3731
3732 if (!uids && !ucookies && !uaddrs && !fids)
3733 return 0;
3734 if (has_progs != !!fids)
3735 return -EINVAL;
3736
3737 if (ucount < tr_link->nodes_cnt)
3738 err = -ENOSPC;
3739 else
3740 ucount = tr_link->nodes_cnt;
3741
3742 has_cookies = !!tr_link->cookies;
3743 show_addrs = kallsyms_show_value(current_cred());
3744
3745 for (int i = 0; i < ucount; i++) {
3746 struct bpf_tracing_multi_node *mnode = &tr_link->nodes[i];
3747 u32 prog_id, btf_id;
3748 u64 addr, cookie;
3749
3750 bpf_trampoline_unpack_key(mnode->trampoline->key, &prog_id, &btf_id);
3751
3752 if (!show_addrs)
3753 addr = 0;
3754 else if (has_progs)
> 3755 addr = (u64) mnode->trampoline->func.addr;
3756 else
3757 addr = mnode->trampoline->ip;
3758 cookie = has_cookies ? tr_link->cookies[i] : 0;
3759
3760 if (uids && put_user(has_progs ? prog_id : btf_id, uids + i))
3761 return -EFAULT;
3762 if (uaddrs && put_user(addr, uaddrs + i))
3763 return -EFAULT;
3764 if (ucookies && put_user(cookie, ucookies + i))
3765 return -EFAULT;
3766 if (has_progs && put_user((u32) mnode->trampoline->key, fids + i))
3767 return -EFAULT;
3768
3769 cond_resched();
3770 }
3771
3772 return err;
3773 }
3774
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
PATCH v2 15/21 selftests/bpf: Add tracing_multi bpf prog attach test
In reply to Leon HwangAttach to two bpf progs with fentry.multi, fexit.multi, and fsession.multi progs.
--- .../selftests/bpf/prog_tests/tracing_multi.c | 74 +++++++++++++++++++ .../selftests/bpf/progs/tracing_multi_bpf.c | 52 +++++++++++++ 2 files changed, 126 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_bpf.c
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +74 -0
@@ -12,6 +12,7 @@#include "tracing_multi_verifier.skel.h" #include "tracing_multi_bench.skel.h" #include "tracing_multi_rollback.skel.h" +#include "tracing_multi_bpf.skel.h" #include "trace_helpers.h" static __u64 bpf_fentry_test_cookies[] = {
@@ -1126,3 +1127,76 @@ void test_tracing_multi_test(void)if (test__start_subtest ("fentry_after_multi" ))test_fentry_after_multi (); } + +static int run_bpf_target (struct bpf_program *prog,__u32 retval,const char *name) +{ +LIBBPF_OPTS (bpf_test_run_opts, topts); +int err; + + err =bpf_prog_test_run_opts (bpf_program__fd (prog), &topts); +if (!ASSERT_OK (err, name)) +return err; + +if (!ASSERT_EQ (topts.retval, retval, name)) +return -EINVAL ; + +return 0 ; +} + +static void test_link_api_bpf_prog (void ) +{ +const char *funcs[] = {"target_1" ,"target_2" }; +LIBBPF_OPTS (bpf_tracing_multi_opts, opts); +struct tracing_multi_bpf *skel =NULL ; +int fds[ARRAY_SIZE (funcs)]; +struct bpf_link *link; + + skel =tracing_multi_bpf__open_and_load (); +if (!ASSERT_OK_PTR (skel,"tracing_multi_bpf__open_and_load" )) +return ; + + fds[0 ] =bpf_program__fd (skel->progs.target_1); + fds[1 ] =bpf_program__fd (skel->progs.target_2); + + opts.fds = fds; + opts.funcs = funcs; + opts.cnt =ARRAY_SIZE (fds); + + link =bpf_program__attach_tracing_multi (skel->progs.test_fentry,NULL , &opts); +if (!ASSERT_OK_PTR (link,"attach_fentry" )) + goto cleanup; + skel->links.test_fentry = link; + + link =bpf_program__attach_tracing_multi (skel->progs.test_fexit,NULL , &opts); +if (!ASSERT_OK_PTR (link,"attach_fexit" )) + goto cleanup; + skel->links.test_fexit = link; + + link =bpf_program__attach_tracing_multi (skel->progs.test_fsession,NULL , &opts); +if (!ASSERT_OK_PTR (link,"attach_fsession" )) + goto cleanup; + skel->links.test_fsession = link; + +if (run_bpf_target (skel->progs.target_1,1 ,"target_1" )) + goto cleanup; +if (run_bpf_target (skel->progs.target_2,2 ,"target_2" )) + goto cleanup; + +ASSERT_EQ (skel->bss->target_1_result,1 ,"target_1_result" ); +ASSERT_EQ (skel->bss->target_2_result,1 ,"target_2_result" ); +ASSERT_EQ (skel->bss->test_result_fentry,ARRAY_SIZE (funcs),"test_result_fentry" ); +ASSERT_EQ (skel->bss->test_result_fexit,ARRAY_SIZE (funcs),"test_result_fexit" ); +ASSERT_EQ (skel->bss->test_result_fsession_entry,ARRAY_SIZE (funcs), +"test_result_fsession_entry" ); +ASSERT_EQ (skel->bss->test_result_fsession_exit,ARRAY_SIZE (funcs), +"test_result_fsession_exit" ); + +cleanup: +tracing_multi_bpf__destroy (skel); +} + +void test_tracing_multi_prog_test (void ) +{ +if (test__start_subtest ("link_api" )) +test_link_api_bpf_prog (); +}
tools/testing/selftests/bpf/progs/tracing_multi_bpf.c new file +52 -0
52 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH v2 16/21 selftests/bpf: Add tracing_multi bpf prog attach failure tests
In reply to Leon HwangTests against two libbpf failures and two kernel failures:
* libbpf: 'ids' + 'fds'.
* libbpf: 'pattern' + 'fds'.
* kernel: 'fds' + invalid 'ids'.
* kernel: duplicate targets via different fd for the same bpf prog + func.
--- tools/lib/bpf/libbpf.c | 2 +- tools/lib/bpf/libbpf_internal.h | 1 + .../selftests/bpf/prog_tests/tracing_multi.c | 100 ++++++++++++++++++ 3 files changed, 102 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf.c +1 -1
@@ -11604,7 +11604,7 @@ int libbpf_find_vmlinux_btf_id(const char *name,return libbpf_err (err); } -static int libbpf_find_prog_btf_id (const char *name,__u32 attach_prog_fd,int token_fd) +int libbpf_find_prog_btf_id (const char *name,__u32 attach_prog_fd,int token_fd) {struct bpf_prog_info info;__u32 info_len =sizeof (info);
tools/lib/bpf/libbpf_internal.h +1 -0
@@ -439,6 +439,7 @@ int btf_load_into_kernel(struct btf *btf, char *log_buf, size_t log_sz, __u32 log_level, int token_fd);struct btf *btf_load_from_kernel (__u32 id,struct btf *base_btf,int token_fd); +int libbpf_find_prog_btf_id (const char *name,__u32 attach_prog_fd,int token_fd);struct btf *btf_get_from_fd (int btf_fd,struct btf *base_btf); void btf_get_kernel_prefix_kind(enum bpf_attach_type attach_type,
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +100 -0
108 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH v2 17/21 selftests/bpf: Add tracing_multi bpf prog cookie test
In reply to Leon HwangVerify that the cookie also works for bpf prog targets of tracing_multi link.
--- .../selftests/bpf/prog_tests/tracing_multi.c | 21 +++++++++++++++++-- .../selftests/bpf/progs/tracing_multi_bpf.c | 13 ++++++++++-- 2 files changed, 30 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +19 -2
@@ -1143,9 +1143,12 @@ static int run_bpf_target(struct bpf_program *prog, __u32 retval, const char *nareturn 0 ; } -static void test_link_api_bpf_prog (void ) +static void test_link_api_bpf_prog (bool test_cookies) {const char *funcs[] = {"target_1" ,"target_2" }; +__u64 fentry_cookies[] = {0xfeed01 ,0xfeed02 }; +__u64 fexit_cookies[] = {0xfeed11 ,0xfeed12 }; +__u64 fsession_cookies[] = {0xfeed21 ,0xfeed22 };LIBBPF_OPTS (bpf_tracing_multi_opts, opts);struct tracing_multi_bpf *skel =NULL ;int fds[ARRAY_SIZE (funcs)];
@@ -1158,27 +1161,39 @@ static void test_link_api_bpf_prog(void) fds[0 ] =bpf_program__fd (skel->progs.target_1); fds[1 ] =bpf_program__fd (skel->progs.target_2); + skel->bss->test_cookies = test_cookies; + opts.fds = fds; opts.funcs = funcs; opts.cnt =ARRAY_SIZE (fds); + opts.cookies = test_cookies ? fentry_cookies :NULL ; link =bpf_program__attach_tracing_multi (skel->progs.test_fentry,NULL , &opts);if (!ASSERT_OK_PTR (link,"attach_fentry" )) goto cleanup; skel->links.test_fentry = link; + opts.cookies = test_cookies ? fexit_cookies :NULL ; link =bpf_program__attach_tracing_multi (skel->progs.test_fexit,NULL , &opts);if (!ASSERT_OK_PTR (link,"attach_fexit" )) goto cleanup; skel->links.test_fexit = link; + opts.cookies = test_cookies ? fsession_cookies :NULL ; link =bpf_program__attach_tracing_multi (skel->progs.test_fsession,NULL , &opts);if (!ASSERT_OK_PTR (link,"attach_fsession" )) goto cleanup; skel->links.test_fsession = link; + skel->bss->fentry_cookie = fentry_cookies[0 ]; + skel->bss->fexit_cookie = fexit_cookies[0 ]; + skel->bss->fsession_cookie = fsession_cookies[0 ];if (run_bpf_target (skel->progs.target_1,1 ,"target_1" )) goto cleanup; + + skel->bss->fentry_cookie = fentry_cookies[1 ]; + skel->bss->fexit_cookie = fexit_cookies[1 ]; + skel->bss->fsession_cookie = fsession_cookies[1 ];if (run_bpf_target (skel->progs.target_2,2 ,"target_2" )) goto cleanup;
@@ -1296,7 +1311,9 @@ static void test_attach_api_bpf_prog_fails(void)void test_tracing_multi_prog_test (void ) {if (test__start_subtest ("link_api" )) -test_link_api_bpf_prog (); +test_link_api_bpf_prog (false);if (test__start_subtest ("attach_api_fails" ))test_attach_api_bpf_prog_fails (); +if (test__start_subtest ("cookies" )) +test_link_api_bpf_prog (true); }
tools/testing/selftests/bpf/progs/tracing_multi_bpf.c +11 -2
@@ -5,6 +5,10 @@ char _license[] SEC("license") = "GPL"; +bool test_cookies; +__u64 fentry_cookie; +__u64 fexit_cookie; +__u64 fsession_cookie;__u64 test_result_fentry;__u64 test_result_fexit;__u64 test_result_fsession_entry;
@@ -29,20 +33,25 @@ int target_2(void *ctx) SEC("fentry.multi")int BPF_PROG (test_fentry ) { - test_result_fentry++; +if (!test_cookies ||bpf_get_attach_cookie (ctx) == fentry_cookie) + test_result_fentry++;return 0 ; } SEC("fexit.multi")int BPF_PROG (test_fexit ) { - test_result_fexit++; +if (!test_cookies ||bpf_get_attach_cookie (ctx) == fexit_cookie) + test_result_fexit++;return 0 ; } SEC("fsession.multi")int BPF_PROG (test_fsession ) { +if (test_cookies &&bpf_get_attach_cookie (ctx) != fsession_cookie) +return 0 ; +if (bpf_session_is_return (ctx)) test_result_fsession_exit++;else
Signature · 1 line
2.55.0
PATCH v2 18/21 selftests/bpf: Add tracing_multi bpf prog rollback test
In reply to Leon HwangWhen rollbacking, the prepared trampolines should be reverted.
In this test, the first bpf prog will be attached with the trampoline, and the second bpf prog won't be attached because the bpf prog has been attached by a fentry.multi prog, that causes the rollback procedure.
--- .../selftests/bpf/prog_tests/tracing_multi.c | 63 +++++++++++++++++++ 1 file changed, 63 insertions(+)
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +63 -0
@@ -1308,6 +1308,67 @@ static void test_attach_api_bpf_prog_fails(void)free (ids); } +static void test_rollback_bpf_prog (void ) +{ +int token_fd, prog_fd, fd =-1 , pre_fd =-1 ; +LIBBPF_OPTS (bpf_link_create_opts, lopts); +struct tracing_multi_bpf *skel =NULL ; +__u32 target_ids[2 ], ids[2 ]; +int target_fds[2 ], fds[2 ]; +struct bpf_program *prog; +int first, second; + + skel =tracing_multi_bpf__open_and_load (); +if (!ASSERT_OK_PTR (skel,"tracing_multi_bpf__open_and_load" )) +return ; + + token_fd =bpf_object__token_fd (skel->obj); + token_fd = token_fd <0 ?0 : token_fd; + target_fds[0 ] =bpf_program__fd (skel->progs.target_1); + target_fds[1 ] =bpf_program__fd (skel->progs.target_2); + target_ids[0 ] =libbpf_find_prog_btf_id ("target_1" , target_fds[0 ], token_fd); +if (!ASSERT_GT (target_ids[0 ],0 ,"target_1_btf_id" )) + goto cleanup; + target_ids[1 ] =libbpf_find_prog_btf_id ("target_2" , target_fds[1 ], token_fd); +if (!ASSERT_GT (target_ids[1 ],0 ,"target_2_btf_id" )) + goto cleanup; + + first = target_fds[0 ] < target_fds[1 ] ?0 :1 ; + second =1 - first; + fds[0 ] = target_fds[first]; + fds[1 ] = target_fds[second]; + ids[0 ] = target_ids[first]; + ids[1 ] = target_ids[second]; + + prog_fd =bpf_program__fd (skel->progs.test_fentry); + lopts.tracing_multi.ids = &ids[1 ]; + lopts.tracing_multi.fds = &fds[1 ]; + lopts.tracing_multi.cnt =1 ; + pre_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_FENTRY_MULTI , &lopts); +if (!ASSERT_GE (pre_fd,0 ,"pre_attach" )) + goto cleanup; + + lopts.tracing_multi.ids = ids; + lopts.tracing_multi.fds = fds; + lopts.tracing_multi.cnt =ARRAY_SIZE (ids); + + fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_FENTRY_MULTI , &lopts); +if (!ASSERT_EQ (fd, -EBUSY ,"bpf_link_create" )) + goto cleanup; + + prog = first ==0 ? skel->progs.target_1 : skel->progs.target_2; +if (run_bpf_target (prog, first +1 ,"rollback_target" )) + goto cleanup; +ASSERT_EQ (skel->bss->test_result_fentry,0 ,"test_result_fentry" ); + +cleanup: +if (fd >=0 ) +close (fd); +if (pre_fd >=0 ) +close (pre_fd); +tracing_multi_bpf__destroy (skel); +} +void test_tracing_multi_prog_test (void ) {if (test__start_subtest ("link_api" ))
@@ -1316,4 +1377,6 @@ void test_tracing_multi_prog_test(void)test_attach_api_bpf_prog_fails ();if (test__start_subtest ("cookies" ))test_link_api_bpf_prog (true); +if (test__start_subtest ("rollback" )) +test_rollback_bpf_prog (); }
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v2 18/21] selftests/bpf: Add tracing_multi bpf prog rollback test
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] bpf: selftest tracing_multi rollback logic relies on unreliable FD ordering --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] bpf: selftest tracing_multi rollback logic relies on unreliable FD ordering --
commit 00cadf9aa091042698069799ffcf7faa2dd5e6a5
Author: Leon Hwang <[email protected]>
selftests/bpf: Add tracing_multi bpf prog rollback test
This commit adds a rollback test for multi-attach fentry BPF programs. It sets up two targets, pre-attaches to one, and then verifies that a subsequent multi-attach to both correctly handles the busy state and rolls back.
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +0
-0@@ -1308,6 +1308,67 @@ static void test_attach_api_bpf_prog_fails(void)
[ ... ]
tools/testing/selftests/bpf/prog_tests/tracing_multi.c static void test_attach_api_bpf_prog_fails(void) · line 1341
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +6
-0+ first = target_fds[0 ] < target_fds[1 ] ?0 :1 ; + second =1 - first; + fds[0 ] = target_fds[first]; + fds[1 ] = target_fds[second]; + ids[0 ] = target_ids[first]; + ids[1 ] = target_ids[second];
[Severity: Medium]
Does this code incorrectly rely on file descriptor order to predict the kernel's internal sort order?
The kernel sorts the targets based on a computed key derived from the program ID and BTF ID: ((u64)prog->aux->id << 32) | btf_id. While program IDs monotonically increase, FD allocation uses the lowest available FD.
If a lower FD becomes available (for example, closed by another thread or process) between loading the first and second targets, the second target receives a smaller FD but a larger ID. This reversed order means the test will incorrectly predict how the kernel sorts the targets.
Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +11
-0+ + prog_fd =bpf_program__fd (skel->progs.test_fentry); + lopts.tracing_multi.ids = &ids[1 ]; + lopts.tracing_multi.fds = &fds[1 ]; + lopts.tracing_multi.cnt =1 ; + pre_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_FENTRY_MULTI , &lopts); +if (!ASSERT_GE (pre_fd,0 ,"pre_attach" )) + goto cleanup; + + lopts.tracing_multi.ids = ids; + lopts.tracing_multi.fds = fds;
tools/testing/selftests/bpf/prog_tests/tracing_multi.c static void test_attach_api_bpf_prog_fails(void) · line 1362
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +10
-0+ lopts.tracing_multi.cnt =ARRAY_SIZE (ids); + + fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_FENTRY_MULTI , &lopts); +if (!ASSERT_EQ (fd, -EBUSY ,"bpf_link_create" )) + goto cleanup; + + prog = first ==0 ? skel->progs.target_1 : skel->progs.target_2; +if (run_bpf_target (prog, first +1 ,"rollback_target" )) + goto cleanup; +ASSERT_EQ (skel->bss->test_result_fentry,0 ,"test_result_fentry" );
[Severity: Medium]
Can this assertion pass without the rollback path ever being tested?
If the FD order doesn't match the ID order as mentioned above, the test will pre-attach the element that the kernel sorts first. The kernel will then fail immediately on the first element of the multi-attach request. Because it failed on the first element, it never proceeds to attach (or need to roll back) the other elements.
The test_result_fentry variable remains 0 in this case, meaning this assertion will pass and silently mask that the rollback path wasn't actually exercised.
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/tracing_multi.c +8
-0+ +cleanup: +if (fd >=0 ) +close (fd); +if (pre_fd >=0 ) +close (pre_fd); +tracing_multi_bpf__destroy (skel); +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=18
Re: [PATCH bpf-next v2 18/21] selftests/bpf: Add tracing_multi bpf prog rollback test
In reply to Leon HwangQuoted 5 lines · Leon Hwang
When rollbacking, the prepared trampolines should be reverted.
In this test, the first bpf prog will be attached with the trampoline, and the second bpf prog won't be attached because the bpf prog has been attached by a fentry.multi prog, that causes the rollback procedure.
there's rollback in bpf_trampoline_multi_prog_attach and in bpf_trampoline_multi_prog_prepare, would be great to cover both with tests
jirka
PATCH v2 19/21 selftests/bpf: Add tracing_multi bpf prog link info test
In reply to Leon HwangVerify the link info works well for bpf prog targets of tracing_multi link.
--- .../selftests/bpf/prog_tests/fill_link_info.c | 169 ++++++++++++++++-- 1 file changed, 151 insertions(+), 18 deletions(-)
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +151 -18
@@ -3,11 +3,13 @@#include <string.h> #include <linux/bpf.h> +#include <linux/kallsyms.h> #include <linux/limits.h> #include <bpf/btf.h> #include <test_progs.h> #include "trace_helpers.h" #include "test_fill_link_info.skel.h" +#include "tracing_multi_bpf.skel.h" #include "bpf/libbpf_internal.h" #define TP_CAT "sched"
@@ -39,8 +41,19 @@ struct tmulti_target {__u64 addr;__u64 cookie;__u32 id; +__u32 func_btf_id; }; +static const char *tmulti_bpf_funcs[] = { +"target_1" , +"target_2" , +}; + +static __u64 tmulti_bpf_fentry_cookies[] = {0xfeed01 ,0xfeed02 }; +static __u64 tmulti_bpf_fexit_cookies[] = {0xfeed11 ,0xfeed12 }; +static __u64 tmulti_bpf_fsession_cookies[] = {0xfeed21 ,0xfeed22 }; +#define TRACING_MULTI_BPF_CNT ARRAY_SIZE(tmulti_bpf_funcs) +#define UPROBE_FILE "/proc/self/exe"static ssize_t uprobe_offset;/* uprobe attach point */
@@ -421,14 +434,28 @@ static int tmulti_target_cmp(const void *a, const void *b)return (ta->id > tb->id) - (ta->id < tb->id); } +static int setup_tmulti_btf_obj_id (const struct bpf_program *prog,__u32 *btf_obj_id) +{ +struct bpf_prog_info prog_info = {}; +__u32 len =sizeof (prog_info); +int err; + + err =bpf_prog_get_info_by_fd (bpf_program__fd (prog), &prog_info, &len); +if (!ASSERT_OK (err,"bpf_prog_get_info_by_fd" )) +return -1 ; +if (!ASSERT_GT (prog_info.attach_btf_obj_id,0 ,"attach_btf_obj_id" )) +return -1 ; + + *btf_obj_id = prog_info.attach_btf_obj_id; +return 0 ; +} +static int setup_tmulti_targets (const struct bpf_program *prog,struct tmulti_target *targets,__u32 *btf_obj_id) { -struct bpf_prog_info prog_info; -__u32 len =sizeof (prog_info);struct btf *btf; -int err, i; +int i;__s32 id; btf =btf__load_vmlinux_btf ();
@@ -446,13 +473,8 @@ static int setup_tmulti_targets(const struct bpf_program *prog, targets[i].id = id; } -memset (&prog_info,0 , len); - err =bpf_prog_get_info_by_fd (bpf_program__fd (prog), &prog_info, &len); -if (!ASSERT_OK (err,"bpf_prog_get_info_by_fd" )) - goto error; -if (!ASSERT_GT (prog_info.attach_btf_obj_id,0 ,"attach_btf_obj_id" )) +if (setup_tmulti_btf_obj_id (prog, btf_obj_id)) goto error; - *btf_obj_id = prog_info.attach_btf_obj_id;/* * The kernel tracing multi attach sorts ids. We sort as well,
@@ -467,13 +489,49 @@ static int setup_tmulti_targets(const struct bpf_program *prog,return -1 ; } +static int setup_tmulti_bpf_prog_targets (const struct bpf_program *prog,int token_fd, +const __u64 *cookies, +const struct bpf_program *const *target_progs, +struct tmulti_target *targets, +__u32 *btf_obj_id) +{ +struct bpf_prog_info prog_info = {}; +__u32 len =sizeof (prog_info), func_btf_id; +int err, i, prog_fd; + +for (i =0 ; i <TRACING_MULTI_BPF_CNT ; i++) { +memset (&prog_info,0 ,sizeof (prog_info)); + len =sizeof (prog_info); + prog_info.jited_ksyms =ptr_to_u64 (&targets[i].addr); + prog_info.nr_jited_ksyms =1 ; + prog_fd =bpf_program__fd (target_progs[i]); + err =bpf_prog_get_info_by_fd (prog_fd, &prog_info, &len); +if (!ASSERT_OK (err,"bpf_prog_get_info_by_fd" )) +return -1 ; + + func_btf_id =libbpf_find_prog_btf_id (tmulti_bpf_funcs[i], prog_fd, token_fd); +if (!ASSERT_GT (func_btf_id,0 ,"libbpf_find_prog_btf_id" )) +return -1 ; + + targets[i].cookie = cookies[i]; + targets[i].id = prog_info.id; + targets[i].func_btf_id = func_btf_id; + } + +qsort (targets,TRACING_MULTI_BPF_CNT ,sizeof (targets[0 ]), tmulti_target_cmp); +return setup_tmulti_btf_obj_id (prog, btf_obj_id); +} +static int verify_tracing_multi_link_info (int fd,const struct bpf_program *prog,const struct tmulti_target *targets, -__u32 btf_obj_id,bool has_cookies) +__u32 btf_obj_id,__u32 count, +bool has_cookies,bool tgt_progs) { +#define TMULTI_CNT (TRACING_MULTI_CNT > TRACING_MULTI_BPF_CNT ? \ + TRACING_MULTI_CNT : TRACING_MULTI_BPF_CNT)enum bpf_attach_type attach_type =bpf_program__expected_attach_type (prog); -__u64 addrs[TRACING_MULTI_CNT ], cookies[TRACING_MULTI_CNT ]; -__u32 ids[TRACING_MULTI_CNT ]; +__u64 addrs[TMULTI_CNT ], cookies[TMULTI_CNT ]; +__u32 ids[TMULTI_CNT ], fids[TMULTI_CNT ];struct bpf_link_info info;__u32 len =sizeof (info);int err, i;
@@ -487,16 +545,21 @@ static int verify_tracing_multi_link_info(int fd, const struct bpf_program *progreturn -1 ;ASSERT_EQ (info.tracing_multi.attach_type, attach_type,"info.tracing_multi.attach_type" ); -ASSERT_EQ (info.tracing_multi.count,TRACING_MULTI_CNT ,"info.tracing_multi.count" ); +ASSERT_EQ (info.tracing_multi.count, count,"info.tracing_multi.count" ); +if (tgt_progs) +ASSERT_TRUE (info.tracing_multi.flags &BPF_F_TRACING_MULTI_TGT_PROGS , +"info.tracing_multi.flags" );memset (ids,0 ,sizeof (ids)); +memset (fids,0 ,sizeof (fids));memset (cookies,0 ,sizeof (cookies));memset (addrs,0 ,sizeof (addrs)); info.tracing_multi.ids =ptr_to_u64 (ids); info.tracing_multi.addrs =ptr_to_u64 (addrs); info.tracing_multi.cookies = has_cookies ?ptr_to_u64 (cookies) :0 ; - info.tracing_multi.count =TRACING_MULTI_CNT ; + info.tracing_multi.func_btf_ids = tgt_progs ?ptr_to_u64 (fids) :0 ; + info.tracing_multi.count = count; err =bpf_link_get_info_by_fd (fd, &info, &len);if (!ASSERT_OK (err,"bpf_link_get_info_by_fd" ))
@@ -506,14 +569,20 @@ static int verify_tracing_multi_link_info(int fd, const struct bpf_program *progreturn -1 ;ASSERT_EQ (info.tracing_multi.attach_type, attach_type,"info.tracing_multi.attach_type" ); -ASSERT_EQ (info.tracing_multi.count,TRACING_MULTI_CNT ,"info.tracing_multi.count" ); +ASSERT_EQ (info.tracing_multi.count, count,"info.tracing_multi.count" );ASSERT_EQ (info.tracing_multi.btf_obj_id, btf_obj_id,"tracing_multi.btf_obj_id" ); +if (tgt_progs) +ASSERT_TRUE (info.tracing_multi.flags &BPF_F_TRACING_MULTI_TGT_PROGS , +"info.tracing_multi.flags" ); -for (i =0 ; i <TRACING_MULTI_CNT ; i++) { +for (i =0 ; i < count; i++) {ASSERT_EQ (ids[i], targets[i].id,"tracing_multi.ids" );ASSERT_EQ (cookies[i], has_cookies ? targets[i].cookie :0 ,"tracing_multi.cookies" ); -if (targets[i].addr) { +if (tgt_progs) { +ASSERT_EQ (addrs[i], targets[i].addr,"tracing_multi.addrs" ); +ASSERT_EQ (fids[i], targets[i].func_btf_id,"tracing_multi.func_btf_ids" ); + }else if (targets[i].addr) {struct ksym *ksym;if (!ASSERT_NEQ (addrs[i],0 ,"tracing_multi.addrs" ))
@@ -624,13 +693,73 @@ static void test_tracing_multi_fill_link_info(struct test_fill_link_info *skel,verify_tracing_multi_invalid_user_buffer (link_fd, targets); }else { err =verify_tracing_multi_link_info (link_fd, skel->progs.tmulti_run, - targets, btf_obj_id, has_cookies); + targets, btf_obj_id,TRACING_MULTI_CNT , + has_cookies, false);ASSERT_OK (err,"verify_tracing_multi_link_info" ); }bpf_link__destroy (link); } +static void test_tracing_multi_bpf_fill_link_info_one (struct tracing_multi_bpf *skel, +struct bpf_program *prog, +__u64 *cookies, +bool has_cookies) +{ +LIBBPF_OPTS (bpf_tracing_multi_opts, opts); +const struct bpf_program *target_progs[] = { + skel->progs.target_1, + skel->progs.target_2, + }; +struct tmulti_target targets[TRACING_MULTI_BPF_CNT ] = {}; +int fds[TRACING_MULTI_BPF_CNT ], token_fd, err, i; +struct bpf_link *link; +__u32 btf_obj_id; + + token_fd =bpf_object__token_fd (skel->obj); + token_fd = token_fd <0 ?0 : token_fd; + +if (setup_tmulti_bpf_prog_targets (prog, token_fd, cookies, target_progs, targets, + &btf_obj_id)) +return ; + +for (i =0 ; i <TRACING_MULTI_BPF_CNT ; i++) + fds[i] =bpf_program__fd (target_progs[i]); + + opts.fds = fds; + opts.funcs = tmulti_bpf_funcs; + opts.cnt =ARRAY_SIZE (fds); + opts.cookies = has_cookies ? cookies :NULL ; + + link =bpf_program__attach_tracing_multi (prog,NULL , &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_tracing_multi" )) +return ; + + err =verify_tracing_multi_link_info (bpf_link__fd (link), prog, targets, btf_obj_id, +TRACING_MULTI_BPF_CNT , has_cookies, true); +ASSERT_OK (err,"verify_tracing_multi_link_info" ); + +bpf_link__destroy (link); +} + +static void test_tracing_multi_bpf_fill_link_info (bool has_cookies) +{ +struct tracing_multi_bpf *skel; + + skel =tracing_multi_bpf__open_and_load (); +if (!ASSERT_OK_PTR (skel,"tracing_multi_bpf__open_and_load" )) +return ; + +test_tracing_multi_bpf_fill_link_info_one (skel, skel->progs.test_fentry, + tmulti_bpf_fentry_cookies, has_cookies); +test_tracing_multi_bpf_fill_link_info_one (skel, skel->progs.test_fexit, + tmulti_bpf_fexit_cookies, has_cookies); +test_tracing_multi_bpf_fill_link_info_one (skel, skel->progs.test_fsession, + tmulti_bpf_fsession_cookies, has_cookies); + +tracing_multi_bpf__destroy (skel); +} +#define SEC (name) __attribute__((section(name), used)) static short uprobe_link_info_sema_1 SEC(".probes");
@@ -879,6 +1008,10 @@ void test_fill_link_info(void)test_tracing_multi_fill_link_info (skel, false, false);test_tracing_multi_fill_link_info (skel, true, false); } +if (test__start_subtest ("tracing_multi_bpf_link_info" )) { +test_tracing_multi_bpf_fill_link_info (false); +test_tracing_multi_bpf_fill_link_info (true); + }if (test__start_subtest ("tracing_multi_invalid_ubuff" ))test_tracing_multi_fill_link_info (skel, true, true);
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v2 19/21] selftests/bpf: Add tracing_multi bpf prog link info test
TL;DR: Nit only: the new <linux/kallsyms.h> include in fill_link_info.c may be unnecessary, since trace_helpers.h already seems to provide the ksym helpers the test uses.
Quoted 1 line
selftests/bpf: Add
tracing_multibpf prog link info test
TL;DR: Nit only: the new <linux/kallsyms.h> include in fill_link_info.c may be unnecessary, since trace_helpers.h already seems to provide the ksym helpers the test uses.
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +0
-0@@ -3,11 +3,13 @@
tools/testing/selftests/bpf/prog_tests/fill_link_info.c line 13
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +2
-0#include <string.h> #include <linux/bpf.h> +#include <linux/kallsyms.h> #include <linux/limits.h> #include <bpf/btf.h> #include <test_progs.h> #include "trace_helpers.h" #include "test_fill_link_info.skel.h" +#include "tracing_multi_bpf.skel.h" #include "bpf/libbpf_internal.h"
this isn't a bug, but is <linux/kallsyms.h> needed for anything here? The ksym helpers used by this test (ksym_search(), ksym_get_addr(), struct ksym) seem to come from trace_helpers.h, which is already included, and nothing in fill_link_info.c appears to use KSYM_NAME_LEN, kallsyms_lookup() or print_ip_sym() from the tools/include/linux/kallsyms.h shim.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
[email protected]
AI reviewRe: [PATCH bpf-next v2 19/21] selftests/bpf: Add tracing_multi bpf prog link info test
In reply to Leon HwangThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] selftests/bpf: Ineffective error check for libbpf_find_prog_btf_id due to __u32 cast --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] selftests/bpf: Ineffective error check for libbpf_find_prog_btf_id due to __u32 cast --
commit 208e2b6f875995feff87983db367727d95ea373b
Author: Leon Hwang <[email protected]>
selftests/bpf: Add tracing_multi bpf prog link info test
Verify the link info works well for bpf prog targets of tracing_multi link.
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +0
-0@@ -467,13 +489,49 @@ static int setup_tmulti_targets(const struct bpf_program *prog,return -1 ;
tools/testing/selftests/bpf/prog_tests/fill_link_info.c static int setup_tmulti_targets(const struct bpf_program *prog, · line 499
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +8
-0} +static int setup_tmulti_bpf_prog_targets (const struct bpf_program *prog,int token_fd, +const __u64 *cookies, +const struct bpf_program *const *target_progs, +struct tmulti_target *targets, +__u32 *btf_obj_id) +{ +struct bpf_prog_info prog_info = {}; +__u32 len =sizeof (prog_info), func_btf_id;
[Severity: Medium]libbpf_find_prog_btf_id() returns an int which can be negative to indicate an error. Since func_btf_id is declared as an unsigned 32-bit integer, does assigning the return value here cast any negative error code to a large positive value?
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +4
-0+int err, i, prog_fd; + +for (i =0 ; i <TRACING_MULTI_BPF_CNT ; i++) { +memset (&prog_info,0 ,sizeof (prog_info));
tools/testing/selftests/bpf/prog_tests/fill_link_info.c static int setup_tmulti_targets(const struct bpf_program *prog, · line 513
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +10
-0+ len =sizeof (prog_info); + prog_info.jited_ksyms =ptr_to_u64 (&targets[i].addr); + prog_info.nr_jited_ksyms =1 ; + prog_fd =bpf_program__fd (target_progs[i]); + err =bpf_prog_get_info_by_fd (prog_fd, &prog_info, &len); +if (!ASSERT_OK (err,"bpf_prog_get_info_by_fd" )) +return -1 ; + + func_btf_id =libbpf_find_prog_btf_id (tmulti_bpf_funcs[i], prog_fd, token_fd); +if (!ASSERT_GT (func_btf_id,0 ,"libbpf_find_prog_btf_id" ))
[Severity: Medium]
If a negative error code is cast to a large positive value in the unsigned func_btf_id, will this greater-than-zero check incorrectly pass, masking test failures during BTF ID lookup and allowing the test to proceed with an invalid BTF ID?
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +6
-0+return -1 ; + + targets[i].cookie = cookies[i]; + targets[i].id = prog_info.id; + targets[i].func_btf_id = func_btf_id; + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=19
PATCH v2 20/21 selftests/bpf: Test tailcall with fentry.multi
In reply to Leon HwangPositively verify that a tail call from a BPF subprog works when the subprog is traced by fentry.multi.
--- .../selftests/bpf/prog_tests/tailcalls.c | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+)
tools/testing/selftests/bpf/prog_tests/tailcalls.c +64 -0
@@ -16,7 +16,9 @@#include "tailcall_sleepable.skel.h" #include "tailcall_callback.skel.h" #include "tailcall_bpf2bpf2.skel.h" +#include "tailcall_bpf2bpf4.skel.h" #include "tailcall_bpf2bpf_fexit.skel.h" +#include "tracing_multi_bpf.skel.h" /* test_tailcall_1 checks basic functionality by patching multiple locations * in a single program for a single tail call slot with nop->jmp, jmp->nop
@@ -2065,6 +2067,66 @@ static void test_tailcall_large_stack(void)tailcall_large_stack__destroy (skel); } +/* +* Verify that a tail call from a BPF subprog works when the subprog is +* traced by fentry.multi. +*/ +static void test_tailcall_bpf2bpf_fentry_multi (void ) +{ +static const char *funcs[] = {"subprog_tail_2" }; +struct tracing_multi_bpf *skel_tracing =NULL ; +LIBBPF_OPTS (bpf_tracing_multi_opts, opts); +struct tailcall_bpf2bpf4 *skel_tc; +struct bpf_link *link =NULL ; +int fds[ARRAY_SIZE (funcs)]; +char buff[128 ] = {}; +int err, key; +LIBBPF_OPTS (bpf_test_run_opts, topts, + .data_in = buff, + .data_size_in = sizeof(buff), + .repeat = 1, + ); + + skel_tc =tailcall_bpf2bpf4__open_and_load (); +if (!ASSERT_OK_PTR (skel_tc,"tailcall_bpf2bpf4__open_and_load" )) +return ; + + skel_tracing =tracing_multi_bpf__open_and_load (); +if (!ASSERT_OK_PTR (skel_tracing,"tracing_multi_bpf__open_and_load" )) + goto out; + + key =0 ; + fds[0 ] =bpf_program__fd (skel_tc->progs.classifier_2); + err =bpf_map_update_elem (bpf_map__fd (skel_tc->maps.jmp_table), &key, &fds[0 ],BPF_ANY ); +if (!ASSERT_OK (err,"bpf_map_update_elem" )) + goto out; + + key =2 ; + err =bpf_map_update_elem (bpf_map__fd (skel_tc->maps.jmp_table), &key, &fds[0 ],BPF_ANY ); +if (!ASSERT_OK (err,"bpf_map_update_elem_loop" )) + goto out; + + opts.fds = fds; + opts.funcs = funcs; + opts.cnt =ARRAY_SIZE (fds); + link =bpf_program__attach_tracing_multi (skel_tracing->progs.test_fentry,NULL , &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_tracing_multi" )) + goto out; + + err =bpf_prog_test_run_opts (bpf_program__fd (skel_tc->progs.entry), &topts); +if (!ASSERT_OK (err,"bpf_prog_test_run_opts" )) + goto out; + +ASSERT_EQ (topts.retval,sizeof (buff) *3 ,"retval" ); +ASSERT_EQ (skel_tc->bss->count,33 ,"count" ); +ASSERT_EQ (skel_tracing->bss->test_result_fentry,33 ,"fentry.multi count" ); + +out: +bpf_link__destroy (link); +tracing_multi_bpf__destroy (skel_tracing); +tailcall_bpf2bpf4__destroy (skel_tc); +} +void test_tailcalls (void ) {if (test__start_subtest ("tailcall_1" ))
@@ -2138,4 +2200,6 @@ void test_tailcalls(void)test_tailcall_bpf2bpf_fexit_links ();if (test__start_subtest ("tailcall_large_stack" ))test_tailcall_large_stack (); +if (test__start_subtest ("tailcall_bpf2bpf_fentry_multi" )) +test_tailcall_bpf2bpf_fentry_multi (); }
Signature · 1 line
2.55.0
PATCH v2 21/21 selftests/bpf: Benchmark tracing_multi bpf progs attachment
In reply to Leon HwangAdd an x86-64 one-shot generic benchmark comparing 1000 individual fentry link creations with one fentry.multi link creation for the same 1000 bpf prog targets.
fentry.multi is much faster than fentry when attaching to 1000 targets on a 16c16g QEMU VM on an AMD EPYC 9745 server.
./bench tracing-multi-attach-progs
Setting up benchmark 'tracing-multi-attach-progs'...
tracing-multi-attach-progs: prepared 1000 identical BPF program targets tracing-multi-attach-progs: fentry created and attached 1000 programs/links in 338.171ms tracing-multi-attach-progs: fentry.multi created one program and attached one 1000-target link in 47.345ms tracing-multi-attach-progs: fentry.multi creation/attachment speedup is 7.14x
--- tools/testing/selftests/bpf/Makefile | 1 + tools/testing/selftests/bpf/bench.c | 5 + .../benchs/bench_tracing_multi_attach_progs.c | 193 ++++++++++++++++++ .../bpf/progs/tracing_multi_attach_progs.c | 24 +++ 4 files changed, 223 insertions(+) create mode 100644 tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_attach_progs.c
tools/testing/selftests/bpf/Makefile +1 -0
@@ -625,6 +625,7 @@ $(OUTPUT)/bench_bloom_filter_map.o: $(OUTPUT)/bloom_filter_bench.skel.h $(OUTPUT)/bench_bpf_loop.o: $(OUTPUT)/bpf_loop_bench.skel.h $(OUTPUT)/bench_bpf_for.o: $(OUTPUT)/bpf_for_bench.skel.h $(OUTPUT)/bench_strncmp.o: $(OUTPUT)/strncmp_bench.skel.h +$(OUTPUT)/bench_tracing_multi_attach_progs.o: $(OUTPUT)/tracing_multi_attach_progs.skel.h $(OUTPUT)/bench_bpf_hashmap_full_update.o: $(OUTPUT)/bpf_hashmap_full_update_bench.skel.h $(OUTPUT)/bench_local_storage.o: $(OUTPUT)/local_storage_bench.skel.h $(OUTPUT)/bench_local_storage_rcu_tasks_trace.o: $(OUTPUT)/local_storage_rcu_tasks_trace_bench.skel.h
tools/testing/selftests/bpf/bench.c +5 -0
@@ -529,6 +529,9 @@ extern const struct bench bench_trig_fmodret;extern const struct bench bench_trig_tp;extern const struct bench bench_trig_rawtp; +/* tracing_multi link attachment benchmark */ +extern const struct bench bench_tracing_multi_attach_progs; +/* uprobe/uretprobe benchmarks */ extern const struct bench bench_trig_uprobe_nop;extern const struct bench bench_trig_uretprobe_nop;
@@ -615,6 +618,8 @@ static const struct bench *benchs[] = { &bench_trig_fmodret, &bench_trig_tp, &bench_trig_rawtp, +/* tracing_multi */ + &bench_tracing_multi_attach_progs,/* uprobes */ &bench_trig_uprobe_nop, &bench_trig_uretprobe_nop,
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +193 -0
193 lines not on this page · raw message
tools/testing/selftests/bpf/progs/tracing_multi_attach_progs.c new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +char _license[] SEC("license") = "GPL"; + +SEC("syscall") +int target (void *ctx) +{ +return 0 ; +} + +SEC("fentry") +int BPF_PROG (dummy_fentry ) +{ +return 0 ; +} + +SEC("fentry.multi") +int BPF_PROG (dummy_fentry_multi ) +{ +return 0 ; +}
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v2 21/21] selftests/bpf: Benchmark tracing_multi bpf progs attachment
TL;DR: - Benchmark may hit EMFILE with default fd limits (about 5000 fds needed) and exit - Reported 7.14x speedup compares unequal work: fentry timing includes load and BTF resolution
Quoted 1 line
selftests/bpf: Benchmark
tracing_multibpf progs attachment
TL;DR:
- Benchmark may hit EMFILE with default fd limits (about 5000 fds needed) and exit
- Reported 7.14x speedup compares unequal work: fentry timing includes load and BTF resolution
Quoted 6 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +0
-0@@ -0,0 +1,193 @@
[ ... ]
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c line 44
Quoted 13 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +10
-0+static void prepare (void ) +{ +bpf_program__set_autoload (target->progs.dummy_fentry, false); +bpf_program__set_autoload (target->progs.dummy_fentry_multi, false); + err =tracing_multi_attach_progs__load (target); +if (err) { +fprintf (stderr,"failed to load target program %d: %s\n" , + i,strerror (-err)); +exit (1 ); + }
Does this benchmark need more than the default RLIMIT_NOFILE?
Every loaded skeleton keeps two fds open until __destroy(), the object BTF fd and the program fd. The BTF fd is not released after load:
bpf_object_load()
bpf_object_post_load_cleanup()
bpf_object_cleanup_btf()
only frees the module/vmlinux BTF, while obj->btf and its fd stay open until bpf_object__close(). So prepare() alone holds about 2 * TARGET_CNT = 2000 fds for the 1000 target objects.
create_and_attach_fentry() then adds roughly 2000 more for the 1000 fentry skeletons, plus 1000 entries in link_fds[], for a peak of about 5000 open fds.
Nothing in bench.c, testing_helpers or libbpf raises RLIMIT_NOFILE (the only RLIMIT_NOFILE user in selftests/bpf is map_tests/sk_storage_map.c). With a 1024 soft limit, './bench tracing-multi-attach-progs' looks like it would hit EMFILE at around target 510 in prepare() and exit(1). Even a 4096 hard limit is below the peak reached in create_and_attach_fentry().
Should setup() or prepare() raise RLIMIT_NOFILE, as sk_storage_map.c does, or should the 'ulimit -n' requirement be documented in the commit message?
[ ... ]
Quoted 15 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +12
-0+static long create_and_attach_fentry (void ) +{ +int link_fds[TARGET_CNT ]; +long start_ns, end_ns; +int err, i, j; + + start_ns =get_time_ns (); + +/* An ordinary fentry program is bound to one target when it is loaded. */ + err =bpf_program__set_attach_target (fentry->progs.dummy_fentry, ctx.target_fds[i], +"target" ); + err = err ?:tracing_multi_attach_progs__load (fentry);
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c line 106
Quoted 10 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +4
-0+for (i =0 ; i <TARGET_CNT ; i++) { + link_fds[i] =bpf_link_create (ctx.fentry_prog_fds[i],0 ,BPF_TRACE_FENTRY ,NULL ); +if (link_fds[i] <0 ) { + end_ns =get_time_ns ();
The commit message describes this as a comparison of "1000 individual fentry link creations with one fentry.multi link creation", but do the two timed regions measure the same work?
The fentry window starts at the get_time_ns() above, before the first loop, so it covers:
1000 x tracing_multi_attach_progs__open()
1000 x bpf_program__set_attach_target()
1000 x tracing_multi_attach_progs__load()
1000 x bpf_link_create()
Each bpf_program__set_attach_target() call goes through libbpf_find_prog_btf_id(), which does bpf_prog_get_info_by_fd() and loads and parses the target BTF from the kernel. The load calls add 1000 object BTF loads and 1000 verifier and JIT runs.
The fentry.multi window in create_and_attach_fentry_multi() covers one open+load and one bpf_link_create(). Its 1000 target BTF IDs were already resolved in prepare(), outside the timed region (ctx.target_btf_ids[i] = btf_id).
So the reported "fentry.multi creation/attachment speedup is 7.14x" is not a link-creation comparison. Part of it comes from BTF ID resolution being timed for fentry but not for fentry.multi, and most of it is probably the cost of loading 1000 programs versus one.
Would it make sense to time only the bpf_link_create() loop on its own and resolve BTF IDs on the same side of the timer for both paths? Otherwise, could the commit message say that the numbers cover program creation and BTF ID resolution as well as attachment?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38273555988
[email protected]
AI reviewRe: [PATCH bpf-next v2 21/21] selftests/bpf: Benchmark tracing_multi bpf progs attachment
In reply to Leon HwangThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] bpf: bench fails with EMFILE due to unraised RLIMIT_NOFILE - [Low] bpf: benchmark time overflow…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] bpf: bench fails with EMFILE due to unraised RLIMIT_NOFILE
- [Low] bpf: benchmark time overflow on 32-bit due to long truncation
--
commit d6a7c4ba0d33ae305098f82b092a318e73ea06fb
Author: Leon Hwang <[email protected]>
selftests/bpf: Benchmark tracing_multi bpf progs attachment
This commit adds an x86-64 one-shot generic benchmark to compare the performance of attaching 1000 individual fentry links versus a single fentry.multi link for the same 1000 BPF program targets.
Quoted 5 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +0
-0
[ ... ]
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c line 65
Quoted 4 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +4
-0+static long create_and_attach_fentry (void ) +{ +int link_fds[TARGET_CNT ]; +long start_ns, end_ns;
[Severity: Low]
Will this truncate the 64-bit nanosecond timestamps returned by get_time_ns() on 32-bit architectures?
get_time_ns() returns a __u64, and storing it in a 32-bit long drops the upper 32 bits. Also, if the elapsed time (end_ns - start_ns) exceeds roughly 2.14 seconds, the signed 32-bit long will overflow into a negative value, resulting in negative benchmark times and malformed speedup calculations.
Loading 1000 targets and links could easily exceed 2.14s on slow or emulated 32-bit hosts.
Quoted 5 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +5
-0+int err, i, j; + + start_ns =get_time_ns (); + +/* An ordinary fentry program is bound to one target when it is loaded. */
[ ... ]
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c line 97
Quoted 3 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +3
-0+for (i =0 ; i <TARGET_CNT ; i++) { + link_fds[i] =bpf_link_create (ctx.fentry_prog_fds[i],0 ,BPF_TRACE_FENTRY ,NULL ); +if (link_fds[i] <0 ) {
[Severity: Medium]
Will this benchmark predictably fail with EMFILE due to exceeding the default 1024 file descriptor limit?
The benchmark attempts to load 1000 target programs, 1000 fentry programs, and 1000 links (over 3000 FDs total) sequentially without raising RLIMIT_NOFILE in setup_libbpf() or bench.c. Running this locally without manually increasing the open file limit via ulimit -n will cause bpf_link_create() or program loading to fail.
Quoted 15 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +15
-0+ err = errno; +for (j =0 ; j < i; j++) +close (link_fds[j]); +fprintf (stderr,"failed to create fentry link %d: %s\n" , + i,strerror (err)); +exit (1 ); + } + } + end_ns =get_time_ns (); + +for (i =0 ; i <TARGET_CNT ; i++) +close (link_fds[i]); + +return end_ns - start_ns; +}
[ ... ]
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c line 169
Quoted 4 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +4
-0+static void setup (void ) +{ +double fentry_multi_ms, fentry_ms; +long fentry_multi_ns, fentry_ns;
[Severity: Low]
Similar to the issue in create_and_attach_fentry() above, will these signed long values overflow and yield negative durations if the elapsed benchmark time exceeds 2.14 seconds on 32-bit architectures?
Quoted 4 lines
tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c new file +4
-0+ +prepare (); + fentry_ns =create_and_attach_fentry (); + fentry_multi_ns =create_and_attach_fentry_multi ();
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=21
Quoted 19 lines · Leon Hwang
Similar to the
tracing_multilink support for kernel functions [1], add support for bpf progs.With this support, I'll be able to enhance the function-graph feature of bpfsnoop [2]. It will trace all bpf progs, including their subprogs, to draw a full function call graph including bpf prog call sites.
The attachment could speed-up when there are hundreds of targets. For example, on high CPU core count servers, it probably runs 200 tc-bpf progs as Kubernetes CNI, and each tc-bpf prog might have 4 subprogs. Hence, there will be 1000 targets to trace for bpfsnoop's function-graph feature.
With
tracing_multilink support for bpf progs, bpfsnoop will be able to attach to all targets quickly.When attaching to 1000 targets, fentry.multi is much faster than fentry on a 16c16g QEMU VM on an AMD EPYC 9745 server.
Quoted 6 lines
./bench tracing-multi-attach-progs
Setting up benchmark 'tracing-multi-attach-progs'...
tracing-multi-attach-progs: prepared 1000 identical BPF program targets tracing-multi-attach-progs: fentry created and attached 1000 programs/links in 338.171ms tracing-multi-attach-progs: fentry.multi created one program and attached one 1000-target link in 47.345ms tracing-multi-attach-progs: fentry.multi creation/attachment speedup is 7.14x
it's faster, but the original attachment is under 1s, so not really a problem imo
I guess the api is better for your use case, like having less file descriptors open ?
jirka
Quoted 27 lines · Jiri Olsa
Quoted 19 lines · Leon Hwang
Similar to the
tracing_multilink support for kernel functions [1], add support for bpf progs.With this support, I'll be able to enhance the function-graph feature of bpfsnoop [2]. It will trace all bpf progs, including their subprogs, to draw a full function call graph including bpf prog call sites.
The attachment could speed-up when there are hundreds of targets. For example, on high CPU core count servers, it probably runs 200 tc-bpf progs as Kubernetes CNI, and each tc-bpf prog might have 4 subprogs. Hence, there will be 1000 targets to trace for bpfsnoop's function-graph feature.
With
tracing_multilink support for bpf progs, bpfsnoop will be able to attach to all targets quickly.When attaching to 1000 targets, fentry.multi is much faster than fentry on a 16c16g QEMU VM on an AMD EPYC 9745 server.
Quoted 6 lines
./bench tracing-multi-attach-progs
Setting up benchmark 'tracing-multi-attach-progs'...
tracing-multi-attach-progs: prepared 1000 identical BPF program targets tracing-multi-attach-progs: fentry created and attached 1000 programs/links in 338.171ms tracing-multi-attach-progs: fentry.multi created one program and attached one 1000-target link in 47.345ms tracing-multi-attach-progs: fentry.multi creation/attachment speedup is 7.14x
Quoted 5 lines
it's faster, but the original attachment is under 1s, so not really a problem imo
I guess the api is better for your use case, like having less file descriptors open ?
Correct.
The original attachment is fast enough. So there is no need to improve attachment performance with a large patch series.
Even though the series brings two benefits:
* One bpf link for multiple bpf prog targets.
* Support for all archs that have trampoline support.
But, I also doubt the value of this series.
Thanks,
Leon
Quoted 39 lines · Leon Hwang
Quoted 27 lines · Jiri Olsa
Quoted 19 lines · Leon Hwang
Similar to the
tracing_multilink support for kernel functions [1], add support for bpf progs.With this support, I'll be able to enhance the function-graph feature of bpfsnoop [2]. It will trace all bpf progs, including their subprogs, to draw a full function call graph including bpf prog call sites.
The attachment could speed-up when there are hundreds of targets. For example, on high CPU core count servers, it probably runs 200 tc-bpf progs as Kubernetes CNI, and each tc-bpf prog might have 4 subprogs. Hence, there will be 1000 targets to trace for bpfsnoop's function-graph feature.
With
tracing_multilink support for bpf progs, bpfsnoop will be able to attach to all targets quickly.When attaching to 1000 targets, fentry.multi is much faster than fentry on a 16c16g QEMU VM on an AMD EPYC 9745 server.
Quoted 6 lines
./bench tracing-multi-attach-progs
Setting up benchmark 'tracing-multi-attach-progs'...
tracing-multi-attach-progs: prepared 1000 identical BPF program targets tracing-multi-attach-progs: fentry created and attached 1000 programs/links in 338.171ms tracing-multi-attach-progs: fentry.multi created one program and attached one 1000-target link in 47.345ms tracing-multi-attach-progs: fentry.multi creation/attachment speedup is 7.14xQuoted 5 lines
it's faster, but the original attachment is under 1s, so not really a problem imo
I guess the api is better for your use case, like having less file descriptors open ?
Correct.
The original attachment is fast enough. So there is no need to improve attachment performance with a large patch series.
Quoted 6 lines
Even though the series brings two benefits:
* One bpf link for multiple bpf prog targets.
* Support for all archs that have trampoline support.But, I also doubt the value of this series.
I'm not sure about the value as well.
You also need to benchmark detach time.
Overall the perf improvement is too minor vs added complexity, but the first couple patches in the set look like a nice cleanup. Pls resend them.
pw-bot: cr