Skip to content

bpf: Add tracing_multi link support for bpf progs

Leon Hwang bpf linux-kselftest linux-trace-kernel lkml oe-kbuild-all 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest21 patches 9 repliesIn review
  2. v113 patches 18 repliesSuperseded

Changes in v2 from the cover letter · 16 lines

* Drop the factoring update_fentry_multi() helper.
* Add bpf_arch_text_poke_batch() to x86_64 JIT backend.
* Add bpf_text_poke_batch() helper.
* Move trampoline_lock_all(), trampoline_unlock_all(), and bpf_trampoline_multi_attach_init() out of the CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard.
* Factor out bpf_trampoline_multi_prog_attach/detach() to get rid of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS.
* Shrink CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard scope for bpf_tracing_multi_attach().
* Explain justification in cover letter. (Andrii)
* Check '!funcs[i]' before libbpf_find_prog_btf_id() in libbpf. (Sashiko)
* Add BPF_F_TRACING_MULTI_TGT_PROGS in link info UAPI. (Sashiko)
* Verify '\0' before skipping '_' when parsing prog name from symbol name in bpftool. (Sashiko)
* Expose libbpf's libbpf_find_prog_btf_id() in attach failure tests. (Sashiko)
* Select the target prog according to the 'first' value. (Sashiko)
* Add qsort() in link info test's setup_tmulti_bpf_prog_targets(). (Sashiko)
* Drop '((u64)tgt_prog->aux->id << 32) | ids[i]' in link fdinfo. (bot+bpf-ci)
* Use different cookie values for different targets in cookie tests. (bot+bpf-ci)
* v1: https://lore.kernel.org/bpf/[email protected]/

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/21bpf: Initialize ftrace_managed in bpf_trampoline_get()kernel/bpf/trampoline.c+2 -91 reply
2/21bpf: Drop unnecessary ftrace_location() in register/unregister/modify_fentry_multi()kernel/bpf/trampoline.c+3 -51 reply
3/21bpf, x86: Factor out bpf_arch_text_poke_prepare_insns() helper on x86_64arch/x86/net/bpf_jit_comp.c+25 -130 replies
4/21bpf, x86: Factor out bpf_arch_text_poke_check_ip() helper on x86_64arch/x86/net/bpf_jit_comp.c+17 -40 replies
5/21bpf, x86: Introduce bpf_arch_text_poke_batch() on x86_64arch/x86/net/bpf_jit_comp.c include/linux/bpf.h kernel/bpf/core.c+55 -00 replies
6/21bpf: Add bpf_text_poke_batch() helper in trampolinekernel/bpf/trampoline.c+51 -00 replies
7/21bpf: Check tgt_prog before is_tracing_multi_id() in verifierkernel/bpf/verifier.c+12 -100 replies
8/21bpf: Move helpers out of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard in trampolinekernel/bpf/trampoline.c+26 -220 replies
9/21bpf: Add bpf_trampoline_multi_prog_attach/detach()include/linux/bpf.h kernel/bpf/trampoline.c+246 -151 reply
10/21bpf: Shrink CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard scope for bpf_tracing_multi_attach()kernel/bpf/trampoline.c kernel/trace/bpf_trace.c+8 -120 replies
11/21bpf: Add tracing_multi link support for bpf progsinclude/uapi/linux/bpf.h kernel/trace/bpf_trace.c tools/include/uapi/linux/bpf.h+156 -251 reply
12/21libbpf: Add tracing_multi link support for bpf progstools/lib/bpf/bpf.c tools/lib/bpf/bpf.h tools/lib/bpf/libbpf.c +1 more+43 -50 replies
13/21bpf: Add tracing_multi link fdinfo support for bpf progskernel/trace/bpf_trace.c+24 -50 replies
14/21bpf: Add tracing_multi link info support for bpf progsinclude/uapi/linux/bpf.h kernel/trace/bpf_trace.c tools/bpf/bpftool/link.c +1 more+82 -211 reply
15/21selftests/bpf: Add tracing_multi bpf prog attach testtools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_bpf.c+126 -00 replies
16/21selftests/bpf: Add tracing_multi bpf prog attach failure teststools/lib/bpf/libbpf.c tools/lib/bpf/libbpf_internal.h tools/testing/selftests/bpf/prog_tests/tracing_multi.c+102 -10 replies
17/21selftests/bpf: Add tracing_multi bpf prog cookie testtools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_bpf.c+30 -40 replies
18/21selftests/bpf: Add tracing_multi bpf prog rollback testtools/testing/selftests/bpf/prog_tests/tracing_multi.c+63 -01 reply
19/21selftests/bpf: Add tracing_multi bpf prog link info testtools/testing/selftests/bpf/prog_tests/fill_link_info.c+151 -180 replies
20/21selftests/bpf: Test tailcall with fentry.multitools/testing/selftests/bpf/prog_tests/tailcalls.c+64 -00 replies
21/21selftests/bpf: Benchmark tracing_multi bpf progs attachmenttools/testing/selftests/bpf/Makefile tools/testing/selftests/bpf/bench.c tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c +1 more+223 -00 replies

Reviews trailers per patch

2 of 21 patches have a Reviewed-by or Acked-by.

PatchJiri Olsa
1/21A-b
2/21A-b
3/21
4/21
5/21
6/21
7/21
8/21
9/21
10/21
11/21
12/21
13/21
14/21
15/21
16/21
17/21
18/21
19/21
20/21
21/21

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH bpf-next v2 00/21] bpf: Add tracing_multi link support for bpf progs

Similar to the tracing_multi link support for kernel functions [1], add support for bpf progs.

With this support, I'll be able to enhance the function-graph feature of bpfsnoop [2]. It will trace all bpf progs, including their subprogs, to draw a full function call graph including bpf prog call sites.

The attachment could speed-up when there are hundreds of targets. For example, on high CPU core count servers, it probably runs 200 tc-bpf progs as Kubernetes CNI, and each tc-bpf prog might have 4 subprogs. Hence, there will be 1000 targets to trace for bpfsnoop's function-graph feature.

With tracing_multi link support for bpf progs, bpfsnoop will be able to attach to all targets quickly.

When attaching to 1000 targets, fentry.multi is much faster than fentry on a 16c16g QEMU VM on an AMD EPYC 9745 server.

./bench tracing-multi-attach-progs
Setting up benchmark 'tracing-multi-attach-progs'...
tracing-multi-attach-progs: prepared 1000 identical BPF program targets tracing-multi-attach-progs: fentry created and attached 1000 programs/links in 338.171ms tracing-multi-attach-progs: fentry.multi created one program and attached one 1000-target link in 47.345ms tracing-multi-attach-progs: fentry.multi creation/attachment speedup is 7.14x

Unlike the tracing_multi link support for kernel functions, the support for bpf progs does not rely on the ftrace CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS. Hence, the support for bpf progs will apply to all archs that have trampoline support by design.

Links:
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://github.com/bpfsnoop/bpfsnoop

Changes:
v1 -> v2:
* Drop the factoring update_fentry_multi() helper.
* Add bpf_arch_text_poke_batch() to x86_64 JIT backend.
* Add bpf_text_poke_batch() helper.
* Move trampoline_lock_all(), trampoline_unlock_all(), and bpf_trampoline_multi_attach_init() out of the CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard.
* Factor out bpf_trampoline_multi_prog_attach/detach() to get rid of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS.
* Shrink CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard scope for bpf_tracing_multi_attach().
* Explain justification in cover letter. (Andrii)
* Check '!funcs[i]' before libbpf_find_prog_btf_id() in libbpf. (Sashiko)
* Add BPF_F_TRACING_MULTI_TGT_PROGS in link info UAPI. (Sashiko)
* Verify '\0' before skipping '_' when parsing prog name from symbol name in bpftool. (Sashiko)
* Expose libbpf's libbpf_find_prog_btf_id() in attach failure tests. (Sashiko)
* Select the target prog according to the 'first' value. (Sashiko)
* Add qsort() in link info test's setup_tmulti_bpf_prog_targets(). (Sashiko)
* Drop '((u64)tgt_prog->aux->id << 32) | ids[i]' in link fdinfo. (bot+bpf-ci)
* Use different cookie values for different targets in cookie tests. (bot+bpf-ci)
* v1: https://lore.kernel.org/bpf/[email protected]/

Leon Hwang (21):
bpf: Initialize ftrace_managed in bpf_trampoline_get()
bpf: Drop unnecessary ftrace_location() in
register/unregister/modify_fentry_multi()
bpf, x86: Factor out bpf_arch_text_poke_prepare_insns() helper on

    x86_64

bpf, x86: Factor out bpf_arch_text_poke_check_ip() helper on x86_64 bpf, x86: Introduce bpf_arch_text_poke_batch() on x86_64
bpf: Add bpf_text_poke_batch() helper in trampoline
bpf: Check tgt_prog before is_tracing_multi_id() in verifier
bpf: Move helpers out of CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard in
trampoline
bpf: Add bpf_trampoline_multi_prog_attach/detach()
bpf: Shrink CONFIG_HAVE_SINGLE_FTRACE_DIRECT_OPS guard scope for
bpf_tracing_multi_attach()
bpf: Add tracing_multi link support for bpf progs
libbpf: Add tracing_multi link support for bpf progs
bpf: Add tracing_multi link fdinfo support for bpf progs
bpf: Add tracing_multi link info support for bpf progs selftests/bpf: Add tracing_multi bpf prog attach test selftests/bpf: Add tracing_multi bpf prog attach failure tests selftests/bpf: Add tracing_multi bpf prog cookie test selftests/bpf: Add tracing_multi bpf prog rollback test selftests/bpf: Add tracing_multi bpf prog link info test selftests/bpf: Test tailcall with fentry.multi selftests/bpf: Benchmark tracing_multi bpf progs attachment

 arch/x86/net/bpf_jit_comp.c                   | 100 +++++-
 include/linux/bpf.h                           |  38 +-
 include/uapi/linux/bpf.h                      |   9 +-
 kernel/bpf/core.c                             |   5 +
 kernel/bpf/trampoline.c                       | 339 ++++++++++++++++--
 kernel/bpf/verifier.c                         |  22 +-
 kernel/trace/bpf_trace.c                      | 243 ++++++++++---
 tools/bpf/bpftool/link.c                      |  63 +++-
 tools/include/uapi/linux/bpf.h                |   9 +-
 tools/lib/bpf/bpf.c                           |   1 +
 tools/lib/bpf/bpf.h                           |   2 +
 tools/lib/bpf/libbpf.c                        |  43 ++-
 tools/lib/bpf/libbpf.h                        |   4 +-
 tools/lib/bpf/libbpf_internal.h               |   1 +
 tools/testing/selftests/bpf/Makefile          |   1 +
 tools/testing/selftests/bpf/bench.c           |   5 +
 .../benchs/bench_tracing_multi_attach_progs.c | 193 ++++++++++
 .../selftests/bpf/prog_tests/fill_link_info.c | 169 ++++++++-
 .../selftests/bpf/prog_tests/tailcalls.c      |  64 ++++
 .../selftests/bpf/prog_tests/tracing_multi.c  | 254 +++++++++++++
 .../bpf/progs/tracing_multi_attach_progs.c    |  24 ++
 .../selftests/bpf/progs/tracing_multi_bpf.c   |  61 ++++
 22 files changed, 1495 insertions(+), 155 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_tracing_multi_attach_progs.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_attach_progs.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_bpf.c
Signature · 1 line
2.55.0