Skip to content

bpf: tracing_multi link

Jiri Olsa bpf linux-trace-kernel 8 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v8 Latest29 patches 2 repliesIn review 28 of 29 applied
  2. v729 patches 15 repliesSuperseded
  3. v629 patches 7 repliesSuperseded
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v528 patches19 repliesSuperseded
v425 patches12 repliesSuperseded
v324 patches16 repliesSuperseded
v223 patches11 repliesSuperseded
v117 patches13 repliesSuperseded

Changes in v8 from the cover letter

- add back the btf_is_union check to btf_get_type_size [sashiko]

Patches in v8 28 of 29 applied

#Subject and filesSizeRepliesApplied
1/29ftrace: Add ftrace_hash_count functioninclude/linux/ftrace.h kernel/trace/ftrace.c+13 -10 repliesApplied
2/29ftrace: Add ftrace_hash_remove functioninclude/linux/ftrace.h kernel/trace/ftrace.c+20 -00 repliesApplied
3/29ftrace: Add add_ftrace_hash_entry functioninclude/linux/ftrace.h kernel/trace/ftrace.c+5 -50 repliesApplied
4/29bpf: Use mutex lock pool for bpf trampolinesinclude/linux/bpf.h kernel/bpf/trampoline.c+53 -260 repliesApplied
5/29bpf: Add struct bpf_trampoline_ops objectkernel/bpf/trampoline.c+41 -180 repliesApplied
6/29bpf: Move trampoline image setup into bpf_trampoline_ops callbackskernel/bpf/trampoline.c+38 -280 repliesApplied
7/29bpf: Add bpf_trampoline_add/remove_prog functionskernel/bpf/trampoline.c+61 -470 repliesApplied
8/29bpf: Add struct bpf_tramp_node objectarch/arm64/net/bpf_jit_comp.c arch/loongarch/net/bpf_jit.c arch/powerpc/net/bpf_jit_comp.c +8 more+294 -2750 repliesApplied
9/29bpf: Factor fsession link to use struct bpf_tramp_nodeinclude/linux/bpf.h kernel/bpf/syscall.c kernel/bpf/trampoline.c+13 -260 repliesApplied
10/29bpf: Add multi tracing attach typesinclude/linux/bpf.h include/linux/btf_ids.h include/uapi/linux/bpf.h +7 more+82 -60 repliesApplied
11/29bpf: Move sleepable verification code to btf_id_allow_sleepablekernel/bpf/verifier.c+50 -320 repliesApplied
12/29bpf: Add bpf_trampoline_multi_attach/detach functionsinclude/linux/bpf.h include/linux/bpf_verifier.h kernel/bpf/trampoline.c +1 more+373 -00 repliesApplied
13/29bpf: Add support for tracing multi linkinclude/linux/bpf_types.h include/linux/trace_events.h include/uapi/linux/bpf.h +4 more+151 -00 repliesApplied
14/29bpf: Add support for tracing_multi link cookiesinclude/linux/bpf.h include/uapi/linux/bpf.h kernel/bpf/trampoline.c +2 more+37 -40 repliesApplied
15/29bpf: Add support for tracing_multi link sessioninclude/linux/bpf.h include/uapi/linux/bpf.h kernel/bpf/fixups.c +7 more+76 -150 repliesApplied
16/29bpf: Add support for tracing_multi link fdinfokernel/trace/bpf_trace.c+30 -00 repliesApplied
17/29libbpf: Add bpf_object_cleanup_btf functiontools/lib/bpf/libbpf.c+12 -40 repliesApplied
18/29libbpf: Add bpf_link_create support for tracing_multi linktools/lib/bpf/bpf.c tools/lib/bpf/bpf.h+14 -00 repliesApplied
19/29libbpf: Add btf_type_is_traceable_func functiontools/lib/bpf/libbpf.c tools/lib/bpf/libbpf_internal.h+80 -00 repliesApplied
20/29libbpf: Add support to create tracing multi linktools/lib/bpf/libbpf.c tools/lib/bpf/libbpf.h tools/lib/bpf/libbpf.map+292 -00 repliesApplied
21/29selftests/bpf: Add tracing multi skel/pattern/ids attach teststools/testing/selftests/bpf/Makefile tools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_attach.c +1 more+450 -10 repliesApplied
22/29selftests/bpf: Add tracing multi skel/pattern/ids module attach teststools/testing/selftests/bpf/Makefile tools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_attach_module.c +1 more+183 -10 repliesApplied
23/29selftests/bpf: Add tracing multi intersect teststools/testing/selftests/bpf/Makefile tools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_intersect_attach.c+143 -10 repliesApplied
24/29selftests/bpf: Add tracing multi cookies testtools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_check.c+35 -30 repliesApplied
25/29selftests/bpf: Add tracing multi session testtools/testing/selftests/bpf/Makefile tools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_session_attach.c+113 -10 repliesApplied
26/29selftests/bpf: Add tracing multi attach fails testtools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_fail.c+114 -00 repliesApplied
27/29selftests/bpf: Add tracing multi verifier fails testtools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_verifier.c+33 -00 repliesApplied
28/29selftests/bpf: Add tracing multi attach benchmark testtools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_bench.c tools/testing/selftests/bpf/trace_helpers.c +1 more+117 -32 replies
29/29selftests/bpf: Add tracing multi attach rollback teststools/testing/selftests/bpf/prog_tests/tracing_multi.c tools/testing/selftests/bpf/progs/tracing_multi_rollback.c+255 -00 repliesApplied

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v8 yet.

Read the v8 thread

Cover letter [PATCHv8 bpf-next 00/29] bpf: tracing_multi link

hi,
adding tracing_multi link support that allows fast attachment of tracing program to many functions.

RFC: https://lore.kernel.org/bpf/[email protected]/
v1: https://lore.kernel.org/bpf/[email protected]/
v2: https://lore.kernel.org/bpf/[email protected]/
v3: https://lore.kernel.org/bpf/[email protected]/
v4: https://lore.kernel.org/bpf/[email protected]/
v5: https://lore.kernel.org/bpf/[email protected]/
v6: https://lore.kernel.org/bpf/[email protected]/
v7: https://lore.kernel.org/bpf/[email protected]/

v8 changes:
- add back the btf_is_union check to btf_get_type_size [sashiko]

v7 changes:
- added ftrace_hash_count stub for !CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS cade [sashiko]
- selftests fixes [sashiko]
- use hash_ptr in select_trampoline_lock [sashiko]
- changed the check duplicate logic in check_dup_ids [sashiko]
- use sort_r_nonatomic in check_dup_ids [sashiko]
- added BPF_TRACE_FSESSION_MULTI to can_be_sleepable,
plus added testcase for sleepable fsession
- make bpf_tracing_multi_opts pointer fields as const
- add ___migrate_enable to trace_blacklist

v6 changes:
- move ftrace_hash_count declaration under CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS [sashiko]
- fix ftrace_hash_remove check/deref [sashiko]
- disable context access for multi programs by using stub function with no arguments
for verification [sashiko]
- add __used for bpf_multi_func, and removed arguments, we do not allow direct access [sashiko]
- rebased on latest loongarch changes, fix ppc build
- guard update_ftrace_direct_del with ftrace_hash_count on rollback [sashiko]
- fix noreturn attachment condition in bpf_check_attach_btf_id_multi [sashiko]
- fail early on multiple same IDs provided by user [sashiko]
- fix selftests error paths [sashiko]
- add MAX_RESOLVE_DEPTH check to btf_get_type_size [sashiko]
- use btf__pointer_size [sashiko]
- fixed compilation on powerpc [sashiko]
- added verifier fails selftest
- after discussing with Song, it was determined that cleaning up FTRACE_OPS_CMD_DISABLE_SHARE_IPMODIFY_PEER is not strictly necessary — keeping the trampoline in the ipmodify_enabled state is acceptable. The race condition this introduces remains unlikely, so the concern raised in [1] will not be addressed at this time.
[1] https://lore.kernel.org/bpf/aec7bAbGlnEo3R1g@krava/

v5 changes:
- add dedicated hashes used for detach, so there's no need to allocate them on detach [sashiko]
- safely release old trampoline images [sashiko]
- add cond_resched() to couple of loops [sashiko]
- validate attr->link_create.target_fd [sashiko]
- allow only bpf_get_func_ret() for return value retrieval [sashiko]
- do not allow attachment of fexit/fsession_multi for noreturn functions [sashiko]
- fixed double free/close in libbpf btf cleanup, in separate patch [sashiko]
- make btf_type_is_traceable_func closer to btf_distill_func_proto [sashiko]
- add prog->attach_btf_obj_fd check to collect_func_ids_by_glob,
to check we don't load module programs for kernel [sashiko]
- make sure program is loaded in bpf_program__attach_tracing_multi [sashiko]
- several selftests fixes [sashiko]
- add attach_type to fdinfo output [Leon Hwang]
- selftests cleanup fixes [Leon Hwang]

v4 changes:
- unlink rollback fix (added ftrace_hash_count) [bot]
- use const for some bpf_link_create_opts tracing_multi members [bot]
- adding missing comment for lockdep keys [bot]
- selftest error path fixes (leaks) and other assorted test fixes [Leon Hwang]
- several compile fixes wrt CONFIG_BPF_SYSCALL and CONFIG_BPF_JIT [kernel test robot]
- make ftrace_hash_clear global, because it's needed in rollback

v3 changes:
- fix module parsing [Leon Hwang]
- use function traceable check from libbpf [Leon Hwang]
- use ptr_to_u64 and fix/updated few comments [ci]
- display cookies as decimal numbers [ci]
- added link_create.flags check [ci]
- fix error path in bpf_trampoline_multi_detach [ci]
- make fentry/fexit.multi not extendable [ci]
- add missing OPTS_VALID to bpf_program__attach_tracing_multi [ci]

v2 changes:
- allocate data.unreg in bpf_trampoline_multi_attach for rollback path [ci] and fixed link count setup in rollback path [ci]
- several small assorted fixes [ci]
- added loongarch and powerpc changes for struct bpf_tramp_node change
- added support to attach functions from modules
- added tests for sleepable programs
- added rollback tests

v1 changes:
- added ftrace_hash_count as wrapper for hash_count [Steven]
- added trampoline mutex pool [Andrii]
- reworked 'struct bpf_tramp_node' separatoin [Andrii]
- the 'struct bpf_tramp_node' now holds pointer to bpf_link, which is similar to what we do for uprobe_multi;
I understand it's not a fundamental change compared to previous version which used bpf_prog pointer instead, but I don't see better way of doing this.. I'm happy to discuss this further if there's better idea
- reworked 'struct bpf_fsession_link' based on bpf_tramp_node
- made btf__find_by_glob_kind function internal helper [Andrii]
- many small assorted fixes [Andrii,CI]
- added session support [Leon Hwang]
- added cookies support
- added more tests

Note I plan to send linkinfo support separately, the patchset is big enough.

thanks,
jirka

Cc Hengqi Chen <[email protected]>

---
Jiri Olsa (29):
ftrace: Add ftrace_hash_count function
ftrace: Add ftrace_hash_remove function
ftrace: Add add_ftrace_hash_entry function
bpf: Use mutex lock pool for bpf trampolines
bpf: Add struct bpf_trampoline_ops object
bpf: Move trampoline image setup into bpf_trampoline_ops callbacks
bpf: Add bpf_trampoline_add/remove_prog functions
bpf: Add struct bpf_tramp_node object
bpf: Factor fsession link to use struct bpf_tramp_node
bpf: Add multi tracing attach types
bpf: Move sleepable verification code to btf_id_allow_sleepable
bpf: Add bpf_trampoline_multi_attach/detach functions
bpf: Add support for tracing multi link
bpf: Add support for tracing_multi link cookies
bpf: Add support for tracing_multi link session
bpf: Add support for tracing_multi link fdinfo
libbpf: Add bpf_object_cleanup_btf function
libbpf: Add bpf_link_create support for tracing_multi link
libbpf: Add btf_type_is_traceable_func function
libbpf: Add support to create tracing multi link selftests/bpf: Add tracing multi skel/pattern/ids attach tests selftests/bpf: Add tracing multi skel/pattern/ids module attach tests selftests/bpf: Add tracing multi intersect tests selftests/bpf: Add tracing multi cookies test selftests/bpf: Add tracing multi session test selftests/bpf: Add tracing multi attach fails test selftests/bpf: Add tracing multi verifier fails test selftests/bpf: Add tracing multi attach benchmark test selftests/bpf: Add tracing multi attach rollback tests

 arch/arm64/net/bpf_jit_comp.c                                      |  58 ++--
 arch/loongarch/net/bpf_jit.c                                       |  52 ++--
 arch/powerpc/net/bpf_jit_comp.c                                    |  54 ++--
 arch/riscv/net/bpf_jit_comp64.c                                    |  52 ++--
 arch/s390/net/bpf_jit_comp.c                                       |  44 +--
 arch/x86/net/bpf_jit_comp.c                                        |  54 ++--
 include/linux/bpf.h                                                | 117 ++++++--
 include/linux/bpf_types.h                                          |   1 +
 include/linux/bpf_verifier.h                                       |   4 +
 include/linux/btf_ids.h                                            |   1 +
 include/linux/ftrace.h                                             |   9 +
 include/linux/trace_events.h                                       |   6 +
 include/uapi/linux/bpf.h                                           |   9 +
 kernel/bpf/bpf_struct_ops.c                                        |  27 +-
 kernel/bpf/fixups.c                                                |   2 +
 kernel/bpf/syscall.c                                               |  83 +++---
 kernel/bpf/trampoline.c                                            | 670 +++++++++++++++++++++++++++++++++----------
 kernel/bpf/verifier.c                                              | 183 ++++++++++--
 kernel/trace/bpf_trace.c                                           | 204 +++++++++++++-
 kernel/trace/ftrace.c                                              |  35 ++-
 net/bpf/bpf_dummy_struct_ops.c                                     |  14 +-
 net/bpf/test_run.c                                                 |   3 +
 tools/include/uapi/linux/bpf.h                                     |  10 +
 tools/lib/bpf/bpf.c                                                |   9 +
 tools/lib/bpf/bpf.h                                                |   5 +
 tools/lib/bpf/libbpf.c                                             | 375 ++++++++++++++++++++++++-
 tools/lib/bpf/libbpf.h                                             |  15 +
 tools/lib/bpf/libbpf.map                                           |   1 +
 tools/lib/bpf/libbpf_internal.h                                    |   1 +
 tools/testing/selftests/bpf/Makefile                               |   9 +-
 tools/testing/selftests/bpf/prog_tests/tracing_multi.c             | 936 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/tracing_multi_attach.c           |  39 +++
 tools/testing/selftests/bpf/progs/tracing_multi_attach_module.c    |  25 ++
 tools/testing/selftests/bpf/progs/tracing_multi_bench.c            |  12 +
 tools/testing/selftests/bpf/progs/tracing_multi_check.c            | 214 ++++++++++++++
 tools/testing/selftests/bpf/progs/tracing_multi_fail.c             |  18 ++
 tools/testing/selftests/bpf/progs/tracing_multi_intersect_attach.c |  41 +++
 tools/testing/selftests/bpf/progs/tracing_multi_rollback.c         |  43 +++
 tools/testing/selftests/bpf/progs/tracing_multi_session_attach.c   |  65 +++++
 tools/testing/selftests/bpf/progs/tracing_multi_verifier.c         |  31 ++
 tools/testing/selftests/bpf/trace_helpers.c                        |   7 +-
 tools/testing/selftests/bpf/trace_helpers.h                        |   1 +
 42 files changed, 3110 insertions(+), 429 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/tracing_multi.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_attach_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_bench.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_check.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_fail.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_intersect_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_rollback.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_session_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_verifier.c