Skip to content

bpf: Add kprobe multi link

Jiri Olsa bpf lkml netdev 2 versions – Applied

Status from this message: This series was applied to bpf/bpf-next.git (master)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest13 patches 11 repliesApplied 13/13
  2. v20 of 8 patches 4 repliesSuperseded

Changes in v3 from the cover letter

- based on latest fprobe post from Masami [2]
- add acks
- add extra comment to kprobe_multi_link_handler wrt entry ip setup [Masami]
- keep swap_words_64 static and swap values directly in bpf_kprobe_multi_cookie_swap [Andrii]
- rearrange locking/migrate setup in kprobe_multi_link_prog_run [Andrii]
- move uapi fields [Andrii]
- add bpf_program__attach_kprobe_multi_opts function [Andrii]
- many small test changes [Andrii]
- added tests for bpf_program__attach_kprobe_multi_opts
- make kallsyms_lookup_name check for empty string [Andrii]

Patches in v3 All 13 applied

#Subject and filesSizeRepliesApplied
1/13lib/sort: Add priv pointer to swap functioninclude/linux/sort.h include/linux/types.h lib/sort.c+32 -110 repliesApplied
2/13kallsyms: Skip the name search for empty stringkernel/kallsyms.c+4 -00 repliesApplied
3/13bpf: Add multi kprobe linkinclude/linux/bpf_types.h include/linux/trace_events.h include/uapi/linux/bpf.h +3 more+266 -52 repliesApplied
4/13bpf: Add bpf_get_func_ip kprobe helper for multi kprobe linkkernel/trace/bpf_trace.c+15 -10 repliesApplied
5/13bpf: Add support to inline bpf_get_func_ip helper on x86kernel/bpf/verifier.c kernel/trace/bpf_trace.c+21 -10 repliesApplied
6/13bpf: Add cookie support to programs attached with kprobe multi linkinclude/uapi/linux/bpf.h kernel/bpf/syscall.c kernel/trace/bpf_trace.c +1 more+116 -20 repliesApplied
7/13libbpf: Add libbpf_kallsyms_parse functiontools/lib/bpf/libbpf.c tools/lib/bpf/libbpf_internal.h+43 -240 repliesApplied
8/13libbpf: Add bpf_link_create support for multi kprobestools/lib/bpf/bpf.c tools/lib/bpf/bpf.h+17 -10 repliesApplied
9/13libbpf: Add bpf_program__attach_kprobe_multi_opts functiontools/lib/bpf/libbpf.c tools/lib/bpf/libbpf.h tools/lib/bpf/libbpf.map+184 -04 repliesApplied
10/13selftests/bpf: Add kprobe_multi attach testtools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c tools/testing/selftests/bpf/progs/kprobe_multi.c tools/testing/selftests/bpf/trace_helpers.c+243 -00 repliesApplied
11/13selftests/bpf: Add kprobe_multi bpf_cookie testtools/testing/selftests/bpf/prog_tests/bpf_cookie.c tools/testing/selftests/bpf/progs/kprobe_multi.c+131 -190 repliesApplied
12/13selftests/bpf: Add attach test for bpf_program__attach_kprobe_multi_optstools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c+193 -110 repliesApplied
13/13selftests/bpf: Add cookie test for bpf_program__attach_kprobe_multi_optstools/testing/selftests/bpf/prog_tests/bpf_cookie.c+68 -00 repliesApplied

Reviews trailers per patch

4 of 13 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchMasami HiramatsuAndrii Nakryiko
SeriesA-b
1/13R-b
2/13
3/13A-b
4/13A-b
5/13
6/13
7/13A-b
8/13
9/13
10/13
11/13
12/13
13/13

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCHv3 bpf-next 00/13] bpf: Add kprobe multi link

hi,
this patchset adds new link type BPF_TRACE_KPROBE_MULTI that attaches kprobe program through fprobe API [1] instroduced by Masami.

The fprobe API allows to attach probe on multiple functions at once very fast, because it works on top of ftrace. On the other hand this limits the probe point to the function entry or return.

With bpftrace support I see following attach speed:

  # perf stat --null -r 5 ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); } '
  Attaching 2 probes...
  Attaching 3342 functions
  ...

1.4960 +- 0.0285 seconds time elapsed ( +- 1.91% )

v3 changes:
- based on latest fprobe post from Masami [2]
- add acks
- add extra comment to kprobe_multi_link_handler wrt entry ip setup [Masami]
- keep swap_words_64 static and swap values directly in bpf_kprobe_multi_cookie_swap [Andrii]
- rearrange locking/migrate setup in kprobe_multi_link_prog_run [Andrii]
- move uapi fields [Andrii]
- add bpf_program__attach_kprobe_multi_opts function [Andrii]
- many small test changes [Andrii]
- added tests for bpf_program__attach_kprobe_multi_opts
- make kallsyms_lookup_name check for empty string [Andrii]

v2 changes:
- based on latest fprobe changes [1]
- renaming the uapi interface to kprobe multi
- adding support for sort_r to pass user pointer for swap functions and using that in cookie support to keep just single functions array
- moving new link to kernel/trace/bpf_trace.c file
- using single fprobe callback function for entry and exit
- using kvzalloc, libbpf_ensure_mem functions
- adding new k[ret]probe.multi sections instead of using current kprobe
- used glob_match from test_progs.c, added '?' matching
- move bpf_get_func_ip verifier inline change to seprate change
- couple of other minor fixes

Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
bpf/kprobe_multi

thanks,
jirka

[1] https://lore.kernel.org/bpf/164458044634.586276.3261555265565111183.stgit@devnote2/
[2] https://lore.kernel.org/bpf/164735281449.1084943.12438881786173547153.stgit@devnote2/
---
Jiri Olsa (13):
lib/sort: Add priv pointer to swap function
kallsyms: Skip the name search for empty string
bpf: Add multi kprobe link
bpf: Add bpf_get_func_ip kprobe helper for multi kprobe link
bpf: Add support to inline bpf_get_func_ip helper on x86
bpf: Add cookie support to programs attached with kprobe multi link
libbpf: Add libbpf_kallsyms_parse function
libbpf: Add bpf_link_create support for multi kprobes
libbpf: Add bpf_program__attach_kprobe_multi_opts function selftests/bpf: Add kprobe_multi attach test
selftests/bpf: Add kprobe_multi bpf_cookie test
selftests/bpf: Add attach test for bpf_program__attach_kprobe_multi_opts selftests/bpf: Add cookie test for bpf_program__attach_kprobe_multi_opts

 include/linux/bpf_types.h                                  |   1 +
 include/linux/sort.h                                       |   2 +-
 include/linux/trace_events.h                               |   7 ++
 include/linux/types.h                                      |   1 +
 include/uapi/linux/bpf.h                                   |  14 ++++
 kernel/bpf/syscall.c                                       |  26 +++++--
 kernel/bpf/verifier.c                                      |  21 +++++-
 kernel/kallsyms.c                                          |   4 ++
 kernel/trace/bpf_trace.c                                   | 342 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 lib/sort.c                                                 |  40 ++++++++---
 tools/include/uapi/linux/bpf.h                             |  14 ++++
 tools/lib/bpf/bpf.c                                        |   9 +++
 tools/lib/bpf/bpf.h                                        |   9 ++-
 tools/lib/bpf/libbpf.c                                     | 222 +++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 tools/lib/bpf/libbpf.h                                     |  23 +++++++
 tools/lib/bpf/libbpf.map                                   |   1 +
 tools/lib/bpf/libbpf_internal.h                            |   5 ++
 tools/testing/selftests/bpf/prog_tests/bpf_cookie.c        | 177 ++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 323 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/kprobe_multi.c           |  98 +++++++++++++++++++++++++++
 tools/testing/selftests/bpf/trace_helpers.c                |   7 ++
 21 files changed, 1302 insertions(+), 44 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi.c