bpf: Add multi uprobe link
Subsystems (8)
Jiri Olsa
reviewerv3 cover bpf: Add multi uprobe link
hi,
this patchset is adding support to attach multiple uprobes and usdt probes through new uprobe_multi link.
The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.
The main reason is that we need to install perf event for each probed function and profile shows perf event installation (perf_install_in_context) as culprit.
The new uprobe_multi link just creates raw uprobes and attaches the bpf program to them without perf event being involved.
In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to.
v3 changes:
- consolidate attach type checks in bpf_prog_attach_check_attach_type [Andrii]
- remove bpf_prog_active check [Alexei]
- change rcu locking [Andrii]
- allocate ref_ctr_offsets conditionally [Andrii]
- remove ctx check from bpf_uprobe_multi_cookie [Andrii]
- move some elf_* functions in elf.c object [Andrii]
- fix uprobe link detection code [Andrii]
- add usdt.s program section [Andrii]
- rename bpf_program__attach_uprobe_multi_opts to bpf_program__attach_uprobe_multi [Andrii]
- remove extra case from attach_uprobe_multi [Andrii]
- rework usdt_manager_attach_usdt [Andrii]
- rework/rename new elf_find_* functions [Andrii]
- elf iterator fixes [Andrii]
- renames
- elf_sym_iter_next loop restruct
- return directly GElf_Sym in elf_sym
- add elf_sym_offset helper
- add st_type arg elf_sym_iter_new
- get rid of '_' prefixed functions
- simplify offsets handling
- other smaller changes
- added acks
- todo:
- seems like more elf_* helpers could go in elf.c object,
I'll send that as follow up
- will send fill_info support when [3] is merged [Andrii]
There's support for bpftrace [2] and tetragon [1].
Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
uprobe_multi
thanks,
jirka
[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (26):
bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
bpf: Add multi uprobe link
bpf: Add cookies support for uprobe_multi link
bpf: Add pid filter support for uprobe_multi link
bpf: Add bpf_get_func_ip helper support for uprobe link
libbpf: Add uprobe_multi attach type and link names
libbpf: Move elf_find_func_offset* functions to elf object
libbpf: Add elf_open/elf_close functions
libbpf: Add elf symbol iterator
libbpf: Add elf_resolve_syms_offsets function
libbpf: Add elf_resolve_pattern_offsets function
libbpf: Add bpf_link_create support for multi uprobes
libbpf: Add bpf_program__attach_uprobe_multi function
libbpf: Add support for u[ret]probe.multi[.s] program sections
libbpf: Add uprobe multi link detection
libbpf: Add uprobe multi link support to bpf_program__attach_usdt selftests/bpf: Add uprobe_multi skel test
selftests/bpf: Add uprobe_multi api test
selftests/bpf: Add uprobe_multi link test
selftests/bpf: Add uprobe_multi test program
selftests/bpf: Add uprobe_multi bench test
selftests/bpf: Add usdt_multi test program
selftests/bpf: Add usdt_multi bench test
selftests/bpf: Add uprobe_multi cookie test
selftests/bpf: Add uprobe_multi pid filter tests
selftests/bpf: Add extra link to uprobe_multi tests
include/linux/trace_events.h | 6 ++ include/uapi/linux/bpf.h | 16 ++++ kernel/bpf/syscall.c | 122 ++++++++++++------------- kernel/trace/bpf_trace.c | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- tools/include/uapi/linux/bpf.h | 16 ++++ tools/lib/bpf/Build | 2 +- tools/lib/bpf/bpf.c | 11 +++ tools/lib/bpf/bpf.h | 11 ++- tools/lib/bpf/elf.c | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 396 ++++++++++++++++++++++++++++++++++++++++++--------------------------------------- tools/lib/bpf/libbpf.h | 27 ++++++ tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_elf.h | 24 +++++ tools/lib/bpf/libbpf_internal.h | 3 + tools/lib/bpf/usdt.c | 109 +++++++++++++++-------- tools/testing/selftests/bpf/Makefile | 10 +++ tools/testing/selftests/bpf/prog_tests/bpf_cookie.c | 78 ++++++++++++++++ tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi.c | 110 +++++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++ tools/testing/selftests/bpf/uprobe_multi.c | 53 +++++++++++ tools/testing/selftests/bpf/usdt_multi.c | 24 +++++ 22 files changed, 1969 insertions(+), 296 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/lib/bpf/libbpf_elf.h create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c create mode 100644 tools/testing/selftests/bpf/usdt_multi.c
Jiri Olsa
reviewerPATCH v3 1/26 bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
Add extra attach_type checks from link_create under bpf_prog_attach_check_attach_type.
--- kernel/bpf/syscall.c | 108 +++++++++++++++++++------------------------ 1 file changed, 47 insertions(+), 61 deletions(-)
kernel/bpf/syscall.c +47 -61
@@ -3502,34 +3502,6 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)return fd; } -static int bpf_prog_attach_check_attach_type (const struct bpf_prog *prog, -enum bpf_attach_type attach_type) -{ -switch (prog->type) { -case BPF_PROG_TYPE_CGROUP_SOCK : -case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : -case BPF_PROG_TYPE_CGROUP_SOCKOPT : -case BPF_PROG_TYPE_SK_LOOKUP : -return attach_type == prog->expected_attach_type ?0 : -EINVAL ; -case BPF_PROG_TYPE_CGROUP_SKB : -if (!capable (CAP_NET_ADMIN )) -/* cg-skb progs can be loaded by unpriv user. -* check permissions at attach time. -*/ -return -EPERM ; -return prog->enforce_expected_attach_type && - prog->expected_attach_type != attach_type ? - -EINVAL :0 ; -case BPF_PROG_TYPE_KPROBE : -if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && - attach_type !=BPF_TRACE_KPROBE_MULTI ) -return -EINVAL ; -return 0 ; -default : -return 0 ; - } -} -static enum bpf_prog_type attach_type_to_prog_type (enum bpf_attach_type attach_type) {
@@ -3593,6 +3565,53 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type) } } +static int bpf_prog_attach_check_attach_type (const struct bpf_prog *prog, +enum bpf_attach_type attach_type) +{ +enum bpf_prog_type ptype; + +switch (prog->type) { +case BPF_PROG_TYPE_CGROUP_SOCK : +case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : +case BPF_PROG_TYPE_CGROUP_SOCKOPT : +case BPF_PROG_TYPE_SK_LOOKUP : +return attach_type == prog->expected_attach_type ?0 : -EINVAL ; +case BPF_PROG_TYPE_CGROUP_SKB : +if (!capable (CAP_NET_ADMIN )) +/* cg-skb progs can be loaded by unpriv user. +* check permissions at attach time. +*/ +return -EPERM ; +return prog->enforce_expected_attach_type && + prog->expected_attach_type != attach_type ? + -EINVAL :0 ; +case BPF_PROG_TYPE_KPROBE : +if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && + attach_type !=BPF_TRACE_KPROBE_MULTI ) +return -EINVAL ; +if (attach_type !=BPF_PERF_EVENT && + attach_type !=BPF_TRACE_KPROBE_MULTI ) +return -EINVAL ; +return 0 ; +case BPF_PROG_TYPE_EXT : +return 0 ; +case BPF_PROG_TYPE_NETFILTER : +if (attach_type !=BPF_NETFILTER ) +return -EINVAL ; +return 0 ; +case BPF_PROG_TYPE_PERF_EVENT : +case BPF_PROG_TYPE_TRACEPOINT : +if (attach_type !=BPF_PERF_EVENT ) +return -EINVAL ; +return 0 ; +default : + ptype =attach_type_to_prog_type (attach_type); +if (ptype ==BPF_PROG_TYPE_UNSPEC || ptype != prog->type) +return -EINVAL ; +return 0 ; + } +} +#define BPF_PROG_ATTACH_LAST_FIELD replace_bpf_fd#define BPF_F_ATTACH_MASK \
@@ -4658,7 +4677,6 @@ static int bpf_map_do_batch(const union bpf_attr *attr,#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookiesstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) { -enum bpf_prog_type ptype;struct bpf_prog *prog;int ret;
@@ -4677,38 +4695,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)if (ret) goto out; -switch (prog->type) { -case BPF_PROG_TYPE_EXT : -break ; -case BPF_PROG_TYPE_NETFILTER : -if (attr->link_create.attach_type !=BPF_NETFILTER ) { - ret = -EINVAL ; - goto out; - } -break ; -case BPF_PROG_TYPE_PERF_EVENT : -case BPF_PROG_TYPE_TRACEPOINT : -if (attr->link_create.attach_type !=BPF_PERF_EVENT ) { - ret = -EINVAL ; - goto out; - } -break ; -case BPF_PROG_TYPE_KPROBE : -if (attr->link_create.attach_type !=BPF_PERF_EVENT && - attr->link_create.attach_type !=BPF_TRACE_KPROBE_MULTI ) { - ret = -EINVAL ; - goto out; - } -break ; -default : - ptype =attach_type_to_prog_type (attr->link_create.attach_type); -if (ptype ==BPF_PROG_TYPE_UNSPEC || ptype != prog->type) { - ret = -EINVAL ; - goto out; - } -break ; - } -switch (prog->type) {case BPF_PROG_TYPE_CGROUP_SKB :case BPF_PROG_TYPE_CGROUP_SOCK :
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 01/26] bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
Quoted 64 lines · Jiri Olsa
Add extra
attach_typechecks fromlink_createunderbpf_prog_attach_check_attach_type.Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- kernel/bpf/syscall.c | 108 +++++++++++++++++++------------------------ 1 file changed, 47 insertions(+), 61 deletions(-)kernel/bpf/syscall.c +11
-28@@ -3502,34 +3502,6 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)return fd; } -static int bpf_prog_attach_check_attach_type (const struct bpf_prog *prog, -enum bpf_attach_type attach_type) -{ -switch (prog->type) { -case BPF_PROG_TYPE_CGROUP_SOCK : -case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : -case BPF_PROG_TYPE_CGROUP_SOCKOPT : -case BPF_PROG_TYPE_SK_LOOKUP : -return attach_type == prog->expected_attach_type ?0 : -EINVAL ; -case BPF_PROG_TYPE_CGROUP_SKB : -if (!capable (CAP_NET_ADMIN )) -/* cg-skb progs can be loaded by unpriv user. -* check permissions at attach time. -*/ -return -EPERM ; -return prog->enforce_expected_attach_type && - prog->expected_attach_type != attach_type ? - -EINVAL :0 ; -case BPF_PROG_TYPE_KPROBE : -if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && - attach_type !=BPF_TRACE_KPROBE_MULTI ) -return -EINVAL ; -return 0 ; -default : -return 0 ; - } -} -static enum bpf_prog_type attach_type_to_prog_type (enum bpf_attach_type attach_type) {@@ -3593,6 +3565,53 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type) } } +static int bpf_prog_attach_check_attach_type (const struct bpf_prog *prog, +enum bpf_attach_type attach_type) +{ +enum bpf_prog_type ptype; + +switch (prog->type) { +case BPF_PROG_TYPE_CGROUP_SOCK : +case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : +case BPF_PROG_TYPE_CGROUP_SOCKOPT : +case BPF_PROG_TYPE_SK_LOOKUP : +return attach_type == prog->expected_attach_type ?0 : -EINVAL ;
kernel/bpf/syscall.c attach_type_to_prog_type(enum bpf_attach_type attach_type) · line 3588
Quoted 10 lines
kernel/bpf/syscall.c +10
-0+case BPF_PROG_TYPE_CGROUP_SKB : +if (!capable (CAP_NET_ADMIN )) +/* cg-skb progs can be loaded by unpriv user. +* check permissions at attach time. +*/ +return -EPERM ; +return prog->enforce_expected_attach_type && + prog->expected_attach_type != attach_type ? + -EINVAL :0 ; +case BPF_PROG_TYPE_KPROBE :
nit, I'd keep KPROBE, TRACEPOINT and PERF_EVENT next to each other in this switch (that will just grow larger over time), as they are pretty closely related
otherwise lgtm:
Quoted 79 lines
kernel/bpf/syscall.c +26
-34+if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && + attach_type !=BPF_TRACE_KPROBE_MULTI ) +return -EINVAL ; +if (attach_type !=BPF_PERF_EVENT && + attach_type !=BPF_TRACE_KPROBE_MULTI ) +return -EINVAL ; +return 0 ; +case BPF_PROG_TYPE_EXT : +return 0 ; +case BPF_PROG_TYPE_NETFILTER : +if (attach_type !=BPF_NETFILTER ) +return -EINVAL ; +return 0 ; +case BPF_PROG_TYPE_PERF_EVENT : +case BPF_PROG_TYPE_TRACEPOINT : +if (attach_type !=BPF_PERF_EVENT ) +return -EINVAL ; +return 0 ; +default : + ptype =attach_type_to_prog_type (attach_type); +if (ptype ==BPF_PROG_TYPE_UNSPEC || ptype != prog->type) +return -EINVAL ; +return 0 ; + } +} +#define BPF_PROG_ATTACH_LAST_FIELD replace_bpf_fd#define BPF_F_ATTACH_MASK \@@ -4658,7 +4677,6 @@ static int bpf_map_do_batch(const union bpf_attr *attr,#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookiesstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) { -enum bpf_prog_type ptype;struct bpf_prog *prog;int ret;@@ -4677,38 +4695,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)if (ret) goto out; -switch (prog->type) { -case BPF_PROG_TYPE_EXT : -break ; -case BPF_PROG_TYPE_NETFILTER : -if (attr->link_create.attach_type !=BPF_NETFILTER ) { - ret = -EINVAL ; - goto out; - } -break ; -case BPF_PROG_TYPE_PERF_EVENT : -case BPF_PROG_TYPE_TRACEPOINT : -if (attr->link_create.attach_type !=BPF_PERF_EVENT ) { - ret = -EINVAL ; - goto out; - } -break ; -case BPF_PROG_TYPE_KPROBE : -if (attr->link_create.attach_type !=BPF_PERF_EVENT && - attr->link_create.attach_type !=BPF_TRACE_KPROBE_MULTI ) { - ret = -EINVAL ; - goto out; - } -break ; -default : - ptype =attach_type_to_prog_type (attr->link_create.attach_type); -if (ptype ==BPF_PROG_TYPE_UNSPEC || ptype != prog->type) { - ret = -EINVAL ; - goto out; - } -break ; - } -switch (prog->type) {case BPF_PROG_TYPE_CGROUP_SKB :case BPF_PROG_TYPE_CGROUP_SOCK : --2.41.0
Jiri Olsa
reviewerPATCH v3 2/26 bpf: Add multi uprobe link
In reply to Jiri OlsaAdding new multi uprobe link that allows to attach bpf program to multiple uprobes.
Uprobes to attach are specified via new link_create uprobe_multi union:
struct {
__u32 flags;
__u32 cnt;
__aligned_u64 path;
__aligned_u64 offsets;
__aligned_u64 ref_ctr_offsets;
} uprobe_multi;Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in ref_ctr_offsets array. All specified arrays have length of 'cnt'.
The 'flags' supports single bit for now that marks the uprobe as return probe.
--- include/linux/trace_events.h | 6 + include/uapi/linux/bpf.h | 14 ++ kernel/bpf/syscall.c | 14 +- kernel/trace/bpf_trace.c | 237 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 14 ++ 5 files changed, 282 insertions(+), 3 deletions(-)
include/linux/trace_events.h +6 -0
@@ -749,6 +749,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id, u32 *fd_type, const char **buf, u64 *probe_offset, u64 *probe_addr);int bpf_kprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog); +int bpf_uprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog);#else static inline unsigned int trace_call_bpf (struct trace_event_call *call,void *ctx) {
@@ -795,6 +796,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) {return -EOPNOTSUPP ; } +static inline int +bpf_uprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog) +{ +return -EOPNOTSUPP ; +}#endif enum {
include/uapi/linux/bpf.h +14 -0
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {BPF_LSM_CGROUP ,BPF_STRUCT_OPS ,BPF_NETFILTER , +BPF_TRACE_UPROBE_MULTI , __MAX_BPF_ATTACH_TYPE };
@@ -1053,6 +1054,7 @@ enum bpf_link_type {BPF_LINK_TYPE_KPROBE_MULTI =8 ,BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 , +BPF_LINK_TYPE_UPROBE_MULTI =11 , MAX_BPF_LINK_TYPE, };
@@ -1170,6 +1172,11 @@ enum bpf_link_type {*/ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +/* link_create.uprobe_multi.flags used in LINK_CREATE command for +* BPF_TRACE_UPROBE_MULTI attach type to create return probe. +*/ +#define BPF_F_UPROBE_MULTI_RETURN (1U << 0) +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: *
@@ -1579,6 +1586,13 @@ union bpf_attr {__s32 priority;__u32 flags; } netfilter; +struct { +__u32 flags; +__u32 cnt; +__aligned_u64 path; +__aligned_u64 offsets; +__aligned_u64 ref_ctr_offsets; + } uprobe_multi; }; } link_create;
kernel/bpf/syscall.c +11 -3
@@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)/* Clean up bpf_link and corresponding anon_inode file and FD. After * anon_inode is created, bpf_link can't be just kfree()'d due to deferred -* anon_inode's release() call. This helper marksbpf_link as +* anon_inode's release() call. This helper marks bpf_link as * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt * is not decremented, it's the responsibility of a calling code that failed * to complete bpf_link initialization. +* This helper eventually calls link's dealloc callback, but does not call +* link's release callback. */ void bpf_link_cleanup (struct bpf_link_primer *primer) {
@@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && attach_type !=BPF_TRACE_KPROBE_MULTI )return -EINVAL ; +if (prog->expected_attach_type ==BPF_TRACE_UPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI ) +return -EINVAL ;if (attach_type !=BPF_PERF_EVENT && - attach_type !=BPF_TRACE_KPROBE_MULTI ) + attach_type !=BPF_TRACE_KPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI )return -EINVAL ;return 0 ;case BPF_PROG_TYPE_EXT :
@@ -4748,8 +4754,10 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)case BPF_PROG_TYPE_KPROBE :if (attr->link_create.attach_type ==BPF_PERF_EVENT ) ret =bpf_perf_link_attach (attr, prog); -else +else if (attr->link_create.attach_type ==BPF_TRACE_KPROBE_MULTI ) ret =bpf_kprobe_multi_link_attach (attr, prog); +else if (attr->link_create.attach_type ==BPF_TRACE_UPROBE_MULTI ) + ret =bpf_uprobe_multi_link_attach (attr, prog);break ;default : ret = -EINVAL ;
kernel/trace/bpf_trace.c +237 -0
246 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +14 -0
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {BPF_LSM_CGROUP ,BPF_STRUCT_OPS ,BPF_NETFILTER , +BPF_TRACE_UPROBE_MULTI , __MAX_BPF_ATTACH_TYPE };
@@ -1053,6 +1054,7 @@ enum bpf_link_type {BPF_LINK_TYPE_KPROBE_MULTI =8 ,BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 , +BPF_LINK_TYPE_UPROBE_MULTI =11 , MAX_BPF_LINK_TYPE, };
@@ -1170,6 +1172,11 @@ enum bpf_link_type {*/ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +/* link_create.uprobe_multi.flags used in LINK_CREATE command for +* BPF_TRACE_UPROBE_MULTI attach type to create return probe. +*/ +#define BPF_F_UPROBE_MULTI_RETURN (1U << 0) +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: *
@@ -1579,6 +1586,13 @@ union bpf_attr {__s32 priority;__u32 flags; } netfilter; +struct { +__u32 flags; +__u32 cnt; +__aligned_u64 path; +__aligned_u64 offsets; +__aligned_u64 ref_ctr_offsets; + } uprobe_multi; }; } link_create;
Signature · 1 line
2.41.0
Quoted 23 lines · Jiri Olsa
Adding new multi uprobe link that allows to attach bpf program to multiple uprobes.
Uprobes to attach are specified via new
link_createuprobe_multiunion:struct { __u32 flags; __u32 cnt; __aligned_u64 path; __aligned_u64 offsets; __aligned_u64 ref_ctr_offsets; } uprobe_multi;Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in
ref_ctr_offsetsarray. All specified arrays have length of 'cnt'.The 'flags' supports single bit for now that marks the uprobe as return probe.
Quoted 9 lines
Signed-off-by Jiri Olsa <[email protected]>--- include/linux/trace_events.h | 6 + include/uapi/linux/bpf.h | 14 ++ kernel/bpf/syscall.c | 14 +- kernel/trace/bpf_trace.c | 237 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 14 ++ 5 files changed, 282 insertions(+), 3 deletions(-)
overall LGTM, but I think there is path leak, please fix that and add my ack
Quoted 45 lines
include/linux/trace_events.h +6
-0@@ -749,6 +749,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id, u32 *fd_type, const char **buf, u64 *probe_offset, u64 *probe_addr);int bpf_kprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog); +int bpf_uprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog);#else static inline unsigned int trace_call_bpf (struct trace_event_call *call,void *ctx) {@@ -795,6 +796,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) {return -EOPNOTSUPP ; } +static inline int +bpf_uprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog) +{ +return -EOPNOTSUPP ; +}#endif enum {include/uapi/linux/bpf.h +2
-0@@ -1036,6 +1036,7 @@ enum bpf_attach_type {BPF_LSM_CGROUP ,BPF_STRUCT_OPS ,BPF_NETFILTER , +BPF_TRACE_UPROBE_MULTI , __MAX_BPF_ATTACH_TYPE };@@ -1053,6 +1054,7 @@ enum bpf_link_type {BPF_LINK_TYPE_KPROBE_MULTI =8 ,BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 , +BPF_LINK_TYPE_UPROBE_MULTI =11 , MAX_BPF_LINK_TYPE, };
include/uapi/linux/bpf.h enum bpf_link_type { · line 1179
Quoted 8 lines
include/uapi/linux/bpf.h +5
-0@@ -1170,6 +1172,11 @@ enum bpf_link_type {*/ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +/* link_create.uprobe_multi.flags used in LINK_CREATE command for +* BPF_TRACE_UPROBE_MULTI attach type to create return probe. +*/ +#define BPF_F_UPROBE_MULTI_RETURN (1U << 0) +
any reason why we don't use anonymous ENUMs for all these UAPI constants? When we need to use these flags from BPF side (e.g., for BPF LSM), having them as #defines will be a PITA, as they won't be present in vmlinux.h
include/uapi/linux/bpf.h union bpf_attr { · line 1591
Quoted 10 lines
include/uapi/linux/bpf.h +3
-0/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1579,6 +1586,13 @@ union bpf_attr {__s32 priority;__u32 flags; } netfilter; +struct { +__u32 flags; +__u32 cnt;
total nit, but I'd move it after path/offsets/ref_ctr_offsets, and make the order cnt (as it applies to previous two offsets/ref_ctr_offsets) and then flags last. Seems like more logical order, but totally subjective
Quoted 12 lines
include/uapi/linux/bpf.h +4
-0+__aligned_u64 path; +__aligned_u64 offsets; +__aligned_u64 ref_ctr_offsets; + } uprobe_multi; }; } link_create;kernel/bpf/syscall.c +0
-0
kernel/bpf/syscall.c static void bpf_link_free_id(int id) · line 2821
Quoted 10 lines
kernel/bpf/syscall.c +3
-1@@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)/* Clean up bpf_link and corresponding anon_inode file and FD. After * anon_inode is created, bpf_link can't be just kfree()'d due to deferred -* anon_inode's release() call. This helper marksbpf_link as +* anon_inode's release() call. This helper marks bpf_link as * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt * is not decremented, it's the responsibility of a calling code that failed * to complete bpf_link initialization. +* This helper eventually calls link's dealloc callback, but does not call +* link's release callback.
Thanks for clarifying comments!
Quoted 4 lines
kernel/bpf/syscall.c +0
-0*/ void bpf_link_cleanup (struct bpf_link_primer *primer) {
kernel/bpf/syscall.c static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, · line 3599
Quoted 10 lines
kernel/bpf/syscall.c +5
-1@@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && attach_type !=BPF_TRACE_KPROBE_MULTI )return -EINVAL ; +if (prog->expected_attach_type ==BPF_TRACE_UPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI ) +return -EINVAL ;if (attach_type !=BPF_PERF_EVENT && - attach_type !=BPF_TRACE_KPROBE_MULTI ) + attach_type !=BPF_TRACE_KPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI )
if this keeps growing, we should think about having a switch in a switch to not repeat BPF_TRACE_UPROBE_MULTI and BPF_TRACE_KPROBE_MULTI twice
Quoted 3 lines
kernel/bpf/syscall.c +0
-0return -EINVAL ;return 0 ;case BPF_PROG_TYPE_EXT :
[...]
Quoted 28 lines
kernel/bpf/syscall.c +28
-0+for (i =0 ; i < cnt; i++) { +if (uref_ctr_offsets &&__get_user (ref_ctr_offsets[i], uref_ctr_offsets + i)) { + err = -EFAULT ; + goto error_free; + } +if (__get_user (uprobes[i].offset, uoffsets + i)) { + err = -EFAULT ; + goto error_free; + } + + uprobes[i].link = link; + +if (flags &BPF_F_UPROBE_MULTI_RETURN ) + uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; +else + uprobes[i].consumer.handler = uprobe_multi_link_handler; + } + + link->cnt = cnt; + link->uprobes = uprobes; + link->path = path; + +bpf_link_init (&link->link,BPF_LINK_TYPE_UPROBE_MULTI , + &bpf_uprobe_multi_link_lops, prog); + + err =bpf_link_prime (&link->link, &link_primer); +if (err) + goto error_free;
kernel/bpf/syscall.c
Quoted 10 lines
kernel/bpf/syscall.c +10
-0+ +for (i =0 ; i < cnt; i++) { + err =uprobe_register_refctr (d_real_inode (link->path.dentry), + uprobes[i].offset, + ref_ctr_offsets ? ref_ctr_offsets[i] :0 , + &uprobes[i].consumer); +if (err) { +bpf_uprobe_unregister (&path, uprobes, i); +bpf_link_cleanup (&link_primer); +kvfree (ref_ctr_offsets);
are we missing path_put() in this error handling path? so maybe goto error_path_put here instead of return?
Quoted 15 lines
kernel/bpf/syscall.c +15
-0+return err; + } + } + +kvfree (ref_ctr_offsets); +return bpf_link_settle (&link_primer); + +error_free: +kvfree (ref_ctr_offsets); +kvfree (uprobes); +kfree (link); +error_path_put: +path_put (&path); +return err; +}
[...]
On Thu, Jul 06, 2023 at 03:34:10PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 24 lines
Quoted 24 lines
include/uapi/linux/bpf.h +2
-0@@ -1036,6 +1036,7 @@ enum bpf_attach_type {BPF_LSM_CGROUP ,BPF_STRUCT_OPS ,BPF_NETFILTER , +BPF_TRACE_UPROBE_MULTI , __MAX_BPF_ATTACH_TYPE };@@ -1053,6 +1054,7 @@ enum bpf_link_type {BPF_LINK_TYPE_KPROBE_MULTI =8 ,BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 , +BPF_LINK_TYPE_UPROBE_MULTI =11 , MAX_BPF_LINK_TYPE, };@@ -1170,6 +1172,11 @@ enum bpf_link_type {*/ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0)
Quoted 10 lines
include/uapi/linux/bpf.h enum bpf_link_type { · line 1179
Quoted 5 lines
include/uapi/linux/bpf.h +5
-0+/* link_create.uprobe_multi.flags used in LINK_CREATE command for +* BPF_TRACE_UPROBE_MULTI attach type to create return probe. +*/ +#define BPF_F_UPROBE_MULTI_RETURN (1U << 0) +any reason why we don't use anonymous ENUMs for all these UAPI constants? When we need to use these flags from BPF side (e.g., for BPF LSM), having them as #defines will be a PITA, as they won't be present in
vmlinux.h
ugh right, we already did that before.. will change
Quoted 17 lines
include/uapi/linux/bpf.h union bpf_attr { · line 1591
Quoted 12 lines
include/uapi/linux/bpf.h +3
-0/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1579,6 +1586,13 @@ union bpf_attr {__s32 priority;__u32 flags; } netfilter; +struct { +__u32 flags; +__u32 cnt;total nit, but I'd move it after path/offsets/ref_ctr_offsets, and make the order cnt (as it applies to previous two offsets/ref_ctr_offsets) and then flags last. Seems like more logical order, but totally subjective
ok
Quoted 36 lines
Quoted 13 lines
include/uapi/linux/bpf.h +4
-0+__aligned_u64 path; +__aligned_u64 offsets; +__aligned_u64 ref_ctr_offsets; + } uprobe_multi; }; } link_create;kernel/bpf/syscall.c +0
-0kernel/bpf/syscall.c static void bpf_link_free_id(int id) · line 2821
Quoted 10 lines
kernel/bpf/syscall.c +3
-1@@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)/* Clean up bpf_link and corresponding anon_inode file and FD. After * anon_inode is created, bpf_link can't be just kfree()'d due to deferred -* anon_inode's release() call. This helper marksbpf_link as +* anon_inode's release() call. This helper marks bpf_link as * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt * is not decremented, it's the responsibility of a calling code that failed * to complete bpf_link initialization. +* This helper eventually calls link's dealloc callback, but does not call +* link's release callback. Thanks for clarifying comments!
Quoted 10 lines
kernel/bpf/syscall.c +3
-0*/ void bpf_link_cleanup (struct bpf_link_primer *primer) {@@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && attach_type !=BPF_TRACE_KPROBE_MULTI )return -EINVAL ; +if (prog->expected_attach_type ==BPF_TRACE_UPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI ) +return -EINVAL ;
Quoted 8 lines
kernel/bpf/syscall.c static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, · line 3599
Quoted 4 lines
kernel/bpf/syscall.c +2
-1if (attach_type !=BPF_PERF_EVENT && - attach_type !=BPF_TRACE_KPROBE_MULTI ) + attach_type !=BPF_TRACE_KPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI )if this keeps growing, we should think about having a switch in a switch to not repeat
BPF_TRACE_UPROBE_MULTIandBPF_TRACE_KPROBE_MULTItwice
ok
SNIP
Quoted 10 lines
kernel/bpf/syscall.c static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
Quoted 9 lines
kernel/bpf/syscall.c +9
-0+for (i =0 ; i < cnt; i++) { + err =uprobe_register_refctr (d_real_inode (link->path.dentry), + uprobes[i].offset, + ref_ctr_offsets ? ref_ctr_offsets[i] :0 , + &uprobes[i].consumer); +if (err) { +bpf_uprobe_unregister (&path, uprobes, i); +bpf_link_cleanup (&link_primer); +kvfree (ref_ctr_offsets);
Quoted 2 lines
are we missing
path_put()in this error handling path? so maybe gotoerror_path_puthere instead of return?
aaaah right path_put needs to go to dealloc callback :-\ will change, thanks
jirka
Quoted 32 lines · Jiri Olsa
Adding new multi uprobe link that allows to attach bpf program to multiple uprobes.
Uprobes to attach are specified via new
link_createuprobe_multiunion:struct { __u32 flags; __u32 cnt; __aligned_u64 path; __aligned_u64 offsets; __aligned_u64 ref_ctr_offsets; } uprobe_multi;Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in
ref_ctr_offsetsarray. All specified arrays have length of 'cnt'.The 'flags' supports single bit for now that marks the uprobe as return probe.
Signed-off-by Jiri Olsa <[email protected]>--- include/linux/trace_events.h | 6 + include/uapi/linux/bpf.h | 14 ++ kernel/bpf/syscall.c | 14 +- kernel/trace/bpf_trace.c | 237 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 14 ++ 5 files changed, 282 insertions(+), 3 deletions(-)
[...]
Quoted 4 lines
kernel/trace/bpf_trace.c +4
-0+ flags = attr->link_create.uprobe_multi.flags; +if (flags & ~BPF_F_UPROBE_MULTI_RETURN ) +return -EINVAL ; +
kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3069
Quoted 10 lines
kernel/trace/bpf_trace.c +10
-0+/* +* path, offsets and cnt are mandatory, +* ref_ctr_offsets is optional +*/ + upath =u64_to_user_ptr (attr->link_create.uprobe_multi.path); + uoffsets =u64_to_user_ptr (attr->link_create.uprobe_multi.offsets); + cnt = attr->link_create.uprobe_multi.cnt; + +if (!upath || !uoffsets || !cnt) +return -EINVAL ;
see below for -EBADF, but we can also, additionally, return -EPROTO here, for example?
Quoted 6 lines
kernel/trace/bpf_trace.c +6
-0+ + uref_ctr_offsets =u64_to_user_ptr (attr->link_create.uprobe_multi.ref_ctr_offsets); + + name =strndup_user (upath,PATH_MAX ); +if (IS_ERR (name)) { + err =PTR_ERR (name);
kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3085
Quoted 10 lines
kernel/trace/bpf_trace.c +10
-0+return err; + } + + err =kern_path (name,LOOKUP_FOLLOW , &path); +kfree (name); +if (err) +return err; + +if (!d_is_reg (path.dentry)) { + err = -EINVAL ;
as I mentioned in another patch, -EBADF here for feature detection (and it makes sense by itself, probably)
Quoted 17 lines
kernel/trace/bpf_trace.c +17
-0+ goto error_path_put; + } + + err = -ENOMEM ; + + link =kzalloc (sizeof (*link),GFP_KERNEL ); + uprobes =kvcalloc (cnt,sizeof (*uprobes),GFP_KERNEL ); + +if (!uprobes || !link) + goto error_free; + +if (uref_ctr_offsets) { + ref_ctr_offsets =kvcalloc (cnt,sizeof (*ref_ctr_offsets),GFP_KERNEL ); +if (!ref_ctr_offsets) + goto error_free; + } +
[...]
On Thu, Jul 06, 2023 at 09:22:29PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 29 lines
Quoted 4 lines
kernel/trace/bpf_trace.c +4
-0+ flags = attr->link_create.uprobe_multi.flags; +if (flags & ~BPF_F_UPROBE_MULTI_RETURN ) +return -EINVAL ; +kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3069
Quoted 10 lines
kernel/trace/bpf_trace.c +10
-0+/* +* path, offsets and cnt are mandatory, +* ref_ctr_offsets is optional +*/ + upath =u64_to_user_ptr (attr->link_create.uprobe_multi.path); + uoffsets =u64_to_user_ptr (attr->link_create.uprobe_multi.offsets); + cnt = attr->link_create.uprobe_multi.cnt; + +if (!upath || !uoffsets || !cnt) +return -EINVAL ;see below for
-EBADF, but we can also, additionally, return-EPROTOhere, for example?Quoted 11 lines
kernel/trace/bpf_trace.c +11
-0+ + uref_ctr_offsets =u64_to_user_ptr (attr->link_create.uprobe_multi.ref_ctr_offsets); + + name =strndup_user (upath,PATH_MAX ); +if (IS_ERR (name)) { + err =PTR_ERR (name); +return err; + } + + err =kern_path (name,LOOKUP_FOLLOW , &path); +kfree (name);
Quoted 8 lines
kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3085
Quoted 5 lines
kernel/trace/bpf_trace.c +5
-0+if (err) +return err; + +if (!d_is_reg (path.dentry)) { + err = -EINVAL ;as I mentioned in another patch,
-EBADFhere for feature detection (and it makes sense by itself, probably)
yes, I like this place better, also -EBADF error makes more sense in here
thanks,
jirka
Quoted 20 lines
Quoted 18 lines
kernel/trace/bpf_trace.c +17
-0+ goto error_path_put; + } + + err = -ENOMEM ; + + link =kzalloc (sizeof (*link),GFP_KERNEL ); + uprobes =kvcalloc (cnt,sizeof (*uprobes),GFP_KERNEL ); + +if (!uprobes || !link) + goto error_free; + +if (uref_ctr_offsets) { + ref_ctr_offsets =kvcalloc (cnt,sizeof (*ref_ctr_offsets),GFP_KERNEL ); +if (!ref_ctr_offsets) + goto error_free; + } +[...]
Jiri Olsa
reviewerPATCH v3 3/26 bpf: Add cookies support for uprobe_multi link
In reply to Jiri OlsaAdding support to specify cookies array for uprobe_multi link.
The cookies array share indexes and length with other uprobe_multi arrays (offsets/ref_ctr_offsets).
The cookies[i] value defines cookie for i-the uprobe and will be returned by bpf_get_attach_cookie helper when called from ebpf program hooked to that specific uprobe.
--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 45 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 44 insertions(+), 5 deletions(-)
include/uapi/linux/bpf.h +1 -0
@@ -1592,6 +1592,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies; } uprobe_multi; }; } link_create;
kernel/bpf/syscall.c +1 -1
@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,return err; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookiesstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) {struct bpf_prog *prog;
kernel/trace/bpf_trace.c +41 -4
115 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +1 -0
@@ -1592,6 +1592,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies; } uprobe_multi; }; } link_create;
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 03/26] bpf: Add cookies support for uprobe_multi link
Quoted 24 lines · Jiri Olsa
Adding support to specify cookies array for
uprobe_multilink.The cookies array share indexes and length with other
uprobe_multiarrays (offsets/ref_ctr_offsets).The cookies[i] value defines cookie for i-the uprobe and will be returned by
bpf_get_attach_cookiehelper when called from ebpf program hooked to that specific uprobe.Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 45 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 44 insertions(+), 5 deletions(-)include/uapi/linux/bpf.h +0
-0
include/uapi/linux/bpf.h union bpf_attr { · line 1595
Quoted 4 lines
include/uapi/linux/bpf.h +1
-0@@ -1592,6 +1592,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies;
Could you pls. explain why the 'cookies' is defined as a pointer ? Are there real use cases to define different cookies for different probes in a single uprobe_multi ? Why can't we just use one cookie for all probes? Per my understanding, the cookie is used to identify the user or the application, rather than each probe.
We also defined it the same way in kprobe_multi...
Quoted 163 lines
include/uapi/linux/bpf.h +0
-0} uprobe_multi; }; } link_create;kernel/bpf/syscall.c +1
-1@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,return err; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookiesstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) {struct bpf_prog *prog;kernel/trace/bpf_trace.c +41
-4@@ -87,6 +87,8 @@ static int bpf_btf_printf_prepare(struct btf_ptr *ptr, u32 btf_ptr_size,static u64 bpf_kprobe_multi_cookie (struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx); +static u64 bpf_uprobe_multi_cookie (struct bpf_run_ctx *ctx); +/** * trace_call_bpf - invoke BPF program * @call: tracepoint event @@ -1099,6 +1101,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs) +{ +return bpf_uprobe_multi_cookie (current->bpf_ctx); +} + +static const struct bpf_func_proto bpf_get_attach_cookie_proto_umulti = { + .func = bpf_get_attach_cookie_uprobe_multi, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , +}; + BPF_CALL_1(bpf_get_attach_cookie_trace, void *, ctx) {struct bpf_trace_run_ctx *run_ctx;@@ -1545,9 +1559,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) &bpf_get_func_ip_proto_kprobe_multi : &bpf_get_func_ip_proto_kprobe;case BPF_FUNC_get_attach_cookie: -return prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI ? - &bpf_get_attach_cookie_proto_kmulti : - &bpf_get_attach_cookie_proto_trace; +if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI ) +return &bpf_get_attach_cookie_proto_kmulti; +if (prog->expected_attach_type ==BPF_TRACE_UPROBE_MULTI ) +return &bpf_get_attach_cookie_proto_umulti; +return &bpf_get_attach_cookie_proto_trace;default :return bpf_tracing_func_proto (func_id, prog); }@@ -2930,6 +2946,7 @@ struct bpf_uprobe_multi_link;struct bpf_uprobe {struct bpf_uprobe_multi_link *link;loff_t offset; +u64 cookie;struct uprobe_consumer consumer; };@@ -2943,6 +2960,7 @@ struct bpf_uprobe_multi_link {struct bpf_uprobe_multi_run_ctx {struct bpf_run_ctx run_ctx;unsigned long entry_ip; +struct bpf_uprobe *uprobe; }; static void bpf_uprobe_unregister(struct path *path, struct bpf_uprobe *uprobes,@@ -2986,6 +3004,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,struct bpf_uprobe_multi_link *link = uprobe->link;struct bpf_uprobe_multi_run_ctx run_ctx = { .entry_ip = entry_ip, + .uprobe = uprobe, };struct bpf_prog *prog = link->link.prog;bool sleepable = prog->aux->sleepable;@@ -3032,6 +3051,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, sreturn uprobe_prog_run (uprobe, func, regs); } +static u64 bpf_uprobe_multi_cookie (struct bpf_run_ctx *ctx) +{ +struct bpf_uprobe_multi_run_ctx *run_ctx; + + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); +return run_ctx->uprobe->cookie; +} +int bpf_uprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog) {struct bpf_uprobe_multi_link *link =NULL ;@@ -3040,6 +3067,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prstruct bpf_link_primer link_primer;struct bpf_uprobe *uprobes =NULL ;unsigned long __user *uoffsets; +u64 __user *ucookies;void __user *upath;u32 flags, cnt, i;struct path path;@@ -3059,7 +3087,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr/* * path, offsets and cnt are mandatory, -* ref_ctr_offsets is optional +* ref_ctr_offsets and cookies are optional */ upath =u64_to_user_ptr (attr->link_create.uprobe_multi.path); uoffsets =u64_to_user_ptr (attr->link_create.uprobe_multi.offsets);@@ -3069,6 +3097,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prreturn -EINVAL ; uref_ctr_offsets =u64_to_user_ptr (attr->link_create.uprobe_multi.ref_ctr_offsets); + ucookies =u64_to_user_ptr (attr->link_create.uprobe_multi.cookies); name =strndup_user (upath,PATH_MAX );if (IS_ERR (name)) {@@ -3101,6 +3130,10 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr }for (i =0 ; i < cnt; i++) { +if (ucookies &&__get_user (uprobes[i].cookie, ucookies + i)) { + err = -EFAULT ; + goto error_free; + }if (uref_ctr_offsets &&__get_user (ref_ctr_offsets[i], uref_ctr_offsets + i)) { err = -EFAULT ; goto error_free;@@ -3158,4 +3191,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr {return -EOPNOTSUPP ; } +static u64 bpf_uprobe_multi_cookie (struct bpf_run_ctx *ctx) +{ +return 0 ; +}#endif /* CONFIG_UPROBES */ tools/include/uapi/linux/bpf.h +1
-1@@ -1592,6 +1592,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies; } uprobe_multi; }; } link_create; --2.41.0
Signature · 2 lines
Regards Yafang
Re: [PATCHv3 bpf-next 03/26] bpf: Add cookies support for uprobe_multi link
Quoted 36 lines · Yafang Shao
Quoted 24 lines · Jiri Olsa
Adding support to specify cookies array for
uprobe_multilink.The cookies array share indexes and length with other
uprobe_multiarrays (offsets/ref_ctr_offsets).The cookies[i] value defines cookie for i-the uprobe and will be returned by
bpf_get_attach_cookiehelper when called from ebpf program hooked to that specific uprobe.Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 45 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 44 insertions(+), 5 deletions(-)include/uapi/linux/bpf.h +0
-0include/uapi/linux/bpf.h union bpf_attr { · line 1595
Quoted 4 lines
include/uapi/linux/bpf.h +1
-0@@ -1592,6 +1592,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies;Could you pls. explain why the 'cookies' is defined as a pointer ? Are there real use cases to define different cookies for different probes in a single
uprobe_multi? Why can't we just use one cookie for all probes? Per my understanding, the cookie is used to identify the user or the application, rather than each probe.
Quoted 1 line
We also defined it the same way in
kprobe_multi...
yes, each probed function (or symbol/offset for uprobe) can get its own cookie and we use that to get probe-specific config in generic bpf program with the bpf_get_attach_cookie helper
jirka
Jiri Olsa
reviewerPATCH v3 4/26 bpf: Add pid filter support for uprobe_multi link
In reply to Jiri OlsaAdding support to specify pid for uprobe_multi link and the uprobes are created only for task with given pid value.
Using the consumer.filter filter callback for that, so the task gets filtered during the uprobe installation.
We still need to check the task during runtime in the uprobe handler, because the handler could get executed if there's another system wide consumer on the same uprobe (thanks Oleg for the insight).
--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 33 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 36 insertions(+), 1 deletion(-)
include/uapi/linux/bpf.h +1 -0
@@ -1593,6 +1593,7 @@ union bpf_attr {__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets;__aligned_u64 cookies; +__u32 pid; } uprobe_multi; }; } link_create;
kernel/bpf/syscall.c +1 -1
@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,return err; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.pidstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) {struct bpf_prog *prog;
kernel/trace/bpf_trace.c +33 -0
@@ -2955,6 +2955,7 @@ struct bpf_uprobe_multi_link {struct bpf_link link;u32 cnt;struct bpf_uprobe *uprobes; +struct task_struct *task; };struct bpf_uprobe_multi_run_ctx {
@@ -2981,6 +2982,8 @@ static void bpf_uprobe_multi_link_release(struct bpf_link *link) umulti_link = container_of(link, struct bpf_uprobe_multi_link, link);bpf_uprobe_unregister (&umulti_link->path, umulti_link->uprobes, umulti_link->cnt);path_put (&umulti_link->path); +if (umulti_link->task) +put_task_struct (umulti_link->task); }static void bpf_uprobe_multi_link_dealloc (struct bpf_link *link)
@@ -3011,6 +3014,9 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,struct bpf_run_ctx *old_run_ctx;int err =0 ; +if (link->task && current != link->task) +return 0 ; +might_fault ();migrate_disable ();
@@ -3033,6 +3039,16 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,return err; } +static bool +uprobe_multi_link_filter (struct uprobe_consumer *con,enum uprobe_filter_ctx ctx, +struct mm_struct *mm) +{ +struct bpf_uprobe *uprobe; + + uprobe = container_of(con, struct bpf_uprobe, consumer); +return uprobe->link->task->mm == mm; +} +static int uprobe_multi_link_handler (struct uprobe_consumer *con,struct pt_regs *regs) {
@@ -3066,12 +3082,14 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prunsigned long *ref_ctr_offsets =NULL ;struct bpf_link_primer link_primer;struct bpf_uprobe *uprobes =NULL ; +struct task_struct *task =NULL ;unsigned long __user *uoffsets;u64 __user *ucookies;void __user *upath;u32 flags, cnt, i;struct path path;char *name; +pid_t pid;int err;/* no support for 32bit archs yet */
@@ -3115,6 +3133,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr goto error_path_put; } + pid = attr->link_create.uprobe_multi.pid; +if (pid) { +rcu_read_lock (); + task =get_pid_task (find_vpid (pid),PIDTYPE_PID ); +rcu_read_unlock (); +if (!task) + goto error_path_put; + } + err = -ENOMEM ; link =kzalloc (sizeof (*link),GFP_KERNEL );
@@ -3149,11 +3176,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; else uprobes[i].consumer.handler = uprobe_multi_link_handler; + +if (pid) + uprobes[i].consumer.filter = uprobe_multi_link_filter; } link->cnt = cnt; link->uprobes = uprobes; link->path = path; + link->task = task;bpf_link_init (&link->link,BPF_LINK_TYPE_UPROBE_MULTI , &bpf_uprobe_multi_link_lops, prog);
@@ -3182,6 +3213,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prkvfree (ref_ctr_offsets);kvfree (uprobes);kfree (link); +if (task) +put_task_struct (task); error_path_put:path_put (&path);return err;
tools/include/uapi/linux/bpf.h +1 -0
@@ -1593,6 +1593,7 @@ union bpf_attr {__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets;__aligned_u64 cookies; +__u32 pid; } uprobe_multi; }; } link_create;
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v3 5/26 bpf: Add bpf_get_func_ip helper support for uprobe link
In reply to Jiri OlsaAdding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.
It returns the ip of the uprobe.
--- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)
kernel/trace/bpf_trace.c +30 -3
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx);static u64 bpf_uprobe_multi_cookie (struct bpf_run_ctx *ctx); +static u64 bpf_uprobe_multi_entry_ip (struct bpf_run_ctx *ctx);/** * trace_call_bpf - invoke BPF program
@@ -1101,6 +1102,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_func_ip_uprobe_multi, struct pt_regs *, regs) +{ +return bpf_uprobe_multi_entry_ip (current->bpf_ctx); +} + +static const struct bpf_func_proto bpf_get_func_ip_proto_uprobe_multi = { + .func = bpf_get_func_ip_uprobe_multi, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , +}; + BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs) {return bpf_uprobe_multi_cookie (current->bpf_ctx);
@@ -1555,9 +1568,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_override_return_proto;#endif case BPF_FUNC_get_func_ip: -return prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI ? - &bpf_get_func_ip_proto_kprobe_multi : - &bpf_get_func_ip_proto_kprobe; +if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI ) +return &bpf_get_func_ip_proto_kprobe_multi; +if (prog->expected_attach_type ==BPF_TRACE_UPROBE_MULTI ) +return &bpf_get_func_ip_proto_uprobe_multi; +return &bpf_get_func_ip_proto_kprobe;case BPF_FUNC_get_attach_cookie:if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI )return &bpf_get_attach_cookie_proto_kmulti;
@@ -3067,6 +3082,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, sreturn uprobe_prog_run (uprobe, func, regs); } +static u64 bpf_uprobe_multi_entry_ip (struct bpf_run_ctx *ctx) +{ +struct bpf_uprobe_multi_run_ctx *run_ctx; + + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); +return run_ctx->entry_ip; +} +static u64 bpf_uprobe_multi_cookie (struct bpf_run_ctx *ctx) {struct bpf_uprobe_multi_run_ctx *run_ctx;
@@ -3228,4 +3251,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) {return 0 ; } +static u64 bpf_uprobe_multi_entry_ip (struct bpf_run_ctx *ctx) +{ +return 0 ; +}#endif /* CONFIG_UPROBES */
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link
Quoted 4 lines · Jiri Olsa
Adding support for
bpf_get_func_iphelper being called from ebpf program attached byuprobe_multilink.
Quoted 8 lines
It returns the ip of the uprobe.
Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)
A slight aside related to bpf_get_func_ip() support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and using bpf_get_func_ip() there with uretprobes, and it seems like it doesn't work.
Is that intentional or we just missed that bpf_get_func_ip() doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them for bpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).
Thanks!
Quoted 7 lines
kernel/trace/bpf_trace.c +0
-0@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx);
[...]
Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link
Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa
Adding support for
bpf_get_func_iphelper being called from ebpf program attached byuprobe_multilink.Quoted 8 lines
It returns the ip of the uprobe.
Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)A slight aside related to
bpf_get_func_ip()support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and usingbpf_get_func_ip()there with uretprobes, and it seems like it doesn't work.
Quoted 5 lines
Is that intentional or we just missed that
bpf_get_func_ip()doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them forbpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).
I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol
we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's get_func_ip helper version:
BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
{
struct kprobe *kp = kprobe_running();
if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
return 0;
return get_entry_ip((uintptr_t)kp->addr);
}I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel
I guess the app needs to store regs->ip and match it against symbol addresses in user space
jirka
Quoted 12 lines
Thanks!
Quoted 7 lines
kernel/trace/bpf_trace.c +0
-0@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx);[...]
Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link
Quoted 44 lines · Jiri Olsa
Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa
Adding support for
bpf_get_func_iphelper being called from ebpf program attached byuprobe_multilink.Quoted 8 lines
It returns the ip of the uprobe.
Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)A slight aside related to
bpf_get_func_ip()support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and usingbpf_get_func_ip()there with uretprobes, and it seems like it doesn't work.Quoted 5 lines
Is that intentional or we just missed that
bpf_get_func_ip()doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them forbpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol
we use
KPROBE_FLAG_ON_FUNC_ENTRYflag for that in kprobe'sget_func_iphelper version:BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { struct kprobe *kp = kprobe_running(); if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY)) return 0; return get_entry_ip((uintptr_t)kp->addr); }
Quoted 2 lines
I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel
what if we just return whatever was the IP where uprobe/uretprobe interrupt instruction was installed at? I haven't tried what does regs->ip contain for u*ret*probe, though, if it already has the IP where we installed uprobe itself, it might be ok as is. But still, feels like it would be nice to have bpf_get_func_ip() working for uprobe/uretprobe (even if semantics might differ for uprobes not at the entry of the function).
Quoted 19 lines
I guess the app needs to store
regs->ipand match it against symbol addresses in user spacejirka
Quoted 12 lines
Thanks!
Quoted 7 lines
kernel/trace/bpf_trace.c +0
-0@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx);[...]
Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link
Quoted 45 lines · Andrii Nakryiko
Quoted 44 lines · Jiri Olsa
Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa
Adding support for
bpf_get_func_iphelper being called from ebpf program attached byuprobe_multilink.Quoted 8 lines
It returns the ip of the uprobe.
Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)A slight aside related to
bpf_get_func_ip()support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and usingbpf_get_func_ip()there with uretprobes, and it seems like it doesn't work.Quoted 5 lines
Is that intentional or we just missed that
bpf_get_func_ip()doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them forbpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol
we use
KPROBE_FLAG_ON_FUNC_ENTRYflag for that in kprobe'sget_func_iphelper version:BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { struct kprobe *kp = kprobe_running(); if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY)) return 0; return get_entry_ip((uintptr_t)kp->addr); }
Quoted 7 lines
Quoted 2 lines
I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel
what if we just return whatever was the IP where uprobe/uretprobe interrupt instruction was installed at? I haven't tried what does
regs->ipcontain for u*ret*probe, though, if it already has the IP where we installed uprobe itself, it might be ok as is. But still,
the entry uprobe gets the IP of the probed instruction, but the uretprobe gets IP of the next instruction after probed func retun - like 4c57f0 in the example below when uprobe is placed on trigger_func_get_func_ip
4c57eb: e8 4b fe ff ff call 4c563b <trigger_func_get_func_ip> 4c57f0: eb 01 jmp 4c57f3 <test_uprobe+0x1b1>
Quoted 3 lines
feels like it would be nice to have
bpf_get_func_ip()working for uprobe/uretprobe (even if semantics might differ for uprobes not at the entry of the function).
but it looks like we actually have the original uprobe address stored in current->utask->vaddr before bpf program is executed (in uretprobe_dispatcher) so we should be able to get that address from there in uretprobe's get_func_ip helper function, I'll check
I don't mind the semantic change for uprobe's get_func_ip, because I think it's unlikely we will be able to change it in future to return the real function entry
jirka
Quoted 20 lines
Quoted 20 lines
I guess the app needs to store
regs->ipand match it against symbol addresses in user spacejirka
Quoted 12 lines
Thanks!
Quoted 7 lines
kernel/trace/bpf_trace.c +0
-0@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx);[...]
Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link
Quoted 62 lines · Jiri Olsa
Quoted 45 lines · Andrii Nakryiko
Quoted 44 lines · Jiri Olsa
Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa
Adding support for
bpf_get_func_iphelper being called from ebpf program attached byuprobe_multilink.Quoted 8 lines
It returns the ip of the uprobe.
Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)A slight aside related to
bpf_get_func_ip()support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and usingbpf_get_func_ip()there with uretprobes, and it seems like it doesn't work.Quoted 5 lines
Is that intentional or we just missed that
bpf_get_func_ip()doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them forbpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol
we use
KPROBE_FLAG_ON_FUNC_ENTRYflag for that in kprobe'sget_func_iphelper version:BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) { struct kprobe *kp = kprobe_running(); if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY)) return 0; return get_entry_ip((uintptr_t)kp->addr); }Quoted 7 lines
Quoted 2 lines
I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel
what if we just return whatever was the IP where uprobe/uretprobe interrupt instruction was installed at? I haven't tried what does
regs->ipcontain for u*ret*probe, though, if it already has the IP where we installed uprobe itself, it might be ok as is. But still,the entry uprobe gets the IP of the probed instruction, but the uretprobe gets IP of the next instruction after probed func retun - like 4c57f0 in the example below when uprobe is placed on
trigger_func_get_func_ip4c57eb: e8 4b fe ff ff call 4c563b <trigger_func_get_func_ip> 4c57f0: eb 01 jmp 4c57f3 <test_uprobe+0x1b1>
Quoted 8 lines
Quoted 3 lines
feels like it would be nice to have
bpf_get_func_ip()working for uprobe/uretprobe (even if semantics might differ for uprobes not at the entry of the function).but it looks like we actually have the original uprobe address stored in
current->utask->vaddrbefore bpf program is executed (inuretprobe_dispatcher) so we should be able to get that address from there in uretprobe'sget_func_iphelper function, I'll check
ok, sounds great, seems like we have a pretty logical behavior for both uprobe and uretprobe.
Quoted 5 lines
I don't mind the semantic change for uprobe's
get_func_ip, because I think it's unlikely we will be able to change it in future to return the real function entry
Yep, I don't think kernel can easily know where the function even begins. But that's ok. We can clarify helper's behavior in UAPI
comments: uprobe/uretprobe always work, but return not function IP, but the traced location. Thanks!
Quoted 22 lines
jirka
Quoted 20 lines
Quoted 20 lines
I guess the app needs to store
regs->ipand match it against symbol addresses in user spacejirka
Quoted 12 lines
Thanks!
Quoted 7 lines
kernel/trace/bpf_trace.c +0
-0@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx);[...]
Jiri Olsa
reviewerPATCH v3 6/26 libbpf: Add uprobe_multi attach type and link names
In reply to Jiri OlsaAdding new uprobe_multi attach type and link names, so the functions can resolve the new values.
--- tools/lib/bpf/libbpf.c | 2 ++ 1 file changed, 2 insertions(+)
tools/lib/bpf/libbpf.c +2 -0
@@ -118,6 +118,7 @@ static const char * const attach_type_name[] = { [BPF_TRACE_KPROBE_MULTI] = "trace_kprobe_multi", [BPF_STRUCT_OPS] = "struct_ops", [BPF_NETFILTER] = "netfilter", + [BPF_TRACE_UPROBE_MULTI] = "trace_uprobe_multi", };static const char *const link_type_name[] = {
@@ -132,6 +133,7 @@ static const char * const link_type_name[] = { [BPF_LINK_TYPE_KPROBE_MULTI] = "kprobe_multi", [BPF_LINK_TYPE_STRUCT_OPS] = "struct_ops", [BPF_LINK_TYPE_NETFILTER] = "netfilter", + [BPF_LINK_TYPE_UPROBE_MULTI] = "uprobe_multi", };static const char *const map_type_name[] = {
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v3 7/26 libbpf: Move elf_find_func_offset* functions to elf object
In reply to Jiri OlsaAdding new elf object that will contain elf related functions. There's no functional change.
--- tools/lib/bpf/Build | 2 +- tools/lib/bpf/elf.c | 198 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 186 +--------------------------------- tools/lib/bpf/libbpf_elf.h | 11 +++ 4 files changed, 211 insertions(+), 186 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/lib/bpf/libbpf_elf.h
tools/lib/bpf/Build +1 -1
@@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o + usdt.o zip.o elf.o
tools/lib/bpf/elf.c new file +198 -0
198 lines not on this page · raw message
tools/lib/bpf/libbpf.c +1 -185
198 lines not on this page · raw message
tools/lib/bpf/libbpf_elf.h new file +11 -0
@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ + +#ifndef __LIBBPF_LIBBPF_ELF_H +#define __LIBBPF_LIBBPF_ELF_H + +#include <libelf.h> + +long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name); +long elf_find_func_offset_from_file (const char *binary_path,const char *name); + +#endif /* *__LIBBPF_LIBBPF_ELF_H */
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object
Quoted 15 lines · Jiri Olsa
Adding new elf object that will contain elf related functions. There's no functional change.
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/Build | 2 +- tools/lib/bpf/elf.c | 198 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 186 +--------------------------------- tools/lib/bpf/libbpf_elf.h | 11 +++ 4 files changed, 211 insertions(+), 186 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/lib/bpf/libbpf_elf.h
[...]
Quoted 7 lines
tools/lib/bpf/libbpf_elf.h new file +1
-0@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
tools/lib/bpf/libbpf_elf.h line 11
Quoted 10 lines
tools/lib/bpf/libbpf_elf.h new file +10
-0+ +#ifndef __LIBBPF_LIBBPF_ELF_H +#define __LIBBPF_LIBBPF_ELF_H + +#include <libelf.h> + +long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name); +long elf_find_func_offset_from_file (const char *binary_path,const char *name); + +#endif /* *__LIBBPF_LIBBPF_ELF_H */
we have libbpf_internal.h, let's put all this there for now, it's already all the internal stuff together, I don't know if separate header with few functions gives us much
Quoted 3 lines
--
2.41.0
Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object
Quoted 32 lines · Andrii Nakryiko
Quoted 15 lines · Jiri Olsa
Adding new elf object that will contain elf related functions. There's no functional change.
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/Build | 2 +- tools/lib/bpf/elf.c | 198 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 186 +--------------------------------- tools/lib/bpf/libbpf_elf.h | 11 +++ 4 files changed, 211 insertions(+), 186 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/lib/bpf/libbpf_elf.h[...]
Quoted 13 lines
tools/lib/bpf/libbpf_elf.h new file +7
-0@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ + +#ifndef __LIBBPF_LIBBPF_ELF_H +#define __LIBBPF_LIBBPF_ELF_H + +#include <libelf.h> +
Quoted 8 lines
tools/lib/bpf/libbpf_elf.h line 11
Quoted 4 lines
tools/lib/bpf/libbpf_elf.h new file +4
-0+long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name); +long elf_find_func_offset_from_file (const char *binary_path,const char *name); + +#endif /* *__LIBBPF_LIBBPF_ELF_H */ we have
libbpf_internal.h, let's put all this there for now, it's already all the internal stuff together, I don't know if separate header with few functions gives us much
there's more functions coming later in the patchset
struct elf_fd {
Elf *elf;
int fd;
};
int elf_open(const char *binary_path, struct elf_fd *elf_fd);
void elf_close(struct elf_fd *elf_fd);
int elf_resolve_syms_offsets(const char *binary_path, int cnt,
const char **syms, unsigned long **poffsets);
int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
unsigned long **poffsets, size_t *pcnt);and there's probably more elf helpers to eventually move in:
libbpf.c:static const char *elf_sym_str(const struct bpf_object *obj, size_t off); libbpf.c:static const char *elf_sec_str(const struct bpf_object *obj, size_t off); libbpf.c:static Elf_Scn *elf_sec_by_idx(const struct bpf_object *obj, size_t idx); libbpf.c:static Elf_Scn *elf_sec_by_name(const struct bpf_object *obj, const char *name); libbpf.c:static Elf64_Shdr *elf_sec_hdr(const struct bpf_object *obj, Elf_Scn *scn); libbpf.c:static const char *elf_sec_name(const struct bpf_object *obj, Elf_Scn *scn); libbpf.c:static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn); libbpf.c:static Elf64_Sym *elf_sym_by_idx(const struct bpf_object *obj, size_t idx); libbpf.c:static Elf64_Rel *elf_rel_by_idx(Elf_Data *data, size_t idx); usdt.c:static int find_elf_sec_by_name(Elf *elf, const char *sec_name, GElf_Shdr *shdr, Elf_Scn **scn) 'struct elf_seg' stuff usdt.c:static int cmp_elf_segs(const void *_a, const void *_b) usdt.c:static int parse_elf_segs(Elf *elf, const char *path, struct elf_seg **segs, size_t *seg_cnt) usdt.c:static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs, size_t *seg_cnt) usdt.c:static struct elf_seg *find_elf_seg(struct elf_seg *segs, size_t seg_cnt, long virtaddr) usdt.c:static struct elf_seg *find_vma_seg(struct elf_seg *segs, size_t seg_cnt, long offset)
but I can add the new header file later in follow up changes when we have more elf functions in
jirka
Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object
Quoted 62 lines · Jiri Olsa
Quoted 32 lines · Andrii Nakryiko
Quoted 15 lines · Jiri Olsa
Adding new elf object that will contain elf related functions. There's no functional change.
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/Build | 2 +- tools/lib/bpf/elf.c | 198 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 186 +--------------------------------- tools/lib/bpf/libbpf_elf.h | 11 +++ 4 files changed, 211 insertions(+), 186 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/lib/bpf/libbpf_elf.h[...]
Quoted 13 lines
tools/lib/bpf/libbpf_elf.h new file +7
-0@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ + +#ifndef __LIBBPF_LIBBPF_ELF_H +#define __LIBBPF_LIBBPF_ELF_H + +#include <libelf.h> +Quoted 8 lines
tools/lib/bpf/libbpf_elf.h line 11
Quoted 4 lines
tools/lib/bpf/libbpf_elf.h new file +4
-0+long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name); +long elf_find_func_offset_from_file (const char *binary_path,const char *name); + +#endif /* *__LIBBPF_LIBBPF_ELF_H */ we have
libbpf_internal.h, let's put all this there for now, it's already all the internal stuff together, I don't know if separate header with few functions gives us muchthere's more functions coming later in the patchset
struct elf_fd { Elf *elf; int fd; }; int elf_open(const char *binary_path, struct elf_fd *elf_fd); void elf_close(struct elf_fd *elf_fd); int elf_resolve_syms_offsets(const char *binary_path, int cnt, const char **syms, unsigned long **poffsets); int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern, unsigned long **poffsets, size_t *pcnt);and there's probably more elf helpers to eventually move in:
Quoted 10 lines
libbpf.c:static const char *elf_sym_str(const struct bpf_object *obj, size_t off);
libbpf.c:static const char *elf_sec_str(const struct bpf_object *obj, size_t off);
libbpf.c:static Elf_Scn *elf_sec_by_idx(const struct bpf_object *obj, size_t idx);
libbpf.c:static Elf_Scn *elf_sec_by_name(const struct bpf_object *obj, const char *name);
libbpf.c:static Elf64_Shdr *elf_sec_hdr(const struct bpf_object *obj, Elf_Scn *scn);
libbpf.c:static const char *elf_sec_name(const struct bpf_object *obj, Elf_Scn *scn);
libbpf.c:static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn);
libbpf.c:static Elf64_Sym *elf_sym_by_idx(const struct bpf_object *obj, size_t idx);
libbpf.c:static Elf64_Rel *elf_rel_by_idx(Elf_Data *data, size_t idx);yep, I was anticipating that these will move as well. But I think it's fine if they all stay in libbpf_internal.h, IMO. I'd rather not have many small internal header for no good reason (like we have str_error.h right now, with a single func declaration)
Quoted 11 lines
usdt.c:static int
find_elf_sec_by_name(Elf *elf, const char *sec_name, GElf_Shdr *shdr, Elf_Scn **scn)'struct
elf_seg' stuffusdt.c:static int
cmp_elf_segs(const void *_a, const void *_b)
usdt.c:static intparse_elf_segs(Elf *elf, const char *path, struct elf_seg **segs, size_t *seg_cnt)usdt.c:static intparse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs, size_t *seg_cnt)usdt.c:static structelf_seg*find_elf_seg(structelf_seg*segs,size_tseg_cnt, long virtaddr) usdt.c:static structelf_seg*find_vma_seg(structelf_seg*segs,size_tseg_cnt, long offset)
Quoted 2 lines
but I can add the new header file later in follow up changes when we have more elf functions in
see above, I don't think we should, let's stick to libbpf_internal.h for now
Quoted 2 lines
jirka
Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object
In reply to Jiri OlsaQuoted 31 lines · Jiri Olsa
Adding new elf object that will contain elf related functions. There's no functional change.
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/Build | 2 +- tools/lib/bpf/elf.c | 198 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 186 +--------------------------------- tools/lib/bpf/libbpf_elf.h | 11 +++ 4 files changed, 211 insertions(+), 186 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/lib/bpf/libbpf_elf.htools/lib/bpf/Build +1
-1@@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o + usdt.o zip.o elf.otools/lib/bpf/elf.c new file +0
-0
tools/lib/bpf/elf.c line 6
Quoted 6 lines
tools/lib/bpf/elf.c new file +6
-0@@ -0,0 +1,198 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +#include <libelf.h> +#include <gelf.h> +#include <fcntl.h> +#include <linux/kernel.h>
do you know why we need linux/kernel.h include? is it to get __u32 and other typedefs?
Quoted 7 lines
tools/lib/bpf/elf.c new file +7
-0+ +#include "libbpf_elf.h" +#include "libbpf_internal.h" +#include "str_error.h" + +#define STRERR_BUFSIZE 128 +
[...]
Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object
Quoted 33 lines · Andrii Nakryiko
Quoted 32 lines · Jiri Olsa
Adding new elf object that will contain elf related functions. There's no functional change.
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/Build | 2 +- tools/lib/bpf/elf.c | 198 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 186 +--------------------------------- tools/lib/bpf/libbpf_elf.h | 11 +++ 4 files changed, 211 insertions(+), 186 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/lib/bpf/libbpf_elf.htools/lib/bpf/Build +1
-1@@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o + usdt.o zip.o elf.otools/lib/bpf/elf.c new file +1
-0@@ -0,0 +1,198 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
Quoted 8 lines
tools/lib/bpf/elf.c line 6
Quoted 5 lines
tools/lib/bpf/elf.c new file +5
-0+ +#include <libelf.h> +#include <gelf.h> +#include <fcntl.h> +#include <linux/kernel.h> do you know why we need
linux/kernel.hinclude? is it to get__u32and other typedefs?
it's for the ARRAY_SIZE macro
jirka
Quoted 10 lines
Quoted 8 lines
tools/lib/bpf/elf.c new file +7
-0+ +#include "libbpf_elf.h" +#include "libbpf_internal.h" +#include "str_error.h" + +#define STRERR_BUFSIZE 128 +[...]
Jiri Olsa
reviewerPATCH v3 8/26 libbpf: Add elf_open/elf_close functions
In reply to Jiri OlsaAdding elf_open/elf_close functions and using it in elf_find_func_offset_from_file function. It will be used in following changes to save some common code.
--- tools/lib/bpf/elf.c | 59 +++++++++++++++++++++++++------------- tools/lib/bpf/libbpf_elf.h | 8 ++++++ tools/lib/bpf/usdt.c | 31 ++++++-------------- 3 files changed, 56 insertions(+), 42 deletions(-)
tools/lib/bpf/elf.c +39 -20
@@ -11,6 +11,40 @@#define STRERR_BUFSIZE 128 +int elf_open (const char *binary_path,struct elf_fd *elf_fd) +{ +char errmsg[STRERR_BUFSIZE ]; +int fd, ret; +Elf *elf; + +if (elf_version (EV_CURRENT ) ==EV_NONE ) { +pr_warn ("elf: failed to init libelf for %s\n" , binary_path); +return -LIBBPF_ERRNO__LIBELF ; + } + fd =open (binary_path,O_RDONLY |O_CLOEXEC ); +if (fd <0 ) { + ret = -errno; +pr_warn ("elf: failed to open %s: %s\n" , binary_path, +libbpf_strerror_r (ret, errmsg,sizeof (errmsg))); +return ret; + } + elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); +if (!elf) { +pr_warn ("elf: could not read elf from %s: %s\n" , binary_path,elf_errmsg (-1 )); +close (fd); +return -LIBBPF_ERRNO__FORMAT ; + } + elf_fd->fd = fd; + elf_fd->elf = elf; +return 0 ; +} + +void elf_close (struct elf_fd *elf_fd) +{ +elf_end (elf_fd->elf); +close (elf_fd->fd); +} +/* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */ static Elf_Scn *elf_find_next_scn_by_type (Elf *elf,int sh_type,Elf_Scn *scn) {
@@ -171,28 +205,13 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)*/ long elf_find_func_offset_from_file (const char *binary_path,const char *name) { -char errmsg[STRERR_BUFSIZE ]; +struct elf_fd elf_fd;long ret = -ENOENT ; -Elf *elf; -int fd; - fd =open (binary_path,O_RDONLY |O_CLOEXEC ); -if (fd <0 ) { - ret = -errno; -pr_warn ("failed to open %s: %s\n" , binary_path, -libbpf_strerror_r (ret, errmsg,sizeof (errmsg))); + ret =elf_open (binary_path, &elf_fd); +if (ret)return ret; - } - elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); -if (!elf) { -pr_warn ("elf: could not read elf from %s: %s\n" , binary_path,elf_errmsg (-1 )); -close (fd); -return -LIBBPF_ERRNO__FORMAT ; - } - - ret =elf_find_func_offset (elf, binary_path, name); -elf_end (elf); -close (fd); + ret =elf_find_func_offset (elf_fd.elf, binary_path, name); +elf_close (&elf_fd);return ret; } -
tools/lib/bpf/libbpf_elf.h +8 -0
@@ -5,6 +5,14 @@#include <libelf.h> +struct elf_fd { +Elf *elf; +int fd; +}; + +int elf_open (const char *binary_path,struct elf_fd *elf_fd); +void elf_close (struct elf_fd *elf_fd); +long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name);long elf_find_func_offset_from_file (const char *binary_path,const char *name);
tools/lib/bpf/usdt.c +9 -22
@@ -19,6 +19,7 @@#include "libbpf.h" #include "libbpf_common.h" #include "libbpf_internal.h" +#include "libbpf_elf.h" #include "hashmap.h" /* libbpf's USDT support consists of BPF-side state/code and user-space
@@ -943,32 +944,22 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct const char *usdt_provider, const char *usdt_name, __u64 usdt_cookie) { - int i, fd, err, spec_map_fd, ip_map_fd; + int i, err, spec_map_fd, ip_map_fd;LIBBPF_OPTS (bpf_uprobe_opts, opts);struct hashmap *specs_hash =NULL ;struct bpf_link_usdt *link =NULL ;struct usdt_target *targets =NULL ; +struct elf_fd elf_fd;size_t target_cnt; -Elf *elf; spec_map_fd =bpf_map__fd (man->specs_map); ip_map_fd =bpf_map__fd (man->ip_to_spec_id_map); - fd =open (path,O_RDONLY |O_CLOEXEC ); -if (fd <0 ) { - err = -errno; -pr_warn ("usdt: failed to open ELF binary '%s': %d\n" , path, err); + err =elf_open (path, &elf_fd); +if (err)return libbpf_err_ptr (err); - } - elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); -if (!elf) { - err = -EBADF ; -pr_warn ("usdt: failed to parse ELF binary '%s': %s\n" , path,elf_errmsg (-1 )); - goto err_out; - } - - err =sanity_check_usdt_elf (elf, path); + err =sanity_check_usdt_elf (elf_fd.elf, path);if (err) goto err_out;
@@ -981,7 +972,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct/* discover USDT in given binary, optionally limiting * activations to a given PID, if pid > 0 */ - err =collect_usdt_targets (man, elf, path, pid, usdt_provider, usdt_name, + err =collect_usdt_targets (man, elf_fd.elf, path, pid, usdt_provider, usdt_name, usdt_cookie, &targets, &target_cnt);if (err <=0 ) { err = (err ==0 ) ? -ENOENT : err;
@@ -1066,9 +1057,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const structfree (targets);hashmap__free (specs_hash); -elf_end (elf); -close (fd); - +elf_close (&elf_fd);return &link->link; err_out:
@@ -1076,9 +1065,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const structbpf_link__destroy (&link->link);free (targets);hashmap__free (specs_hash); -if (elf) -elf_end (elf); -close (fd); +elf_close (&elf_fd);return libbpf_err_ptr (err); }
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 08/26] libbpf: Add elf_open/elf_close functions
Quoted 5 lines · Jiri Olsa
Adding elf_open/elf_close functions and using it in
elf_find_func_offset_from_filefunction. It will be used in following changes to save some common code.
Quoted 7 lines
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 59 +++++++++++++++++++++++++------------- tools/lib/bpf/libbpf_elf.h | 8 ++++++ tools/lib/bpf/usdt.c | 31 ++++++-------------- 3 files changed, 56 insertions(+), 42 deletions(-)
one nit below
Quoted 30 lines
tools/lib/bpf/elf.c +22
-0@@ -11,6 +11,40 @@#define STRERR_BUFSIZE 128 +int elf_open (const char *binary_path,struct elf_fd *elf_fd) +{ +char errmsg[STRERR_BUFSIZE ]; +int fd, ret; +Elf *elf; + +if (elf_version (EV_CURRENT ) ==EV_NONE ) { +pr_warn ("elf: failed to init libelf for %s\n" , binary_path); +return -LIBBPF_ERRNO__LIBELF ; + } + fd =open (binary_path,O_RDONLY |O_CLOEXEC ); +if (fd <0 ) { + ret = -errno; +pr_warn ("elf: failed to open %s: %s\n" , binary_path, +libbpf_strerror_r (ret, errmsg,sizeof (errmsg))); +return ret; + } + elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); +if (!elf) { +pr_warn ("elf: could not read elf from %s: %s\n" , binary_path,elf_errmsg (-1 )); +close (fd); +return -LIBBPF_ERRNO__FORMAT ;
tools/lib/bpf/elf.c line 45
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+ } + elf_fd->fd = fd; + elf_fd->elf = elf; +return 0 ; +} + +void elf_close (struct elf_fd *elf_fd) +{ +elf_end (elf_fd->elf); +close (elf_fd->fd);
nit: I'd make elf_close() work correctly with a) NULL elf_fd and b) NULL elf_fd->elf, just to never have to think about this
Quoted 5 lines
tools/lib/bpf/elf.c +2
-0+} +/* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */ static Elf_Scn *elf_find_next_scn_by_type (Elf *elf,int sh_type,Elf_Scn *scn) {
[...]
Re: [PATCHv3 bpf-next 08/26] libbpf: Add elf_open/elf_close functions
Quoted 52 lines · Andrii Nakryiko
Quoted 5 lines · Jiri Olsa
Adding elf_open/elf_close functions and using it in
elf_find_func_offset_from_filefunction. It will be used in following changes to save some common code.Quoted 7 lines
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 59 +++++++++++++++++++++++++------------- tools/lib/bpf/libbpf_elf.h | 8 ++++++ tools/lib/bpf/usdt.c | 31 ++++++-------------- 3 files changed, 56 insertions(+), 42 deletions(-)one nit below
Acked-by Andrii Nakryiko <[email protected]>Quoted 34 lines
tools/lib/bpf/elf.c +26
-0@@ -11,6 +11,40 @@#define STRERR_BUFSIZE 128 +int elf_open (const char *binary_path,struct elf_fd *elf_fd) +{ +char errmsg[STRERR_BUFSIZE ]; +int fd, ret; +Elf *elf; + +if (elf_version (EV_CURRENT ) ==EV_NONE ) { +pr_warn ("elf: failed to init libelf for %s\n" , binary_path); +return -LIBBPF_ERRNO__LIBELF ; + } + fd =open (binary_path,O_RDONLY |O_CLOEXEC ); +if (fd <0 ) { + ret = -errno; +pr_warn ("elf: failed to open %s: %s\n" , binary_path, +libbpf_strerror_r (ret, errmsg,sizeof (errmsg))); +return ret; + } + elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); +if (!elf) { +pr_warn ("elf: could not read elf from %s: %s\n" , binary_path,elf_errmsg (-1 )); +close (fd); +return -LIBBPF_ERRNO__FORMAT ; + } + elf_fd->fd = fd; + elf_fd->elf = elf; +return 0 ;
Quoted 8 lines
tools/lib/bpf/elf.c line 45
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+} + +void elf_close (struct elf_fd *elf_fd) +{ +elf_end (elf_fd->elf); +close (elf_fd->fd);nit: I'd make
elf_close()work correctly with a) NULLelf_fdand b)
right, ok
Quoted 1 line
NULL
elf_fd->elf, just to never have to think about this
there's NULL check in elf_end
thanks,
jirka
Jiri Olsa
reviewerPATCH v3 9/26 libbpf: Add elf symbol iterator
In reply to Jiri OlsaAdding elf symbol iterator object (and some functions) that follow open-coded iterator pattern and some functions to ease up iterating elf object symbols.
The idea is to iterate single symbol section with:
struct elf_sym_iter iter;
struct elf_sym *sym;
if (elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM))
goto error;
while ((sym = elf_sym_iter_next(&iter))) {
...
}I considered opening the elf inside the iterator and iterate all symbol sections, but then it gets more complicated wrt user checks for when the next section is processed.
Plus side is the we don't need 'exit' function, because caller/user is in charge of that.
The returned iterated symbol object from elf_sym_iter_next function is placed inside the struct elf_sym_iter, so no extra allocation or argument is needed.
--- tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++--------------- 1 file changed, 117 insertions(+), 61 deletions(-)
tools/lib/bpf/elf.c +117 -61
@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)return NULL ; } +struct elf_sym { +const char *name; +GElf_Sym sym; +GElf_Shdr sh; +}; + +struct elf_sym_iter { +Elf *elf; +Elf_Data *syms; +size_t nr_syms; +size_t strtabidx; +size_t next_sym_idx; +struct elf_sym sym; +int st_type; +}; + +static int elf_sym_iter_new (struct elf_sym_iter *iter, +Elf *elf,const char *binary_path, +int sh_type,int st_type) +{ +Elf_Scn *scn =NULL ; +GElf_Ehdr ehdr; +GElf_Shdr sh; + +memset (iter,0 ,sizeof (*iter)); + +if (!gelf_getehdr (elf, &ehdr)) { +pr_warn ("elf: failed to get ehdr from %s: %s\n" , binary_path,elf_errmsg (-1 )); +return -EINVAL ; + } + + scn =elf_find_next_scn_by_type (elf, sh_type,NULL ); +if (!scn) { +pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , + binary_path); +return -ENOENT ; + } + +if (!gelf_getshdr (scn, &sh)) +return -EINVAL ; + + iter->strtabidx = sh.sh_link; + iter->syms =elf_getdata (scn,0 ); +if (!iter->syms) { +pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , + binary_path,elf_errmsg (-1 )); +return -EINVAL ; + } + iter->nr_syms = iter->syms->d_size / sh.sh_entsize; + iter->elf = elf; + iter->st_type = st_type; +return 0 ; +} + +static struct elf_sym *elf_sym_iter_next (struct elf_sym_iter *iter) +{ +struct elf_sym *ret = &iter->sym; +GElf_Sym *sym = &ret->sym; +const char *name =NULL ; +Elf_Scn *sym_scn; +size_t idx; + +for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) { +if (!gelf_getsym (iter->syms, idx, sym)) +continue ; +if (GELF_ST_TYPE (sym->st_info) != iter->st_type) +continue ; + name =elf_strptr (iter->elf, iter->strtabidx, sym->st_name); +if (!name) +continue ; + +/* Transform symbol's virtual address (absolute for +* binaries and relative for shared libs) into file +* offset, which is what kernel is expecting for +* uprobe/uretprobe attachment. +* See Documentation/trace/uprobetracer.rst for more +* details. +* This is done by looking up symbol's containing +* section's header and using iter's virtual address +* (sh_addr) and corresponding file offset (sh_offset) +* to transform sym.st_value (virtual address) into +* desired final file offset. +*/ + sym_scn =elf_getscn (iter->elf, sym->st_shndx); +if (!sym_scn) +continue ; +if (!gelf_getshdr (sym_scn, &ret->sh)) +continue ; + + iter->next_sym_idx = idx +1 ; + ret->name = name; +return ret; + } + +return NULL ; +} + +static unsigned long elf_sym_offset (struct elf_sym *sym) +{ +return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset; +} +/* Find offset of function name in the provided ELF object. "binary_path" is * the path to the ELF binary represented by "elf", and only used for error * reporting matters. "name" matches symbol name or name@@LIB for library
@@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)* reported as a warning/error. */ for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { -size_t nr_syms, strtabidx, idx; -Elf_Data *symbols =NULL ; -Elf_Scn *scn =NULL ; +struct elf_sym_iter iter; +struct elf_sym *sym;int last_bind =-1 ; -const char *sname; -GElf_Shdr sh; +int curr_bind; - scn =elf_find_next_scn_by_type (elf, sh_types[i],NULL ); -if (!scn) { -pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , - binary_path); -continue ; - } -if (!gelf_getshdr (scn, &sh)) -continue ; - strtabidx = sh.sh_link; - symbols =elf_getdata (scn,0 ); -if (!symbols) { -pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , - binary_path,elf_errmsg (-1 )); - ret = -LIBBPF_ERRNO__FORMAT ; + ret =elf_sym_iter_new (&iter, elf, binary_path, sh_types[i],STT_FUNC ); +if (ret) { +if (ret == -ENOENT ) +continue ; goto out; } - nr_syms = symbols->d_size / sh.sh_entsize; - -for (idx =0 ; idx < nr_syms; idx++) { -int curr_bind; -GElf_Sym sym; -Elf_Scn *sym_scn; -GElf_Shdr sym_sh; - -if (!gelf_getsym (symbols, idx, &sym)) -continue ; - -if (GELF_ST_TYPE (sym.st_info) !=STT_FUNC ) -continue ; - - sname =elf_strptr (elf, strtabidx, sym.st_name); -if (!sname) -continue ; - - curr_bind =GELF_ST_BIND (sym.st_info); +while ((sym =elf_sym_iter_next (&iter))) {/* User can specify func, func@@LIB or func@@LIB_VERSION. */ -if (strncmp (sname, name, name_len) !=0 ) +if (strncmp (sym->name, name, name_len) !=0 )continue ;/* ...but we don't want a search for "foo" to match 'foo2" also, so any * additional characters in sname should be of the form "@@LIB". */ -if (!is_name_qualified && sname[name_len] !='\0' && sname[name_len] !='@' ) +if (!is_name_qualified && sym->name[name_len] !='\0' && sym->name[name_len] !='@' )continue ; -if (ret >=0 ) { + curr_bind =GELF_ST_BIND (sym->sym.st_info); + +if (ret >0 ) {/* handle multiple matches */ if (last_bind !=STB_WEAK && curr_bind !=STB_WEAK ) {/* Only accept one non-weak bind. */ pr_warn ("elf: ambiguous match for '%s', '%s' in '%s'\n" , - sname, name, binary_path); + sym->name, name, binary_path); ret = -LIBBPF_ERRNO__FORMAT ; goto out; }else if (curr_bind ==STB_WEAK ) {
@@ -158,25 +232,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) } } -/* Transform symbol's virtual address (absolute for -* binaries and relative for shared libs) into file -* offset, which is what kernel is expecting for -* uprobe/uretprobe attachment. -* See Documentation/trace/uprobetracer.rst for more -* details. -* This is done by looking up symbol's containing -* section's header and using it's virtual address -* (sh_addr) and corresponding file offset (sh_offset) -* to transform sym.st_value (virtual address) into -* desired final file offset. -*/ - sym_scn =elf_getscn (elf, sym.st_shndx); -if (!sym_scn) -continue ; -if (!gelf_getshdr (sym_scn, &sym_sh)) -continue ; - - ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset; + ret =elf_sym_offset (sym); last_bind = curr_bind; }if (ret >0 )
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator
Quoted 28 lines · Jiri Olsa
Adding elf symbol iterator object (and some functions) that follow open-coded iterator pattern and some functions to ease up iterating elf object symbols.
The idea is to iterate single symbol section with:
struct
elf_sym_iteriter;
structelf_sym*sym;if (
elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM))goto error;while ((sym = elf_sym_iter_next(&iter))) { ... }I considered opening the elf inside the iterator and iterate all symbol sections, but then it gets more complicated wrt user checks for when the next section is processed.
Plus side is the we don't need 'exit' function, because caller/user is in charge of that.
The returned iterated symbol object from
elf_sym_iter_nextfunction is placed inside the structelf_sym_iter, so no extra allocation or argument is needed.
Quoted 6 lines
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++--------------- 1 file changed, 117 insertions(+), 61 deletions(-)
A bunch of nits, but overall looks good. Please address nits, and add my ack
Quoted 5 lines
tools/lib/bpf/elf.c +0
-0
tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 67
Quoted 9 lines
tools/lib/bpf/elf.c +6
-0@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)return NULL ; } +struct elf_sym { +const char *name; +GElf_Sym sym; +GElf_Shdr sh; +}; +
if we want to use elf_sym_iter outside of elf.c, this should be in libbpf_internal.h?
Quoted 67 lines
tools/lib/bpf/elf.c +67
-0+struct elf_sym_iter { +Elf *elf; +Elf_Data *syms; +size_t nr_syms; +size_t strtabidx; +size_t next_sym_idx; +struct elf_sym sym; +int st_type; +}; + +static int elf_sym_iter_new (struct elf_sym_iter *iter, +Elf *elf,const char *binary_path, +int sh_type,int st_type) +{ +Elf_Scn *scn =NULL ; +GElf_Ehdr ehdr; +GElf_Shdr sh; + +memset (iter,0 ,sizeof (*iter)); + +if (!gelf_getehdr (elf, &ehdr)) { +pr_warn ("elf: failed to get ehdr from %s: %s\n" , binary_path,elf_errmsg (-1 )); +return -EINVAL ; + } + + scn =elf_find_next_scn_by_type (elf, sh_type,NULL ); +if (!scn) { +pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , + binary_path); +return -ENOENT ; + } + +if (!gelf_getshdr (scn, &sh)) +return -EINVAL ; + + iter->strtabidx = sh.sh_link; + iter->syms =elf_getdata (scn,0 ); +if (!iter->syms) { +pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , + binary_path,elf_errmsg (-1 )); +return -EINVAL ; + } + iter->nr_syms = iter->syms->d_size / sh.sh_entsize; + iter->elf = elf; + iter->st_type = st_type; +return 0 ; +} + +static struct elf_sym *elf_sym_iter_next (struct elf_sym_iter *iter) +{ +struct elf_sym *ret = &iter->sym; +GElf_Sym *sym = &ret->sym; +const char *name =NULL ; +Elf_Scn *sym_scn; +size_t idx; + +for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) { +if (!gelf_getsym (iter->syms, idx, sym)) +continue ; +if (GELF_ST_TYPE (sym->st_info) != iter->st_type) +continue ; + name =elf_strptr (iter->elf, iter->strtabidx, sym->st_name); +if (!name) +continue ; + +/* Transform symbol's virtual address (absolute for +* binaries and relative for shared libs) into file
tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 144
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+* offset, which is what kernel is expecting for +* uprobe/uretprobe attachment. +* See Documentation/trace/uprobetracer.rst for more +* details. +* This is done by looking up symbol's containing +* section's header and using iter's virtual address +* (sh_addr) and corresponding file offset (sh_offset) +* to transform sym.st_value (virtual address) into +* desired final file offset. +*/
this comment is misplaced? we don't do the translation here
Quoted 25 lines
tools/lib/bpf/elf.c +19
-0+ sym_scn =elf_getscn (iter->elf, sym->st_shndx); +if (!sym_scn) +continue ; +if (!gelf_getshdr (sym_scn, &ret->sh)) +continue ; + + iter->next_sym_idx = idx +1 ; + ret->name = name; +return ret; + } + +return NULL ; +} + +static unsigned long elf_sym_offset (struct elf_sym *sym) +{ +return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset; +} +/* Find offset of function name in the provided ELF object. "binary_path" is * the path to the ELF binary represented by "elf", and only used for error * reporting matters. "name" matches symbol name or name@@LIB for library @@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)* reported as a warning/error. */
tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 198
Quoted 10 lines
tools/lib/bpf/elf.c +3
-5for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { -size_t nr_syms, strtabidx, idx; -Elf_Data *symbols =NULL ; -Elf_Scn *scn =NULL ; +struct elf_sym_iter iter; +struct elf_sym *sym;int last_bind =-1 ; -const char *sname; -GElf_Shdr sh; +int curr_bind;
OCD nit:
$ rg 'curr(_|\b)' | wc -l 8 $ rg 'cur(_|\b)' | wc -l 148
and those 8 I consider an unfortunate accident ;) let's standardize on using "cur" consistently
Quoted 10 lines
tools/lib/bpf/elf.c +0
-9- scn =elf_find_next_scn_by_type (elf, sh_types[i],NULL ); -if (!scn) { -pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , - binary_path); -continue ; - } -if (!gelf_getshdr (scn, &sh)) -continue ; - strtabidx = sh.sh_link;
tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 204
Quoted 10 lines
tools/lib/bpf/elf.c +4
-5- symbols =elf_getdata (scn,0 ); -if (!symbols) { -pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , - binary_path,elf_errmsg (-1 )); - ret = -LIBBPF_ERRNO__FORMAT ; + ret =elf_sym_iter_new (&iter, elf, binary_path, sh_types[i],STT_FUNC ); +if (ret) { +if (ret == -ENOENT ) +continue ; goto out;
another styling nit: let's avoid unnecessary nesting of ifs:
if (ret == -ENOENT)
continue;
if (ret)
goto out;simple and clean
Quoted 27 lines
tools/lib/bpf/elf.c +2
-20} - nr_syms = symbols->d_size / sh.sh_entsize; - -for (idx =0 ; idx < nr_syms; idx++) { -int curr_bind; -GElf_Sym sym; -Elf_Scn *sym_scn; -GElf_Shdr sym_sh; - -if (!gelf_getsym (symbols, idx, &sym)) -continue ; - -if (GELF_ST_TYPE (sym.st_info) !=STT_FUNC ) -continue ; - - sname =elf_strptr (elf, strtabidx, sym.st_name); -if (!sname) -continue ; - - curr_bind =GELF_ST_BIND (sym.st_info); +while ((sym =elf_sym_iter_next (&iter))) {/* User can specify func, func@@LIB or func@@LIB_VERSION. */ -if (strncmp (sname, name, name_len) !=0 ) +if (strncmp (sym->name, name, name_len) !=0 )continue ;/* ...but we don't want a search for "foo" to match 'foo2" also, so any
tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 219
Quoted 10 lines
tools/lib/bpf/elf.c +4
-2* additional characters in sname should be of the form "@@LIB". */ -if (!is_name_qualified && sname[name_len] !='\0' && sname[name_len] !='@' ) +if (!is_name_qualified && sym->name[name_len] !='\0' && sym->name[name_len] !='@' )continue ; -if (ret >=0 ) { + curr_bind =GELF_ST_BIND (sym->sym.st_info); + +if (ret >0 ) {
used to be >=, why the change?
Quoted 39 lines
tools/lib/bpf/elf.c +2
-21/* handle multiple matches */ if (last_bind !=STB_WEAK && curr_bind !=STB_WEAK ) {/* Only accept one non-weak bind. */ pr_warn ("elf: ambiguous match for '%s', '%s' in '%s'\n" , - sname, name, binary_path); + sym->name, name, binary_path); ret = -LIBBPF_ERRNO__FORMAT ; goto out; }else if (curr_bind ==STB_WEAK ) {@@ -158,25 +232,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) } } -/* Transform symbol's virtual address (absolute for -* binaries and relative for shared libs) into file -* offset, which is what kernel is expecting for -* uprobe/uretprobe attachment. -* See Documentation/trace/uprobetracer.rst for more -* details. -* This is done by looking up symbol's containing -* section's header and using it's virtual address -* (sh_addr) and corresponding file offset (sh_offset) -* to transform sym.st_value (virtual address) into -* desired final file offset. -*/ - sym_scn =elf_getscn (elf, sym.st_shndx); -if (!sym_scn) -continue ; -if (!gelf_getshdr (sym_scn, &sym_sh)) -continue ; - - ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset; + ret =elf_sym_offset (sym); last_bind = curr_bind; }if (ret >0 ) --2.41.0
Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator
On Thu, Jul 06, 2023 at 04:24:48PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 19 lines
Quoted 6 lines
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++--------------- 1 file changed, 117 insertions(+), 61 deletions(-)A bunch of nits, but overall looks good. Please address nits, and add my ack
Acked-by Andrii Nakryiko <[email protected]>Quoted 8 lines
tools/lib/bpf/elf.c +0
-0@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)return NULL ; }
Quoted 9 lines
tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 67
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+struct elf_sym { +const char *name; +GElf_Sym sym; +GElf_Shdr sh; +}; +if we want to use
elf_sym_iteroutside ofelf.c, this should be inlibbpf_internal.h?
yes eventually, but all the helper functions using elf_sym_iter that I added later are in elf.c, so there's no need atm
SNIP
Quoted 24 lines
Quoted 24 lines
tools/lib/bpf/elf.c +24
-0+ +static struct elf_sym *elf_sym_iter_next (struct elf_sym_iter *iter) +{ +struct elf_sym *ret = &iter->sym; +GElf_Sym *sym = &ret->sym; +const char *name =NULL ; +Elf_Scn *sym_scn; +size_t idx; + +for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) { +if (!gelf_getsym (iter->syms, idx, sym)) +continue ; +if (GELF_ST_TYPE (sym->st_info) != iter->st_type) +continue ; + name =elf_strptr (iter->elf, iter->strtabidx, sym->st_name); +if (!name) +continue ; + +/* Transform symbol's virtual address (absolute for +* binaries and relative for shared libs) into file +* offset, which is what kernel is expecting for +* uprobe/uretprobe attachment. +* See Documentation/trace/uprobetracer.rst for more +* details.
Quoted 8 lines
tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 144
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+* This is done by looking up symbol's containing +* section's header and using iter's virtual address +* (sh_addr) and corresponding file offset (sh_offset) +* to transform sym.st_value (virtual address) into +* desired final file offset. +*/ this comment is misplaced? we don't do the translation here
right, should be placed at the elf_sym_offset function
Quoted 39 lines
Quoted 26 lines
tools/lib/bpf/elf.c +19
-0+ sym_scn =elf_getscn (iter->elf, sym->st_shndx); +if (!sym_scn) +continue ; +if (!gelf_getshdr (sym_scn, &ret->sh)) +continue ; + + iter->next_sym_idx = idx +1 ; + ret->name = name; +return ret; + } + +return NULL ; +} + +static unsigned long elf_sym_offset (struct elf_sym *sym) +{ +return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset; +} +/* Find offset of function name in the provided ELF object. "binary_path" is * the path to the ELF binary represented by "elf", and only used for error * reporting matters. "name" matches symbol name or name@@LIB for library @@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)* reported as a warning/error. */ tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 198
Quoted 10 lines
tools/lib/bpf/elf.c +3
-5for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { -size_t nr_syms, strtabidx, idx; -Elf_Data *symbols =NULL ; -Elf_Scn *scn =NULL ; +struct elf_sym_iter iter; +struct elf_sym *sym;int last_bind =-1 ; -const char *sname; -GElf_Shdr sh; +int curr_bind;OCD nit:
Quoted 7 lines
$ rg 'curr(_|\b)' | wc -l 8 $ rg 'cur(_|\b)' | wc -l 148and those 8 I consider an unfortunate accident ;) let's standardize on using "cur" consistently
ok.. probably easier to make that change than treat ocd ;-)
Quoted 22 lines
Quoted 11 lines
tools/lib/bpf/elf.c +0
-9- scn =elf_find_next_scn_by_type (elf, sh_types[i],NULL ); -if (!scn) { -pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , - binary_path); -continue ; - } -if (!gelf_getshdr (scn, &sh)) -continue ; - strtabidx = sh.sh_link;tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 204
Quoted 10 lines
tools/lib/bpf/elf.c +4
-5- symbols =elf_getdata (scn,0 ); -if (!symbols) { -pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , - binary_path,elf_errmsg (-1 )); - ret = -LIBBPF_ERRNO__FORMAT ; + ret =elf_sym_iter_new (&iter, elf, binary_path, sh_types[i],STT_FUNC ); +if (ret) { +if (ret == -ENOENT ) +continue ; goto out;
Quoted 8 lines
another styling nit: let's avoid unnecessary nesting of ifs:
if (ret ==
-ENOENT)continue; if (ret) goto out;simple and clean
ok
Quoted 35 lines
Quoted 35 lines
tools/lib/bpf/elf.c +3
-21} - nr_syms = symbols->d_size / sh.sh_entsize; - -for (idx =0 ; idx < nr_syms; idx++) { -int curr_bind; -GElf_Sym sym; -Elf_Scn *sym_scn; -GElf_Shdr sym_sh; - -if (!gelf_getsym (symbols, idx, &sym)) -continue ; - -if (GELF_ST_TYPE (sym.st_info) !=STT_FUNC ) -continue ; - - sname =elf_strptr (elf, strtabidx, sym.st_name); -if (!sname) -continue ; - - curr_bind =GELF_ST_BIND (sym.st_info); +while ((sym =elf_sym_iter_next (&iter))) {/* User can specify func, func@@LIB or func@@LIB_VERSION. */ -if (strncmp (sname, name, name_len) !=0 ) +if (strncmp (sym->name, name, name_len) !=0 )continue ;/* ...but we don't want a search for "foo" to match 'foo2" also, so any * additional characters in sname should be of the form "@@LIB". */ -if (!is_name_qualified && sname[name_len] !='\0' && sname[name_len] !='@' ) +if (!is_name_qualified && sym->name[name_len] !='\0' && sym->name[name_len] !='@' )continue ;
Quoted 6 lines
tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 219
Quoted 4 lines
tools/lib/bpf/elf.c +3
-1-if (ret >=0 ) { + curr_bind =GELF_ST_BIND (sym->sym.st_info); + +if (ret >0 ) {used to be >=, why the change?
the original code initialized ret with -ENOENT and did not change its value till this point, so the condition never triggered for the first loop, but it did for the new code because we now use ret earlier in:
ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
also the check makes sense to me only if ret > 0 .. when we find a duplicate value for symbol
jirka
Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator
Quoted 118 lines · Jiri Olsa
On Thu, Jul 06, 2023 at 04:24:48PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 19 lines
Quoted 6 lines
Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++--------------- 1 file changed, 117 insertions(+), 61 deletions(-)A bunch of nits, but overall looks good. Please address nits, and add my ack
Acked-by Andrii Nakryiko <[email protected]>Quoted 8 lines
tools/lib/bpf/elf.c +0
-0@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)return NULL ; }Quoted 9 lines
tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 67
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+struct elf_sym { +const char *name; +GElf_Sym sym; +GElf_Shdr sh; +}; +if we want to use
elf_sym_iteroutside ofelf.c, this should be inlibbpf_internal.h?yes eventually, but all the helper functions using
elf_sym_iterthat I added later are inelf.c, so there's no need atmSNIP
Quoted 24 lines
Quoted 24 lines
tools/lib/bpf/elf.c +24
-0+ +static struct elf_sym *elf_sym_iter_next (struct elf_sym_iter *iter) +{ +struct elf_sym *ret = &iter->sym; +GElf_Sym *sym = &ret->sym; +const char *name =NULL ; +Elf_Scn *sym_scn; +size_t idx; + +for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) { +if (!gelf_getsym (iter->syms, idx, sym)) +continue ; +if (GELF_ST_TYPE (sym->st_info) != iter->st_type) +continue ; + name =elf_strptr (iter->elf, iter->strtabidx, sym->st_name); +if (!name) +continue ; + +/* Transform symbol's virtual address (absolute for +* binaries and relative for shared libs) into file +* offset, which is what kernel is expecting for +* uprobe/uretprobe attachment. +* See Documentation/trace/uprobetracer.rst for more +* details. Quoted 8 lines
tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 144
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+* This is done by looking up symbol's containing +* section's header and using iter's virtual address +* (sh_addr) and corresponding file offset (sh_offset) +* to transform sym.st_value (virtual address) into +* desired final file offset. +*/ this comment is misplaced? we don't do the translation here
right, should be placed at the
elf_sym_offsetfunctionQuoted 44 lines
Quoted 26 lines
tools/lib/bpf/elf.c +19
-0+ sym_scn =elf_getscn (iter->elf, sym->st_shndx); +if (!sym_scn) +continue ; +if (!gelf_getshdr (sym_scn, &ret->sh)) +continue ; + + iter->next_sym_idx = idx +1 ; + ret->name = name; +return ret; + } + +return NULL ; +} + +static unsigned long elf_sym_offset (struct elf_sym *sym) +{ +return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset; +} +/* Find offset of function name in the provided ELF object. "binary_path" is * the path to the ELF binary represented by "elf", and only used for error * reporting matters. "name" matches symbol name or name@@LIB for library @@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)* reported as a warning/error. */ tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 198
Quoted 10 lines
tools/lib/bpf/elf.c +3
-5for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { -size_t nr_syms, strtabidx, idx; -Elf_Data *symbols =NULL ; -Elf_Scn *scn =NULL ; +struct elf_sym_iter iter; +struct elf_sym *sym;int last_bind =-1 ; -const char *sname; -GElf_Shdr sh; +int curr_bind;OCD nit:
$ rg 'curr(_|\b)' | wc -l 8 $ rg 'cur(_|\b)' | wc -l 148
Quoted 5 lines
Quoted 2 lines
and those 8 I consider an unfortunate accident ;) let's standardize on using "cur" consistently
ok.. probably easier to make that change than treat ocd ;-)
thanks :)
Quoted 80 lines
Quoted 22 lines
Quoted 11 lines
tools/lib/bpf/elf.c +0
-9- scn =elf_find_next_scn_by_type (elf, sh_types[i],NULL ); -if (!scn) { -pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , - binary_path); -continue ; - } -if (!gelf_getshdr (scn, &sh)) -continue ; - strtabidx = sh.sh_link;tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 204
Quoted 10 lines
tools/lib/bpf/elf.c +4
-5- symbols =elf_getdata (scn,0 ); -if (!symbols) { -pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , - binary_path,elf_errmsg (-1 )); - ret = -LIBBPF_ERRNO__FORMAT ; + ret =elf_sym_iter_new (&iter, elf, binary_path, sh_types[i],STT_FUNC ); +if (ret) { +if (ret == -ENOENT ) +continue ; goto out;Quoted 8 lines
another styling nit: let's avoid unnecessary nesting of ifs:
if (ret ==
-ENOENT)continue; if (ret) goto out;simple and clean
ok
Quoted 35 lines
Quoted 35 lines
tools/lib/bpf/elf.c +3
-21} - nr_syms = symbols->d_size / sh.sh_entsize; - -for (idx =0 ; idx < nr_syms; idx++) { -int curr_bind; -GElf_Sym sym; -Elf_Scn *sym_scn; -GElf_Shdr sym_sh; - -if (!gelf_getsym (symbols, idx, &sym)) -continue ; - -if (GELF_ST_TYPE (sym.st_info) !=STT_FUNC ) -continue ; - - sname =elf_strptr (elf, strtabidx, sym.st_name); -if (!sname) -continue ; - - curr_bind =GELF_ST_BIND (sym.st_info); +while ((sym =elf_sym_iter_next (&iter))) {/* User can specify func, func@@LIB or func@@LIB_VERSION. */ -if (strncmp (sname, name, name_len) !=0 ) +if (strncmp (sym->name, name, name_len) !=0 )continue ;/* ...but we don't want a search for "foo" to match 'foo2" also, so any * additional characters in sname should be of the form "@@LIB". */ -if (!is_name_qualified && sname[name_len] !='\0' && sname[name_len] !='@' ) +if (!is_name_qualified && sym->name[name_len] !='\0' && sym->name[name_len] !='@' )continue ;Quoted 6 lines
tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 219
Quoted 4 lines
tools/lib/bpf/elf.c +3
-1-if (ret >=0 ) { + curr_bind =GELF_ST_BIND (sym->sym.st_info); + +if (ret >0 ) {used to be >=, why the change?
the original code initialized ret with
-ENOENTand did not change its value till this point, so the condition never triggered for the first loop, but it did for the new code because we now use ret earlier in:
Quoted 4 lines
ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);also the check makes sense to me only if ret > 0 .. when we find a duplicate value for symbol
there is a subtle difference if we have some unresolved func with addr 0 or something. But you are right, it's probably not very sensical, we can keep it as > 0.
Quoted 2 lines
jirka
Jiri Olsa
reviewerPATCH v3 10/26 libbpf: Add elf_resolve_syms_offsets function
In reply to Jiri OlsaAdding elf_resolve_syms_offsets function that looks up offsets for symbols specified in syms array argument.
Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller.
--- tools/lib/bpf/elf.c | 105 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_elf.h | 2 + 2 files changed, 107 insertions(+)
tools/lib/bpf/elf.c +105 -0
@@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)elf_close (&elf_fd);return ret; } + +struct symbol { +const char *name; +int bind; +int idx; +}; + +static int symbol_cmp (const void *_a,const void *_b) +{ +const struct symbol *a = _a; +const struct symbol *b = _b; + +return strcmp (a->name, b->name); +} + +int elf_resolve_syms_offsets (const char *binary_path,int cnt, +const char **syms,unsigned long **poffsets) +{ +int sh_types[2 ] = {SHT_DYNSYM ,SHT_SYMTAB }; +int err =0 , i, cnt_done =0 ; +unsigned long *offsets; +struct symbol *symbols; +struct elf_fd elf_fd; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; + + offsets =calloc (cnt,sizeof (*offsets)); + symbols =calloc (cnt,sizeof (*symbols)); + +if (!offsets || !symbols) { + err = -ENOMEM ; + goto out; + } + +for (i =0 ; i < cnt; i++) { + symbols[i].name = syms[i]; + symbols[i].idx = i; + } + +qsort (symbols, cnt,sizeof (*symbols), symbol_cmp); + +for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; + + err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err) { +if (err == -ENOENT ) +continue ; + goto out; + } + +while ((sym =elf_sym_iter_next (&iter))) { +int bind =GELF_ST_BIND (sym->sym.st_info); +struct symbol *found, tmp = { + .name = sym->name, + }; +unsigned long *offset; + + found =bsearch (&tmp, symbols, cnt,sizeof (*symbols), symbol_cmp); +if (!found) +continue ; + + offset = &offsets[found->idx]; +if (*offset >0 ) { +/* same offset, no problem */ +if (*offset ==elf_sym_offset (sym)) +continue ; +/* handle multiple matches */ +if (found->bind !=STB_WEAK && bind !=STB_WEAK ) { +/* Only accept one non-weak bind. */ +pr_warn ("elf: ambiguous match foundr '%s', '%s' in '%s'\n" , + sym->name, found->name, binary_path); + err = -LIBBPF_ERRNO__FORMAT ; + goto out; + }else if (bind ==STB_WEAK ) { +/* already have a non-weak bind, and +* this is a weak bind, so ignore. +*/ +continue ; + } + }else { + cnt_done++; + } + *offset =elf_sym_offset (sym); + found->bind = bind; + } + } + +if (cnt != cnt_done) { + err = -ENOENT ; + goto out; + } + + *poffsets = offsets; + +out: +free (symbols); +if (err) +free (offsets); +elf_close (&elf_fd); +return err; +}
tools/lib/bpf/libbpf_elf.h +2 -0
@@ -16,4 +16,6 @@ void elf_close(struct elf_fd *elf_fd);long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name);long elf_find_func_offset_from_file (const char *binary_path,const char *name); +int elf_resolve_syms_offsets (const char *binary_path,int cnt, +const char **syms,unsigned long **poffsets);#endif /* *__LIBBPF_LIBBPF_ELF_H */
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 10/26] libbpf: Add elf_resolve_syms_offsets function
Quoted 22 lines · Jiri Olsa
Adding
elf_resolve_syms_offsetsfunction that looks up offsets for symbols specified in syms array argument.Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller.
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 105 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_elf.h | 2 + 2 files changed, 107 insertions(+)tools/lib/bpf/elf.c +1
-0@@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)elf_close (&elf_fd);return ret; } +
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 284
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+struct symbol { +const char *name; +int bind; +int idx; +}; + +static int symbol_cmp (const void *_a,const void *_b) +{ +const struct symbol *a = _a; +const struct symbol *b = _b;
please, let's not (over)use leading underscores, x/y, s1/s2, whatever
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 288
Quoted 4 lines
tools/lib/bpf/elf.c +4
-0+ +return strcmp (a->name, b->name); +} +
probably worth leaving a comment that the caller should free offsets on success?
Quoted 28 lines
tools/lib/bpf/elf.c +28
-0+int elf_resolve_syms_offsets (const char *binary_path,int cnt, +const char **syms,unsigned long **poffsets) +{ +int sh_types[2 ] = {SHT_DYNSYM ,SHT_SYMTAB }; +int err =0 , i, cnt_done =0 ; +unsigned long *offsets; +struct symbol *symbols; +struct elf_fd elf_fd; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; + + offsets =calloc (cnt,sizeof (*offsets)); + symbols =calloc (cnt,sizeof (*symbols)); + +if (!offsets || !symbols) { + err = -ENOMEM ; + goto out; + } + +for (i =0 ; i < cnt; i++) { + symbols[i].name = syms[i]; + symbols[i].idx = i; + } + +qsort (symbols, cnt,sizeof (*symbols), symbol_cmp); +
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 326
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; + + err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err) { +if (err == -ENOENT ) +continue ; + goto out; + }
same nit, no need for nested ifs
Quoted 11 lines
tools/lib/bpf/elf.c +11
-0+ +while ((sym =elf_sym_iter_next (&iter))) { +int bind =GELF_ST_BIND (sym->sym.st_info); +struct symbol *found, tmp = { + .name = sym->name, + }; +unsigned long *offset; + + found =bsearch (&tmp, symbols, cnt,sizeof (*symbols), symbol_cmp); +if (!found) +continue ;
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 347
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+ + offset = &offsets[found->idx]; +if (*offset >0 ) { +/* same offset, no problem */ +if (*offset ==elf_sym_offset (sym)) +continue ; +/* handle multiple matches */ +if (found->bind !=STB_WEAK && bind !=STB_WEAK ) { +/* Only accept one non-weak bind. */ + pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",
typo: found
but also wouldn't sym->name and found->name be always the same? Maybe log sym->name, previous *offset and newly calculated elf_sym_offset(sym) instead?
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 349
Quoted 2 lines
tools/lib/bpf/elf.c +2
-0+ sym->name, found->name, binary_path); + err = -LIBBPF_ERRNO__FORMAT ;
I'd minimize using those custom libbpf-only errors, why not -ESRCH here?
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 360
Quoted 11 lines
tools/lib/bpf/elf.c +11
-0+ goto out; + }else if (bind ==STB_WEAK ) { +/* already have a non-weak bind, and +* this is a weak bind, so ignore. +*/ +continue ; + } + }else { + cnt_done++; + } + *offset =elf_sym_offset (sym);
maybe remember elf_sym_offset() result in a variable? you are using it in two (and with my suggestion above it will be three) places already
Quoted 32 lines
tools/lib/bpf/elf.c +18
-0+ found->bind = bind; + } + } + +if (cnt != cnt_done) { + err = -ENOENT ; + goto out; + } + + *poffsets = offsets; + +out: +free (symbols); +if (err) +free (offsets); +elf_close (&elf_fd); +return err; +}tools/lib/bpf/libbpf_elf.h +2
-1@@ -16,4 +16,6 @@ void elf_close(struct elf_fd *elf_fd);long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name);long elf_find_func_offset_from_file (const char *binary_path,const char *name); +int elf_resolve_syms_offsets (const char *binary_path,int cnt, +const char **syms,unsigned long **poffsets);#endif /* *__LIBBPF_LIBBPF_ELF_H */ --2.41.0
Re: [PATCHv3 bpf-next 10/26] libbpf: Add elf_resolve_syms_offsets function
Quoted 27 lines · Andrii Nakryiko
Quoted 26 lines · Jiri Olsa
Adding
elf_resolve_syms_offsetsfunction that looks up offsets for symbols specified in syms array argument.Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller.
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 105 +++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_elf.h | 2 + 2 files changed, 107 insertions(+)tools/lib/bpf/elf.c +5
-0@@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)elf_close (&elf_fd);return ret; } + +struct symbol { +const char *name; +int bind; +int idx;
Quoted 8 lines
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 284
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+}; + +static int symbol_cmp (const void *_a,const void *_b) +{ +const struct symbol *a = _a; +const struct symbol *b = _b;please, let's not (over)use leading underscores, x/y, s1/s2, whatever
ok
Quoted 7 lines
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 288
Quoted 5 lines
tools/lib/bpf/elf.c +4
-0+ +return strcmp (a->name, b->name); +} +probably worth leaving a comment that the caller should free offsets on success?
ook
Quoted 33 lines
Quoted 33 lines
tools/lib/bpf/elf.c +32
-0+int elf_resolve_syms_offsets (const char *binary_path,int cnt, +const char **syms,unsigned long **poffsets) +{ +int sh_types[2 ] = {SHT_DYNSYM ,SHT_SYMTAB }; +int err =0 , i, cnt_done =0 ; +unsigned long *offsets; +struct symbol *symbols; +struct elf_fd elf_fd; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; + + offsets =calloc (cnt,sizeof (*offsets)); + symbols =calloc (cnt,sizeof (*symbols)); + +if (!offsets || !symbols) { + err = -ENOMEM ; + goto out; + } + +for (i =0 ; i < cnt; i++) { + symbols[i].name = syms[i]; + symbols[i].idx = i; + } + +qsort (symbols, cnt,sizeof (*symbols), symbol_cmp); + +for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; +
Quoted 8 lines
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 326
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+ err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err) { +if (err == -ENOENT ) +continue ; + goto out; + }same nit, no need for nested ifs
ok
Quoted 24 lines
Quoted 11 lines
tools/lib/bpf/elf.c +11
-0+ +while ((sym =elf_sym_iter_next (&iter))) { +int bind =GELF_ST_BIND (sym->sym.st_info); +struct symbol *found, tmp = { + .name = sym->name, + }; +unsigned long *offset; + + found =bsearch (&tmp, symbols, cnt,sizeof (*symbols), symbol_cmp); +if (!found) +continue ;tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 347
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+ + offset = &offsets[found->idx]; +if (*offset >0 ) { +/* same offset, no problem */ +if (*offset ==elf_sym_offset (sym)) +continue ; +/* handle multiple matches */ +if (found->bind !=STB_WEAK && bind !=STB_WEAK ) { +/* Only accept one non-weak bind. */ + pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",typo: found
Quoted 3 lines
but also wouldn't
sym->nameandfound->namebe always the same? Maybe logsym->name, previous *offset and newly calculatedelf_sym_offset(sym)instead?
ok
Quoted 5 lines
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 349
Quoted 3 lines
tools/lib/bpf/elf.c +2
-0+ sym->name, found->name, binary_path); + err = -LIBBPF_ERRNO__FORMAT ;I'd minimize using those custom libbpf-only errors, why not
-ESRCHhere?
ok
Quoted 13 lines
tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 360
Quoted 12 lines
tools/lib/bpf/elf.c +11
-0+ goto out; + }else if (bind ==STB_WEAK ) { +/* already have a non-weak bind, and +* this is a weak bind, so ignore. +*/ +continue ; + } + }else { + cnt_done++; + } + *offset =elf_sym_offset (sym);
Quoted 2 lines
maybe remember
elf_sym_offset()result in a variable? you are using it in two (and with my suggestion above it will be three) places already
ok
jirka
Jiri Olsa
reviewerPATCH v3 11/26 libbpf: Add elf_resolve_pattern_offsets function
In reply to Jiri OlsaAdding elf_resolve_pattern_offsets function that looks up offsets for symbols specified by pattern argument.
The 'pattern' argument allows wildcards (*?' supported).
Offsets are returned in allocated array together with its size and needs to be released by the caller.
--- tools/lib/bpf/elf.c | 57 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 2 +- tools/lib/bpf/libbpf_elf.h | 3 ++ tools/lib/bpf/libbpf_internal.h | 1 + 4 files changed, 62 insertions(+), 1 deletion(-)
tools/lib/bpf/elf.c +57 -0
@@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,elf_close (&elf_fd);return err; } + +int elf_resolve_pattern_offsets (const char *binary_path,const char *pattern, +unsigned long **poffsets,size_t *pcnt) +{ +int sh_types[2 ] = {SHT_DYNSYM ,SHT_SYMTAB }; +unsigned long *offsets =NULL ; +size_t cap =0 , cnt =0 ; +struct elf_fd elf_fd; +int err =0 , i; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; + +for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; + + err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err) { +if (err == -ENOENT ) +continue ; + goto out; + } + +while ((sym =elf_sym_iter_next (&iter))) { +if (!glob_match (sym->name, pattern)) +continue ; + + err =libbpf_ensure_mem ((void **) &offsets, &cap,sizeof (*offsets), + cnt +1 ); +if (err) + goto out; + + offsets[cnt++] =elf_sym_offset (sym); + } + +/* If we found anything in the first symbol section, +* do not search others to avoid duplicates. +*/ +if (cnt) +break ; + } + +if (cnt) { + *poffsets = offsets; + *pcnt = cnt; + }else { + err = -ENOENT ; + } + +out: +if (err) +free (offsets); +elf_close (&elf_fd); +return err; +}
tools/lib/bpf/libbpf.c +1 -1
@@ -10509,7 +10509,7 @@ struct bpf_link *bpf_program__attach_ksyscall(const struct bpf_program *prog, }/* Adapted from perf/util/string.c */ -static bool glob_match (const char *str,const char *pat) +bool glob_match (const char *str,const char *pat) {while (*str && *pat && *pat !='*' ) {if (*pat =='?' ) {/* Matches any single character */
tools/lib/bpf/libbpf_elf.h +3 -0
@@ -18,4 +18,7 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name);int elf_resolve_syms_offsets (const char *binary_path,int cnt,const char **syms,unsigned long **poffsets); + +int elf_resolve_pattern_offsets (const char *binary_path,const char *pattern, +unsigned long **poffsets,size_t *pcnt);#endif /* *__LIBBPF_LIBBPF_ELF_H */
tools/lib/bpf/libbpf_internal.h +1 -0
@@ -577,4 +577,5 @@ static inline bool is_pow_of_2(size_t x)#define PROG_LOAD_ATTEMPTS 5int sys_bpf_prog_load (union bpf_attr *attr,unsigned int size,int attempts); +bool glob_match (const char *str,const char *pat);#endif /* __LIBBPF_LIBBPF_INTERNAL_H */
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 11/26] libbpf: Add elf_resolve_pattern_offsets function
Quoted 22 lines · Jiri Olsa
Adding
elf_resolve_pattern_offsetsfunction that looks up offsets for symbols specified by pattern argument.The 'pattern' argument allows wildcards (*?' supported).
Offsets are returned in allocated array together with its size and needs to be released by the caller.
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 57 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 2 +- tools/lib/bpf/libbpf_elf.h | 3 ++ tools/lib/bpf/libbpf_internal.h | 1 + 4 files changed, 62 insertions(+), 1 deletion(-)tools/lib/bpf/elf.c +0
-0
tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 379
Quoted 4 lines
tools/lib/bpf/elf.c +1
-0@@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,elf_close (&elf_fd);return err; } +
same, leave comment that caller should free offsets on success?
Quoted 13 lines
tools/lib/bpf/elf.c +13
-0+int elf_resolve_pattern_offsets (const char *binary_path,const char *pattern, +unsigned long **poffsets,size_t *pcnt) +{ +int sh_types[2 ] = {SHT_DYNSYM ,SHT_SYMTAB }; +unsigned long *offsets =NULL ; +size_t cap =0 , cnt =0 ; +struct elf_fd elf_fd; +int err =0 , i; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; +
tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 402
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; + + err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err) { +if (err == -ENOENT ) +continue ; + goto out; + }
ditto, minimize nesting, please
Quoted 5 lines
tools/lib/bpf/elf.c +5
-0+ +while ((sym =elf_sym_iter_next (&iter))) { +if (!glob_match (sym->name, pattern)) +continue ; +
tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 417
Quoted 10 lines
tools/lib/bpf/elf.c +10
-0+ err =libbpf_ensure_mem ((void **) &offsets, &cap,sizeof (*offsets), + cnt +1 ); +if (err) + goto out; + + offsets[cnt++] =elf_sym_offset (sym); + } + +/* If we found anything in the first symbol section, +* do not search others to avoid duplicates.
DYNSYM is going to have only exposed symbols, so for this pattern matching, maybe it's best to start with SYMTAB and only fallback to DYNSYM if we didn't find anything in SYMTAB (more realistically it would be that SYMTAB section is missing, so we fallback to DYNSYM; otherwise neither DYNSYM nor SYMTAB will have matching symbols, most probably, but that's minor)
other than that, LGTM
Quoted 56 lines
tools/lib/bpf/elf.c +18
-0+*/ +if (cnt) +break ; + } + +if (cnt) { + *poffsets = offsets; + *pcnt = cnt; + }else { + err = -ENOENT ; + } + +out: +if (err) +free (offsets); +elf_close (&elf_fd); +return err; +}tools/lib/bpf/libbpf.c +1
-1@@ -10509,7 +10509,7 @@ struct bpf_link *bpf_program__attach_ksyscall(const struct bpf_program *prog, }/* Adapted from perf/util/string.c */ -static bool glob_match (const char *str,const char *pat) +bool glob_match (const char *str,const char *pat) {while (*str && *pat && *pat !='*' ) {if (*pat =='?' ) {/* Matches any single character */ tools/lib/bpf/libbpf_elf.h +3
-0@@ -18,4 +18,7 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name);int elf_resolve_syms_offsets (const char *binary_path,int cnt,const char **syms,unsigned long **poffsets); + +int elf_resolve_pattern_offsets (const char *binary_path,const char *pattern, +unsigned long **poffsets,size_t *pcnt);#endif /* *__LIBBPF_LIBBPF_ELF_H */ tools/lib/bpf/libbpf_internal.h +1
-1@@ -577,4 +577,5 @@ static inline bool is_pow_of_2(size_t x)#define PROG_LOAD_ATTEMPTS 5int sys_bpf_prog_load (union bpf_attr *attr,unsigned int size,int attempts); +bool glob_match (const char *str,const char *pat);#endif /* __LIBBPF_LIBBPF_INTERNAL_H */ --2.41.0
Re: [PATCHv3 bpf-next 11/26] libbpf: Add elf_resolve_pattern_offsets function
Quoted 18 lines · Andrii Nakryiko
Quoted 17 lines · Jiri Olsa
Adding
elf_resolve_pattern_offsetsfunction that looks up offsets for symbols specified by pattern argument.The 'pattern' argument allows wildcards (*?' supported).
Offsets are returned in allocated array together with its size and needs to be released by the caller.
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/elf.c | 57 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 2 +- tools/lib/bpf/libbpf_elf.h | 3 ++ tools/lib/bpf/libbpf_internal.h | 1 + 4 files changed, 62 insertions(+), 1 deletion(-)
Quoted 11 lines
tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 379
Quoted 9 lines
tools/lib/bpf/elf.c +1
-0@@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,elf_close (&elf_fd);return err; } +same, leave comment that caller should free offsets on success?
yes, will add
Quoted 18 lines
Quoted 18 lines
tools/lib/bpf/elf.c +17
-0+int elf_resolve_pattern_offsets (const char *binary_path,const char *pattern, +unsigned long **poffsets,size_t *pcnt) +{ +int sh_types[2 ] = {SHT_DYNSYM ,SHT_SYMTAB }; +unsigned long *offsets =NULL ; +size_t cap =0 , cnt =0 ; +struct elf_fd elf_fd; +int err =0 , i; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; + +for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; +
Quoted 8 lines
tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 402
Quoted 6 lines
tools/lib/bpf/elf.c +6
-0+ err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err) { +if (err == -ENOENT ) +continue ; + goto out; + }ditto, minimize nesting, please
ok
Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/elf.c +14
-0+ +while ((sym =elf_sym_iter_next (&iter))) { +if (!glob_match (sym->name, pattern)) +continue ; + + err =libbpf_ensure_mem ((void **) &offsets, &cap,sizeof (*offsets), + cnt +1 ); +if (err) + goto out; + + offsets[cnt++] =elf_sym_offset (sym); + } + +/* If we found anything in the first symbol section,
Quoted 8 lines
tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 417
Quoted 1 line
tools/lib/bpf/elf.c +1
-0+* do not search others to avoid duplicates. DYNSYM is going to have only exposed symbols, so for this pattern matching, maybe it's best to start with SYMTAB and only fallback to DYNSYM if we didn't find anything in SYMTAB (more realistically it would be that SYMTAB section is missing, so we fallback to DYNSYM; otherwise neither DYNSYM nor SYMTAB will have matching symbols, most probably, but that's minor)
makes sense, will switch
Quoted 2 lines
other than that, LGTM
thanks,
jirka
Jiri Olsa
reviewerPATCH v3 12/26 libbpf: Add bpf_link_create support for multi uprobes
In reply to Jiri OlsaAdding new uprobe_multi struct to bpf_link_create_opts object to pass multiple uprobe data to link_create attr uapi.
--- tools/lib/bpf/bpf.c | 11 +++++++++++ tools/lib/bpf/bpf.h | 11 ++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-)
tools/lib/bpf/bpf.c +11 -0
@@ -733,6 +733,17 @@ int bpf_link_create(int prog_fd, int target_fd,if (!OPTS_ZEROED (opts, kprobe_multi))return libbpf_err (-EINVAL );break ; +case BPF_TRACE_UPROBE_MULTI : + attr.link_create.uprobe_multi.flags =OPTS_GET (opts, uprobe_multi.flags,0 ); + attr.link_create.uprobe_multi.cnt =OPTS_GET (opts, uprobe_multi.cnt,0 ); + attr.link_create.uprobe_multi.path =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.path,0 )); + attr.link_create.uprobe_multi.offsets =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.offsets,0 )); + attr.link_create.uprobe_multi.ref_ctr_offsets =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.ref_ctr_offsets,0 )); + attr.link_create.uprobe_multi.cookies =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.cookies,0 )); + attr.link_create.uprobe_multi.pid =OPTS_GET (opts, uprobe_multi.pid,0 ); +if (!OPTS_ZEROED (opts, uprobe_multi)) +return libbpf_err (-EINVAL ); +break ;case BPF_TRACE_FENTRY :case BPF_TRACE_FEXIT :case BPF_MODIFY_RETURN :
tools/lib/bpf/bpf.h +10 -1
@@ -346,13 +346,22 @@ struct bpf_link_create_opts {const unsigned long *addrs;const __u64 *cookies; } kprobe_multi; +struct { +__u32 flags; +__u32 cnt; +const char *path; +const unsigned long *offsets; +const unsigned long *ref_ctr_offsets; +const __u64 *cookies; +__u32 pid; + } uprobe_multi;struct {__u64 cookie; } tracing; }; size_t :0; }; -#define bpf_link_create_opts__last_field kprobe_multi.cookies +#define bpf_link_create_opts__last_field uprobe_multi.pid LIBBPF_API int bpf_link_create(int prog_fd, int target_fd, enum bpf_attach_type attach_type,
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v3 13/26 libbpf: Add bpf_program__attach_uprobe_multi function
In reply to Jiri OlsaAdding bpf_program__attach_uprobe_multi function that allows to attach multiple uprobes with uprobe_multi link.
The user can specify uprobes with direct arguments:
binary_path/func_pattern/pid
or with struct bpf_uprobe_multi_opts opts argument fields:
const char **syms;
const unsigned long *offsets;
const unsigned long *ref_ctr_offsets;
const __u64 *cookies;
User can specify 2 mutually exclusive set of inputs:
1) use only path/func_pattern/pid arguments
2) use path/pid with allowed combinations of:
syms/offsets/ref_ctr_offsets/cookies/cnt
- syms and offsets are mutually exclusive
- ref_ctr_offsets and cookies are optional
Any other usage results in error.
--- tools/lib/bpf/libbpf.c | 122 +++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 27 +++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 150 insertions(+)
tools/lib/bpf/libbpf.c +122 -0
@@ -10954,6 +10954,128 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz)return -ENOENT ; } +struct bpf_link * +bpf_program__attach_uprobe_multi (const struct bpf_program *prog, +pid_t pid, +const char *path, +const char *func_pattern, +const struct bpf_uprobe_multi_opts *opts) +{ +const unsigned long *ref_ctr_offsets =NULL , *offsets =NULL ; +LIBBPF_OPTS (bpf_link_create_opts, lopts); +unsigned long *resolved_offsets =NULL ; +int err =0 , link_fd, prog_fd; +struct bpf_link *link =NULL ; +char errmsg[STRERR_BUFSIZE ]; +char full_path[PATH_MAX ]; +const __u64 *cookies; +const char **syms; +bool has_pattern; +bool retprobe; +size_t cnt; + +if (!OPTS_VALID (opts, bpf_uprobe_multi_opts)) +return libbpf_err_ptr (-EINVAL ); + + syms =OPTS_GET (opts, syms,NULL ); + offsets =OPTS_GET (opts, offsets,NULL ); + ref_ctr_offsets =OPTS_GET (opts, ref_ctr_offsets,NULL ); + cookies =OPTS_GET (opts, cookies,NULL ); + cnt =OPTS_GET (opts, cnt,0 ); + +/* +* User can specify 2 mutually exclusive set of inputs: +* +* 1) use only path/func_pattern/pid arguments +* +* 2) use path/pid with allowed combinations of: +* syms/offsets/ref_ctr_offsets/cookies/cnt +* +* - syms and offsets are mutually exclusive +* - ref_ctr_offsets and cookies are optional +* +* Any other usage results in error. +*/ + +if (!path && !func_pattern && !cnt) +return libbpf_err_ptr (-EINVAL ); +if (func_pattern && !path) +return libbpf_err_ptr (-EINVAL ); + + has_pattern = path && func_pattern; + +if (has_pattern) { +if (syms || offsets || ref_ctr_offsets || cookies || cnt) +return libbpf_err_ptr (-EINVAL ); + }else { +if (!cnt) +return libbpf_err_ptr (-EINVAL ); +if (!!syms == !!offsets) +return libbpf_err_ptr (-EINVAL ); + } + +if (has_pattern) { +if (!strchr (path,'/' )) { + err =resolve_full_path (path, full_path,sizeof (full_path)); +if (err) { +pr_warn ("prog '%s': failed to resolve full path for '%s': %d\n" , + prog->name, path, err); +return libbpf_err_ptr (err); + } + path = full_path; + } + + err =elf_resolve_pattern_offsets (path, func_pattern, + &resolved_offsets, &cnt); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets; + }else if (syms) { + err =elf_resolve_syms_offsets (path, cnt, syms, &resolved_offsets); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets; + } + + retprobe =OPTS_GET (opts, retprobe, false); + + lopts.uprobe_multi.path = path; + lopts.uprobe_multi.offsets = offsets; + lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; + lopts.uprobe_multi.cookies = cookies; + lopts.uprobe_multi.cnt = cnt; + lopts.uprobe_multi.flags = retprobe ?BPF_F_UPROBE_MULTI_RETURN :0 ; + +if (pid ==0 ) + pid =getpid (); +if (pid >0 ) + lopts.uprobe_multi.pid = pid; + + link =calloc (1 ,sizeof (*link)); +if (!link) { + err = -ENOMEM ; + goto error; + } + link->detach = &bpf_link__detach_fd; + + prog_fd =bpf_program__fd (prog); + link_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &lopts); +if (link_fd <0 ) { + err = -errno; +pr_warn ("prog '%s': failed to attach: %s\n" , + prog->name,libbpf_strerror_r (err, errmsg,sizeof (errmsg))); + goto error; + } + link->fd = link_fd; +free (resolved_offsets); +return link; + +error: +free (resolved_offsets); +free (link); +return libbpf_err_ptr (err); +} + LIBBPF_API struct bpf_link * bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, const char *binary_path, size_t func_offset,
tools/lib/bpf/libbpf.h +27 -0
@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,const char *pattern, const struct bpf_kprobe_multi_opts *opts); +struct bpf_uprobe_multi_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; +/* array of function symbols to attach */ +const char **syms; +/* array of function addresses to attach */ +const unsigned long *offsets; +/* array of refctr offsets to attach */ +const unsigned long *ref_ctr_offsets; +/* array of user-provided values fetchable through bpf_get_attach_cookie */ +const __u64 *cookies; +/* number of elements in syms/addrs/cookies arrays */ +size_t cnt; +/* create return uprobes */ +bool retprobe; + size_t :0; +}; + +#define bpf_uprobe_multi_opts__last_field retprobe + +LIBBPF_API struct bpf_link * +bpf_program__attach_uprobe_multi (const struct bpf_program *prog, +pid_t pid, +const char *binary_path, +const char *func_pattern, +const struct bpf_uprobe_multi_opts *opts); +struct bpf_ksyscall_opts {/* size of this struct, for forward/backward compatibility */ size_t sz;
tools/lib/bpf/libbpf.map +1 -0
@@ -395,4 +395,5 @@ LIBBPF_1.2.0 { LIBBPF_1.3.0 { global: bpf_obj_pin_opts; + bpf_program__attach_uprobe_multi; } LIBBPF_1.2.0;
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function
Quoted 76 lines · Jiri Olsa
Adding
bpf_program__attach_uprobe_multifunction that allows to attach multiple uprobes withuprobe_multilink.The user can specify uprobes with direct arguments:
binary_path/func_pattern/pid
or with struct
bpf_uprobe_multi_optsopts argument fields:const char **syms;
const unsigned long *offsets;
const unsigned long*ref_ctr_offsets;
const__u64*cookies;User can specify 2 mutually exclusive set of inputs:
1) use only path/func_pattern/pid arguments
2) use path/pid with allowed combinations of:
syms/offsets/ref_ctr_offsets/cookies/cnt- syms and offsets are mutually exclusive
-ref_ctr_offsetsand cookies are optionalAny other usage results in error.
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/libbpf.c | 122 +++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 27 +++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 150 insertions(+)tools/lib/bpf/libbpf.c +34
-0@@ -10954,6 +10954,128 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz)return -ENOENT ; } +struct bpf_link * +bpf_program__attach_uprobe_multi (const struct bpf_program *prog, +pid_t pid, +const char *path, +const char *func_pattern, +const struct bpf_uprobe_multi_opts *opts) +{ +const unsigned long *ref_ctr_offsets =NULL , *offsets =NULL ; +LIBBPF_OPTS (bpf_link_create_opts, lopts); +unsigned long *resolved_offsets =NULL ; +int err =0 , link_fd, prog_fd; +struct bpf_link *link =NULL ; +char errmsg[STRERR_BUFSIZE ]; +char full_path[PATH_MAX ]; +const __u64 *cookies; +const char **syms; +bool has_pattern; +bool retprobe; +size_t cnt; + +if (!OPTS_VALID (opts, bpf_uprobe_multi_opts)) +return libbpf_err_ptr (-EINVAL ); + + syms =OPTS_GET (opts, syms,NULL ); + offsets =OPTS_GET (opts, offsets,NULL ); + ref_ctr_offsets =OPTS_GET (opts, ref_ctr_offsets,NULL ); + cookies =OPTS_GET (opts, cookies,NULL ); + cnt =OPTS_GET (opts, cnt,0 ); + +/* +* User can specify 2 mutually exclusive set of inputs: +* +* 1) use only path/func_pattern/pid arguments +*
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11000
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+* 2) use path/pid with allowed combinations of: +* syms/offsets/ref_ctr_offsets/cookies/cnt +* +* - syms and offsets are mutually exclusive +* - ref_ctr_offsets and cookies are optional +* +* Any other usage results in error. +*/ + +if (!path && !func_pattern && !cnt)
weird, I'd expect separate if (!path) return error (already bad, regardless of func_pattern or cnt)
then if (!func_pattern && cnt == 0) return error
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11005
Quoted 5 lines
tools/lib/bpf/libbpf.c +5
-0+return libbpf_err_ptr (-EINVAL ); +if (func_pattern && !path) +return libbpf_err_ptr (-EINVAL ); + + has_pattern = path && func_pattern;
this and above check must be some leftovers from previous version. path should always be present. and so you don't need has_pattern variable, just use "func_pattern" check
Quoted 22 lines
tools/lib/bpf/libbpf.c +22
-0+ +if (has_pattern) { +if (syms || offsets || ref_ctr_offsets || cookies || cnt) +return libbpf_err_ptr (-EINVAL ); + }else { +if (!cnt) +return libbpf_err_ptr (-EINVAL ); +if (!!syms == !!offsets) +return libbpf_err_ptr (-EINVAL ); + } + +if (has_pattern) { +if (!strchr (path,'/' )) { + err =resolve_full_path (path, full_path,sizeof (full_path)); +if (err) { +pr_warn ("prog '%s': failed to resolve full path for '%s': %d\n" , + prog->name, path, err); +return libbpf_err_ptr (err); + } + path = full_path; + } +
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11037
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+ err =elf_resolve_pattern_offsets (path, func_pattern, + &resolved_offsets, &cnt); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets; + }else if (syms) { + err =elf_resolve_syms_offsets (path, cnt, syms, &resolved_offsets); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets;
you can extract this common error checking and offsets = resolved_offsets; to after if, it's common for both branches
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11047
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+ } + + retprobe =OPTS_GET (opts, retprobe, false); + + lopts.uprobe_multi.path = path; + lopts.uprobe_multi.offsets = offsets; + lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; + lopts.uprobe_multi.cookies = cookies; + lopts.uprobe_multi.cnt = cnt; + lopts.uprobe_multi.flags = retprobe ?BPF_F_UPROBE_MULTI_RETURN :0 ;
retprobe is another unnecessary var, just inline check here to keep it simpler
Quoted 8 lines
tools/lib/bpf/libbpf.c +8
-0+ +if (pid ==0 ) + pid =getpid (); +if (pid >0 ) + lopts.uprobe_multi.pid = pid; + + link =calloc (1 ,sizeof (*link)); +if (!link) {
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11065
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+ err = -ENOMEM ; + goto error; + } + link->detach = &bpf_link__detach_fd; + + prog_fd =bpf_program__fd (prog); + link_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &lopts); +if (link_fd <0 ) { + err = -errno; + pr_warn("prog '%s': failed to attach: %s\n",
"failed to attach multi-uprobe"? We probably should have added "failed to attach multi-kprobe" in bpf_program__attach_kprobe_multi_opts as well?
Quoted 21 lines
tools/lib/bpf/libbpf.c +13
-0+ prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg))); + goto error; + } + link->fd = link_fd; +free (resolved_offsets); +return link; + +error: +free (resolved_offsets); +free (link); +return libbpf_err_ptr (err); +} + LIBBPF_API struct bpf_link * bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, const char *binary_path, size_t func_offset,tools/lib/bpf/libbpf.h +0
-0
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 535
Quoted 7 lines
tools/lib/bpf/libbpf.h +4
-0@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,const char *pattern, const struct bpf_kprobe_multi_opts *opts); +struct bpf_uprobe_multi_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; +/* array of function symbols to attach */
attach to?
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 537
Quoted 2 lines
tools/lib/bpf/libbpf.h +2
-0+const char **syms; +/* array of function addresses to attach */
attach to?
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 539
Quoted 2 lines
tools/lib/bpf/libbpf.h +2
-0+const unsigned long *offsets; +/* array of refctr offsets to attach */
we don't really attach to ref counters, so maybe "optional, array of associated ref counter offsets" or something along those lines ?
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 541
Quoted 2 lines
tools/lib/bpf/libbpf.h +2
-0+const unsigned long *ref_ctr_offsets; +/* array of user-provided values fetchable through bpf_get_attach_cookie */
"array of associated BPF cookies"? we can't keep explaining what BPF cookie is in every possible API :)
Quoted 7 lines
tools/lib/bpf/libbpf.h +7
-0+const __u64 *cookies; +/* number of elements in syms/addrs/cookies arrays */ +size_t cnt; +/* create return uprobes */ +bool retprobe; + size_t :0; +};
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 558
Quoted 10 lines
tools/lib/bpf/libbpf.h +10
-0+ +#define bpf_uprobe_multi_opts__last_field retprobe + +LIBBPF_API struct bpf_link * +bpf_program__attach_uprobe_multi (const struct bpf_program *prog, +pid_t pid, +const char *binary_path, +const char *func_pattern, +const struct bpf_uprobe_multi_opts *opts); +
ok, let's be good citizens and add documentation for this new API. Those comments about valid combinations belong here as well. Please take a look at existing doccomments for the format and conventions. Thanks!
Quoted 16 lines
tools/lib/bpf/libbpf.h +0
-0struct bpf_ksyscall_opts {/* size of this struct, for forward/backward compatibility */ size_t sz;tools/lib/bpf/libbpf.map +1
-1@@ -395,4 +395,5 @@ LIBBPF_1.2.0 { LIBBPF_1.3.0 { global: bpf_obj_pin_opts; + bpf_program__attach_uprobe_multi; } LIBBPF_1.2.0; --2.41.0
Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function
On Thu, Jul 06, 2023 at 09:05:23PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 36 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14
-0+if (!OPTS_VALID (opts, bpf_uprobe_multi_opts)) +return libbpf_err_ptr (-EINVAL ); + + syms =OPTS_GET (opts, syms,NULL ); + offsets =OPTS_GET (opts, offsets,NULL ); + ref_ctr_offsets =OPTS_GET (opts, ref_ctr_offsets,NULL ); + cookies =OPTS_GET (opts, cookies,NULL ); + cnt =OPTS_GET (opts, cnt,0 ); + +/* +* User can specify 2 mutually exclusive set of inputs: +* +* 1) use only path/func_pattern/pid arguments +* tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11000
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+* 2) use path/pid with allowed combinations of: +* syms/offsets/ref_ctr_offsets/cookies/cnt +* +* - syms and offsets are mutually exclusive +* - ref_ctr_offsets and cookies are optional +* +* Any other usage results in error. +*/ + +if (!path && !func_pattern && !cnt)weird, I'd expect separate if (!path) return error (already bad, regardless of
func_patternor cnt)then if (!func_pattern && cnt == 0) return error
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11005
Quoted 5 lines
tools/lib/bpf/libbpf.c +5
-0+return libbpf_err_ptr (-EINVAL ); +if (func_pattern && !path) +return libbpf_err_ptr (-EINVAL ); + + has_pattern = path && func_pattern;
Quoted 3 lines
this and above check must be some leftovers from previous version. path should always be present. and so you don't need
has_patternvariable, just use "func_pattern" check
hum, right, previous version had 2 paths, now there's just one, I'll change that together with the suggested change above
Quoted 28 lines
Quoted 28 lines
tools/lib/bpf/libbpf.c +27
-0+ +if (has_pattern) { +if (syms || offsets || ref_ctr_offsets || cookies || cnt) +return libbpf_err_ptr (-EINVAL ); + }else { +if (!cnt) +return libbpf_err_ptr (-EINVAL ); +if (!!syms == !!offsets) +return libbpf_err_ptr (-EINVAL ); + } + +if (has_pattern) { +if (!strchr (path,'/' )) { + err =resolve_full_path (path, full_path,sizeof (full_path)); +if (err) { +pr_warn ("prog '%s': failed to resolve full path for '%s': %d\n" , + prog->name, path, err); +return libbpf_err_ptr (err); + } + path = full_path; + } + + err =elf_resolve_pattern_offsets (path, func_pattern, + &resolved_offsets, &cnt); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets;
Quoted 8 lines
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11037
Quoted 5 lines
tools/lib/bpf/libbpf.c +5
-0+ }else if (syms) { + err =elf_resolve_syms_offsets (path, cnt, syms, &resolved_offsets); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets;you can extract this common error checking and
offsets = resolved_offsets;to after if, it's common for both branches
not sure what you mean in here, offsets can be also provided by OPTS_GET(opts, offsets, NULL) earlier
Quoted 4 lines
Quoted 4 lines
tools/lib/bpf/libbpf.c +4
-0+ } + + retprobe =OPTS_GET (opts, retprobe, false); +
Quoted 8 lines
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11047
Quoted 6 lines
tools/lib/bpf/libbpf.c +6
-0+ lopts.uprobe_multi.path = path; + lopts.uprobe_multi.offsets = offsets; + lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; + lopts.uprobe_multi.cookies = cookies; + lopts.uprobe_multi.cnt = cnt; + lopts.uprobe_multi.flags = retprobe ?BPF_F_UPROBE_MULTI_RETURN :0 ;retprobe is another unnecessary var, just inline check here to keep it simpler
ok
Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.c +14
-0+ +if (pid ==0 ) + pid =getpid (); +if (pid >0 ) + lopts.uprobe_multi.pid = pid; + + link =calloc (1 ,sizeof (*link)); +if (!link) { + err = -ENOMEM ; + goto error; + } + link->detach = &bpf_link__detach_fd; + + prog_fd =bpf_program__fd (prog);
Quoted 8 lines
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11065
Quoted 4 lines
tools/lib/bpf/libbpf.c +4
-0+ link_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &lopts); +if (link_fd <0 ) { + err = -errno; + pr_warn("prog '%s': failed to attach: %s\n","failed to attach multi-uprobe"? We probably should have added "failed to attach multi-kprobe" in
bpf_program__attach_kprobe_multi_optsas well?
ook, will add
Quoted 25 lines
Quoted 25 lines
tools/lib/bpf/libbpf.c +13
-0+ prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg))); + goto error; + } + link->fd = link_fd; +free (resolved_offsets); +return link; + +error: +free (resolved_offsets); +free (link); +return libbpf_err_ptr (err); +} + LIBBPF_API struct bpf_link * bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, const char *binary_path, size_t func_offset,tools/lib/bpf/libbpf.h +0
-0@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,const char *pattern, const struct bpf_kprobe_multi_opts *opts);
Quoted 6 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 535
Quoted 4 lines
tools/lib/bpf/libbpf.h +4
-0+struct bpf_uprobe_multi_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; +/* array of function symbols to attach */ attach to?
ok
Quoted 5 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 537
Quoted 3 lines
tools/lib/bpf/libbpf.h +2
-0+const char **syms; +/* array of function addresses to attach */ attach to?
ook
Quoted 6 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 539
Quoted 3 lines
tools/lib/bpf/libbpf.h +2
-0+const unsigned long *offsets; +/* array of refctr offsets to attach */ we don't really attach to ref counters, so maybe "optional, array of associated ref counter offsets" or something along those lines ?
ok
Quoted 6 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 541
Quoted 3 lines
tools/lib/bpf/libbpf.h +2
-0+const unsigned long *ref_ctr_offsets; +/* array of user-provided values fetchable through bpf_get_attach_cookie */ "array of associated BPF cookies"? we can't keep explaining what BPF cookie is in every possible API :)
ook
Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.h +14
-0+const __u64 *cookies; +/* number of elements in syms/addrs/cookies arrays */ +size_t cnt; +/* create return uprobes */ +bool retprobe; + size_t :0; +}; + +#define bpf_uprobe_multi_opts__last_field retprobe + +LIBBPF_API struct bpf_link * +bpf_program__attach_uprobe_multi(const struct bpf_program *prog, + pid_t pid, + const char *binary_path,
Quoted 8 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 558
Quoted 3 lines
tools/lib/bpf/libbpf.h +3
-0+const char *func_pattern, + const struct bpf_uprobe_multi_opts *opts); +ok, let's be good citizens and add documentation for this new API. Those comments about valid combinations belong here as well. Please take a look at existing doccomments for the format and conventions. Thanks!
ok, will add
thanks,
jirka
Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function
Quoted 85 lines · Jiri Olsa
On Thu, Jul 06, 2023 at 09:05:23PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 36 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14
-0+if (!OPTS_VALID (opts, bpf_uprobe_multi_opts)) +return libbpf_err_ptr (-EINVAL ); + + syms =OPTS_GET (opts, syms,NULL ); + offsets =OPTS_GET (opts, offsets,NULL ); + ref_ctr_offsets =OPTS_GET (opts, ref_ctr_offsets,NULL ); + cookies =OPTS_GET (opts, cookies,NULL ); + cnt =OPTS_GET (opts, cnt,0 ); + +/* +* User can specify 2 mutually exclusive set of inputs: +* +* 1) use only path/func_pattern/pid arguments +* tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11000
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+* 2) use path/pid with allowed combinations of: +* syms/offsets/ref_ctr_offsets/cookies/cnt +* +* - syms and offsets are mutually exclusive +* - ref_ctr_offsets and cookies are optional +* +* Any other usage results in error. +*/ + +if (!path && !func_pattern && !cnt)weird, I'd expect separate if (!path) return error (already bad, regardless of
func_patternor cnt)then if (!func_pattern && cnt == 0) return error
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11005
Quoted 5 lines
tools/lib/bpf/libbpf.c +5
-0+return libbpf_err_ptr (-EINVAL ); +if (func_pattern && !path) +return libbpf_err_ptr (-EINVAL ); + + has_pattern = path && func_pattern;Quoted 3 lines
this and above check must be some leftovers from previous version. path should always be present. and so you don't need
has_patternvariable, just use "func_pattern" checkhum, right, previous version had 2 paths, now there's just one, I'll change that together with the suggested change above
Quoted 28 lines
Quoted 28 lines
tools/lib/bpf/libbpf.c +27
-0+ +if (has_pattern) { +if (syms || offsets || ref_ctr_offsets || cookies || cnt) +return libbpf_err_ptr (-EINVAL ); + }else { +if (!cnt) +return libbpf_err_ptr (-EINVAL ); +if (!!syms == !!offsets) +return libbpf_err_ptr (-EINVAL ); + } + +if (has_pattern) { +if (!strchr (path,'/' )) { + err =resolve_full_path (path, full_path,sizeof (full_path)); +if (err) { +pr_warn ("prog '%s': failed to resolve full path for '%s': %d\n" , + prog->name, path, err); +return libbpf_err_ptr (err); + } + path = full_path; + } + + err =elf_resolve_pattern_offsets (path, func_pattern, + &resolved_offsets, &cnt); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets;Quoted 8 lines
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11037
Quoted 5 lines
tools/lib/bpf/libbpf.c +5
-0+ }else if (syms) { + err =elf_resolve_syms_offsets (path, cnt, syms, &resolved_offsets); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets;you can extract this common error checking and
offsets = resolved_offsets;to after if, it's common for both branches
Quoted 2 lines
not sure what you mean in here, offsets can be also provided by
OPTS_GET(opts, offsets, NULL)earlier
ah, I missed that it's else if, not just else. Never mind, it's good as is then.
Quoted 130 lines
Quoted 13 lines
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11047
Quoted 11 lines
tools/lib/bpf/libbpf.c +10
-0+ } + + retprobe =OPTS_GET (opts, retprobe, false); + + lopts.uprobe_multi.path = path; + lopts.uprobe_multi.offsets = offsets; + lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; + lopts.uprobe_multi.cookies = cookies; + lopts.uprobe_multi.cnt = cnt; + lopts.uprobe_multi.flags = retprobe ?BPF_F_UPROBE_MULTI_RETURN :0 ;retprobe is another unnecessary var, just inline check here to keep it simpler
ok
Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.c +14
-0+ +if (pid ==0 ) + pid =getpid (); +if (pid >0 ) + lopts.uprobe_multi.pid = pid; + + link =calloc (1 ,sizeof (*link)); +if (!link) { + err = -ENOMEM ; + goto error; + } + link->detach = &bpf_link__detach_fd; + + prog_fd =bpf_program__fd (prog);Quoted 8 lines
tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11065
Quoted 4 lines
tools/lib/bpf/libbpf.c +4
-0+ link_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &lopts); +if (link_fd <0 ) { + err = -errno; + pr_warn("prog '%s': failed to attach: %s\n","failed to attach multi-uprobe"? We probably should have added "failed to attach multi-kprobe" in
bpf_program__attach_kprobe_multi_optsas well?ook, will add
Quoted 25 lines
Quoted 25 lines
tools/lib/bpf/libbpf.c +13
-0+ prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg))); + goto error; + } + link->fd = link_fd; +free (resolved_offsets); +return link; + +error: +free (resolved_offsets); +free (link); +return libbpf_err_ptr (err); +} + LIBBPF_API struct bpf_link * bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, const char *binary_path, size_t func_offset,tools/lib/bpf/libbpf.h +0
-0@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,const char *pattern, const struct bpf_kprobe_multi_opts *opts);Quoted 6 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 535
Quoted 4 lines
tools/lib/bpf/libbpf.h +4
-0+struct bpf_uprobe_multi_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; +/* array of function symbols to attach */ attach to?
ok
Quoted 5 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 537
Quoted 3 lines
tools/lib/bpf/libbpf.h +2
-0+const char **syms; +/* array of function addresses to attach */ attach to?
ook
Quoted 6 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 539
Quoted 3 lines
tools/lib/bpf/libbpf.h +2
-0+const unsigned long *offsets; +/* array of refctr offsets to attach */ we don't really attach to ref counters, so maybe "optional, array of associated ref counter offsets" or something along those lines ?
ok
Quoted 6 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 541
Quoted 3 lines
tools/lib/bpf/libbpf.h +2
-0+const unsigned long *ref_ctr_offsets; +/* array of user-provided values fetchable through bpf_get_attach_cookie */ "array of associated BPF cookies"? we can't keep explaining what BPF cookie is in every possible API :)
ook
Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.h +14
-0+const __u64 *cookies; +/* number of elements in syms/addrs/cookies arrays */ +size_t cnt; +/* create return uprobes */ +bool retprobe; + size_t :0; +}; + +#define bpf_uprobe_multi_opts__last_field retprobe + +LIBBPF_API struct bpf_link * +bpf_program__attach_uprobe_multi(const struct bpf_program *prog, + pid_t pid, + const char *binary_path,Quoted 8 lines
tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 558
Quoted 3 lines
tools/lib/bpf/libbpf.h +3
-0+const char *func_pattern, + const struct bpf_uprobe_multi_opts *opts); +ok, let's be good citizens and add documentation for this new API. Those comments about valid combinations belong here as well. Please take a look at existing doccomments for the format and conventions. Thanks!
ok, will add
thanks,
jirka
Jiri Olsa
reviewerPATCH v3 14/26 libbpf: Add support for u[ret]probe.multi[.s] program sections
In reply to Jiri OlsaAdding support for several uprobe_multi program sections to allow auto attach of multi_uprobe programs.
--- tools/lib/bpf/libbpf.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
tools/lib/bpf/libbpf.c +36 -0
@@ -8654,6 +8654,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_linstatic int attach_raw_tp (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_trace (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_kprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link); +static int attach_uprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_lsm (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_iter (const struct bpf_program *prog,long cookie,struct bpf_link **link);
@@ -8669,6 +8670,10 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("uretprobe.s+" ,KPROBE ,0 ,SEC_SLEEPABLE , attach_uprobe),SEC_DEF ("kprobe.multi+" ,KPROBE ,BPF_TRACE_KPROBE_MULTI ,SEC_NONE , attach_kprobe_multi),SEC_DEF ("kretprobe.multi+" ,KPROBE ,BPF_TRACE_KPROBE_MULTI ,SEC_NONE , attach_kprobe_multi), +SEC_DEF ("uprobe.multi+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_NONE , attach_uprobe_multi), +SEC_DEF ("uretprobe.multi+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_NONE , attach_uprobe_multi), +SEC_DEF ("uprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi), +SEC_DEF ("uretprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi),SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall), SEC_DEF("usdt+", KPROBE, 0, SEC_NONE, attach_usdt),
@@ -10730,6 +10735,37 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, strureturn libbpf_get_error (*link); } +static int attach_uprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link) +{ +char *probe_type =NULL , *binary_path =NULL , *func_name =NULL ; +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); +int n, ret = -EINVAL ; + + *link =NULL ; + + n =sscanf (prog->sec_name,"%m[^/]/%m[^:]:%ms" , + &probe_type, &binary_path, &func_name); +switch (n) { +case 1 : +/* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */ + ret =0 ; +break ; +case 3 : + opts.retprobe =strcmp (probe_type,"uretprobe.multi" ) ==0 ; + *link =bpf_program__attach_uprobe_multi (prog,-1 , binary_path, func_name, &opts); + ret =libbpf_get_error (*link); +break ; +default : +pr_warn ("prog '%s': invalid format of section definition '%s'\n" , prog->name, + prog->sec_name); +break ; + } +free (probe_type); +free (binary_path); +free (func_name); +return ret; +} +static void gen_uprobe_legacy_event_name (char *buf,size_t buf_sz,const char *binary_path,uint64_t offset) {
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 14/26] libbpf: Add support for u[ret]probe.multi[.s] program sections
Quoted 9 lines · Jiri Olsa
Adding support for several
uprobe_multiprogram sections to allow auto attach ofmulti_uprobeprograms.Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/libbpf.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
LGTM
Quoted 64 lines
tools/lib/bpf/libbpf.c +36
-1@@ -8654,6 +8654,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_linstatic int attach_raw_tp (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_trace (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_kprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link); +static int attach_uprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_lsm (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_iter (const struct bpf_program *prog,long cookie,struct bpf_link **link);@@ -8669,6 +8670,10 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("uretprobe.s+" ,KPROBE ,0 ,SEC_SLEEPABLE , attach_uprobe),SEC_DEF ("kprobe.multi+" ,KPROBE ,BPF_TRACE_KPROBE_MULTI ,SEC_NONE , attach_kprobe_multi),SEC_DEF ("kretprobe.multi+" ,KPROBE ,BPF_TRACE_KPROBE_MULTI ,SEC_NONE , attach_kprobe_multi), +SEC_DEF ("uprobe.multi+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_NONE , attach_uprobe_multi), +SEC_DEF ("uretprobe.multi+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_NONE , attach_uprobe_multi), +SEC_DEF ("uprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi), +SEC_DEF ("uretprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi),SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall), SEC_DEF("usdt+", KPROBE, 0, SEC_NONE, attach_usdt),@@ -10730,6 +10735,37 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, strureturn libbpf_get_error (*link); } +static int attach_uprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link) +{ +char *probe_type =NULL , *binary_path =NULL , *func_name =NULL ; +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); +int n, ret = -EINVAL ; + + *link =NULL ; + + n =sscanf (prog->sec_name,"%m[^/]/%m[^:]:%ms" , + &probe_type, &binary_path, &func_name); +switch (n) { +case 1 : +/* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */ + ret =0 ; +break ; +case 3 : + opts.retprobe =strcmp (probe_type,"uretprobe.multi" ) ==0 ; + *link =bpf_program__attach_uprobe_multi (prog,-1 , binary_path, func_name, &opts); + ret =libbpf_get_error (*link); +break ; +default : +pr_warn ("prog '%s': invalid format of section definition '%s'\n" , prog->name, + prog->sec_name); +break ; + } +free (probe_type); +free (binary_path); +free (func_name); +return ret; +} +static void gen_uprobe_legacy_event_name (char *buf,size_t buf_sz,const char *binary_path,uint64_t offset) { --2.41.0
Jiri Olsa
reviewerPATCH v3 15/26 libbpf: Add uprobe multi link detection
In reply to Jiri OlsaAdding uprobe-multi link detection. It will be used later in bpf_program__attach_usdt function to check and use uprobe_multi link over standard uprobe links.
--- tools/lib/bpf/libbpf.c | 35 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 2 files changed, 37 insertions(+)
tools/lib/bpf/libbpf.c +35 -0
@@ -4817,6 +4817,38 @@ static int probe_perf_link(void)return link_fd <0 && err == -EBADF ; } +static int probe_uprobe_multi_link (void ) +{ +LIBBPF_OPTS (bpf_prog_load_opts, load_opts, + .expected_attach_type = BPF_TRACE_UPROBE_MULTI, + ); +LIBBPF_OPTS (bpf_link_create_opts, link_opts); +struct bpf_insn insns[] = { +BPF_MOV64_IMM (BPF_REG_0 ,0 ), +BPF_EXIT_INSN (), + }; +unsigned long offset =0 ; +int prog_fd, link_fd; + + prog_fd =bpf_prog_load (BPF_PROG_TYPE_KPROBE ,NULL ,"GPL" , + insns,ARRAY_SIZE (insns), &load_opts); +if (prog_fd <0 ) +return -errno; + +/* create single uprobe on offset 0 in current process */ + link_opts.uprobe_multi.path ="/proc/self/exe" ; + link_opts.uprobe_multi.offsets = &offset; + link_opts.uprobe_multi.cnt =1 ; + + link_fd =bpf_link_create (prog_fd,-1 ,BPF_TRACE_UPROBE_MULTI , &link_opts); + +if (link_fd >=0 ) +close (link_fd); +close (prog_fd); + +return link_fd >=0 ; +} +static int probe_kern_bpf_cookie (void ) {struct bpf_insn insns[] = {
@@ -4913,6 +4945,9 @@ static struct kern_feature_desc { [FEAT_SYSCALL_WRAPPER ] = {"Kernel using syscall wrapper" , probe_kern_syscall_wrapper, }, + [FEAT_UPROBE_MULTI_LINK ] = { +"BPF uprobe multi link support" , probe_uprobe_multi_link, + }, };bool kernel_supports (const struct bpf_object *obj,enum kern_feature_id feat_id)
tools/lib/bpf/libbpf_internal.h +2 -0
@@ -354,6 +354,8 @@ enum kern_feature_id {FEAT_BTF_ENUM64 ,/* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */ FEAT_SYSCALL_WRAPPER , +/* BPF uprobe_multi link support */ +FEAT_UPROBE_MULTI_LINK , __FEAT_CNT, };
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 15/26] libbpf: Add uprobe multi link detection
Quoted 34 lines · Jiri Olsa
Adding uprobe-multi link detection. It will be used later in
bpf_program__attach_usdtfunction to check and useuprobe_multilink over standard uprobe links.Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/libbpf.c | 35 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 2 files changed, 37 insertions(+)tools/lib/bpf/libbpf.c +15
-0@@ -4817,6 +4817,38 @@ static int probe_perf_link(void)return link_fd <0 && err == -EBADF ; } +static int probe_uprobe_multi_link (void ) +{ +LIBBPF_OPTS (bpf_prog_load_opts, load_opts, + .expected_attach_type = BPF_TRACE_UPROBE_MULTI, + ); +LIBBPF_OPTS (bpf_link_create_opts, link_opts); +struct bpf_insn insns[] = { +BPF_MOV64_IMM (BPF_REG_0 ,0 ), +BPF_EXIT_INSN (), + }; +unsigned long offset =0 ; +int prog_fd, link_fd; + + prog_fd =bpf_prog_load (BPF_PROG_TYPE_KPROBE ,NULL ,"GPL" , + insns,ARRAY_SIZE (insns), &load_opts);
tools/lib/bpf/libbpf.c static int probe_perf_link(void) · line 4844
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+if (prog_fd <0 ) +return -errno; + +/* create single uprobe on offset 0 in current process */ + link_opts.uprobe_multi.path ="/proc/self/exe" ; + link_opts.uprobe_multi.offsets = &offset; + link_opts.uprobe_multi.cnt =1 ; + + link_fd =bpf_link_create (prog_fd,-1 ,BPF_TRACE_UPROBE_MULTI , &link_opts); +
so I'd like us to avoid successfully attaching anything. This might have unintended consequences (e.g., unintentionally breaking backing huge pages into normal pages, just because we happen to successfully attach briefly). So let's work on feature detection that fails to create a link, but does it in a way that we know that the feature itself is supported by the kernel
Some ideas we could do:
1. Pass invalid file (e.g., just root, "/" as path), but modify kernel-side logic to return not -EINVAL, but -EBADF (and I think it would be good to do this anyway). Then expect -EBADF as a signal that the feature is supported.
2. Also, we can return -EPROTO instead of -EINVAL on invalid combination of paramers or something like that
I'd start with -EBADF change.
In general, we should write kernel-side code in such a way that allows simple and efficient feature-detection. We shouldn't repeat the nightmare of memcg-based mem accounting :(
Quoted 11 lines
tools/lib/bpf/libbpf.c +7
-0+if (link_fd >=0 ) +close (link_fd); +close (prog_fd); + +return link_fd >=0 ; +} +static int probe_kern_bpf_cookie (void ) {struct bpf_insn insns[] = {
tools/lib/bpf/libbpf.c static struct kern_feature_desc { · line 4949
Quoted 5 lines
tools/lib/bpf/libbpf.c +2
-0@@ -4913,6 +4945,9 @@ static struct kern_feature_desc { [FEAT_SYSCALL_WRAPPER] = { "Kernel using syscall wrapper", probe_kern_syscall_wrapper, }, + [FEAT_UPROBE_MULTI_LINK] = { + "BPF uprobe multi link support", probe_uprobe_multi_link,
nit: BPF multi-uprobe link support
tools/lib/bpf/libbpf_internal.h enum kern_feature_id { · line 357
Quoted 13 lines
tools/lib/bpf/libbpf.c +1
-0+ }, };bool kernel_supports (const struct bpf_object *obj,enum kern_feature_id feat_id)tools/lib/bpf/libbpf_internal.h +1
-0@@ -354,6 +354,8 @@ enum kern_feature_id {FEAT_BTF_ENUM64 ,/* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */ FEAT_SYSCALL_WRAPPER, +/* BPF uprobe_multi link support */
same, multi-uprobe link support
Quoted 7 lines
tools/lib/bpf/libbpf_internal.h +1
-1+FEAT_UPROBE_MULTI_LINK , __FEAT_CNT, }; --2.41.0
Re: [PATCHv3 bpf-next 15/26] libbpf: Add uprobe multi link detection
Quoted 53 lines · Andrii Nakryiko
Quoted 34 lines · Jiri Olsa
Adding uprobe-multi link detection. It will be used later in
bpf_program__attach_usdtfunction to check and useuprobe_multilink over standard uprobe links.Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/libbpf.c | 35 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 2 files changed, 37 insertions(+)tools/lib/bpf/libbpf.c +15
-0@@ -4817,6 +4817,38 @@ static int probe_perf_link(void)return link_fd <0 && err == -EBADF ; } +static int probe_uprobe_multi_link (void ) +{ +LIBBPF_OPTS (bpf_prog_load_opts, load_opts, + .expected_attach_type = BPF_TRACE_UPROBE_MULTI, + ); +LIBBPF_OPTS (bpf_link_create_opts, link_opts); +struct bpf_insn insns[] = { +BPF_MOV64_IMM (BPF_REG_0 ,0 ), +BPF_EXIT_INSN (), + }; +unsigned long offset =0 ; +int prog_fd, link_fd; + + prog_fd =bpf_prog_load (BPF_PROG_TYPE_KPROBE ,NULL ,"GPL" , + insns,ARRAY_SIZE (insns), &load_opts);tools/lib/bpf/libbpf.c static int probe_perf_link(void) · line 4844
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+if (prog_fd <0 ) +return -errno; + +/* create single uprobe on offset 0 in current process */ + link_opts.uprobe_multi.path ="/proc/self/exe" ; + link_opts.uprobe_multi.offsets = &offset; + link_opts.uprobe_multi.cnt =1 ; + + link_fd =bpf_link_create (prog_fd,-1 ,BPF_TRACE_UPROBE_MULTI , &link_opts); +so I'd like us to avoid successfully attaching anything. This might have unintended consequences (e.g., unintentionally breaking backing huge pages into normal pages, just because we happen to successfully attach briefly). So let's work on feature detection that fails to create a link, but does it in a way that we know that the feature itself is supported by the kernel
Quoted 6 lines
Some ideas we could do:
1. Pass invalid file (e.g., just root, "/" as path), but modify kernel-side logic to return not
-EINVAL, but-EBADF(and I think it would be good to do this anyway). Then expect-EBADFas a signal that the feature is supported.
ah ok, so -EBADF from inside uprobe_multi link setup code would mean it's supported, anything else means it's not.. should work also for old kernels, I don't think we have -EBADF in related paths, will check
Quoted 32 lines
2. Also, we can return
-EPROTOinstead of-EINVALon invalid combination of paramers or something like thatI'd start with
-EBADFchange.In general, we should write kernel-side code in such a way that allows simple and efficient feature-detection. We shouldn't repeat the nightmare of memcg-based mem accounting :(
Quoted 11 lines
tools/lib/bpf/libbpf.c +7
-0+if (link_fd >=0 ) +close (link_fd); +close (prog_fd); + +return link_fd >=0 ; +} +static int probe_kern_bpf_cookie (void ) {struct bpf_insn insns[] = {tools/lib/bpf/libbpf.c static struct kern_feature_desc { · line 4949
Quoted 5 lines
tools/lib/bpf/libbpf.c +2
-0@@ -4913,6 +4945,9 @@ static struct kern_feature_desc { [FEAT_SYSCALL_WRAPPER] = { "Kernel using syscall wrapper", probe_kern_syscall_wrapper, }, + [FEAT_UPROBE_MULTI_LINK] = { + "BPF uprobe multi link support", probe_uprobe_multi_link,nit: BPF multi-uprobe link support
Quoted 3 lines
tools/lib/bpf/libbpf.c +1
-0+ }, };
Quoted 12 lines
tools/lib/bpf/libbpf_internal.h enum kern_feature_id { · line 357
Quoted 10 lines
tools/lib/bpf/libbpf.c +0
-0bool kernel_supports (const struct bpf_object *obj,enum kern_feature_id feat_id)tools/lib/bpf/libbpf_internal.h +1
-0@@ -354,6 +354,8 @@ enum kern_feature_id {FEAT_BTF_ENUM64 ,/* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */ FEAT_SYSCALL_WRAPPER, +/* BPF uprobe_multi link support */ same, multi-uprobe link support
ok, thanks
jirka
Quoted 10 lines
Quoted 10 lines
tools/lib/bpf/libbpf_internal.h +1
-1+FEAT_UPROBE_MULTI_LINK , __FEAT_CNT, }; --2.41.0
Jiri Olsa
reviewerPATCH v3 16/26 libbpf: Add uprobe multi link support to bpf_program__attach_usdt
In reply to Jiri OlsaAdding support for usdt_manager_attach_usdt to use uprobe_multi link to attach to usdt probes.
The uprobe_multi support is detected before the usdt program is loaded and its expected_attach_type is set accordingly.
If uprobe_multi support is detected the usdt_manager_attach_usdt gathers uprobes info and calls bpf_program__attach_uprobe to create all needed uprobes.
If uprobe_multi support is not detected the old behaviour stays.
Also adding usdt.s program section for sleepable usdt probes.
--- tools/lib/bpf/libbpf.c | 13 +++++-- tools/lib/bpf/usdt.c | 78 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 75 insertions(+), 16 deletions(-)
tools/lib/bpf/libbpf.c +11 -2
@@ -365,6 +365,8 @@ enum sec_def_flags {SEC_SLEEPABLE =8 ,/* BPF program support non-linear XDP buffer */ SEC_XDP_FRAGS =16 , +/* Setup proper attach type for usdt probes. */ + SEC_USDT = 32, }; struct bpf_sec_def {
@@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,if (prog->type ==BPF_PROG_TYPE_XDP && (def &SEC_XDP_FRAGS )) opts->prog_flags |=BPF_F_XDP_HAS_FRAGS ; +/* special check for usdt to use uprobe_multi link */ +if ((def &SEC_USDT ) &&kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK )) + prog->expected_attach_type =BPF_TRACE_UPROBE_MULTI ; +if ((def &SEC_ATTACH_BTF ) && !prog->attach_btf_id) {int btf_obj_fd =0 , btf_type_id =0 , err;const char *attach_name;
@@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *progif (!insns || !insns_cnt)return -EINVAL ; - load_attr.expected_attach_type = prog->expected_attach_type;if (kernel_supports (obj,FEAT_PROG_NAME )) prog_name = prog->name; load_attr.attach_prog_fd = prog->attach_prog_fd;
@@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog insns_cnt = prog->insns_cnt; } +/* allow prog_prepare_load_fn to change expected_attach_type */ + load_attr.expected_attach_type = prog->expected_attach_type; +if (obj->gen_loader) { bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name, license, insns, insns_cnt, &load_attr,
@@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("uretprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi),SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall), -SEC_DEF ("usdt+" ,KPROBE ,0 ,SEC_NONE , attach_usdt), +SEC_DEF ("usdt+" ,KPROBE ,0 ,SEC_USDT , attach_usdt), +SEC_DEF ("usdt.s+" ,KPROBE ,0 ,SEC_USDT |SEC_SLEEPABLE , attach_usdt),SEC_DEF ("tc" ,SCHED_CLS ,0 ,SEC_NONE ),SEC_DEF ("classifier" ,SCHED_CLS ,0 ,SEC_NONE ), SEC_DEF("action", SCHED_ACT, 0, SEC_NONE),
tools/lib/bpf/usdt.c +64 -14
123 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 16/26] libbpf: Add uprobe multi link support to bpf_program__attach_usdt
Quoted 35 lines · Jiri Olsa
Adding support for
usdt_manager_attach_usdtto useuprobe_multilink to attach to usdt probes.The
uprobe_multisupport is detected before the usdt program is loaded and itsexpected_attach_typeis set accordingly.If
uprobe_multisupport is detected theusdt_manager_attach_usdtgathers uprobes info and callsbpf_program__attach_uprobeto create all needed uprobes.If
uprobe_multisupport is not detected the old behaviour stays.Also adding usdt.s program section for sleepable usdt probes.
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/libbpf.c | 13 +++++-- tools/lib/bpf/usdt.c | 78 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 75 insertions(+), 16 deletions(-)tools/lib/bpf/libbpf.c +2
-0@@ -365,6 +365,8 @@ enum sec_def_flags {SEC_SLEEPABLE =8 ,/* BPF program support non-linear XDP buffer */ SEC_XDP_FRAGS =16 , +/* Setup proper attach type for usdt probes. */ + SEC_USDT = 32, }; struct bpf_sec_def {
tools/lib/bpf/libbpf.c static int libbpf_prepare_prog_load(struct bpf_program *prog, · line 6813
Quoted 5 lines
tools/lib/bpf/libbpf.c +2
-0@@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,if (prog->type ==BPF_PROG_TYPE_XDP && (def &SEC_XDP_FRAGS )) opts->prog_flags |=BPF_F_XDP_HAS_FRAGS ; +/* special check for usdt to use uprobe_multi link */ +if ((def &SEC_USDT ) &&kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK ))
please pass prog->obj to kernel_supports(), it will be especially important later with BPF token stuff
Quoted 24 lines
tools/lib/bpf/libbpf.c +5
-1+ prog->expected_attach_type =BPF_TRACE_UPROBE_MULTI ; +if ((def &SEC_ATTACH_BTF ) && !prog->attach_btf_id) {int btf_obj_fd =0 , btf_type_id =0 , err;const char *attach_name;@@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *progif (!insns || !insns_cnt)return -EINVAL ; - load_attr.expected_attach_type = prog->expected_attach_type;if (kernel_supports (obj,FEAT_PROG_NAME )) prog_name = prog->name; load_attr.attach_prog_fd = prog->attach_prog_fd;@@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog insns_cnt = prog->insns_cnt; } +/* allow prog_prepare_load_fn to change expected_attach_type */ + load_attr.expected_attach_type = prog->expected_attach_type; +if (obj->gen_loader) { bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name, license, insns, insns_cnt, &load_attr,
tools/lib/bpf/libbpf.c static const struct bpf_sec_def section_defs[] = { · line 8723
Quoted 6 lines
tools/lib/bpf/libbpf.c +2
-1@@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("uretprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi),SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall), - SEC_DEF("usdt+", KPROBE, 0, SEC_NONE, attach_usdt), +SEC_DEF ("usdt+" ,KPROBE ,0 ,SEC_USDT , attach_usdt), + SEC_DEF("usdt.s+", KPROBE, 0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),
spaces around |
Quoted 17 lines
tools/lib/bpf/libbpf.c +0
-0SEC_DEF ("tc" ,SCHED_CLS ,0 ,SEC_NONE ),SEC_DEF ("classifier" ,SCHED_CLS ,0 ,SEC_NONE ), SEC_DEF("action", SCHED_ACT, 0, SEC_NONE),tools/lib/bpf/usdt.c +2
-0@@ -809,6 +809,8 @@ struct bpf_link_usdt {long abs_ip;struct bpf_link *link; } *uprobes; + +struct bpf_link *multi_link; };static int bpf_link_usdt_detach (struct bpf_link *link)
tools/lib/bpf/usdt.c static int bpf_link_usdt_detach(struct bpf_link *link) · line 822
Quoted 4 lines
tools/lib/bpf/usdt.c +1
-0@@ -817,6 +819,9 @@ static int bpf_link_usdt_detach(struct bpf_link *link)struct usdt_manager *man = usdt_link->usdt_man;int i; +/* When having multi_link, uprobe_cnt is 0 */
misplaced comment, move down to for() loop?
Quoted 20 lines
tools/lib/bpf/usdt.c +4
-0+bpf_link__destroy (usdt_link->multi_link); +for (i =0 ; i < usdt_link->uprobe_cnt; i++) {/* detach underlying uprobe link */ bpf_link__destroy (usdt_link->uprobes[i].link);@@ -944,11 +949,13 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct const char *usdt_provider, const char *usdt_name, __u64 usdt_cookie) { + unsigned long *offsets = NULL, *ref_ctr_offsets = NULL;int i, err, spec_map_fd, ip_map_fd;LIBBPF_OPTS (bpf_uprobe_opts, opts);struct hashmap *specs_hash =NULL ;struct bpf_link_usdt *link =NULL ;struct usdt_target *targets =NULL ; +__u64 *cookies =NULL ;struct elf_fd elf_fd;size_t target_cnt;
tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1005
Quoted 8 lines
tools/lib/bpf/usdt.c +1
-4@@ -995,10 +1002,21 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct link->link.detach = &bpf_link_usdt_detach; link->link.dealloc = &bpf_link_usdt_dealloc; - link->uprobes =calloc (target_cnt,sizeof (*link->uprobes)); -if (!link->uprobes) { - err = -ENOMEM ; - goto err_out; +if (kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK )) {
see how we feature-detect has_sema_refcnt and has_bpf_cookie, let's do the same with UPROBE_MULTI_LINK, detect once, remember, consistently use it (it also matters later for BPF token)
Quoted 39 lines
tools/lib/bpf/usdt.c +27
-5+ offsets =calloc (target_cnt,sizeof (*offsets)); + cookies =calloc (target_cnt,sizeof (*cookies)); + ref_ctr_offsets =calloc (target_cnt,sizeof (*ref_ctr_offsets)); + +if (!offsets || !ref_ctr_offsets || !cookies) { + err = -ENOMEM ; + goto err_out; + } + }else { + link->uprobes =calloc (target_cnt,sizeof (*link->uprobes)); +if (!link->uprobes) { + err = -ENOMEM ; + goto err_out; + } }for (i =0 ; i < target_cnt; i++) {@@ -1039,20 +1057,48 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct goto err_out; } - opts.ref_ctr_offset = target->sema_off; - opts.bpf_cookie = man->has_bpf_cookie ? spec_id :0 ; - uprobe_link =bpf_program__attach_uprobe_opts (prog, pid, path, - target->rel_ip, &opts); - err =libbpf_get_error (uprobe_link); +if (kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK )) { + offsets[i] = target->rel_ip; + ref_ctr_offsets[i] = target->sema_off; + cookies[i] = spec_id; + }else { + opts.ref_ctr_offset = target->sema_off; + opts.bpf_cookie = man->has_bpf_cookie ? spec_id :0 ; + uprobe_link =bpf_program__attach_uprobe_opts (prog, pid, path, + target->rel_ip, &opts); + err =libbpf_get_error (uprobe_link); +if (err) { +pr_warn ("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n" , + i, usdt_provider, usdt_name, path, err);
tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1082
Quoted 10 lines
tools/lib/bpf/usdt.c +10
-0+ goto err_out; + } + + link->uprobes[i].link = uprobe_link; + link->uprobes[i].abs_ip = target->abs_ip; + link->uprobe_cnt++; + } + } + +if (kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK )) {
same as above, we should feature-detect once per usdt_manager while we have associated bpf_object
tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1092
Quoted 10 lines
tools/lib/bpf/usdt.c +10
-0+LIBBPF_OPTS (bpf_uprobe_multi_opts, opts_multi, + .cnt = target_cnt, + .offsets = offsets, + .ref_ctr_offsets = ref_ctr_offsets, + .cookies = cookies, + ); + + link->multi_link =bpf_program__attach_uprobe_multi (prog, pid, path, +NULL , &opts_multi); + err =libbpf_get_error (link->multi_link);
let's not use libbpf_get_error() in new code, there is no need, just err = -errno and if (!link->multi_link)
Quoted 31 lines
tools/lib/bpf/usdt.c +9
-6if (err) { -pr_warn ("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n" , - i, usdt_provider, usdt_name, path, err); +pr_warn ("usdt: failed to attach uprobe multi for '%s:%s' in '%s': %d\n" , + usdt_provider, usdt_name, path, err); goto err_out; } - link->uprobes[i].link = uprobe_link; - link->uprobes[i].abs_ip = target->abs_ip; - link->uprobe_cnt++; +free (offsets); +free (ref_ctr_offsets); +free (cookies); }free (targets);@@ -1061,6 +1107,10 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const structreturn &link->link; err_out: +free (offsets); +free (ref_ctr_offsets); +free (cookies); +if (link)bpf_link__destroy (&link->link);free (targets); --2.41.0
Re: [PATCHv3 bpf-next 16/26] libbpf: Add uprobe multi link support to bpf_program__attach_usdt
Quoted 39 lines · Andrii Nakryiko
Quoted 38 lines · Jiri Olsa
Adding support for
usdt_manager_attach_usdtto useuprobe_multilink to attach to usdt probes.The
uprobe_multisupport is detected before the usdt program is loaded and itsexpected_attach_typeis set accordingly.If
uprobe_multisupport is detected theusdt_manager_attach_usdtgathers uprobes info and callsbpf_program__attach_uprobeto create all needed uprobes.If
uprobe_multisupport is not detected the old behaviour stays.Also adding usdt.s program section for sleepable usdt probes.
Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/bpf/libbpf.c | 13 +++++-- tools/lib/bpf/usdt.c | 78 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 75 insertions(+), 16 deletions(-)tools/lib/bpf/libbpf.c +2
-0@@ -365,6 +365,8 @@ enum sec_def_flags {SEC_SLEEPABLE =8 ,/* BPF program support non-linear XDP buffer */ SEC_XDP_FRAGS =16 , +/* Setup proper attach type for usdt probes. */ + SEC_USDT = 32, }; struct bpf_sec_def {@@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,if (prog->type ==BPF_PROG_TYPE_XDP && (def &SEC_XDP_FRAGS )) opts->prog_flags |=BPF_F_XDP_HAS_FRAGS ;
Quoted 5 lines
tools/lib/bpf/libbpf.c static int libbpf_prepare_prog_load(struct bpf_program *prog, · line 6813
Quoted 2 lines
tools/lib/bpf/libbpf.c +2
-0+/* special check for usdt to use uprobe_multi link */ +if ((def &SEC_USDT ) &&kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK ))please pass
prog->objtokernel_supports(), it will be especially important later with BPF token stuff
did not realize I have it in prog->obj ;-) ok
Quoted 18 lines
Quoted 18 lines
tools/lib/bpf/libbpf.c +2
-1+ prog->expected_attach_type =BPF_TRACE_UPROBE_MULTI ; +if ((def &SEC_ATTACH_BTF ) && !prog->attach_btf_id) {int btf_obj_fd =0 , btf_type_id =0 , err;const char *attach_name;@@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *progif (!insns || !insns_cnt)return -EINVAL ; - load_attr.expected_attach_type = prog->expected_attach_type;if (kernel_supports (obj,FEAT_PROG_NAME )) prog_name = prog->name; load_attr.attach_prog_fd = prog->attach_prog_fd;@@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog insns_cnt = prog->insns_cnt; }
Quoted 15 lines
tools/lib/bpf/libbpf.c static const struct bpf_sec_def section_defs[] = { · line 8723
Quoted 13 lines
tools/lib/bpf/libbpf.c +5
-1+/* allow prog_prepare_load_fn to change expected_attach_type */ + load_attr.expected_attach_type = prog->expected_attach_type; +if (obj->gen_loader) { bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name, license, insns, insns_cnt, &load_attr,@@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("uretprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi),SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall), - SEC_DEF("usdt+", KPROBE, 0, SEC_NONE, attach_usdt), +SEC_DEF ("usdt+" ,KPROBE ,0 ,SEC_USDT , attach_usdt), + SEC_DEF("usdt.s+", KPROBE, 0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),spaces around |
ook
Quoted 16 lines
Quoted 16 lines
tools/lib/bpf/libbpf.c +0
-0SEC_DEF ("tc" ,SCHED_CLS ,0 ,SEC_NONE ),SEC_DEF ("classifier" ,SCHED_CLS ,0 ,SEC_NONE ), SEC_DEF("action", SCHED_ACT, 0, SEC_NONE),tools/lib/bpf/usdt.c +2
-0@@ -809,6 +809,8 @@ struct bpf_link_usdt {long abs_ip;struct bpf_link *link; } *uprobes; + +struct bpf_link *multi_link; };
Quoted 8 lines
tools/lib/bpf/usdt.c static int bpf_link_usdt_detach(struct bpf_link *link) · line 822
Quoted 6 lines
tools/lib/bpf/usdt.c +1
-0static int bpf_link_usdt_detach (struct bpf_link *link)@@ -817,6 +819,9 @@ static int bpf_link_usdt_detach(struct bpf_link *link)struct usdt_manager *man = usdt_link->usdt_man;int i; +/* When having multi_link, uprobe_cnt is 0 */ misplaced comment, move down to
for()loop?
right, will move
Quoted 24 lines
Quoted 24 lines
tools/lib/bpf/usdt.c +4
-0+bpf_link__destroy (usdt_link->multi_link); +for (i =0 ; i < usdt_link->uprobe_cnt; i++) {/* detach underlying uprobe link */ bpf_link__destroy (usdt_link->uprobes[i].link);@@ -944,11 +949,13 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct const char *usdt_provider, const char *usdt_name, __u64 usdt_cookie) { + unsigned long *offsets = NULL, *ref_ctr_offsets = NULL;int i, err, spec_map_fd, ip_map_fd;LIBBPF_OPTS (bpf_uprobe_opts, opts);struct hashmap *specs_hash =NULL ;struct bpf_link_usdt *link =NULL ;struct usdt_target *targets =NULL ; +__u64 *cookies =NULL ;struct elf_fd elf_fd;size_t target_cnt;@@ -995,10 +1002,21 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct link->link.detach = &bpf_link_usdt_detach; link->link.dealloc = &bpf_link_usdt_dealloc;
Quoted 9 lines
tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1005
Quoted 5 lines
tools/lib/bpf/usdt.c +1
-4- link->uprobes =calloc (target_cnt,sizeof (*link->uprobes)); -if (!link->uprobes) { - err = -ENOMEM ; - goto err_out; +if (kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK )) {see how we feature-detect
has_sema_refcntandhas_bpf_cookie, let's do the same withUPROBE_MULTI_LINK, detect once, remember, consistently use it (it also matters later for BPF token)
ok
Quoted 45 lines
Quoted 45 lines
tools/lib/bpf/usdt.c +32
-5+ offsets =calloc (target_cnt,sizeof (*offsets)); + cookies =calloc (target_cnt,sizeof (*cookies)); + ref_ctr_offsets =calloc (target_cnt,sizeof (*ref_ctr_offsets)); + +if (!offsets || !ref_ctr_offsets || !cookies) { + err = -ENOMEM ; + goto err_out; + } + }else { + link->uprobes =calloc (target_cnt,sizeof (*link->uprobes)); +if (!link->uprobes) { + err = -ENOMEM ; + goto err_out; + } }for (i =0 ; i < target_cnt; i++) {@@ -1039,20 +1057,48 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct goto err_out; } - opts.ref_ctr_offset = target->sema_off; - opts.bpf_cookie = man->has_bpf_cookie ? spec_id :0 ; - uprobe_link =bpf_program__attach_uprobe_opts (prog, pid, path, - target->rel_ip, &opts); - err =libbpf_get_error (uprobe_link); +if (kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK )) { + offsets[i] = target->rel_ip; + ref_ctr_offsets[i] = target->sema_off; + cookies[i] = spec_id; + }else { + opts.ref_ctr_offset = target->sema_off; + opts.bpf_cookie = man->has_bpf_cookie ? spec_id :0 ; + uprobe_link =bpf_program__attach_uprobe_opts (prog, pid, path, + target->rel_ip, &opts); + err =libbpf_get_error (uprobe_link); +if (err) { +pr_warn ("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n" , + i, usdt_provider, usdt_name, path, err); + goto err_out; + } + + link->uprobes[i].link = uprobe_link; + link->uprobes[i].abs_ip = target->abs_ip;
Quoted 8 lines
tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1082
Quoted 5 lines
tools/lib/bpf/usdt.c +5
-0+ link->uprobe_cnt++; + } + } + +if (kernel_supports (NULL ,FEAT_UPROBE_MULTI_LINK )) {same as above, we should feature-detect once per
usdt_managerwhile we have associatedbpf_object
ook
Quoted 12 lines
tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1092
Quoted 11 lines
tools/lib/bpf/usdt.c +10
-0+LIBBPF_OPTS (bpf_uprobe_multi_opts, opts_multi, + .cnt = target_cnt, + .offsets = offsets, + .ref_ctr_offsets = ref_ctr_offsets, + .cookies = cookies, + ); + + link->multi_link =bpf_program__attach_uprobe_multi (prog, pid, path, +NULL , &opts_multi); + err =libbpf_get_error (link->multi_link);
Quoted 2 lines
let's not use
libbpf_get_error()in new code, there is no need, justerr = -errnoandif (!link->multi_link)
ah right, did not see its comment that says it's no longer recomended, will change
thanks,
jirka
Jiri Olsa
reviewerPATCH v3 17/26 selftests/bpf: Add uprobe_multi skel test
In reply to Jiri OlsaAdding uprobe_multi test for skeleton load/attach functions, to test skeleton auto attach for uprobe_multi link.
Test that bpf_get_func_ip works properly for uprobe_multi attachment.
--- .../bpf/prog_tests/uprobe_multi_test.c | 76 ++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 91 +++++++++++++++++++ 2 files changed, 167 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c new file +76 -0
@@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <unistd.h> +#include <test_progs.h> +#include "uprobe_multi.skel.h" + +static char test_data[] ="test_data" ; + +noinline void uprobe_multi_func_1(void) +{ + asmvolatile ("" ); +} + +noinline void uprobe_multi_func_2(void) +{ + asmvolatile ("" ); +} + +noinline void uprobe_multi_func_3(void) +{ + asmvolatile ("" ); +} + +static void uprobe_multi_test_run (struct uprobe_multi *skel) +{ + skel->bss->uprobe_multi_func_1_addr = (__u64 ) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (__u64 ) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (__u64 ) uprobe_multi_func_3; + + skel->bss->user_ptr = test_data; + skel->bss->pid =getpid (); + +/* trigger all probes */ +uprobe_multi_func_1 (); +uprobe_multi_func_2 (); +uprobe_multi_func_3 (); + +/* +* There are 2 entry and 2 exit probe called for each uprobe_multi_func_[123] +* function and each slepable probe (6) increments uprobe_multi_sleep_result. +*/ +ASSERT_EQ (skel->bss->uprobe_multi_func_1_result,2 ,"uprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_2_result,2 ,"uprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_3_result,2 ,"uprobe_multi_func_3_result" ); + +ASSERT_EQ (skel->bss->uretprobe_multi_func_1_result,2 ,"uretprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_2_result,2 ,"uretprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_3_result,2 ,"uretprobe_multi_func_3_result" ); + +ASSERT_EQ (skel->bss->uprobe_multi_sleep_result,6 ,"uprobe_multi_sleep_result" ); +} + +static void test_skel_api (void ) +{ +struct uprobe_multi *skel =NULL ; +int err; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + err =uprobe_multi__attach (skel); +if (!ASSERT_OK (err,"uprobe_multi__attach" )) + goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +uprobe_multi__destroy (skel); +} + +void test_uprobe_multi_test (void ) +{ +if (test__start_subtest ("skel_api" )) +test_skel_api (); +}
tools/testing/selftests/bpf/progs/uprobe_multi.c new file +91 -0
91 lines not on this page · raw message
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v3 18/26 selftests/bpf: Add uprobe_multi api test
In reply to Jiri OlsaAdding uprobe_multi test for bpf_program__attach_uprobe_multi attach function.
Testing attachment using glob patterns and via bpf_uprobe_multi_opts paths/syms fields.
--- .../bpf/prog_tests/uprobe_multi_test.c | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +71 -0
@@ -69,8 +69,79 @@ static void test_skel_api(void)uprobe_multi__destroy (skel); } +static void +test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +{ +struct bpf_link *link1 =NULL , *link2 =NULL ; +struct bpf_link *link3 =NULL , *link4 =NULL ; +struct uprobe_multi *skel =NULL ; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + opts->retprobe = false; + link1 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + + opts->retprobe = true; + link2 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) + goto cleanup; + + opts->retprobe = false; + link3 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_sleep,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + + opts->retprobe = true; + link4 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe_sleep,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) + goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +bpf_link__destroy (link4); +bpf_link__destroy (link3); +bpf_link__destroy (link2); +bpf_link__destroy (link1); +uprobe_multi__destroy (skel); +} + +static void test_attach_api_pattern (void ) +{ +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); + +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_*" , &opts); +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_?" , &opts); +} + +static void test_attach_api_syms (void ) +{ +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); +const char *syms[3 ] = { +"uprobe_multi_func_1" , +"uprobe_multi_func_2" , +"uprobe_multi_func_3" , + }; + + opts.syms = syms; + opts.cnt =ARRAY_SIZE (syms); +test_attach_api ("/proc/self/exe" ,NULL , &opts); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))test_skel_api (); +if (test__start_subtest ("attach_api_pattern" )) +test_attach_api_pattern (); +if (test__start_subtest ("attach_api_syms" )) +test_attach_api_syms (); }
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 18/26] selftests/bpf: Add uprobe_multi api test
Quoted 37 lines · Jiri Olsa
Adding
uprobe_multitest forbpf_program__attach_uprobe_multiattach function.Testing attachment using glob patterns and via
bpf_uprobe_multi_optspaths/syms fields.Signed-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/uprobe_multi_test.c | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+)tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +17
-0@@ -69,8 +69,79 @@ static void test_skel_api(void)uprobe_multi__destroy (skel); } +static void +test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +{ +struct bpf_link *link1 =NULL , *link2 =NULL ; +struct bpf_link *link3 =NULL , *link4 =NULL ; +struct uprobe_multi *skel =NULL ; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + opts->retprobe = false; + link1 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) + goto cleanup; +
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 98
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10
-0+ opts->retprobe = true; + link2 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) + goto cleanup; + + opts->retprobe = false; + link3 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_sleep,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" ))
link3
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 104
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6
-0+ goto cleanup; + + opts->retprobe = true; + link4 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe_sleep,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" ))
link4
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 113
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +9
-0+ goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +bpf_link__destroy (link4); +bpf_link__destroy (link3); +bpf_link__destroy (link2); +bpf_link__destroy (link1);
you could have used skel->links.{test_uprobe,test_uretprobe,test_uprobe_sleep,test_uretprobe_sleep} "containers" to not manage destruction of these links manually
Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +11
-0+uprobe_multi__destroy (skel); +} + +static void test_attach_api_pattern (void ) +{ +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); + +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_*" , &opts); +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_?" , &opts); +} +
[...]
Re: [PATCHv3 bpf-next 18/26] selftests/bpf: Add uprobe_multi api test
Quoted 60 lines · Andrii Nakryiko
Quoted 37 lines · Jiri Olsa
Adding
uprobe_multitest forbpf_program__attach_uprobe_multiattach function.Testing attachment using glob patterns and via
bpf_uprobe_multi_optspaths/syms fields.Signed-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/uprobe_multi_test.c | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+)tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +17
-0@@ -69,8 +69,79 @@ static void test_skel_api(void)uprobe_multi__destroy (skel); } +static void +test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +{ +struct bpf_link *link1 =NULL , *link2 =NULL ; +struct bpf_link *link3 =NULL , *link4 =NULL ; +struct uprobe_multi *skel =NULL ; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + opts->retprobe = false; + link1 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) + goto cleanup; +tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 98
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10
-0+ opts->retprobe = true; + link2 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) + goto cleanup; + + opts->retprobe = false; + link3 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_sleep,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" ))link3
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 104
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6
-0+ goto cleanup; + + opts->retprobe = true; + link4 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe_sleep,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" ))link4
Quoted 13 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 113
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +9
-0+ goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +bpf_link__destroy (link4); +bpf_link__destroy (link3); +bpf_link__destroy (link2); +bpf_link__destroy (link1);you could have used skel->links.{test_uprobe,test_uretprobe,test_uprobe_sleep,test_uretprobe_sleep} "containers" to not manage destruction of these links manually
ah right, I keep forgetting we have those variables ready in there :-\
will change
thanks,
jirka
Quoted 14 lines
Quoted 12 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +11
-0+uprobe_multi__destroy (skel); +} + +static void test_attach_api_pattern (void ) +{ +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); + +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_*" , &opts); +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_?" , &opts); +} +[...]
Jiri Olsa
reviewerPATCH v3 19/26 selftests/bpf: Add uprobe_multi link test
In reply to Jiri OlsaAdding uprobe_multi test for bpf_link_create attach function.
Testing attachment using the struct bpf_link_create_opts.
--- .../bpf/prog_tests/uprobe_multi_test.c | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +68 -0
@@ -3,6 +3,7 @@#include <unistd.h> #include <test_progs.h> #include "uprobe_multi.skel.h" +#include "bpf/libbpf_elf.h" static char test_data[] ="test_data" ;
@@ -136,6 +137,71 @@ static void test_attach_api_syms(void)test_attach_api ("/proc/self/exe" ,NULL , &opts); } +static void test_link_api (void ) +{ +int prog_fd, link1_fd =-1 , link2_fd =-1 , link3_fd =-1 , link4_fd =-1 ; +LIBBPF_OPTS (bpf_link_create_opts, opts); +const char *path ="/proc/self/exe" ; +struct uprobe_multi *skel =NULL ; +unsigned long *offsets =NULL ; +const char *syms[3 ] = { +"uprobe_multi_func_1" , +"uprobe_multi_func_2" , +"uprobe_multi_func_3" , + }; +int err; + + err =elf_resolve_syms_offsets (path,3 , syms, (unsigned long **) &offsets); +if (!ASSERT_OK (err,"elf_resolve_syms_offsets" )) +return ; + + opts.uprobe_multi.path = path; + opts.uprobe_multi.offsets = offsets; + opts.uprobe_multi.cnt =ARRAY_SIZE (syms); + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + opts.kprobe_multi.flags =0 ; + prog_fd =bpf_program__fd (skel->progs.test_uprobe); + link1_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link1_fd,0 ,"link1_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.test_uretprobe); + link2_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link2_fd,0 ,"link2_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =0 ; + prog_fd =bpf_program__fd (skel->progs.test_uprobe_sleep); + link3_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link1_fd,0 ,"link3_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.test_uretprobe_sleep); + link4_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link2_fd,0 ,"link4_fd" )) + goto cleanup; +uprobe_multi_test_run (skel); + +cleanup: +if (link1_fd >=0 ) +close (link1_fd); +if (link2_fd >=0 ) +close (link2_fd); +if (link3_fd >=0 ) +close (link3_fd); +if (link4_fd >=0 ) +close (link4_fd); + +uprobe_multi__destroy (skel); +free (offsets); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))
@@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)test_attach_api_pattern ();if (test__start_subtest ("attach_api_syms" ))test_attach_api_syms (); +if (test__start_subtest ("link_api" )) +test_link_api (); }
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 19/26] selftests/bpf: Add uprobe_multi link test
Quoted 10 lines · Jiri Olsa
Adding
uprobe_multitest forbpf_link_createattach function.Testing attachment using the struct
bpf_link_create_opts.Signed-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/uprobe_multi_test.c | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+)
[...]
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 181
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10
-0+ opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.test_uretprobe); + link2_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link2_fd,0 ,"link2_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =0 ; + prog_fd =bpf_program__fd (skel->progs.test_uprobe_sleep); + link3_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link1_fd,0 ,"link3_fd" ))
link3_fd
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 187
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6
-0+ goto cleanup; + + opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.test_uretprobe_sleep); + link4_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link2_fd,0 ,"link4_fd" ))
link4_fd
Quoted 30 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +19
-1+ goto cleanup; +uprobe_multi_test_run (skel); + +cleanup: +if (link1_fd >=0 ) +close (link1_fd); +if (link2_fd >=0 ) +close (link2_fd); +if (link3_fd >=0 ) +close (link3_fd); +if (link4_fd >=0 ) +close (link4_fd); + +uprobe_multi__destroy (skel); +free (offsets); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))@@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)test_attach_api_pattern ();if (test__start_subtest ("attach_api_syms" ))test_attach_api_syms (); +if (test__start_subtest ("link_api" )) +test_link_api (); } --2.41.0
Re: [PATCHv3 bpf-next 19/26] selftests/bpf: Add uprobe_multi link test
Quoted 27 lines · Andrii Nakryiko
Quoted 10 lines · Jiri Olsa
Adding
uprobe_multitest forbpf_link_createattach function.Testing attachment using the struct
bpf_link_create_opts.Signed-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/uprobe_multi_test.c | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+)[...]
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 181
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10
-0+ opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.test_uretprobe); + link2_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link2_fd,0 ,"link2_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =0 ; + prog_fd =bpf_program__fd (skel->progs.test_uprobe_sleep); + link3_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link1_fd,0 ,"link3_fd" ))
link3_fd
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 187
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6
-0+ goto cleanup; + + opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.test_uretprobe_sleep); + link4_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link2_fd,0 ,"link4_fd" ))
link4_fd
right, will change
thanks,
jirka
Quoted 31 lines
Quoted 31 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +19
-1+ goto cleanup; +uprobe_multi_test_run (skel); + +cleanup: +if (link1_fd >=0 ) +close (link1_fd); +if (link2_fd >=0 ) +close (link2_fd); +if (link3_fd >=0 ) +close (link3_fd); +if (link4_fd >=0 ) +close (link4_fd); + +uprobe_multi__destroy (skel); +free (offsets); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))@@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)test_attach_api_pattern ();if (test__start_subtest ("attach_api_syms" ))test_attach_api_syms (); +if (test__start_subtest ("link_api" )) +test_link_api (); } --2.41.0
Jiri Olsa
reviewerPATCH v3 20/26 selftests/bpf: Add uprobe_multi test program
In reply to Jiri OlsaAdding uprobe_multi test program that defines 50k uprobe_multi_func_* functions and will serve as attach point for uprobe_multi bench test in following patch.
--- tools/testing/selftests/bpf/Makefile | 5 ++ tools/testing/selftests/bpf/uprobe_multi.c | 53 ++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
tools/testing/selftests/bpf/Makefile +5 -0
@@ -567,6 +567,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/liburandom_read.so \ $(OUTPUT)/xdp_synproxy \ $(OUTPUT)/sign-file \ + $(OUTPUT)/uprobe_multi \ ima_setup.sh \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \
@@ -670,6 +671,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@ +$(OUTPUT)/uprobe_multi: uprobe_multi.c + $(call msg,BINARY,,$@) + $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ + EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool \
tools/testing/selftests/bpf/uprobe_multi.c new file +53 -0
53 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 20/26] selftests/bpf: Add uprobe_multi test program
Quoted 17 lines · Jiri Olsa
Adding
uprobe_multitest program that defines 50k uprobe_multi_func_* functions and will serve as attach point foruprobe_multibench test in following patch.Signed-off-by Jiri Olsa <[email protected]>--- tools/testing/selftests/bpf/Makefile | 5 ++ tools/testing/selftests/bpf/uprobe_multi.c | 53 ++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100644 tools/testing/selftests/bpf/uprobe_multi.ctools/testing/selftests/bpf/Makefile +0
-0
tools/testing/selftests/bpf/Makefile TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ · line 570
Quoted 4 lines
tools/testing/selftests/bpf/Makefile +1
-0@@ -567,6 +567,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/liburandom_read.so \ $(OUTPUT)/xdp_synproxy \ $(OUTPUT)/sign-file \ + $(OUTPUT)/uprobe_multi \
would adding all this to urandom_read be too bad in terms of size and performance? I'm just not sure we won't yet more custom binaries and rules in Makefile
Quoted 76 lines
tools/testing/selftests/bpf/Makefile +4
-0ima_setup.sh \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \@@ -670,6 +671,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@ +$(OUTPUT)/uprobe_multi: uprobe_multi.c + $(call msg,BINARY,,$@) + $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ + EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool \tools/testing/selftests/bpf/uprobe_multi.c new file +53
-1@@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <stdio.h> + +#define __PASTE (a, b) a##b +#define PASTE (a, b) __PASTE(a, b) + +#define NAME (name, idx) PASTE(name, idx) + +#define DEF (name, idx) int NAME(name, idx)(void) { return 0; } +#define CALL (name, idx) NAME(name, idx)(); + +#define F (body, name, idx) body(name, idx) + +#define F10 (body, name, idx) \ + F(body, PASTE(name, idx), 0) F(body, PASTE(name, idx), 1) F(body, PASTE(name, idx), 2) \ + F(body, PASTE(name, idx), 3) F(body, PASTE(name, idx), 4) F(body, PASTE(name, idx), 5) \ + F(body, PASTE(name, idx), 6) F(body, PASTE(name, idx), 7) F(body, PASTE(name, idx), 8) \ + F(body, PASTE(name, idx), 9) + +#define F100 (body, name, idx) \ + F10(body, PASTE(name, idx), 0) F10(body, PASTE(name, idx), 1) F10(body, PASTE(name, idx), 2) \ + F10(body, PASTE(name, idx), 3) F10(body, PASTE(name, idx), 4) F10(body, PASTE(name, idx), 5) \ + F10(body, PASTE(name, idx), 6) F10(body, PASTE(name, idx), 7) F10(body, PASTE(name, idx), 8) \ + F10(body, PASTE(name, idx), 9) + +#define F1000 (body, name, idx) \ + F100(body, PASTE(name, idx), 0) F100(body, PASTE(name, idx), 1) F100(body, PASTE(name, idx), 2) \ + F100(body, PASTE(name, idx), 3) F100(body, PASTE(name, idx), 4) F100(body, PASTE(name, idx), 5) \ + F100(body, PASTE(name, idx), 6) F100(body, PASTE(name, idx), 7) F100(body, PASTE(name, idx), 8) \ + F100(body, PASTE(name, idx), 9) + +#define F10000 (body, name, idx) \ + F1000(body, PASTE(name, idx), 0) F1000(body, PASTE(name, idx), 1) F1000(body, PASTE(name, idx), 2) \ + F1000(body, PASTE(name, idx), 3) F1000(body, PASTE(name, idx), 4) F1000(body, PASTE(name, idx), 5) \ + F1000(body, PASTE(name, idx), 6) F1000(body, PASTE(name, idx), 7) F1000(body, PASTE(name, idx), 8) \ + F1000(body, PASTE(name, idx), 9) + +F10000(DEF, uprobe_multi_func_, 0) +F10000(DEF, uprobe_multi_func_, 1) +F10000(DEF, uprobe_multi_func_, 2) +F10000(DEF, uprobe_multi_func_, 3) +F10000(DEF, uprobe_multi_func_, 4) + +int main (void ) +{ + F10000(CALL, uprobe_multi_func_, 0) + F10000(CALL, uprobe_multi_func_, 1) + F10000(CALL, uprobe_multi_func_, 2) + F10000(CALL, uprobe_multi_func_, 3) + F10000(CALL, uprobe_multi_func_, 4) +return 0 ; +} --2.41.0
Jiri Olsa
reviewerPATCH v3 21/26 selftests/bpf: Add uprobe_multi bench test
In reply to Jiri OlsaAdding test that attaches 50k uprobes in uprobe_multi binary.
After the attach is done we run the binary and make sure we get proper amount of hits.
--- .../bpf/prog_tests/uprobe_multi_test.c | 56 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 9 +++ 2 files changed, 65 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +56 -0
@@ -202,6 +202,60 @@ static void test_link_api(void)free (offsets); } +static inline __u64 get_time_ns (void ) +{ +struct timespec t; + +clock_gettime (CLOCK_MONOTONIC , &t); +return (__u64 ) t.tv_sec *1000000000 + t.tv_nsec; +} + +static void test_bench_attach_uprobe (void ) +{ +long attach_start_ns, attach_end_ns; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +struct uprobe_multi *skel =NULL ; +struct bpf_program *prog; +int err; + + skel =uprobe_multi__open (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open" )) + goto cleanup; + +bpf_object__for_each_program (prog, skel->obj) +bpf_program__set_autoload (prog, false); + +bpf_program__set_autoload (skel->progs.test_uprobe_bench, true); + + err =uprobe_multi__load (skel); +if (!ASSERT_EQ (err,0 ,"uprobe_multi__load" )) + goto cleanup; + + attach_start_ns =get_time_ns (); + + err =uprobe_multi__attach (skel); +if (!ASSERT_OK (err,"uprobe_multi__attach" )) + goto cleanup; + + attach_end_ns =get_time_ns (); + +system ("./uprobe_multi" ); + +ASSERT_EQ (skel->bss->count,50000 ,"uprobes_count" ); + +cleanup: + detach_start_ns =get_time_ns (); +uprobe_multi__destroy (skel); + detach_end_ns =get_time_ns (); + + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +printf ("%s: attached in %7.3lfs\n" , __func__, attach_delta); +printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))
@@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)test_attach_api_syms ();if (test__start_subtest ("link_api" ))test_link_api (); +if (test__start_subtest ("bench_uprobe" )) +test_bench_attach_uprobe (); }
tools/testing/selftests/bpf/progs/uprobe_multi.c +9 -0
@@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)uprobe_multi_check (ctx, true, true);return 0 ; } + +int count; + +SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*") +int test_uprobe_bench (struct pt_regs *ctx) +{ + count++; +return 0 ; +}
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 21/26] selftests/bpf: Add uprobe_multi bench test
Quoted 18 lines · Jiri Olsa
Adding test that attaches 50k uprobes in
uprobe_multibinary.After the attach is done we run the binary and make sure we get proper amount of hits.
Signed-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/uprobe_multi_test.c | 56 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 9 +++ 2 files changed, 65 insertions(+)tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +0
-0@@ -202,6 +202,60 @@ static void test_link_api(void)free (offsets);
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 212
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +8
-0} +static inline __u64 get_time_ns (void ) +{ +struct timespec t; + +clock_gettime (CLOCK_MONOTONIC , &t); +return (__u64 ) t.tv_sec *1000000000 + t.tv_nsec; +} +
hmm.. I would expect we have this helper somewhere in common headers. If not, we should probably move all such helpers into one header and use it everywhere
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +7
-0+static void test_bench_attach_uprobe (void ) +{ +long attach_start_ns, attach_end_ns; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +struct uprobe_multi *skel =NULL ; +struct bpf_program *prog;
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 229
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10
-0+int err; + + skel =uprobe_multi__open (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open" )) + goto cleanup; + +bpf_object__for_each_program (prog, skel->obj) +bpf_program__set_autoload (prog, false); + +bpf_program__set_autoload (skel->progs.test_uprobe_bench, true);
I don't get why you bothered adding this test_uprobe_bench into progs/uprobe_multi and go through this manual auto-load setting/resetting, instead of just having test_uprobe_bench in a separate skeleton?...
Quoted 17 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +17
-0+ + err =uprobe_multi__load (skel); +if (!ASSERT_EQ (err,0 ,"uprobe_multi__load" )) + goto cleanup; + + attach_start_ns =get_time_ns (); + + err =uprobe_multi__attach (skel); +if (!ASSERT_OK (err,"uprobe_multi__attach" )) + goto cleanup; + + attach_end_ns =get_time_ns (); + +system ("./uprobe_multi" ); + +ASSERT_EQ (skel->bss->count,50000 ,"uprobes_count" ); +
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 256
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10
-0+cleanup: + detach_start_ns =get_time_ns (); +uprobe_multi__destroy (skel); + detach_end_ns =get_time_ns (); + + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +printf ("%s: attached in %7.3lfs\n" , __func__, attach_delta); +printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta);
and for us lazy folks, what are the numbers you see on your machine?
Quoted 32 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +4
-0+} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))@@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)test_attach_api_syms ();if (test__start_subtest ("link_api" ))test_link_api (); +if (test__start_subtest ("bench_uprobe" )) +test_bench_attach_uprobe (); }tools/testing/selftests/bpf/progs/uprobe_multi.c +9
-1@@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)uprobe_multi_check (ctx, true, true);return 0 ; } + +int count; + +SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*") +int test_uprobe_bench (struct pt_regs *ctx) +{ + count++; +return 0 ; +} --2.41.0
Re: [PATCHv3 bpf-next 21/26] selftests/bpf: Add uprobe_multi bench test
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Jiri Olsa
Adding test that attaches 50k uprobes in
uprobe_multibinary.After the attach is done we run the binary and make sure we get proper amount of hits.
Signed-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/uprobe_multi_test.c | 56 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 9 +++ 2 files changed, 65 insertions(+)tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +0
-0@@ -202,6 +202,60 @@ static void test_link_api(void)free (offsets); }
Quoted 12 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 212
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +8
-0+static inline __u64 get_time_ns (void ) +{ +struct timespec t; + +clock_gettime (CLOCK_MONOTONIC , &t); +return (__u64 ) t.tv_sec *1000000000 + t.tv_nsec; +} +hmm.. I would expect we have this helper somewhere in common headers. If not, we should probably move all such helpers into one header and use it everywhere
hum, now I see it's also defined in bench.h, I'll try to use it from there ;-) also perhaps in kprobe multi test
Quoted 15 lines
Quoted 15 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +14
-0+static void test_bench_attach_uprobe (void ) +{ +long attach_start_ns, attach_end_ns; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +struct uprobe_multi *skel =NULL ; +struct bpf_program *prog; +int err; + + skel =uprobe_multi__open (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open" )) + goto cleanup; + +bpf_object__for_each_program (prog, skel->obj)
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 229
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +3
-0+bpf_program__set_autoload (prog, false); + +bpf_program__set_autoload (skel->progs.test_uprobe_bench, true);I don't get why you bothered adding this
test_uprobe_benchinto progs/uprobe_multi and go through this manual auto-load setting/resetting, instead of just havingtest_uprobe_benchin a separate skeleton?...
ok, will add separate file for that.. will be simpler
Quoted 22 lines
Quoted 22 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +21
-0+ + err =uprobe_multi__load (skel); +if (!ASSERT_EQ (err,0 ,"uprobe_multi__load" )) + goto cleanup; + + attach_start_ns =get_time_ns (); + + err =uprobe_multi__attach (skel); +if (!ASSERT_OK (err,"uprobe_multi__attach" )) + goto cleanup; + + attach_end_ns =get_time_ns (); + +system ("./uprobe_multi" ); + +ASSERT_EQ (skel->bss->count,50000 ,"uprobes_count" ); + +cleanup: + detach_start_ns =get_time_ns (); +uprobe_multi__destroy (skel); + detach_end_ns =get_time_ns ();
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 256
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6
-0+ + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +printf ("%s: attached in %7.3lfs\n" , __func__, attach_delta); +printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta);and for us lazy folks, what are the numbers you see on your machine?
right, will put it to the changelog
# ./test_progs -n 260/5 -v bpf_testmod.ko is already unloaded. Loading bpf_testmod.ko... Successfully loaded bpf_testmod.ko. test_bench_attach_uprobe:PASS:uprobe_multi__open 0 nsec test_bench_attach_uprobe:PASS:uprobe_multi__load 0 nsec test_bench_attach_uprobe:PASS:uprobe_multi__attach 0 nsec test_bench_attach_uprobe:PASS:uprobes_count 0 nsec test_bench_attach_uprobe: attached in 0.398s test_bench_attach_uprobe: detached in 0.443s #260/5 uprobe_multi_test/bench_uprobe:OK #260 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Successfully unloaded bpf_testmod.ko.
thanks,
jirka
Quoted 33 lines
Quoted 33 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +4
-0+} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))@@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)test_attach_api_syms ();if (test__start_subtest ("link_api" ))test_link_api (); +if (test__start_subtest ("bench_uprobe" )) +test_bench_attach_uprobe (); }tools/testing/selftests/bpf/progs/uprobe_multi.c +9
-1@@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)uprobe_multi_check (ctx, true, true);return 0 ; } + +int count; + +SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*") +int test_uprobe_bench (struct pt_regs *ctx) +{ + count++; +return 0 ; +} --2.41.0
Jiri Olsa
reviewerPATCH v3 22/26 selftests/bpf: Add usdt_multi test program
In reply to Jiri OlsaAdding usdt_multi test program that defines 50k usdts and will serve as attach point for uprobe_multi usdt bench test in following patch.
--- tools/testing/selftests/bpf/Makefile | 5 +++++ tools/testing/selftests/bpf/usdt_multi.c | 24 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+) create mode 100644 tools/testing/selftests/bpf/usdt_multi.c
tools/testing/selftests/bpf/Makefile +5 -0
@@ -568,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/xdp_synproxy \ $(OUTPUT)/sign-file \ $(OUTPUT)/uprobe_multi \ + $(OUTPUT)/usdt_multi \ ima_setup.sh \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \
@@ -675,6 +676,10 @@ $(OUTPUT)/uprobe_multi: uprobe_multi.c $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ +$(OUTPUT)/usdt_multi: usdt_multi.c + $(call msg,BINARY,,$@) + $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ + EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool \
tools/testing/selftests/bpf/usdt_multi.c new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <sdt.h> + +#define PROBE STAP_PROBE(test, usdt); + +#define PROBE10 PROBE PROBE PROBE PROBE PROBE \ + PROBE PROBE PROBE PROBE PROBE +#define PROBE100 PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \ + PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 +#define PROBE1000 PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \ + PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 +#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \ + PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 + +int main (void ) +{ +PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 +return 0 ; +}
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 22/26] selftests/bpf: Add usdt_multi test program
Quoted 17 lines · Jiri Olsa
Adding
usdt_multitest program that defines 50k usdts and will serve as attach point foruprobe_multiusdt bench test in following patch.Signed-off-by Jiri Olsa <[email protected]>--- tools/testing/selftests/bpf/Makefile | 5 +++++ tools/testing/selftests/bpf/usdt_multi.c | 24 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+) create mode 100644 tools/testing/selftests/bpf/usdt_multi.ctools/testing/selftests/bpf/Makefile +0
-0
tools/testing/selftests/bpf/Makefile TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ · line 571
Quoted 4 lines
tools/testing/selftests/bpf/Makefile +1
-0@@ -568,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/xdp_synproxy \ $(OUTPUT)/sign-file \ $(OUTPUT)/uprobe_multi \ + $(OUTPUT)/usdt_multi \
that's a bit too much, if you can't put everything into urandom_read, let's at least combine uprobe_multi and usdt_multi
Quoted 47 lines
tools/testing/selftests/bpf/Makefile +4
-0ima_setup.sh \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \@@ -675,6 +676,10 @@ $(OUTPUT)/uprobe_multi: uprobe_multi.c $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ +$(OUTPUT)/usdt_multi: usdt_multi.c + $(call msg,BINARY,,$@) + $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ + EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool \tools/testing/selftests/bpf/usdt_multi.c new file +24
-1@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <sdt.h> + +#define PROBE STAP_PROBE(test, usdt); + +#define PROBE10 PROBE PROBE PROBE PROBE PROBE \ + PROBE PROBE PROBE PROBE PROBE +#define PROBE100 PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \ + PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 +#define PROBE1000 PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \ + PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 +#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \ + PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 + +int main (void ) +{ +PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 +return 0 ; +} --2.41.0
Jiri Olsa
reviewerPATCH v3 23/26 selftests/bpf: Add usdt_multi bench test
In reply to Jiri OlsaAdding test that attaches 50k usdt probes in usdt_multi binary.
After the attach is done we run the binary and make sure we get proper amount of hits.
With current uprobes:
# perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
Performance counter stats for './test_progs -n 254/6':
1353.659680562 seconds time elapsed
With uprobe_multi link:
# perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
Performance counter stats for './test_progs -n 254/6':
0.322046364 seconds time elapsed
--- .../bpf/prog_tests/uprobe_multi_test.c | 50 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++++ 2 files changed, 66 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +50 -0
@@ -4,6 +4,7 @@#include <test_progs.h> #include "uprobe_multi.skel.h" #include "bpf/libbpf_elf.h" +#include "uprobe_multi_usdt.skel.h" static char test_data[] ="test_data" ;
@@ -256,6 +257,53 @@ static void test_bench_attach_uprobe(void)printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); } +static void test_bench_attach_usdt (void ) +{ +struct uprobe_multi_usdt *skel =NULL ; +long attach_start_ns, attach_end_ns; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +struct bpf_program *prog; +int err; + + skel =uprobe_multi_usdt__open (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open" )) + goto cleanup; + +bpf_object__for_each_program (prog, skel->obj) +bpf_program__set_autoload (prog, false); + +bpf_program__set_autoload (skel->progs.usdt0, true); + + err =uprobe_multi_usdt__load (skel); +if (!ASSERT_EQ (err,0 ,"uprobe_multi_usdt__load" )) + goto cleanup; + + attach_start_ns =get_time_ns (); + + skel->links.usdt0 =bpf_program__attach_usdt (skel->progs.usdt0,-1 ,"./usdt_multi" , +"test" ,"usdt" ,NULL ); +if (!ASSERT_OK_PTR (skel->links.usdt0,"bpf_program__attach_usdt" )) + goto cleanup; + + attach_end_ns =get_time_ns (); + +system ("./usdt_multi" ); + +ASSERT_EQ (skel->bss->count,50000 ,"usdt_count" ); + +cleanup: + detach_start_ns =get_time_ns (); +uprobe_multi_usdt__destroy (skel); + detach_end_ns =get_time_ns (); + + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +printf ("%s: attached in %7.3lfs\n" , __func__, attach_delta); +printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))
@@ -268,4 +316,6 @@ void test_uprobe_multi_test(void)test_link_api ();if (test__start_subtest ("bench_uprobe" ))test_bench_attach_uprobe (); +if (test__start_subtest ("bench_usdt" )) +test_bench_attach_usdt (); }
tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/usdt.bpf.h> + +char _license[] SEC("license") = "GPL"; + +int count; + +SEC("usdt") +int usdt0 (struct pt_regs *ctx) +{ + count++; +return 0 ; +}
Signature · 1 line
2.41.0
Re: [PATCHv3 bpf-next 23/26] selftests/bpf: Add usdt_multi bench test
Quoted 8 lines · Jiri Olsa
Adding test that attaches 50k usdt probes in
usdt_multibinary.After the attach is done we run the binary and make sure we get proper amount of hits.
With current uprobes:
Quoted 9 lines
# perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILEDPerformance counter stats for './test_progs -n 254/6':
1353.659680562 seconds time elapsed
20 minutes, I commend your patience!
Quoted 10 lines
With
uprobe_multilink:# perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILEDPerformance counter stats for './test_progs -n 254/6':
0.322046364 seconds time elapsed
This is an impressive speed up, especially taking into account that there is no batch attachment magic that we had to do for kprobes. Very nice!
Quoted 31 lines
Signed-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/uprobe_multi_test.c | 50 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++++ 2 files changed, 66 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.ctools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +8
-0@@ -4,6 +4,7 @@#include <test_progs.h> #include "uprobe_multi.skel.h" #include "bpf/libbpf_elf.h" +#include "uprobe_multi_usdt.skel.h" static char test_data[] ="test_data" ;@@ -256,6 +257,53 @@ static void test_bench_attach_uprobe(void)printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); } +static void test_bench_attach_usdt (void ) +{ +struct uprobe_multi_usdt *skel =NULL ; +long attach_start_ns, attach_end_ns; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +struct bpf_program *prog;
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_bench_attach_uprobe(void) · line 276
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10
-0+int err; + + skel =uprobe_multi_usdt__open (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open" )) + goto cleanup; + +bpf_object__for_each_program (prog, skel->obj) +bpf_program__set_autoload (prog, false); + +bpf_program__set_autoload (skel->progs.usdt0, true);
there is nothing else in that skeleton, why this set_autoload() business?
Quoted 65 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +32
-0+ + err =uprobe_multi_usdt__load (skel); +if (!ASSERT_EQ (err,0 ,"uprobe_multi_usdt__load" )) + goto cleanup; + + attach_start_ns =get_time_ns (); + + skel->links.usdt0 =bpf_program__attach_usdt (skel->progs.usdt0,-1 ,"./usdt_multi" , +"test" ,"usdt" ,NULL ); +if (!ASSERT_OK_PTR (skel->links.usdt0,"bpf_program__attach_usdt" )) + goto cleanup; + + attach_end_ns =get_time_ns (); + +system ("./usdt_multi" ); + +ASSERT_EQ (skel->bss->count,50000 ,"usdt_count" ); + +cleanup: + detach_start_ns =get_time_ns (); +uprobe_multi_usdt__destroy (skel); + detach_end_ns =get_time_ns (); + + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +printf ("%s: attached in %7.3lfs\n" , __func__, attach_delta); +printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))@@ -268,4 +316,6 @@ void test_uprobe_multi_test(void)test_link_api ();if (test__start_subtest ("bench_uprobe" ))test_bench_attach_uprobe (); +if (test__start_subtest ("bench_usdt" )) +test_bench_attach_usdt (); }tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c new file +16
-1@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/usdt.bpf.h> + +char _license[] SEC("license") = "GPL"; + +int count; + +SEC("usdt") +int usdt0 (struct pt_regs *ctx) +{ + count++; +return 0 ; +} --2.41.0
Re: [PATCHv3 bpf-next 23/26] selftests/bpf: Add usdt_multi bench test
On Thu, Jul 06, 2023 at 09:42:05PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +11
-0+static void test_bench_attach_usdt (void ) +{ +struct uprobe_multi_usdt *skel =NULL ; +long attach_start_ns, attach_end_ns; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +struct bpf_program *prog; +int err; + + skel =uprobe_multi_usdt__open (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open" ))
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_bench_attach_uprobe(void) · line 276
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6
-0+ goto cleanup; + +bpf_object__for_each_program (prog, skel->obj) +bpf_program__set_autoload (prog, false); + +bpf_program__set_autoload (skel->progs.usdt0, true);there is nothing else in that skeleton, why this
set_autoload()business?
nah copy&paste error from the uprobe bench test.. will change ;-)
jirka
SNIP
Jiri Olsa
reviewerPATCH v3 24/26 selftests/bpf: Add uprobe_multi cookie test
In reply to Jiri OlsaAdding test for cookies setup/retrieval in uprobe_link uprobes and making sure bpf_get_attach_cookie works properly.
--- .../selftests/bpf/prog_tests/bpf_cookie.c | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+)
tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +78 -0
@@ -11,6 +11,7 @@#include <bpf/btf.h> #include "test_bpf_cookie.skel.h" #include "kprobe_multi.skel.h" +#include "uprobe_multi.skel.h" /* uprobe attach point */ static noinline void trigger_func(void)
@@ -239,6 +240,81 @@ static void kprobe_multi_attach_api_subtest(void)bpf_link__destroy (link1);kprobe_multi__destroy (skel); } + +/* defined in prog_tests/uprobe_multi_test.c */ +void uprobe_multi_func_1 (void ); +void uprobe_multi_func_2 (void ); +void uprobe_multi_func_3 (void ); + +static void uprobe_multi_test_run (struct uprobe_multi *skel) +{ + skel->bss->uprobe_multi_func_1_addr = (__u64 ) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (__u64 ) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (__u64 ) uprobe_multi_func_3; + + skel->bss->pid =getpid (); + skel->bss->test_cookie = true; + +uprobe_multi_func_1 (); +uprobe_multi_func_2 (); +uprobe_multi_func_3 (); + +ASSERT_EQ (skel->bss->uprobe_multi_func_1_result,1 ,"uprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_2_result,1 ,"uprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_3_result,1 ,"uprobe_multi_func_3_result" ); + +ASSERT_EQ (skel->bss->uretprobe_multi_func_1_result,1 ,"uretprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_2_result,1 ,"uretprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_3_result,1 ,"uretprobe_multi_func_3_result" ); +} + +static void uprobe_multi_attach_api_subtest (void ) +{ +struct bpf_link *link1 =NULL , *link2 =NULL ; +struct uprobe_multi *skel =NULL ; +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); +const char *syms[3 ] = { +"uprobe_multi_func_1" , +"uprobe_multi_func_2" , +"uprobe_multi_func_3" , + }; +__u64 cookies[3 ]; + + cookies[0 ] =3 ;/* uprobe_multi_func_1 */ + cookies[1 ] =1 ;/* uprobe_multi_func_2 */ + cookies[2 ] =2 ;/* uprobe_multi_func_3 */ + + opts.syms = syms; + opts.cnt =ARRAY_SIZE (syms); + opts.cookies = &cookies[0 ]; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi" )) + goto cleanup; + + link1 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe,-1 , +"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + + cookies[0 ] =2 ;/* uprobe_multi_func_1 */ + cookies[1 ] =3 ;/* uprobe_multi_func_2 */ + cookies[2 ] =1 ;/* uprobe_multi_func_3 */ + + opts.retprobe = true; + link2 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe,-1 , +"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) + goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +bpf_link__destroy (link2); +bpf_link__destroy (link1); +uprobe_multi__destroy (skel); +} +static void uprobe_subtest (struct test_bpf_cookie *skel) {DECLARE_LIBBPF_OPTS (bpf_uprobe_opts, opts);
@@ -515,6 +591,8 @@ void test_bpf_cookie(void)kprobe_multi_attach_api_subtest ();if (test__start_subtest ("uprobe" ))uprobe_subtest (skel); +if (test__start_subtest ("multi_uprobe_attach_api" )) +uprobe_multi_attach_api_subtest ();if (test__start_subtest ("tracepoint" ))tp_subtest (skel);if (test__start_subtest ("perf_event" ))
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v3 25/26 selftests/bpf: Add uprobe_multi pid filter tests
In reply to Jiri OlsaRunning api and link tests also with pid filter and checking the probe gets executed only for specific pid.
Spawning extra process to trigger attached uprobes and checking we get correct counts from executed programs.
--- .../bpf/prog_tests/uprobe_multi_test.c | 131 ++++++++++++++++-- .../selftests/bpf/progs/uprobe_multi.c | 6 +- 2 files changed, 125 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +120 -11
@@ -23,14 +23,86 @@ noinline void uprobe_multi_func_3(void) asmvolatile ("" ); } -static void uprobe_multi_test_run (struct uprobe_multi *skel) +struct child { +int go[2 ]; +int pid; +}; + +static void release_child (struct child *child) +{ +int child_status; + +if (!child) +return ; +close (child->go[1 ]); +close (child->go[0 ]); +if (child->pid >0 ) +waitpid (child->pid, &child_status,0 ); +} + +static void kick_child (struct child *child) +{ +char c =1 ; + +if (child) { +write (child->go[1 ], &c,1 ); +release_child (child); + } +fflush (NULL ); +} + +static struct child *spawn_child (void ) +{ +static struct child child; +int err; +int c; + +/* pid filter */ +if (!ASSERT_OK (pipe (child.go),"pipe" )) +return NULL ; + + child.pid =fork (); +if (child.pid <0 ) { +release_child (&child); +return NULL ; + } + +/* child */ +if (child.pid ==0 ) { +close (child.go[1 ]); +fflush (NULL ); +/* wait for parent's kick */ + err =read (child.go[0 ], &c,1 ); +if (!ASSERT_EQ (err,1 ,"child_read_pipe" )) +exit (err); + +uprobe_multi_func_1 (); +uprobe_multi_func_2 (); +uprobe_multi_func_3 (); + +exit (errno); + } + +return &child; +} + +static void uprobe_multi_test_run (struct uprobe_multi *skel,struct child *child) { skel->bss->uprobe_multi_func_1_addr = (__u64 ) uprobe_multi_func_1; skel->bss->uprobe_multi_func_2_addr = (__u64 ) uprobe_multi_func_2; skel->bss->uprobe_multi_func_3_addr = (__u64 ) uprobe_multi_func_3; skel->bss->user_ptr = test_data; - skel->bss->pid =getpid (); + +/* +* Disable pid check in bpf program if we are pid filter test, +* because the probe should be executed only by child->pid +* passed at the probe attach. +*/ + skel->bss->pid = child ?0 :getpid (); + +if (child) +kick_child (child);/* trigger all probes */ uprobe_multi_func_1 ();
@@ -50,6 +122,9 @@ static void uprobe_multi_test_run(struct uprobe_multi *skel)ASSERT_EQ (skel->bss->uretprobe_multi_func_3_result,2 ,"uretprobe_multi_func_3_result" );ASSERT_EQ (skel->bss->uprobe_multi_sleep_result,6 ,"uprobe_multi_sleep_result" ); + +if (child) +ASSERT_EQ (skel->bss->child_pid, child->pid,"uprobe_multi_child_pid" ); }static void test_skel_api (void )
@@ -65,17 +140,19 @@ static void test_skel_api(void)if (!ASSERT_OK (err,"uprobe_multi__attach" )) goto cleanup; -uprobe_multi_test_run (skel); +uprobe_multi_test_run (skel,NULL ); cleanup:uprobe_multi__destroy (skel); }static void -test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +__test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts, +struct child *child) {struct bpf_link *link1 =NULL , *link2 =NULL ;struct bpf_link *link3 =NULL , *link4 =NULL ; +pid_t pid = child ? child->pid :-1 ;struct uprobe_multi *skel =NULL ; skel =uprobe_multi__open_and_load ();
@@ -83,30 +160,30 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi goto cleanup; opts->retprobe = false; - link1 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe,-1 , + link1 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe, pid, binary, pattern, opts);if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) goto cleanup; opts->retprobe = true; - link2 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe,-1 , + link2 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe, pid, binary, pattern, opts);if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) goto cleanup; opts->retprobe = false; - link3 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_sleep,-1 , + link3 =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_sleep, pid, binary, pattern, opts);if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) goto cleanup; opts->retprobe = true; - link4 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe_sleep,-1 , + link4 =bpf_program__attach_uprobe_multi (skel->progs.test_uretprobe_sleep, pid, binary, pattern, opts);if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) goto cleanup; -uprobe_multi_test_run (skel); +uprobe_multi_test_run (skel, child); cleanup:bpf_link__destroy (link4);
@@ -116,6 +193,22 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multiuprobe_multi__destroy (skel); } +static void +test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +{ +struct child *child; + +/* no pid filter */ +__test_attach_api (binary, pattern, opts,NULL ); + +/* pid filter */ + child =spawn_child (); +if (!child) +return ; + +__test_attach_api (binary, pattern, opts, child); +} +static void test_attach_api_pattern (void ) {LIBBPF_OPTS (bpf_uprobe_multi_opts, opts);
@@ -138,7 +231,7 @@ static void test_attach_api_syms(void)test_attach_api ("/proc/self/exe" ,NULL , &opts); } -static void test_link_api (void ) +static void __test_link_api (struct child *child) {int prog_fd, link1_fd =-1 , link2_fd =-1 , link3_fd =-1 , link4_fd =-1 ;LIBBPF_OPTS (bpf_link_create_opts, opts);
@@ -159,6 +252,7 @@ static void test_link_api(void) opts.uprobe_multi.path = path; opts.uprobe_multi.offsets = offsets; opts.uprobe_multi.cnt =ARRAY_SIZE (syms); + opts.uprobe_multi.pid = child ? child->pid :0 ; skel =uprobe_multi__open_and_load ();if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" ))
@@ -187,7 +281,7 @@ static void test_link_api(void) link4_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts);if (!ASSERT_GE (link2_fd,0 ,"link4_fd" )) goto cleanup; -uprobe_multi_test_run (skel); +uprobe_multi_test_run (skel, child); cleanup:if (link1_fd >=0 )
@@ -203,6 +297,21 @@ static void test_link_api(void)free (offsets); } +void test_link_api (void ) +{ +struct child *child; + +/* no pid filter */ +__test_link_api (NULL ); + +/* pid filter */ + child =spawn_child (); +if (!child) +return ; + +__test_link_api (child); +} +static inline __u64 get_time_ns (void ) {struct timespec t;
tools/testing/selftests/bpf/progs/uprobe_multi.c +5 -1
@@ -21,6 +21,8 @@ __u64 uretprobe_multi_func_3_result = 0;__u64 uprobe_multi_sleep_result =0 ;int pid =0 ; +int child_pid =0 ; +bool test_cookie = false;void *user_ptr =0 ;
@@ -34,7 +36,9 @@ static __always_inline bool verify_sleepable_user_copy(void)static void uprobe_multi_check (void *ctx,bool is_return,bool is_sleep) { -if (bpf_get_current_pid_tgid () >>32 != pid) + child_pid =bpf_get_current_pid_tgid () >>32 ; + +if (pid && child_pid != pid)return ;__u64 cookie = test_cookie ?bpf_get_attach_cookie (ctx) :0 ;
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v3 26/26 selftests/bpf: Add extra link to uprobe_multi tests
In reply to Jiri OlsaAttaching extra program to same functions system wide for api and link tests.
This way we can test the pid filter works properly when there's extra system wide consumer on the same uprobe that will trigger the original uprobe handler.
We expect to have the same counts as before.
--- .../bpf/prog_tests/uprobe_multi_test.c | 19 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 6 ++++++ 2 files changed, 25 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +19 -0
@@ -153,6 +153,7 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mulstruct bpf_link *link1 =NULL , *link2 =NULL ;struct bpf_link *link3 =NULL , *link4 =NULL ;pid_t pid = child ? child->pid :-1 ; +struct bpf_link *link_extra =NULL ;struct uprobe_multi *skel =NULL ; skel =uprobe_multi__open_and_load ();
@@ -183,9 +184,16 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mulif (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) goto cleanup; + opts->retprobe = false; + link_extra =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_extra,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (link_extra,"bpf_program__attach_uprobe_multi" )) + goto cleanup; +uprobe_multi_test_run (skel, child); cleanup: +bpf_link__destroy (link_extra);bpf_link__destroy (link4);bpf_link__destroy (link3);bpf_link__destroy (link2);
@@ -243,6 +251,7 @@ static void __test_link_api(struct child *child)"uprobe_multi_func_2" , "uprobe_multi_func_3", }; +int link_extra_fd =-1 ;int err; err =elf_resolve_syms_offsets (path,3 , syms, (unsigned long **) &offsets);
@@ -281,6 +290,14 @@ static void __test_link_api(struct child *child) link4_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts);if (!ASSERT_GE (link2_fd,0 ,"link4_fd" )) goto cleanup; + + opts.kprobe_multi.flags =0 ; + opts.uprobe_multi.pid =0 ; + prog_fd =bpf_program__fd (skel->progs.test_uprobe_extra); + link_extra_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link_extra_fd,0 ,"link_extra_fd" )) + goto cleanup; +uprobe_multi_test_run (skel, child); cleanup:
@@ -292,6 +309,8 @@ static void __test_link_api(struct child *child)close (link3_fd);if (link4_fd >=0 )close (link4_fd); +if (link_extra_fd >=0 ) +close (link_extra_fd);uprobe_multi__destroy (skel);free (offsets);
tools/testing/selftests/bpf/progs/uprobe_multi.c +6 -0
@@ -102,3 +102,9 @@ int test_uprobe_bench(struct pt_regs *ctx) count++;return 0 ; } + +SEC("uprobe.multi//proc/self/exe:uprobe_multi_func_*") +int test_uprobe_extra (struct pt_regs *ctx) +{ +return 0 ; +}
Signature · 1 line
2.41.0
Daniel Borkmann
maintainerQuoted 111 lines · Jiri Olsa
hi,
this patchset is adding support to attach multiple uprobes and usdt probes through newuprobe_multilink.The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.
The main reason is that we need to install perf event for each probed function and profile shows perf event installation (
perf_install_in_context) as culprit.The new
uprobe_multilink just creates raw uprobes and attaches the bpf program to them without perf event being involved.In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to.
v3 changes:
- consolidate attach type checks inbpf_prog_attach_check_attach_type[Andrii]
- removebpf_prog_activecheck [Alexei]
- change rcu locking [Andrii]
- allocateref_ctr_offsetsconditionally [Andrii]
- remove ctx check frombpf_uprobe_multi_cookie[Andrii]
- move some elf_* functions inelf.cobject [Andrii]
- fix uprobe link detection code [Andrii]
- add usdt.s program section [Andrii]
- renamebpf_program__attach_uprobe_multi_optstobpf_program__attach_uprobe_multi[Andrii]
- remove extra case fromattach_uprobe_multi[Andrii]
- reworkusdt_manager_attach_usdt[Andrii]
- rework/rename new elf_find_* functions [Andrii]
- elf iterator fixes [Andrii]
- renames
-elf_sym_iter_nextloop restruct
- return directlyGElf_Syminelf_sym
- addelf_sym_offsethelper
- addst_typeargelf_sym_iter_new
- get rid of '_' prefixed functions
- simplify offsets handling
- other smaller changes
- added acks
- todo:
- seems like more elf_* helpers could go inelf.cobject,
I'll send that as follow up
- will sendfill_infosupport when [3] is merged [Andrii]There's support for bpftrace [2] and tetragon [1].
Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
uprobe_multithanks,
jirka[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (26):
bpf: Addattach_typechecks underbpf_prog_attach_check_attach_type
bpf: Add multi uprobe link
bpf: Add cookies support foruprobe_multilink
bpf: Add pid filter support foruprobe_multilink
bpf: Addbpf_get_func_iphelper support for uprobe link
libbpf: Adduprobe_multiattach type and link names
libbpf: Move elf_find_func_offset* functions to elf object
libbpf: Add elf_open/elf_close functions
libbpf: Add elf symbol iterator
libbpf: Addelf_resolve_syms_offsetsfunction
libbpf: Addelf_resolve_pattern_offsetsfunction
libbpf: Addbpf_link_createsupport for multi uprobes
libbpf: Addbpf_program__attach_uprobe_multifunction
libbpf: Add support for u[ret]probe.multi[.s] program sections
libbpf: Add uprobe multi link detection
libbpf: Add uprobe multi link support tobpf_program__attach_usdtselftests/bpf: Adduprobe_multiskel test
selftests/bpf: Adduprobe_multiapi test
selftests/bpf: Adduprobe_multilink test
selftests/bpf: Adduprobe_multitest program
selftests/bpf: Adduprobe_multibench test
selftests/bpf: Addusdt_multitest program
selftests/bpf: Addusdt_multibench test
selftests/bpf: Adduprobe_multicookie test
selftests/bpf: Adduprobe_multipid filter tests
selftests/bpf: Add extra link touprobe_multitestsinclude/linux/trace_events.h | 6 ++ include/uapi/linux/bpf.h | 16 ++++ kernel/bpf/syscall.c | 122 ++++++++++++------------- kernel/trace/bpf_trace.c | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- tools/include/uapi/linux/bpf.h | 16 ++++ tools/lib/bpf/Build | 2 +- tools/lib/bpf/bpf.c | 11 +++ tools/lib/bpf/bpf.h | 11 ++- tools/lib/bpf/elf.c | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 396 ++++++++++++++++++++++++++++++++++++++++++--------------------------------------- tools/lib/bpf/libbpf.h | 27 ++++++ tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_elf.h | 24 +++++ tools/lib/bpf/libbpf_internal.h | 3 + tools/lib/bpf/usdt.c | 109 +++++++++++++++-------- tools/testing/selftests/bpf/Makefile | 10 +++ tools/testing/selftests/bpf/prog_tests/bpf_cookie.c | 78 ++++++++++++++++ tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi.c | 110 +++++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++ tools/testing/selftests/bpf/uprobe_multi.c | 53 +++++++++++ tools/testing/selftests/bpf/usdt_multi.c | 24 +++++ 22 files changed, 1969 insertions(+), 296 deletions(-)
Quoted 7 lines
create mode 100644
tools/lib/bpf/elf.c
create mode 100644tools/lib/bpf/libbpf_elf.h
create mode 100644tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.ccreate mode 100644tools/testing/selftests/bpf/progs/uprobe_multi.ccreate mode 100644tools/testing/selftests/bpf/progs/uprobe_multi_usdt.ccreate mode 100644tools/testing/selftests/bpf/uprobe_multi.ccreate mode 100644tools/testing/selftests/bpf/usdt_multi.c
Looks like BPF CI did not trigger because of build error with LLVM:
https://github.com/kernel-patches/bpf/actions/runs/5423873866/jobs/9908376097
[...]
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_OK(err, "uprobe_multi__attach"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:2: note: remove the 'if' if its condition is always false
if (!ASSERT_OK(err, "uprobe_multi__attach"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:37: note: initialize the variable 'attach_end_ns' to silence this warning
long attach_start_ns, attach_end_ns;
^
= 0
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:22: note: initialize the variable 'attach_start_ns' to silence this warning
long attach_start_ns, attach_end_ns;
^
= 0
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:2: note: remove the 'if' if its condition is always false
if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:37: note: initialize the variable 'attach_end_ns' to silence this warning
long attach_start_ns, attach_end_ns;
^
= 0
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
^~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:22: note: initialize the variable 'attach_start_ns' to silence this warning
long attach_start_ns, attach_end_ns;
^
= 0 10 errors generated.
TEST-OBJ [test_progs] lookup_key.test.o
make: *** [Makefile:578: /tmp/work/bpf/bpf/tools/testing/selftests/bpf/uprobe_multi_test.test.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/tmp/work/bpf/bpf/tools/testing/selftests/bpf'
Error: Process completed with exit code 2.
Quoted 120 lines · Daniel Borkmann
Quoted 111 lines · Jiri Olsa
hi,
this patchset is adding support to attach multiple uprobes and usdt probes through newuprobe_multilink.The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.
The main reason is that we need to install perf event for each probed function and profile shows perf event installation (
perf_install_in_context) as culprit.The new
uprobe_multilink just creates raw uprobes and attaches the bpf program to them without perf event being involved.In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to.
v3 changes:
- consolidate attach type checks inbpf_prog_attach_check_attach_type[Andrii]
- removebpf_prog_activecheck [Alexei]
- change rcu locking [Andrii]
- allocateref_ctr_offsetsconditionally [Andrii]
- remove ctx check frombpf_uprobe_multi_cookie[Andrii]
- move some elf_* functions inelf.cobject [Andrii]
- fix uprobe link detection code [Andrii]
- add usdt.s program section [Andrii]
- renamebpf_program__attach_uprobe_multi_optstobpf_program__attach_uprobe_multi[Andrii]
- remove extra case fromattach_uprobe_multi[Andrii]
- reworkusdt_manager_attach_usdt[Andrii]
- rework/rename new elf_find_* functions [Andrii]
- elf iterator fixes [Andrii]
- renames
-elf_sym_iter_nextloop restruct
- return directlyGElf_Syminelf_sym
- addelf_sym_offsethelper
- addst_typeargelf_sym_iter_new
- get rid of '_' prefixed functions
- simplify offsets handling
- other smaller changes
- added acks
- todo:
- seems like more elf_* helpers could go inelf.cobject,
I'll send that as follow up
- will sendfill_infosupport when [3] is merged [Andrii]There's support for bpftrace [2] and tetragon [1].
Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
uprobe_multithanks,
jirka[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (26):
bpf: Addattach_typechecks underbpf_prog_attach_check_attach_type
bpf: Add multi uprobe link
bpf: Add cookies support foruprobe_multilink
bpf: Add pid filter support foruprobe_multilink
bpf: Addbpf_get_func_iphelper support for uprobe link
libbpf: Adduprobe_multiattach type and link names
libbpf: Move elf_find_func_offset* functions to elf object
libbpf: Add elf_open/elf_close functions
libbpf: Add elf symbol iterator
libbpf: Addelf_resolve_syms_offsetsfunction
libbpf: Addelf_resolve_pattern_offsetsfunction
libbpf: Addbpf_link_createsupport for multi uprobes
libbpf: Addbpf_program__attach_uprobe_multifunction
libbpf: Add support for u[ret]probe.multi[.s] program sections
libbpf: Add uprobe multi link detection
libbpf: Add uprobe multi link support tobpf_program__attach_usdtselftests/bpf: Adduprobe_multiskel test
selftests/bpf: Adduprobe_multiapi test
selftests/bpf: Adduprobe_multilink test
selftests/bpf: Adduprobe_multitest program
selftests/bpf: Adduprobe_multibench test
selftests/bpf: Addusdt_multitest program
selftests/bpf: Addusdt_multibench test
selftests/bpf: Adduprobe_multicookie test
selftests/bpf: Adduprobe_multipid filter tests
selftests/bpf: Add extra link touprobe_multitestsinclude/linux/trace_events.h | 6 ++ include/uapi/linux/bpf.h | 16 ++++ kernel/bpf/syscall.c | 122 ++++++++++++------------- kernel/trace/bpf_trace.c | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- tools/include/uapi/linux/bpf.h | 16 ++++ tools/lib/bpf/Build | 2 +- tools/lib/bpf/bpf.c | 11 +++ tools/lib/bpf/bpf.h | 11 ++- tools/lib/bpf/elf.c | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 396 ++++++++++++++++++++++++++++++++++++++++++--------------------------------------- tools/lib/bpf/libbpf.h | 27 ++++++ tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_elf.h | 24 +++++ tools/lib/bpf/libbpf_internal.h | 3 + tools/lib/bpf/usdt.c | 109 +++++++++++++++-------- tools/testing/selftests/bpf/Makefile | 10 +++ tools/testing/selftests/bpf/prog_tests/bpf_cookie.c | 78 ++++++++++++++++ tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi.c | 110 +++++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++ tools/testing/selftests/bpf/uprobe_multi.c | 53 +++++++++++ tools/testing/selftests/bpf/usdt_multi.c | 24 +++++ 22 files changed, 1969 insertions(+), 296 deletions(-)Quoted 7 lines
create mode 100644
tools/lib/bpf/elf.c
create mode 100644tools/lib/bpf/libbpf_elf.h
create mode 100644tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.ccreate mode 100644tools/testing/selftests/bpf/progs/uprobe_multi.ccreate mode 100644tools/testing/selftests/bpf/progs/uprobe_multi_usdt.ccreate mode 100644tools/testing/selftests/bpf/uprobe_multi.ccreate mode 100644tools/testing/selftests/bpf/usdt_multi.c
Quoted 3 lines
Looks like BPF CI did not trigger because of build error with LLVM:
https://github.com/kernel-patches/bpf/actions/runs/5423873866/jobs/9908376097
will check, thanks
jirka
Quoted 114 lines
[...]
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]if (!ASSERT_OK(err, "uprobe_multi__attach")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:2: note: remove the 'if' if its condition is always false if (!ASSERT_OK(err, "uprobe_multi__attach")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_EQ(err, 0, "uprobe_multi__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false if (!ASSERT_EQ(err, 0, "uprobe_multi__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:37: note: initialize the variable 'attach_end_ns' to silence this warning long attach_start_ns, attach_end_ns; ^ = 0 /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_EQ(err, 0, "uprobe_multi__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false if (!ASSERT_EQ(err, 0, "uprobe_multi__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:22: note: initialize the variable 'attach_start_ns' to silence this warning long attach_start_ns, attach_end_ns; ^ = 0 /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:2: note: remove the 'if' if its condition is always false if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:37: note: initialize the variable 'attach_end_ns' to silence this warning long attach_start_ns, attach_end_ns; ^ = 0 /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0; ^~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false if (!ASSERT_OK_PTR(skel, "uprobe_multi__open")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:22: note: initialize the variable 'attach_start_ns' to silence this warning long attach_start_ns, attach_end_ns; ^ = 010 errors generated.
TEST-OBJ [test_progs] lookup_key.test.o
make: *** [Makefile:578:/tmp/work/bpf/bpf/tools/testing/selftests/bpf/uprobe_multi_test.test.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/tmp/work/bpf/bpf/tools/testing/selftests/bpf'
Error: Process completed with exit code 2.