bpf: Add multi uprobe link
Subsystems (8)
Jiri Olsa
reviewerv7 cover bpf: Add multi uprobe link
hi,
this patchset is adding support to attach multiple uprobes and usdt probes through new uprobe_multi link.
The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.
The main reason is that we need to install perf event for each probed function and profile shows perf event installation (perf_install_in_context) as culprit.
The new uprobe_multi link just creates raw uprobes and attaches the bpf program to them without perf event being involved.
In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to.
v7 changes:
- fixed task release on error path and re-org the error path to be more straightforward [Yonghong]
- re-organized uprobe_prog_run locking to follow general pattern and removed might_fault check as it's not needed in uprobe/task context [Yonghong]
There's support for bpftrace [2] and tetragon [1].
Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
uprobe_multi
thanks,
jirka
[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (28):
bpf: Switch BPF_F_KPROBE_MULTI_RETURN macro to enum
bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
bpf: Add multi uprobe link
bpf: Add cookies support for uprobe_multi link
bpf: Add pid filter support for uprobe_multi link
bpf: Add bpf_get_func_ip helper support for uprobe link
libbpf: Add uprobe_multi attach type and link names
libbpf: Move elf_find_func_offset* functions to elf object
libbpf: Add elf_open/elf_close functions
libbpf: Add elf symbol iterator
libbpf: Add elf_resolve_syms_offsets function
libbpf: Add elf_resolve_pattern_offsets function
libbpf: Add bpf_link_create support for multi uprobes
libbpf: Add bpf_program__attach_uprobe_multi function
libbpf: Add support for u[ret]probe.multi[.s] program sections
libbpf: Add uprobe multi link detection
libbpf: Add uprobe multi link support to bpf_program__attach_usdt selftests/bpf: Move get_time_ns to testing_helpers.h
selftests/bpf: Add uprobe_multi skel test
selftests/bpf: Add uprobe_multi api test
selftests/bpf: Add uprobe_multi link test
selftests/bpf: Add uprobe_multi test program
selftests/bpf: Add uprobe_multi bench test
selftests/bpf: Add uprobe_multi usdt test code
selftests/bpf: Add uprobe_multi usdt bench test
selftests/bpf: Add uprobe_multi cookie test
selftests/bpf: Add uprobe_multi pid filter tests
selftests/bpf: Add extra link to uprobe_multi tests
include/linux/trace_events.h | 6 ++ include/uapi/linux/bpf.h | 22 ++++- kernel/bpf/syscall.c | 134 ++++++++++++++-------------- kernel/trace/bpf_trace.c | 342 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- tools/include/uapi/linux/bpf.h | 22 ++++- tools/lib/bpf/Build | 2 +- tools/lib/bpf/bpf.c | 11 +++ tools/lib/bpf/bpf.h | 11 ++- tools/lib/bpf/elf.c | 440 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 388 ++++++++++++++++++++++++++++++++++++++++++--------------------------------------- tools/lib/bpf/libbpf.h | 51 +++++++++++ tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_internal.h | 21 +++++ tools/lib/bpf/usdt.c | 116 ++++++++++++++++-------- tools/testing/selftests/bpf/Makefile | 5 ++ tools/testing/selftests/bpf/bench.h | 9 -- tools/testing/selftests/bpf/prog_tests/bpf_cookie.c | 78 +++++++++++++++++ tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 8 -- tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 415 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi.c | 101 +++++++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi_bench.c | 15 ++++ tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++ tools/testing/selftests/bpf/testing_helpers.h | 10 +++ tools/testing/selftests/bpf/uprobe_multi.c | 91 +++++++++++++++++++ 24 files changed, 1992 insertions(+), 323 deletions(-) create mode 100644 tools/lib/bpf/elf.c create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_bench.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
Jiri Olsa
reviewerPATCH v7 1/28 bpf: Switch BPF_F_KPROBE_MULTI_RETURN macro to enum
Switching BPF_F_KPROBE_MULTI_RETURN macro to anonymous enum, so it'd show up in vmlinux.h. There's not functional change compared to having this as macro.
--- include/uapi/linux/bpf.h | 4 +++- tools/include/uapi/linux/bpf.h | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +3 -1
@@ -1186,7 +1186,9 @@ enum bpf_perf_event_type {/* link_create.kprobe_multi.flags used in LINK_CREATE command for * BPF_TRACE_KPROBE_MULTI attach type to create return probe. */ -#define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +enum { +BPF_F_KPROBE_MULTI_RETURN = (1U <<0 ) +};/* link_create.netfilter.flags used in LINK_CREATE command for * BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation.
tools/include/uapi/linux/bpf.h +3 -1
@@ -1186,7 +1186,9 @@ enum bpf_perf_event_type {/* link_create.kprobe_multi.flags used in LINK_CREATE command for * BPF_TRACE_KPROBE_MULTI attach type to create return probe. */ -#define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +enum { +BPF_F_KPROBE_MULTI_RETURN = (1U <<0 ) +};/* link_create.netfilter.flags used in LINK_CREATE command for * BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation.
Signature · 1 line
2.41.0
Re: [PATCHv7 bpf-next 01/28] bpf: Switch BPF_F_KPROBE_MULTI_RETURN macro to enum
Quoted 7 lines · Jiri Olsa
Switching
BPF_F_KPROBE_MULTI_RETURNmacro to anonymous enum, so it'd show up invmlinux.h. There's not functional change compared to having this as macro.Acked-by Yafang Shao <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
Jiri Olsa
reviewerPATCH v7 2/28 bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
In reply to Jiri OlsaAdd extra attach_type checks from link_create under bpf_prog_attach_check_attach_type.
--- kernel/bpf/syscall.c | 120 +++++++++++++++++++------------------------ 1 file changed, 52 insertions(+), 68 deletions(-)
kernel/bpf/syscall.c +52 -68
@@ -3656,34 +3656,6 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)return fd; } -static int bpf_prog_attach_check_attach_type (const struct bpf_prog *prog, -enum bpf_attach_type attach_type) -{ -switch (prog->type) { -case BPF_PROG_TYPE_CGROUP_SOCK : -case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : -case BPF_PROG_TYPE_CGROUP_SOCKOPT : -case BPF_PROG_TYPE_SK_LOOKUP : -return attach_type == prog->expected_attach_type ?0 : -EINVAL ; -case BPF_PROG_TYPE_CGROUP_SKB : -if (!capable (CAP_NET_ADMIN )) -/* cg-skb progs can be loaded by unpriv user. -* check permissions at attach time. -*/ -return -EPERM ; -return prog->enforce_expected_attach_type && - prog->expected_attach_type != attach_type ? - -EINVAL :0 ; -case BPF_PROG_TYPE_KPROBE : -if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && - attach_type !=BPF_TRACE_KPROBE_MULTI ) -return -EINVAL ; -return 0 ; -default : -return 0 ; - } -} -static enum bpf_prog_type attach_type_to_prog_type (enum bpf_attach_type attach_type) {
@@ -3750,6 +3722,58 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type) } } +static int bpf_prog_attach_check_attach_type (const struct bpf_prog *prog, +enum bpf_attach_type attach_type) +{ +enum bpf_prog_type ptype; + +switch (prog->type) { +case BPF_PROG_TYPE_CGROUP_SOCK : +case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : +case BPF_PROG_TYPE_CGROUP_SOCKOPT : +case BPF_PROG_TYPE_SK_LOOKUP : +return attach_type == prog->expected_attach_type ?0 : -EINVAL ; +case BPF_PROG_TYPE_CGROUP_SKB : +if (!capable (CAP_NET_ADMIN )) +/* cg-skb progs can be loaded by unpriv user. +* check permissions at attach time. +*/ +return -EPERM ; +return prog->enforce_expected_attach_type && + prog->expected_attach_type != attach_type ? + -EINVAL :0 ; +case BPF_PROG_TYPE_EXT : +return 0 ; +case BPF_PROG_TYPE_NETFILTER : +if (attach_type !=BPF_NETFILTER ) +return -EINVAL ; +return 0 ; +case BPF_PROG_TYPE_PERF_EVENT : +case BPF_PROG_TYPE_TRACEPOINT : +if (attach_type !=BPF_PERF_EVENT ) +return -EINVAL ; +return 0 ; +case BPF_PROG_TYPE_KPROBE : +if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && + attach_type !=BPF_TRACE_KPROBE_MULTI ) +return -EINVAL ; +if (attach_type !=BPF_PERF_EVENT && + attach_type !=BPF_TRACE_KPROBE_MULTI ) +return -EINVAL ; +return 0 ; +case BPF_PROG_TYPE_SCHED_CLS : +if (attach_type !=BPF_TCX_INGRESS && + attach_type !=BPF_TCX_EGRESS ) +return -EINVAL ; +return 0 ; +default : + ptype =attach_type_to_prog_type (attach_type); +if (ptype ==BPF_PROG_TYPE_UNSPEC || ptype != prog->type) +return -EINVAL ; +return 0 ; + } +} +#define BPF_PROG_ATTACH_LAST_FIELD expected_revision#define BPF_F_ATTACH_MASK_BASE \
@@ -4856,7 +4880,6 @@ static int bpf_map_do_batch(const union bpf_attr *attr,#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookiesstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) { -enum bpf_prog_type ptype;struct bpf_prog *prog;int ret;
@@ -4875,45 +4898,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)if (ret) goto out; -switch (prog->type) { -case BPF_PROG_TYPE_EXT : -break ; -case BPF_PROG_TYPE_NETFILTER : -if (attr->link_create.attach_type !=BPF_NETFILTER ) { - ret = -EINVAL ; - goto out; - } -break ; -case BPF_PROG_TYPE_PERF_EVENT : -case BPF_PROG_TYPE_TRACEPOINT : -if (attr->link_create.attach_type !=BPF_PERF_EVENT ) { - ret = -EINVAL ; - goto out; - } -break ; -case BPF_PROG_TYPE_KPROBE : -if (attr->link_create.attach_type !=BPF_PERF_EVENT && - attr->link_create.attach_type !=BPF_TRACE_KPROBE_MULTI ) { - ret = -EINVAL ; - goto out; - } -break ; -case BPF_PROG_TYPE_SCHED_CLS : -if (attr->link_create.attach_type !=BPF_TCX_INGRESS && - attr->link_create.attach_type !=BPF_TCX_EGRESS ) { - ret = -EINVAL ; - goto out; - } -break ; -default : - ptype =attach_type_to_prog_type (attr->link_create.attach_type); -if (ptype ==BPF_PROG_TYPE_UNSPEC || ptype != prog->type) { - ret = -EINVAL ; - goto out; - } -break ; - } -switch (prog->type) {case BPF_PROG_TYPE_CGROUP_SKB :case BPF_PROG_TYPE_CGROUP_SOCK :
Signature · 1 line
2.41.0
Re: [PATCHv7 bpf-next 02/28] bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
Quoted 7 lines · Jiri Olsa
Add extra
attach_typechecks fromlink_createunderbpf_prog_attach_check_attach_type.Suggested-by Andrii Nakryiko <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Acked-by Yafang Shao <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
Jiri Olsa
reviewerPATCH v7 3/28 bpf: Add multi uprobe link
In reply to Jiri OlsaAdding new multi uprobe link that allows to attach bpf program to multiple uprobes.
Uprobes to attach are specified via new link_create uprobe_multi union:
struct {
__aligned_u64 path;
__aligned_u64 offsets;
__aligned_u64 ref_ctr_offsets;
__u32 cnt;
__u32 flags;
} uprobe_multi;Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in ref_ctr_offsets array. All specified arrays have length of 'cnt'.
The 'flags' supports single bit for now that marks the uprobe as return probe.
--- include/linux/trace_events.h | 6 + include/uapi/linux/bpf.h | 16 +++ kernel/bpf/syscall.c | 14 +- kernel/trace/bpf_trace.c | 233 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 16 +++ 5 files changed, 282 insertions(+), 3 deletions(-)
include/linux/trace_events.h +6 -0
@@ -752,6 +752,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id, u32 *fd_type, const char **buf, u64 *probe_offset, u64 *probe_addr);int bpf_kprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog); +int bpf_uprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog);#else static inline unsigned int trace_call_bpf (struct trace_event_call *call,void *ctx) {
@@ -798,6 +799,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) {return -EOPNOTSUPP ; } +static inline int +bpf_uprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog) +{ +return -EOPNOTSUPP ; +}#endif enum {
include/uapi/linux/bpf.h +16 -0
@@ -1039,6 +1039,7 @@ enum bpf_attach_type {BPF_NETFILTER ,BPF_TCX_INGRESS ,BPF_TCX_EGRESS , +BPF_TRACE_UPROBE_MULTI , __MAX_BPF_ATTACH_TYPE };
@@ -1057,6 +1058,7 @@ enum bpf_link_type {BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 ,BPF_LINK_TYPE_TCX =11 , +BPF_LINK_TYPE_UPROBE_MULTI =12 , MAX_BPF_LINK_TYPE, };
@@ -1190,6 +1192,13 @@ enum { BPF_F_KPROBE_MULTI_RETURN = (1U << 0) }; +/* link_create.uprobe_multi.flags used in LINK_CREATE command for +* BPF_TRACE_UPROBE_MULTI attach type to create return probe. +*/ +enum { +BPF_F_UPROBE_MULTI_RETURN = (1U <<0 ) +}; +/* link_create.netfilter.flags used in LINK_CREATE command for * BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation. */
@@ -1626,6 +1635,13 @@ union bpf_attr { };__u64 expected_revision; } tcx; +struct { +__aligned_u64 path; +__aligned_u64 offsets; +__aligned_u64 ref_ctr_offsets; +__u32 cnt; +__u32 flags; + } uprobe_multi; }; } link_create;
kernel/bpf/syscall.c +11 -3
@@ -2815,10 +2815,12 @@ static void bpf_link_free_id(int id)/* Clean up bpf_link and corresponding anon_inode file and FD. After * anon_inode is created, bpf_link can't be just kfree()'d due to deferred -* anon_inode's release() call. This helper marksbpf_link as +* anon_inode's release() call. This helper marks bpf_link as * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt * is not decremented, it's the responsibility of a calling code that failed * to complete bpf_link initialization. +* This helper eventually calls link's dealloc callback, but does not call +* link's release callback. */ void bpf_link_cleanup (struct bpf_link_primer *primer) {
@@ -3757,8 +3759,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI && attach_type !=BPF_TRACE_KPROBE_MULTI )return -EINVAL ; +if (prog->expected_attach_type ==BPF_TRACE_UPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI ) +return -EINVAL ;if (attach_type !=BPF_PERF_EVENT && - attach_type !=BPF_TRACE_KPROBE_MULTI ) + attach_type !=BPF_TRACE_KPROBE_MULTI && + attach_type !=BPF_TRACE_UPROBE_MULTI )return -EINVAL ;return 0 ;case BPF_PROG_TYPE_SCHED_CLS :
@@ -4954,8 +4960,10 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)case BPF_PROG_TYPE_KPROBE :if (attr->link_create.attach_type ==BPF_PERF_EVENT ) ret =bpf_perf_link_attach (attr, prog); -else +else if (attr->link_create.attach_type ==BPF_TRACE_KPROBE_MULTI ) ret =bpf_kprobe_multi_link_attach (attr, prog); +else if (attr->link_create.attach_type ==BPF_TRACE_UPROBE_MULTI ) + ret =bpf_uprobe_multi_link_attach (attr, prog);break ;default : ret = -EINVAL ;
kernel/trace/bpf_trace.c +233 -0
242 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +16 -0
@@ -1039,6 +1039,7 @@ enum bpf_attach_type {BPF_NETFILTER ,BPF_TCX_INGRESS ,BPF_TCX_EGRESS , +BPF_TRACE_UPROBE_MULTI , __MAX_BPF_ATTACH_TYPE };
@@ -1057,6 +1058,7 @@ enum bpf_link_type {BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 ,BPF_LINK_TYPE_TCX =11 , +BPF_LINK_TYPE_UPROBE_MULTI =12 , MAX_BPF_LINK_TYPE, };
@@ -1190,6 +1192,13 @@ enum { BPF_F_KPROBE_MULTI_RETURN = (1U << 0) }; +/* link_create.uprobe_multi.flags used in LINK_CREATE command for +* BPF_TRACE_UPROBE_MULTI attach type to create return probe. +*/ +enum { +BPF_F_UPROBE_MULTI_RETURN = (1U <<0 ) +}; +/* link_create.netfilter.flags used in LINK_CREATE command for * BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation. */
@@ -1626,6 +1635,13 @@ union bpf_attr { };__u64 expected_revision; } tcx; +struct { +__aligned_u64 path; +__aligned_u64 offsets; +__aligned_u64 ref_ctr_offsets; +__u32 cnt; +__u32 flags; + } uprobe_multi; }; } link_create;
Signature · 1 line
2.41.0
Quoted 19 lines · Jiri Olsa
Adding new multi uprobe link that allows to attach bpf program to multiple uprobes.
Uprobes to attach are specified via new
link_createuprobe_multiunion:struct { __aligned_u64 path; __aligned_u64 offsets; __aligned_u64 ref_ctr_offsets; __u32 cnt; __u32 flags; } uprobe_multi;Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in
ref_ctr_offsetsarray. All specified arrays have length of 'cnt'.
Quoted 6 lines
The 'flags' supports single bit for now that marks the uprobe as return probe.
Acked-by Andrii Nakryiko <[email protected]>Acked-by Yafang Shao <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
Jiri Olsa
reviewerPATCH v7 4/28 bpf: Add cookies support for uprobe_multi link
In reply to Jiri OlsaAdding support to specify cookies array for uprobe_multi link.
The cookies array share indexes and length with other uprobe_multi arrays (offsets/ref_ctr_offsets).
The cookies[i] value defines cookie for i-the uprobe and will be returned by bpf_get_attach_cookie helper when called from ebpf program hooked to that specific uprobe.
--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 45 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 44 insertions(+), 5 deletions(-)
include/uapi/linux/bpf.h +1 -0
@@ -1639,6 +1639,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies;__u32 cnt;__u32 flags; } uprobe_multi;
kernel/bpf/syscall.c +1 -1
@@ -4883,7 +4883,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,return err; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.flagsstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) {struct bpf_prog *prog;
kernel/trace/bpf_trace.c +41 -4
115 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +1 -0
@@ -1639,6 +1639,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies;__u32 cnt;__u32 flags; } uprobe_multi;
Signature · 1 line
2.41.0
Re: [PATCHv7 bpf-next 04/28] bpf: Add cookies support for uprobe_multi link
Quoted 5 lines · Jiri Olsa
Adding support to specify cookies array for
uprobe_multilink.The cookies array share indexes and length with other
uprobe_multiarrays (offsets/ref_ctr_offsets).
Quoted 7 lines
The cookies[i] value defines cookie for i-the uprobe and will be returned by
bpf_get_attach_cookiehelper when called from ebpf program hooked to that specific uprobe.Acked-by Andrii Nakryiko <[email protected]>Acked-by Yafang Shao <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
Ack with a minor nit below.
Quoted 12 lines
--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 45 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 44 insertions(+), 5 deletions(-)include/uapi/linux/bpf.h +0
-0
include/uapi/linux/bpf.h union bpf_attr { · line 1645
Quoted 7 lines
include/uapi/linux/bpf.h +1
-0@@ -1639,6 +1639,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies;__u32 cnt;__u32 flags; } uprobe_multi;
The 'cookies' field is inserted into the middle of 'uprobe_multi' struct. Not sure whether this may cause bug bisecting issue or not.
Re: [PATCHv7 bpf-next 04/28] bpf: Add cookies support for uprobe_multi link
Quoted 27 lines · Yonghong Song
Quoted 5 lines · Jiri Olsa
Adding support to specify cookies array for
uprobe_multilink.The cookies array share indexes and length with other
uprobe_multiarrays (offsets/ref_ctr_offsets).Quoted 7 lines
The cookies[i] value defines cookie for i-the uprobe and will be returned by
bpf_get_attach_cookiehelper when called from ebpf program hooked to that specific uprobe.Acked-by Andrii Nakryiko <[email protected]>Acked-by Yafang Shao <[email protected]>Signed-off-by Jiri Olsa <[email protected]>Ack with a minor nit below.
Acked-by Yonghong Song <[email protected]>Quoted 7 lines
--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 45 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 44 insertions(+), 5 deletions(-)
Quoted 15 lines
include/uapi/linux/bpf.h union bpf_attr { · line 1645
Quoted 12 lines
include/uapi/linux/bpf.h +1
-0@@ -1639,6 +1639,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies;__u32 cnt;__u32 flags; } uprobe_multi;The 'cookies' field is inserted into the middle of '
uprobe_multi' struct. Not sure whether this may cause bug bisecting issue or not.
the idea is to have all fields related to the 'cnt' field grouped
so the problem is that the bisect reproducer would fail on previous patch because of the changed uprobe_multi layout, but it only got introduced in previous patch, so we are close ;-)
I don't think it's real problem but I guess I could add 'cookie' field in previous patch
thanks,
jirka
Yonghong Song
reviewerRe: [PATCHv7 bpf-next 04/28] bpf: Add cookies support for uprobe_multi link
Quoted 44 lines · Jiri Olsa
Quoted 27 lines · Yonghong Song
Quoted 5 lines · Jiri Olsa
Adding support to specify cookies array for
uprobe_multilink.The cookies array share indexes and length with other
uprobe_multiarrays (offsets/ref_ctr_offsets).Quoted 7 lines
The cookies[i] value defines cookie for i-the uprobe and will be returned by
bpf_get_attach_cookiehelper when called from ebpf program hooked to that specific uprobe.Acked-by Andrii Nakryiko <[email protected]>Acked-by Yafang Shao <[email protected]>Signed-off-by Jiri Olsa <[email protected]>Ack with a minor nit below.
Acked-by Yonghong Song <[email protected]>Quoted 7 lines
--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 45 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 44 insertions(+), 5 deletions(-)Quoted 15 lines
include/uapi/linux/bpf.h union bpf_attr { · line 1645
Quoted 12 lines
include/uapi/linux/bpf.h +1
-0@@ -1639,6 +1639,7 @@ union bpf_attr {__aligned_u64 path;__aligned_u64 offsets;__aligned_u64 ref_ctr_offsets; +__aligned_u64 cookies;__u32 cnt;__u32 flags; } uprobe_multi;The 'cookies' field is inserted into the middle of '
uprobe_multi' struct. Not sure whether this may cause bug bisecting issue or not.
Quoted 8 lines
the idea is to have all fields related to the 'cnt' field grouped
so the problem is that the bisect reproducer would fail on previous patch because of the changed
uprobe_multilayout, but it only got introduced in previous patch, so we are close ;-)I don't think it's real problem but I guess I could add 'cookie' field in previous patch
I agree it should not be a real issue. Maybe let maintainers to step in with their opinion.
Quoted 4 lines
thanks,
jirka
Jiri Olsa
reviewerPATCH v7 5/28 bpf: Add pid filter support for uprobe_multi link
In reply to Jiri OlsaAdding support to specify pid for uprobe_multi link and the uprobes are created only for task with given pid value.
Using the consumer.filter filter callback for that, so the task gets filtered during the uprobe installation.
We still need to check the task during runtime in the uprobe handler, because the handler could get executed if there's another system wide consumer on the same uprobe (thanks Oleg for the insight).
--- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 2 +- kernel/trace/bpf_trace.c | 33 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 36 insertions(+), 1 deletion(-)
include/uapi/linux/bpf.h +1 -0
@@ -1642,6 +1642,7 @@ union bpf_attr {__aligned_u64 cookies;__u32 cnt;__u32 flags; +__u32 pid; } uprobe_multi; }; } link_create;
kernel/bpf/syscall.c +1 -1
@@ -4883,7 +4883,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,return err; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.flags +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.pidstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) {struct bpf_prog *prog;
kernel/trace/bpf_trace.c +33 -0
@@ -3003,6 +3003,7 @@ struct bpf_uprobe_multi_link {struct bpf_link link;u32 cnt;struct bpf_uprobe *uprobes; +struct task_struct *task; };struct bpf_uprobe_multi_run_ctx {
@@ -3035,6 +3036,8 @@ static void bpf_uprobe_multi_link_dealloc(struct bpf_link *link)struct bpf_uprobe_multi_link *umulti_link; umulti_link = container_of(link, struct bpf_uprobe_multi_link, link); +if (umulti_link->task) +put_task_struct (umulti_link->task);path_put (&umulti_link->path);kvfree (umulti_link->uprobes);kfree (umulti_link);
@@ -3059,6 +3062,9 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,struct bpf_run_ctx *old_run_ctx;int err =0 ; +if (link->task && current != link->task) +return 0 ; +if (sleepable)rcu_read_lock_trace ();else
@@ -3079,6 +3085,16 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,return err; } +static bool +uprobe_multi_link_filter (struct uprobe_consumer *con,enum uprobe_filter_ctx ctx, +struct mm_struct *mm) +{ +struct bpf_uprobe *uprobe; + + uprobe = container_of(con, struct bpf_uprobe, consumer); +return uprobe->link->task->mm == mm; +} +static int uprobe_multi_link_handler (struct uprobe_consumer *con,struct pt_regs *regs) {
@@ -3112,12 +3128,14 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prunsigned long *ref_ctr_offsets =NULL ;struct bpf_link_primer link_primer;struct bpf_uprobe *uprobes =NULL ; +struct task_struct *task =NULL ;unsigned long __user *uoffsets;u64 __user *ucookies;void __user *upath;u32 flags, cnt, i;struct path path;char *name; +pid_t pid;int err;/* no support for 32bit archs yet */
@@ -3161,6 +3179,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr goto error_path_put; } + pid = attr->link_create.uprobe_multi.pid; +if (pid) { +rcu_read_lock (); + task =get_pid_task (find_vpid (pid),PIDTYPE_PID ); +rcu_read_unlock (); +if (!task) + goto error_path_put; + } + err = -ENOMEM ; link =kzalloc (sizeof (*link),GFP_KERNEL );
@@ -3195,11 +3222,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler; else uprobes[i].consumer.handler = uprobe_multi_link_handler; + +if (pid) + uprobes[i].consumer.filter = uprobe_multi_link_filter; } link->cnt = cnt; link->uprobes = uprobes; link->path = path; + link->task = task;bpf_link_init (&link->link,BPF_LINK_TYPE_UPROBE_MULTI , &bpf_uprobe_multi_link_lops, prog);
@@ -3226,6 +3257,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prkvfree (ref_ctr_offsets);kvfree (uprobes);kfree (link); +if (task) +put_task_struct (task); error_path_put:path_put (&path);return err;
tools/include/uapi/linux/bpf.h +1 -0
@@ -1642,6 +1642,7 @@ union bpf_attr {__aligned_u64 cookies;__u32 cnt;__u32 flags; +__u32 pid; } uprobe_multi; }; } link_create;
Signature · 1 line
2.41.0
Re: [PATCHv7 bpf-next 05/28] bpf: Add pid filter support for uprobe_multi link
Quoted 6 lines · Jiri Olsa
Adding support to specify pid for
uprobe_multilink and the uprobes are created only for task with given pid value.Using the consumer.filter filter callback for that, so the task gets filtered during the uprobe installation.
Quoted 7 lines
We still need to check the task during runtime in the uprobe handler, because the handler could get executed if there's another system wide consumer on the same uprobe (thanks Oleg for the insight).
Cc Oleg Nesterov <[email protected]>Reviewed-by Oleg Nesterov <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
Jiri Olsa
reviewerPATCH v7 6/28 bpf: Add bpf_get_func_ip helper support for uprobe link
In reply to Jiri OlsaAdding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.
It returns the ip of the uprobe.
--- kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)
kernel/trace/bpf_trace.c +30 -3
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);static u64 bpf_kprobe_multi_entry_ip (struct bpf_run_ctx *ctx);static u64 bpf_uprobe_multi_cookie (struct bpf_run_ctx *ctx); +static u64 bpf_uprobe_multi_entry_ip (struct bpf_run_ctx *ctx);/** * trace_call_bpf - invoke BPF program
@@ -1106,6 +1107,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_func_ip_uprobe_multi, struct pt_regs *, regs) +{ +return bpf_uprobe_multi_entry_ip (current->bpf_ctx); +} + +static const struct bpf_func_proto bpf_get_func_ip_proto_uprobe_multi = { + .func = bpf_get_func_ip_uprobe_multi, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , +}; + BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs) {return bpf_uprobe_multi_cookie (current->bpf_ctx);
@@ -1560,9 +1573,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_override_return_proto;#endif case BPF_FUNC_get_func_ip: -return prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI ? - &bpf_get_func_ip_proto_kprobe_multi : - &bpf_get_func_ip_proto_kprobe; +if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI ) +return &bpf_get_func_ip_proto_kprobe_multi; +if (prog->expected_attach_type ==BPF_TRACE_UPROBE_MULTI ) +return &bpf_get_func_ip_proto_uprobe_multi; +return &bpf_get_func_ip_proto_kprobe;case BPF_FUNC_get_attach_cookie:if (prog->expected_attach_type ==BPF_TRACE_KPROBE_MULTI )return &bpf_get_attach_cookie_proto_kmulti;
@@ -3113,6 +3128,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, sreturn uprobe_prog_run (uprobe, func, regs); } +static u64 bpf_uprobe_multi_entry_ip (struct bpf_run_ctx *ctx) +{ +struct bpf_uprobe_multi_run_ctx *run_ctx; + + run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx); +return run_ctx->entry_ip; +} +static u64 bpf_uprobe_multi_cookie (struct bpf_run_ctx *ctx) {struct bpf_uprobe_multi_run_ctx *run_ctx;
@@ -3272,4 +3295,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx) {return 0 ; } +static u64 bpf_uprobe_multi_entry_ip (struct bpf_run_ctx *ctx) +{ +return 0 ; +}#endif /* CONFIG_UPROBES */
Signature · 1 line
2.41.0
Re: [PATCHv7 bpf-next 06/28] bpf: Add bpf_get_func_ip helper support for uprobe link
Quoted 7 lines · Jiri Olsa
Adding support for
bpf_get_func_iphelper being called from ebpf program attached byuprobe_multilink.It returns the ip of the uprobe.
Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
Jiri Olsa
reviewerPATCH v7 7/28 libbpf: Add uprobe_multi attach type and link names
In reply to Jiri OlsaAdding new uprobe_multi attach type and link names, so the functions can resolve the new values.
--- tools/lib/bpf/libbpf.c | 2 ++ 1 file changed, 2 insertions(+)
tools/lib/bpf/libbpf.c +2 -0
@@ -120,6 +120,7 @@ static const char * const attach_type_name[] = { [BPF_NETFILTER] = "netfilter", [BPF_TCX_INGRESS] = "tcx_ingress", [BPF_TCX_EGRESS] = "tcx_egress", + [BPF_TRACE_UPROBE_MULTI] = "trace_uprobe_multi", };static const char *const link_type_name[] = {
@@ -135,6 +136,7 @@ static const char * const link_type_name[] = { [BPF_LINK_TYPE_STRUCT_OPS] = "struct_ops", [BPF_LINK_TYPE_NETFILTER] = "netfilter", [BPF_LINK_TYPE_TCX] = "tcx", + [BPF_LINK_TYPE_UPROBE_MULTI] = "uprobe_multi", };static const char *const map_type_name[] = {
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 8/28 libbpf: Move elf_find_func_offset* functions to elf object
In reply to Jiri OlsaAdding new elf object that will contain elf related functions. There's no functional change.
--- tools/lib/bpf/Build | 2 +- tools/lib/bpf/elf.c | 197 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 185 ------------------------------ tools/lib/bpf/libbpf_internal.h | 4 + 4 files changed, 202 insertions(+), 186 deletions(-) create mode 100644 tools/lib/bpf/elf.c
tools/lib/bpf/Build +1 -1
@@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o + usdt.o zip.o elf.o
tools/lib/bpf/elf.c new file +197 -0
197 lines not on this page · raw message
tools/lib/bpf/libbpf.c +0 -185
191 lines not on this page · raw message
tools/lib/bpf/libbpf_internal.h +4 -0
@@ -15,6 +15,7 @@#include <linux/err.h> #include <fcntl.h> #include <unistd.h> +#include <libelf.h> #include "relo_core.h" /* make sure libbpf doesn't use kernel-only integer typedefs */
@@ -577,4 +578,7 @@ static inline bool is_pow_of_2(size_t x)#define PROG_LOAD_ATTEMPTS 5int sys_bpf_prog_load (union bpf_attr *attr,unsigned int size,int attempts); +long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name); +long elf_find_func_offset_from_file (const char *binary_path,const char *name); +#endif /* __LIBBPF_LIBBPF_INTERNAL_H */
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 9/28 libbpf: Add elf_open/elf_close functions
In reply to Jiri OlsaAdding elf_open/elf_close functions and using it in elf_find_func_offset_from_file function. It will be used in following changes to save some common code.
--- tools/lib/bpf/elf.c | 61 ++++++++++++++++++++++----------- tools/lib/bpf/libbpf_internal.h | 8 +++++ tools/lib/bpf/usdt.c | 30 +++++----------- 3 files changed, 57 insertions(+), 42 deletions(-)
tools/lib/bpf/elf.c +41 -20
@@ -10,6 +10,42 @@#define STRERR_BUFSIZE 128 +int elf_open (const char *binary_path,struct elf_fd *elf_fd) +{ +char errmsg[STRERR_BUFSIZE ]; +int fd, ret; +Elf *elf; + +if (elf_version (EV_CURRENT ) ==EV_NONE ) { +pr_warn ("elf: failed to init libelf for %s\n" , binary_path); +return -LIBBPF_ERRNO__LIBELF ; + } + fd =open (binary_path,O_RDONLY |O_CLOEXEC ); +if (fd <0 ) { + ret = -errno; +pr_warn ("elf: failed to open %s: %s\n" , binary_path, +libbpf_strerror_r (ret, errmsg,sizeof (errmsg))); +return ret; + } + elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); +if (!elf) { +pr_warn ("elf: could not read elf from %s: %s\n" , binary_path,elf_errmsg (-1 )); +close (fd); +return -LIBBPF_ERRNO__FORMAT ; + } + elf_fd->fd = fd; + elf_fd->elf = elf; +return 0 ; +} + +void elf_close (struct elf_fd *elf_fd) +{ +if (!elf_fd) +return ; +elf_end (elf_fd->elf); +close (elf_fd->fd); +} +/* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */ static Elf_Scn *elf_find_next_scn_by_type (Elf *elf,int sh_type,Elf_Scn *scn) {
@@ -170,28 +206,13 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)*/ long elf_find_func_offset_from_file (const char *binary_path,const char *name) { -char errmsg[STRERR_BUFSIZE ]; +struct elf_fd elf_fd;long ret = -ENOENT ; -Elf *elf; -int fd; - fd =open (binary_path,O_RDONLY |O_CLOEXEC ); -if (fd <0 ) { - ret = -errno; -pr_warn ("failed to open %s: %s\n" , binary_path, -libbpf_strerror_r (ret, errmsg,sizeof (errmsg))); + ret =elf_open (binary_path, &elf_fd); +if (ret)return ret; - } - elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); -if (!elf) { -pr_warn ("elf: could not read elf from %s: %s\n" , binary_path,elf_errmsg (-1 )); -close (fd); -return -LIBBPF_ERRNO__FORMAT ; - } - - ret =elf_find_func_offset (elf, binary_path, name); -elf_end (elf); -close (fd); + ret =elf_find_func_offset (elf_fd.elf, binary_path, name); +elf_close (&elf_fd);return ret; } -
tools/lib/bpf/libbpf_internal.h +8 -0
@@ -581,4 +581,12 @@ int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts);long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name);long elf_find_func_offset_from_file (const char *binary_path,const char *name); +struct elf_fd { +Elf *elf; +int fd; +}; + +int elf_open (const char *binary_path,struct elf_fd *elf_fd); +void elf_close (struct elf_fd *elf_fd); +#endif /* __LIBBPF_LIBBPF_INTERNAL_H */
tools/lib/bpf/usdt.c +8 -22
@@ -946,32 +946,22 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct const char *usdt_provider, const char *usdt_name, __u64 usdt_cookie) { - int i, fd, err, spec_map_fd, ip_map_fd; + int i, err, spec_map_fd, ip_map_fd;LIBBPF_OPTS (bpf_uprobe_opts, opts);struct hashmap *specs_hash =NULL ;struct bpf_link_usdt *link =NULL ;struct usdt_target *targets =NULL ; +struct elf_fd elf_fd;size_t target_cnt; -Elf *elf; spec_map_fd =bpf_map__fd (man->specs_map); ip_map_fd =bpf_map__fd (man->ip_to_spec_id_map); - fd =open (path,O_RDONLY |O_CLOEXEC ); -if (fd <0 ) { - err = -errno; -pr_warn ("usdt: failed to open ELF binary '%s': %d\n" , path, err); + err =elf_open (path, &elf_fd); +if (err)return libbpf_err_ptr (err); - } - elf =elf_begin (fd,ELF_C_READ_MMAP ,NULL ); -if (!elf) { - err = -EBADF ; -pr_warn ("usdt: failed to parse ELF binary '%s': %s\n" , path,elf_errmsg (-1 )); - goto err_out; - } - - err =sanity_check_usdt_elf (elf, path); + err =sanity_check_usdt_elf (elf_fd.elf, path);if (err) goto err_out;
@@ -984,7 +974,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct/* discover USDT in given binary, optionally limiting * activations to a given PID, if pid > 0 */ - err =collect_usdt_targets (man, elf, path, pid, usdt_provider, usdt_name, + err =collect_usdt_targets (man, elf_fd.elf, path, pid, usdt_provider, usdt_name, usdt_cookie, &targets, &target_cnt);if (err <=0 ) { err = (err ==0 ) ? -ENOENT : err;
@@ -1069,9 +1059,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const structfree (targets);hashmap__free (specs_hash); -elf_end (elf); -close (fd); - +elf_close (&elf_fd);return &link->link; err_out:
@@ -1079,9 +1067,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const structbpf_link__destroy (&link->link);free (targets);hashmap__free (specs_hash); -if (elf) -elf_end (elf); -close (fd); +elf_close (&elf_fd);return libbpf_err_ptr (err); }
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 10/28 libbpf: Add elf symbol iterator
In reply to Jiri OlsaAdding elf symbol iterator object (and some functions) that follow open-coded iterator pattern and some functions to ease up iterating elf object symbols.
The idea is to iterate single symbol section with:
struct elf_sym_iter iter;
struct elf_sym *sym;
if (elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM))
goto error;
while ((sym = elf_sym_iter_next(&iter))) {
...
}I considered opening the elf inside the iterator and iterate all symbol sections, but then it gets more complicated wrt user checks for when the next section is processed.
Plus side is the we don't need 'exit' function, because caller/user is in charge of that.
The returned iterated symbol object from elf_sym_iter_next function is placed inside the struct elf_sym_iter, so no extra allocation or argument is needed.
--- tools/lib/bpf/elf.c | 179 ++++++++++++++++++++++++++++---------------- 1 file changed, 115 insertions(+), 64 deletions(-)
tools/lib/bpf/elf.c +115 -64
@@ -60,6 +60,104 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)return NULL ; } +struct elf_sym { +const char *name; +GElf_Sym sym; +GElf_Shdr sh; +}; + +struct elf_sym_iter { +Elf *elf; +Elf_Data *syms; +size_t nr_syms; +size_t strtabidx; +size_t next_sym_idx; +struct elf_sym sym; +int st_type; +}; + +static int elf_sym_iter_new (struct elf_sym_iter *iter, +Elf *elf,const char *binary_path, +int sh_type,int st_type) +{ +Elf_Scn *scn =NULL ; +GElf_Ehdr ehdr; +GElf_Shdr sh; + +memset (iter,0 ,sizeof (*iter)); + +if (!gelf_getehdr (elf, &ehdr)) { +pr_warn ("elf: failed to get ehdr from %s: %s\n" , binary_path,elf_errmsg (-1 )); +return -EINVAL ; + } + + scn =elf_find_next_scn_by_type (elf, sh_type,NULL ); +if (!scn) { +pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , + binary_path); +return -ENOENT ; + } + +if (!gelf_getshdr (scn, &sh)) +return -EINVAL ; + + iter->strtabidx = sh.sh_link; + iter->syms =elf_getdata (scn,0 ); +if (!iter->syms) { +pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , + binary_path,elf_errmsg (-1 )); +return -EINVAL ; + } + iter->nr_syms = iter->syms->d_size / sh.sh_entsize; + iter->elf = elf; + iter->st_type = st_type; +return 0 ; +} + +static struct elf_sym *elf_sym_iter_next (struct elf_sym_iter *iter) +{ +struct elf_sym *ret = &iter->sym; +GElf_Sym *sym = &ret->sym; +const char *name =NULL ; +Elf_Scn *sym_scn; +size_t idx; + +for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) { +if (!gelf_getsym (iter->syms, idx, sym)) +continue ; +if (GELF_ST_TYPE (sym->st_info) != iter->st_type) +continue ; + name =elf_strptr (iter->elf, iter->strtabidx, sym->st_name); +if (!name) +continue ; + sym_scn =elf_getscn (iter->elf, sym->st_shndx); +if (!sym_scn) +continue ; +if (!gelf_getshdr (sym_scn, &ret->sh)) +continue ; + + iter->next_sym_idx = idx +1 ; + ret->name = name; +return ret; + } + +return NULL ; +} + + +/* Transform symbol's virtual address (absolute for binaries and relative +* for shared libs) into file offset, which is what kernel is expecting +* for uprobe/uretprobe attachment. +* See Documentation/trace/uprobetracer.rst for more details. This is done +* by looking up symbol's containing section's header and using iter's virtual +* address (sh_addr) and corresponding file offset (sh_offset) to transform +* sym.st_value (virtual address) into desired final file offset. +*/ +static unsigned long elf_sym_offset (struct elf_sym *sym) +{ +return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset; +} +/* Find offset of function name in the provided ELF object. "binary_path" is * the path to the ELF binary represented by "elf", and only used for error * reporting matters. "name" matches symbol name or name@@LIB for library
@@ -91,67 +189,38 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)* reported as a warning/error. */ for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { -size_t nr_syms, strtabidx, idx; -Elf_Data *symbols =NULL ; -Elf_Scn *scn =NULL ; +struct elf_sym_iter iter; +struct elf_sym *sym;int last_bind =-1 ; -const char *sname; -GElf_Shdr sh; +int cur_bind; - scn =elf_find_next_scn_by_type (elf, sh_types[i],NULL ); -if (!scn) { -pr_debug ("elf: failed to find symbol table ELF sections in '%s'\n" , - binary_path); + ret =elf_sym_iter_new (&iter, elf, binary_path, sh_types[i],STT_FUNC ); +if (ret == -ENOENT )continue ; - } -if (!gelf_getshdr (scn, &sh)) -continue ; - strtabidx = sh.sh_link; - symbols =elf_getdata (scn,0 ); -if (!symbols) { -pr_warn ("elf: failed to get symbols for symtab section in '%s': %s\n" , - binary_path,elf_errmsg (-1 )); - ret = -LIBBPF_ERRNO__FORMAT ; +if (ret) goto out; - } - nr_syms = symbols->d_size / sh.sh_entsize; - -for (idx =0 ; idx < nr_syms; idx++) { -int curr_bind; -GElf_Sym sym; -Elf_Scn *sym_scn; -GElf_Shdr sym_sh; - -if (!gelf_getsym (symbols, idx, &sym)) -continue ; - -if (GELF_ST_TYPE (sym.st_info) !=STT_FUNC ) -continue ; - - sname =elf_strptr (elf, strtabidx, sym.st_name); -if (!sname) -continue ; - - curr_bind =GELF_ST_BIND (sym.st_info); +while ((sym =elf_sym_iter_next (&iter))) {/* User can specify func, func@@LIB or func@@LIB_VERSION. */ -if (strncmp (sname, name, name_len) !=0 ) +if (strncmp (sym->name, name, name_len) !=0 )continue ;/* ...but we don't want a search for "foo" to match 'foo2" also, so any * additional characters in sname should be of the form "@@LIB". */ -if (!is_name_qualified && sname[name_len] !='\0' && sname[name_len] !='@' ) +if (!is_name_qualified && sym->name[name_len] !='\0' && sym->name[name_len] !='@' )continue ; -if (ret >=0 ) { + cur_bind =GELF_ST_BIND (sym->sym.st_info); + +if (ret >0 ) {/* handle multiple matches */ -if (last_bind !=STB_WEAK && curr_bind !=STB_WEAK ) { +if (last_bind !=STB_WEAK && cur_bind !=STB_WEAK ) {/* Only accept one non-weak bind. */ pr_warn ("elf: ambiguous match for '%s', '%s' in '%s'\n" , - sname, name, binary_path); + sym->name, name, binary_path); ret = -LIBBPF_ERRNO__FORMAT ; goto out; - }else if (curr_bind ==STB_WEAK ) { + }else if (cur_bind ==STB_WEAK ) {/* already have a non-weak bind, and * this is a weak bind, so ignore. */
@@ -159,26 +228,8 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) } } -/* Transform symbol's virtual address (absolute for -* binaries and relative for shared libs) into file -* offset, which is what kernel is expecting for -* uprobe/uretprobe attachment. -* See Documentation/trace/uprobetracer.rst for more -* details. -* This is done by looking up symbol's containing -* section's header and using it's virtual address -* (sh_addr) and corresponding file offset (sh_offset) -* to transform sym.st_value (virtual address) into -* desired final file offset. -*/ - sym_scn =elf_getscn (elf, sym.st_shndx); -if (!sym_scn) -continue ; -if (!gelf_getshdr (sym_scn, &sym_sh)) -continue ; - - ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset; - last_bind = curr_bind; + ret =elf_sym_offset (sym); + last_bind = cur_bind; }if (ret >0 )break ;
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 11/28 libbpf: Add elf_resolve_syms_offsets function
In reply to Jiri OlsaAdding elf_resolve_syms_offsets function that looks up offsets for symbols specified in syms array argument.
Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller.
--- tools/lib/bpf/elf.c | 110 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_internal.h | 2 + 2 files changed, 112 insertions(+)
tools/lib/bpf/elf.c +110 -0
@@ -267,3 +267,113 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)elf_close (&elf_fd);return ret; } + +struct symbol { +const char *name; +int bind; +int idx; +}; + +static int symbol_cmp (const void *a,const void *b) +{ +const struct symbol *sym_a = a; +const struct symbol *sym_b = b; + +return strcmp (sym_a->name, sym_b->name); +} + +/* +* Return offsets in @poffsets for symbols specified in @syms array argument. +* On success returns 0 and offsets are returned in allocated array with @cnt +* size, that needs to be released by the caller. +*/ +int elf_resolve_syms_offsets (const char *binary_path,int cnt, +const char **syms,unsigned long **poffsets) +{ +int sh_types[2 ] = {SHT_DYNSYM ,SHT_SYMTAB }; +int err =0 , i, cnt_done =0 ; +unsigned long *offsets; +struct symbol *symbols; +struct elf_fd elf_fd; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; + + offsets =calloc (cnt,sizeof (*offsets)); + symbols =calloc (cnt,sizeof (*symbols)); + +if (!offsets || !symbols) { + err = -ENOMEM ; + goto out; + } + +for (i =0 ; i < cnt; i++) { + symbols[i].name = syms[i]; + symbols[i].idx = i; + } + +qsort (symbols, cnt,sizeof (*symbols), symbol_cmp); + +for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; + + err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err == -ENOENT ) +continue ; +if (err) + goto out; + +while ((sym =elf_sym_iter_next (&iter))) { +unsigned long sym_offset =elf_sym_offset (sym); +int bind =GELF_ST_BIND (sym->sym.st_info); +struct symbol *found, tmp = { + .name = sym->name, + }; +unsigned long *offset; + + found =bsearch (&tmp, symbols, cnt,sizeof (*symbols), symbol_cmp); +if (!found) +continue ; + + offset = &offsets[found->idx]; +if (*offset >0 ) { +/* same offset, no problem */ +if (*offset == sym_offset) +continue ; +/* handle multiple matches */ +if (found->bind !=STB_WEAK && bind !=STB_WEAK ) { +/* Only accept one non-weak bind. */ +pr_warn ("elf: ambiguous match found '%s@%lu' in '%s' previous offset %lu\n" , + sym->name, sym_offset, binary_path, *offset); + err = -ESRCH ; + goto out; + }else if (bind ==STB_WEAK ) { +/* already have a non-weak bind, and +* this is a weak bind, so ignore. +*/ +continue ; + } + }else { + cnt_done++; + } + *offset = sym_offset; + found->bind = bind; + } + } + +if (cnt != cnt_done) { + err = -ENOENT ; + goto out; + } + + *poffsets = offsets; + +out: +free (symbols); +if (err) +free (offsets); +elf_close (&elf_fd); +return err; +}
tools/lib/bpf/libbpf_internal.h +2 -0
@@ -589,4 +589,6 @@ struct elf_fd {int elf_open (const char *binary_path,struct elf_fd *elf_fd);void elf_close (struct elf_fd *elf_fd); +int elf_resolve_syms_offsets (const char *binary_path,int cnt, +const char **syms,unsigned long **poffsets);#endif /* __LIBBPF_LIBBPF_INTERNAL_H */
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 12/28 libbpf: Add elf_resolve_pattern_offsets function
In reply to Jiri OlsaAdding elf_resolve_pattern_offsets function that looks up offsets for symbols specified by pattern argument.
The 'pattern' argument allows wildcards (*?' supported).
Offsets are returned in allocated array together with its size and needs to be released by the caller.
--- tools/lib/bpf/elf.c | 61 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 2 +- tools/lib/bpf/libbpf_internal.h | 5 +++ 3 files changed, 67 insertions(+), 1 deletion(-)
tools/lib/bpf/elf.c +61 -0
@@ -377,3 +377,64 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,elf_close (&elf_fd);return err; } + +/* +* Return offsets in @poffsets for symbols specified by @pattern argument. +* On success returns 0 and offsets are returned in allocated @poffsets +* array with the @pctn size, that needs to be released by the caller. +*/ +int elf_resolve_pattern_offsets (const char *binary_path,const char *pattern, +unsigned long **poffsets,size_t *pcnt) +{ +int sh_types[2 ] = {SHT_SYMTAB ,SHT_DYNSYM }; +unsigned long *offsets =NULL ; +size_t cap =0 , cnt =0 ; +struct elf_fd elf_fd; +int err =0 , i; + + err =elf_open (binary_path, &elf_fd); +if (err) +return err; + +for (i =0 ; i <ARRAY_SIZE (sh_types); i++) { +struct elf_sym_iter iter; +struct elf_sym *sym; + + err =elf_sym_iter_new (&iter, elf_fd.elf, binary_path, sh_types[i],STT_FUNC ); +if (err == -ENOENT ) +continue ; +if (err) + goto out; + +while ((sym =elf_sym_iter_next (&iter))) { +if (!glob_match (sym->name, pattern)) +continue ; + + err =libbpf_ensure_mem ((void **) &offsets, &cap,sizeof (*offsets), + cnt +1 ); +if (err) + goto out; + + offsets[cnt++] =elf_sym_offset (sym); + } + +/* If we found anything in the first symbol section, +* do not search others to avoid duplicates. +*/ +if (cnt) +break ; + } + +if (cnt) { + *poffsets = offsets; + *pcnt = cnt; + }else { + err = -ENOENT ; + } + +out: +if (err) +free (offsets); +elf_close (&elf_fd); +return err; +}
tools/lib/bpf/libbpf.c +1 -1
@@ -10551,7 +10551,7 @@ struct bpf_link *bpf_program__attach_ksyscall(const struct bpf_program *prog, }/* Adapted from perf/util/string.c */ -static bool glob_match (const char *str,const char *pat) +bool glob_match (const char *str,const char *pat) {while (*str && *pat && *pat !='*' ) {if (*pat =='?' ) {/* Matches any single character */
tools/lib/bpf/libbpf_internal.h +5 -0
@@ -578,6 +578,8 @@ static inline bool is_pow_of_2(size_t x)#define PROG_LOAD_ATTEMPTS 5int sys_bpf_prog_load (union bpf_attr *attr,unsigned int size,int attempts); +bool glob_match (const char *str,const char *pat); +long elf_find_func_offset (Elf *elf,const char *binary_path,const char *name);long elf_find_func_offset_from_file (const char *binary_path,const char *name);
@@ -591,4 +593,7 @@ void elf_close(struct elf_fd *elf_fd);int elf_resolve_syms_offsets (const char *binary_path,int cnt,const char **syms,unsigned long **poffsets); +int elf_resolve_pattern_offsets (const char *binary_path,const char *pattern, +unsigned long **poffsets,size_t *pcnt); +#endif /* __LIBBPF_LIBBPF_INTERNAL_H */
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 13/28 libbpf: Add bpf_link_create support for multi uprobes
In reply to Jiri OlsaAdding new uprobe_multi struct to bpf_link_create_opts object to pass multiple uprobe data to link_create attr uapi.
--- tools/lib/bpf/bpf.c | 11 +++++++++++ tools/lib/bpf/bpf.h | 11 ++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-)
tools/lib/bpf/bpf.c +11 -0
@@ -767,6 +767,17 @@ int bpf_link_create(int prog_fd, int target_fd,if (!OPTS_ZEROED (opts, kprobe_multi))return libbpf_err (-EINVAL );break ; +case BPF_TRACE_UPROBE_MULTI : + attr.link_create.uprobe_multi.flags =OPTS_GET (opts, uprobe_multi.flags,0 ); + attr.link_create.uprobe_multi.cnt =OPTS_GET (opts, uprobe_multi.cnt,0 ); + attr.link_create.uprobe_multi.path =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.path,0 )); + attr.link_create.uprobe_multi.offsets =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.offsets,0 )); + attr.link_create.uprobe_multi.ref_ctr_offsets =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.ref_ctr_offsets,0 )); + attr.link_create.uprobe_multi.cookies =ptr_to_u64 (OPTS_GET (opts, uprobe_multi.cookies,0 )); + attr.link_create.uprobe_multi.pid =OPTS_GET (opts, uprobe_multi.pid,0 ); +if (!OPTS_ZEROED (opts, uprobe_multi)) +return libbpf_err (-EINVAL ); +break ;case BPF_TRACE_FENTRY :case BPF_TRACE_FEXIT :case BPF_MODIFY_RETURN :
tools/lib/bpf/bpf.h +10 -1
@@ -392,6 +392,15 @@ struct bpf_link_create_opts {const unsigned long *addrs;const __u64 *cookies; } kprobe_multi; +struct { +__u32 flags; +__u32 cnt; +const char *path; +const unsigned long *offsets; +const unsigned long *ref_ctr_offsets; +const __u64 *cookies; +__u32 pid; + } uprobe_multi;struct {__u64 cookie; } tracing;
@@ -409,7 +418,7 @@ struct bpf_link_create_opts { }; size_t :0; }; -#define bpf_link_create_opts__last_field kprobe_multi.cookies +#define bpf_link_create_opts__last_field uprobe_multi.pid LIBBPF_API int bpf_link_create(int prog_fd, int target_fd, enum bpf_attach_type attach_type,
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 14/28 libbpf: Add bpf_program__attach_uprobe_multi function
In reply to Jiri OlsaAdding bpf_program__attach_uprobe_multi function that allows to attach multiple uprobes with uprobe_multi link.
The user can specify uprobes with direct arguments:
binary_path/func_pattern/pid
or with struct bpf_uprobe_multi_opts opts argument fields:
const char **syms;
const unsigned long *offsets;
const unsigned long *ref_ctr_offsets;
const __u64 *cookies;
User can specify 2 mutually exclusive set of inputs:
1) use only path/func_pattern/pid arguments
2) use path/pid with allowed combinations of:
syms/offsets/ref_ctr_offsets/cookies/cnt
- syms and offsets are mutually exclusive
- ref_ctr_offsets and cookies are optional
Any other usage results in error.
--- tools/lib/bpf/libbpf.c | 114 +++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 51 ++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 166 insertions(+)
tools/lib/bpf/libbpf.c +114 -0
@@ -11128,6 +11128,120 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz)return -ENOENT ; } +struct bpf_link * +bpf_program__attach_uprobe_multi (const struct bpf_program *prog, +pid_t pid, +const char *path, +const char *func_pattern, +const struct bpf_uprobe_multi_opts *opts) +{ +const unsigned long *ref_ctr_offsets =NULL , *offsets =NULL ; +LIBBPF_OPTS (bpf_link_create_opts, lopts); +unsigned long *resolved_offsets =NULL ; +int err =0 , link_fd, prog_fd; +struct bpf_link *link =NULL ; +char errmsg[STRERR_BUFSIZE ]; +char full_path[PATH_MAX ]; +const __u64 *cookies; +const char **syms; +size_t cnt; + +if (!OPTS_VALID (opts, bpf_uprobe_multi_opts)) +return libbpf_err_ptr (-EINVAL ); + + syms =OPTS_GET (opts, syms,NULL ); + offsets =OPTS_GET (opts, offsets,NULL ); + ref_ctr_offsets =OPTS_GET (opts, ref_ctr_offsets,NULL ); + cookies =OPTS_GET (opts, cookies,NULL ); + cnt =OPTS_GET (opts, cnt,0 ); + +/* +* User can specify 2 mutually exclusive set of inputs: +* +* 1) use only path/func_pattern/pid arguments +* +* 2) use path/pid with allowed combinations of: +* syms/offsets/ref_ctr_offsets/cookies/cnt +* +* - syms and offsets are mutually exclusive +* - ref_ctr_offsets and cookies are optional +* +* Any other usage results in error. +*/ + +if (!path) +return libbpf_err_ptr (-EINVAL ); +if (!func_pattern && cnt ==0 ) +return libbpf_err_ptr (-EINVAL ); + +if (func_pattern) { +if (syms || offsets || ref_ctr_offsets || cookies || cnt) +return libbpf_err_ptr (-EINVAL ); + }else { +if (!!syms == !!offsets) +return libbpf_err_ptr (-EINVAL ); + } + +if (func_pattern) { +if (!strchr (path,'/' )) { + err =resolve_full_path (path, full_path,sizeof (full_path)); +if (err) { +pr_warn ("prog '%s': failed to resolve full path for '%s': %d\n" , + prog->name, path, err); +return libbpf_err_ptr (err); + } + path = full_path; + } + + err =elf_resolve_pattern_offsets (path, func_pattern, + &resolved_offsets, &cnt); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets; + }else if (syms) { + err =elf_resolve_syms_offsets (path, cnt, syms, &resolved_offsets); +if (err <0 ) +return libbpf_err_ptr (err); + offsets = resolved_offsets; + } + + lopts.uprobe_multi.path = path; + lopts.uprobe_multi.offsets = offsets; + lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets; + lopts.uprobe_multi.cookies = cookies; + lopts.uprobe_multi.cnt = cnt; + lopts.uprobe_multi.flags =OPTS_GET (opts, retprobe, false) ?BPF_F_UPROBE_MULTI_RETURN :0 ; + +if (pid ==0 ) + pid =getpid (); +if (pid >0 ) + lopts.uprobe_multi.pid = pid; + + link =calloc (1 ,sizeof (*link)); +if (!link) { + err = -ENOMEM ; + goto error; + } + link->detach = &bpf_link__detach_fd; + + prog_fd =bpf_program__fd (prog); + link_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &lopts); +if (link_fd <0 ) { + err = -errno; +pr_warn ("prog '%s': failed to attach multi-uprobe: %s\n" , + prog->name,libbpf_strerror_r (err, errmsg,sizeof (errmsg))); + goto error; + } + link->fd = link_fd; +free (resolved_offsets); +return link; + +error: +free (resolved_offsets); +free (link); +return libbpf_err_ptr (err); +} + LIBBPF_API struct bpf_link * bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid, const char *binary_path, size_t func_offset,
tools/lib/bpf/libbpf.h +51 -0
57 lines not on this page · raw message
tools/lib/bpf/libbpf.map +1 -0
@@ -398,4 +398,5 @@ LIBBPF_1.3.0 { bpf_prog_detach_opts; bpf_program__attach_netfilter; bpf_program__attach_tcx; + bpf_program__attach_uprobe_multi; } LIBBPF_1.2.0;
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 15/28 libbpf: Add support for u[ret]probe.multi[.s] program sections
In reply to Jiri OlsaAdding support for several uprobe_multi program sections to allow auto attach of multi_uprobe programs.
--- tools/lib/bpf/libbpf.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
tools/lib/bpf/libbpf.c +36 -0
@@ -8683,6 +8683,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_linstatic int attach_raw_tp (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_trace (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_kprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link); +static int attach_uprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_lsm (const struct bpf_program *prog,long cookie,struct bpf_link **link);static int attach_iter (const struct bpf_program *prog,long cookie,struct bpf_link **link);
@@ -8698,6 +8699,10 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("uretprobe.s+" ,KPROBE ,0 ,SEC_SLEEPABLE , attach_uprobe),SEC_DEF ("kprobe.multi+" ,KPROBE ,BPF_TRACE_KPROBE_MULTI ,SEC_NONE , attach_kprobe_multi),SEC_DEF ("kretprobe.multi+" ,KPROBE ,BPF_TRACE_KPROBE_MULTI ,SEC_NONE , attach_kprobe_multi), +SEC_DEF ("uprobe.multi+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_NONE , attach_uprobe_multi), +SEC_DEF ("uretprobe.multi+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_NONE , attach_uprobe_multi), +SEC_DEF ("uprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi), +SEC_DEF ("uretprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi),SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall), SEC_DEF("usdt+", KPROBE, 0, SEC_NONE, attach_usdt),
@@ -10904,6 +10909,37 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, strureturn libbpf_get_error (*link); } +static int attach_uprobe_multi (const struct bpf_program *prog,long cookie,struct bpf_link **link) +{ +char *probe_type =NULL , *binary_path =NULL , *func_name =NULL ; +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); +int n, ret = -EINVAL ; + + *link =NULL ; + + n =sscanf (prog->sec_name,"%m[^/]/%m[^:]:%ms" , + &probe_type, &binary_path, &func_name); +switch (n) { +case 1 : +/* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */ + ret =0 ; +break ; +case 3 : + opts.retprobe =strcmp (probe_type,"uretprobe.multi" ) ==0 ; + *link =bpf_program__attach_uprobe_multi (prog,-1 , binary_path, func_name, &opts); + ret =libbpf_get_error (*link); +break ; +default : +pr_warn ("prog '%s': invalid format of section definition '%s'\n" , prog->name, + prog->sec_name); +break ; + } +free (probe_type); +free (binary_path); +free (func_name); +return ret; +} +static void gen_uprobe_legacy_event_name (char *buf,size_t buf_sz,const char *binary_path,uint64_t offset) {
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 16/28 libbpf: Add uprobe multi link detection
In reply to Jiri OlsaAdding uprobe-multi link detection. It will be used later in bpf_program__attach_usdt function to check and use uprobe_multi link over standard uprobe links.
--- tools/lib/bpf/libbpf.c | 36 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 2 files changed, 38 insertions(+)
tools/lib/bpf/libbpf.c +36 -0
@@ -4819,6 +4819,39 @@ static int probe_perf_link(void)return link_fd <0 && err == -EBADF ; } +static int probe_uprobe_multi_link (void ) +{ +LIBBPF_OPTS (bpf_prog_load_opts, load_opts, + .expected_attach_type = BPF_TRACE_UPROBE_MULTI, + ); +LIBBPF_OPTS (bpf_link_create_opts, link_opts); +struct bpf_insn insns[] = { +BPF_MOV64_IMM (BPF_REG_0 ,0 ), +BPF_EXIT_INSN (), + }; +int prog_fd, link_fd, err; +unsigned long offset =0 ; + + prog_fd =bpf_prog_load (BPF_PROG_TYPE_KPROBE ,NULL ,"GPL" , + insns,ARRAY_SIZE (insns), &load_opts); +if (prog_fd <0 ) +return -errno; + +/* Creating uprobe in '/' binary should fail with -EBADF. */ + link_opts.uprobe_multi.path ="/" ; + link_opts.uprobe_multi.offsets = &offset; + link_opts.uprobe_multi.cnt =1 ; + + link_fd =bpf_link_create (prog_fd,-1 ,BPF_TRACE_UPROBE_MULTI , &link_opts); + err = -errno;/* close() can clobber errno */ + +if (link_fd >=0 ) +close (link_fd); +close (prog_fd); + +return link_fd <0 && err == -EBADF ; +} +static int probe_kern_bpf_cookie (void ) {struct bpf_insn insns[] = {
@@ -4915,6 +4948,9 @@ static struct kern_feature_desc { [FEAT_SYSCALL_WRAPPER ] = {"Kernel using syscall wrapper" , probe_kern_syscall_wrapper, }, + [FEAT_UPROBE_MULTI_LINK ] = { +"BPF multi-uprobe link support" , probe_uprobe_multi_link, + }, };bool kernel_supports (const struct bpf_object *obj,enum kern_feature_id feat_id)
tools/lib/bpf/libbpf_internal.h +2 -0
@@ -355,6 +355,8 @@ enum kern_feature_id {FEAT_BTF_ENUM64 ,/* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */ FEAT_SYSCALL_WRAPPER , +/* BPF multi-uprobe link support */ +FEAT_UPROBE_MULTI_LINK , __FEAT_CNT, };
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 17/28 libbpf: Add uprobe multi link support to bpf_program__attach_usdt
In reply to Jiri OlsaAdding support for usdt_manager_attach_usdt to use uprobe_multi link to attach to usdt probes.
The uprobe_multi support is detected before the usdt program is loaded and its expected_attach_type is set accordingly.
If uprobe_multi support is detected the usdt_manager_attach_usdt gathers uprobes info and calls bpf_program__attach_uprobe to create all needed uprobes.
If uprobe_multi support is not detected the old behaviour stays.
Also adding usdt.s program section for sleepable usdt probes.
--- tools/lib/bpf/libbpf.c | 13 ++++++- tools/lib/bpf/usdt.c | 86 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 82 insertions(+), 17 deletions(-)
tools/lib/bpf/libbpf.c +11 -2
@@ -367,6 +367,8 @@ enum sec_def_flags {SEC_SLEEPABLE =8 ,/* BPF program support non-linear XDP buffer */ SEC_XDP_FRAGS =16 , +/* Setup proper attach type for usdt probes. */ + SEC_USDT = 32, }; struct bpf_sec_def {
@@ -6818,6 +6820,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,if (prog->type ==BPF_PROG_TYPE_XDP && (def &SEC_XDP_FRAGS )) opts->prog_flags |=BPF_F_XDP_HAS_FRAGS ; +/* special check for usdt to use uprobe_multi link */ +if ((def &SEC_USDT ) &&kernel_supports (prog->obj,FEAT_UPROBE_MULTI_LINK )) + prog->expected_attach_type =BPF_TRACE_UPROBE_MULTI ; +if ((def &SEC_ATTACH_BTF ) && !prog->attach_btf_id) {int btf_obj_fd =0 , btf_type_id =0 , err;const char *attach_name;
@@ -6886,7 +6892,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *progif (!insns || !insns_cnt)return -EINVAL ; - load_attr.expected_attach_type = prog->expected_attach_type;if (kernel_supports (obj,FEAT_PROG_NAME )) prog_name = prog->name; load_attr.attach_prog_fd = prog->attach_prog_fd;
@@ -6922,6 +6927,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog insns_cnt = prog->insns_cnt; } +/* allow prog_prepare_load_fn to change expected_attach_type */ + load_attr.expected_attach_type = prog->expected_attach_type; +if (obj->gen_loader) { bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name, license, insns, insns_cnt, &load_attr,
@@ -8741,7 +8749,8 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("uretprobe.multi.s+" ,KPROBE ,BPF_TRACE_UPROBE_MULTI ,SEC_SLEEPABLE , attach_uprobe_multi),SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall), -SEC_DEF ("usdt+" ,KPROBE ,0 ,SEC_NONE , attach_usdt), +SEC_DEF ("usdt+" ,KPROBE ,0 ,SEC_USDT , attach_usdt), +SEC_DEF ("usdt.s+" ,KPROBE ,0 ,SEC_USDT |SEC_SLEEPABLE , attach_usdt),SEC_DEF ("tc/ingress" ,SCHED_CLS ,BPF_TCX_INGRESS ,SEC_NONE ),/* alias for tcx */ SEC_DEF ("tc/egress" ,SCHED_CLS ,BPF_TCX_EGRESS ,SEC_NONE ),/* alias for tcx */ SEC_DEF("tcx/ingress", SCHED_CLS, BPF_TCX_INGRESS, SEC_NONE),
tools/lib/bpf/usdt.c +71 -15
142 lines not on this page · raw message
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 18/28 selftests/bpf: Move get_time_ns to testing_helpers.h
In reply to Jiri OlsaWe'd like to have single copy of get_time_ns used b bench and test_progs, but we can't just include bench.h, because of conflicting 'struct env' objects.
Moving get_time_ns to testing_helpers.h which is being included by both bench and test_progs objects.
--- tools/testing/selftests/bpf/bench.h | 9 --------- .../selftests/bpf/prog_tests/kprobe_multi_test.c | 8 -------- tools/testing/selftests/bpf/testing_helpers.h | 10 ++++++++++ 3 files changed, 10 insertions(+), 17 deletions(-)
tools/testing/selftests/bpf/bench.h +0 -9
@@ -81,15 +81,6 @@ void grace_period_latency_basic_stats(struct bench_res res[], int res_cnt,void grace_period_ticks_basic_stats (struct bench_res res[],int res_cnt,struct basic_stats *gp_stat); -static inline __u64 get_time_ns (void ) -{ -struct timespec t; - -clock_gettime (CLOCK_MONOTONIC , &t); - -return (u64 )t.tv_sec *1000000000 + t.tv_nsec; -} -static inline void atomic_inc (long *value) { (void )__atomic_add_fetch (value,1 , __ATOMIC_RELAXED);
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +0 -8
@@ -304,14 +304,6 @@ static void test_attach_api_fails(void)kprobe_multi__destroy (skel); } -static inline __u64 get_time_ns (void ) -{ -struct timespec t; - -clock_gettime (CLOCK_MONOTONIC , &t); -return (__u64 ) t.tv_sec *1000000000 + t.tv_nsec; -} -static size_t symbol_hash (long key,void *ctx__maybe_unused ) {return str_hash ((const char *) key);
tools/testing/selftests/bpf/testing_helpers.h +10 -0
@@ -7,6 +7,7 @@#include <stdbool.h> #include <bpf/bpf.h> #include <bpf/libbpf.h> +#include <time.h> int parse_num_list (const char *s,bool **set,int *set_len);__u32 link_info_prog_id (const struct bpf_link *link,struct bpf_link_info *info);
@@ -33,4 +34,13 @@ int load_bpf_testmod(bool verbose);int unload_bpf_testmod (bool verbose);int kern_sync_rcu (void ); +static inline __u64 get_time_ns (void ) +{ +struct timespec t; + +clock_gettime (CLOCK_MONOTONIC , &t); + +return (u64 )t.tv_sec *1000000000 + t.tv_nsec; +} +#endif /* __TESTING_HELPERS_H */
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 19/28 selftests/bpf: Add uprobe_multi skel test
In reply to Jiri OlsaAdding uprobe_multi test for skeleton load/attach functions, to test skeleton auto attach for uprobe_multi link.
Test that bpf_get_func_ip works properly for uprobe_multi attachment.
--- .../bpf/prog_tests/uprobe_multi_test.c | 76 ++++++++++++++++ .../selftests/bpf/progs/uprobe_multi.c | 91 +++++++++++++++++++ 2 files changed, 167 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c new file +76 -0
@@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <unistd.h> +#include <test_progs.h> +#include "uprobe_multi.skel.h" + +static char test_data[] ="test_data" ; + +noinline void uprobe_multi_func_1(void) +{ + asmvolatile ("" ); +} + +noinline void uprobe_multi_func_2(void) +{ + asmvolatile ("" ); +} + +noinline void uprobe_multi_func_3(void) +{ + asmvolatile ("" ); +} + +static void uprobe_multi_test_run (struct uprobe_multi *skel) +{ + skel->bss->uprobe_multi_func_1_addr = (__u64 ) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (__u64 ) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (__u64 ) uprobe_multi_func_3; + + skel->bss->user_ptr = test_data; + skel->bss->pid =getpid (); + +/* trigger all probes */ +uprobe_multi_func_1 (); +uprobe_multi_func_2 (); +uprobe_multi_func_3 (); + +/* +* There are 2 entry and 2 exit probe called for each uprobe_multi_func_[123] +* function and each slepable probe (6) increments uprobe_multi_sleep_result. +*/ +ASSERT_EQ (skel->bss->uprobe_multi_func_1_result,2 ,"uprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_2_result,2 ,"uprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_3_result,2 ,"uprobe_multi_func_3_result" ); + +ASSERT_EQ (skel->bss->uretprobe_multi_func_1_result,2 ,"uretprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_2_result,2 ,"uretprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_3_result,2 ,"uretprobe_multi_func_3_result" ); + +ASSERT_EQ (skel->bss->uprobe_multi_sleep_result,6 ,"uprobe_multi_sleep_result" ); +} + +static void test_skel_api (void ) +{ +struct uprobe_multi *skel =NULL ; +int err; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + err =uprobe_multi__attach (skel); +if (!ASSERT_OK (err,"uprobe_multi__attach" )) + goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +uprobe_multi__destroy (skel); +} + +void test_uprobe_multi_test (void ) +{ +if (test__start_subtest ("skel_api" )) +test_skel_api (); +}
tools/testing/selftests/bpf/progs/uprobe_multi.c new file +91 -0
91 lines not on this page · raw message
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 20/28 selftests/bpf: Add uprobe_multi api test
In reply to Jiri OlsaAdding uprobe_multi test for bpf_program__attach_uprobe_multi attach function.
Testing attachment using glob patterns and via bpf_uprobe_multi_opts paths/syms fields.
--- .../bpf/prog_tests/uprobe_multi_test.c | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +65 -0
@@ -69,8 +69,73 @@ static void test_skel_api(void)uprobe_multi__destroy (skel); } +static void +test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +{ +struct uprobe_multi *skel =NULL ; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + opts->retprobe = false; + skel->links.uprobe =bpf_program__attach_uprobe_multi (skel->progs.uprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (skel->links.uprobe,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + + opts->retprobe = true; + skel->links.uretprobe =bpf_program__attach_uprobe_multi (skel->progs.uretprobe,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (skel->links.uretprobe,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + + opts->retprobe = false; + skel->links.uprobe_sleep =bpf_program__attach_uprobe_multi (skel->progs.uprobe_sleep,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (skel->links.uprobe_sleep,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + + opts->retprobe = true; + skel->links.uretprobe_sleep =bpf_program__attach_uprobe_multi (skel->progs.uretprobe_sleep, +-1 , binary, pattern, opts); +if (!ASSERT_OK_PTR (skel->links.uretprobe_sleep,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +uprobe_multi__destroy (skel); +} + +static void test_attach_api_pattern (void ) +{ +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); + +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_*" , &opts); +test_attach_api ("/proc/self/exe" ,"uprobe_multi_func_?" , &opts); +} + +static void test_attach_api_syms (void ) +{ +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); +const char *syms[3 ] = { +"uprobe_multi_func_1" , +"uprobe_multi_func_2" , +"uprobe_multi_func_3" , + }; + + opts.syms = syms; + opts.cnt =ARRAY_SIZE (syms); +test_attach_api ("/proc/self/exe" ,NULL , &opts); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))test_skel_api (); +if (test__start_subtest ("attach_api_pattern" )) +test_attach_api_pattern (); +if (test__start_subtest ("attach_api_syms" )) +test_attach_api_syms (); }
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 21/28 selftests/bpf: Add uprobe_multi link test
In reply to Jiri OlsaAdding uprobe_multi test for bpf_link_create attach function.
Testing attachment using the struct bpf_link_create_opts.
--- .../bpf/prog_tests/uprobe_multi_test.c | 69 +++++++++++++++++++ 1 file changed, 69 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +69 -0
@@ -3,6 +3,7 @@#include <unistd.h> #include <test_progs.h> #include "uprobe_multi.skel.h" +#include "bpf/libbpf_internal.h" static char test_data[] ="test_data" ;
@@ -130,6 +131,72 @@ static void test_attach_api_syms(void)test_attach_api ("/proc/self/exe" ,NULL , &opts); } +static void test_link_api (void ) +{ +int prog_fd, link1_fd =-1 , link2_fd =-1 , link3_fd =-1 , link4_fd =-1 ; +LIBBPF_OPTS (bpf_link_create_opts, opts); +const char *path ="/proc/self/exe" ; +struct uprobe_multi *skel =NULL ; +unsigned long *offsets =NULL ; +const char *syms[3 ] = { +"uprobe_multi_func_1" , +"uprobe_multi_func_2" , +"uprobe_multi_func_3" , + }; +int err; + + err =elf_resolve_syms_offsets (path,3 , syms, (unsigned long **) &offsets); +if (!ASSERT_OK (err,"elf_resolve_syms_offsets" )) +return ; + + opts.uprobe_multi.path = path; + opts.uprobe_multi.offsets = offsets; + opts.uprobe_multi.cnt =ARRAY_SIZE (syms); + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" )) + goto cleanup; + + opts.kprobe_multi.flags =0 ; + prog_fd =bpf_program__fd (skel->progs.uprobe); + link1_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link1_fd,0 ,"link1_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.uretprobe); + link2_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link2_fd,0 ,"link2_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =0 ; + prog_fd =bpf_program__fd (skel->progs.uprobe_sleep); + link3_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link3_fd,0 ,"link3_fd" )) + goto cleanup; + + opts.kprobe_multi.flags =BPF_F_UPROBE_MULTI_RETURN ; + prog_fd =bpf_program__fd (skel->progs.uretprobe_sleep); + link4_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link4_fd,0 ,"link4_fd" )) + goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +if (link1_fd >=0 ) +close (link1_fd); +if (link2_fd >=0 ) +close (link2_fd); +if (link3_fd >=0 ) +close (link3_fd); +if (link4_fd >=0 ) +close (link4_fd); + +uprobe_multi__destroy (skel); +free (offsets); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))
@@ -138,4 +205,6 @@ void test_uprobe_multi_test(void)test_attach_api_pattern ();if (test__start_subtest ("attach_api_syms" ))test_attach_api_syms (); +if (test__start_subtest ("link_api" )) +test_link_api (); }
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 22/28 selftests/bpf: Add uprobe_multi test program
In reply to Jiri OlsaAdding uprobe_multi test program that defines 50k uprobe_multi_func_* functions and will serve as attach point for uprobe_multi bench test in following patch.
--- tools/testing/selftests/bpf/Makefile | 5 ++ tools/testing/selftests/bpf/uprobe_multi.c | 67 ++++++++++++++++++++++ 2 files changed, 72 insertions(+) create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
tools/testing/selftests/bpf/Makefile +5 -0
@@ -585,6 +585,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/liburandom_read.so \ $(OUTPUT)/xdp_synproxy \ $(OUTPUT)/sign-file \ + $(OUTPUT)/uprobe_multi \ ima_setup.sh \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \
@@ -698,6 +699,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@ +$(OUTPUT)/uprobe_multi: uprobe_multi.c + $(call msg,BINARY,,$@) + $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@ + EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool \
tools/testing/selftests/bpf/uprobe_multi.c new file +67 -0
67 lines not on this page · raw message
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 23/28 selftests/bpf: Add uprobe_multi bench test
In reply to Jiri OlsaAdding test that attaches 50k uprobes in uprobe_multi binary.
After the attach is done we run the binary and make sure we get proper amount of hits.
The resulting attach/detach times on my setup:
test_bench_attach_uprobe:PASS:uprobe_multi__open 0 nsec test_bench_attach_uprobe:PASS:uprobe_multi__attach 0 nsec test_bench_attach_uprobe:PASS:uprobes_count 0 nsec
test_bench_attach_uprobe: attached in 0.346s
test_bench_attach_uprobe: detached in 0.419s
#262/5 uprobe_multi_test/bench_uprobe:OK
--- .../bpf/prog_tests/uprobe_multi_test.c | 40 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi_bench.c | 15 +++++++ 2 files changed, 55 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_bench.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +40 -0
@@ -3,7 +3,9 @@#include <unistd.h> #include <test_progs.h> #include "uprobe_multi.skel.h" +#include "uprobe_multi_bench.skel.h" #include "bpf/libbpf_internal.h" +#include "testing_helpers.h" static char test_data[] ="test_data" ;
@@ -197,6 +199,42 @@ static void test_link_api(void)free (offsets); } +static void test_bench_attach_uprobe (void ) +{ +long attach_start_ns =0 , attach_end_ns =0 ; +struct uprobe_multi_bench *skel =NULL ; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +int err; + + skel =uprobe_multi_bench__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi_bench__open_and_load" )) + goto cleanup; + + attach_start_ns =get_time_ns (); + + err =uprobe_multi_bench__attach (skel); +if (!ASSERT_OK (err,"uprobe_multi_bench__attach" )) + goto cleanup; + + attach_end_ns =get_time_ns (); + +system ("./uprobe_multi bench" ); + +ASSERT_EQ (skel->bss->count,50000 ,"uprobes_count" ); + +cleanup: + detach_start_ns =get_time_ns (); +uprobe_multi_bench__destroy (skel); + detach_end_ns =get_time_ns (); + + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +printf ("%s: attached in %7.3lfs\n" , __func__, attach_delta); +printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))
@@ -207,4 +245,6 @@ void test_uprobe_multi_test(void)test_attach_api_syms ();if (test__start_subtest ("link_api" ))test_link_api (); +if (test__start_subtest ("bench_uprobe" )) +test_bench_attach_uprobe (); }
tools/testing/selftests/bpf/progs/uprobe_multi_bench.c new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +char _license[] SEC("license") = "GPL"; + +int count; + +SEC("uprobe.multi/./uprobe_multi:uprobe_multi_func_*") +int uprobe_bench (struct pt_regs *ctx) +{ + count++; +return 0 ; +}
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 24/28 selftests/bpf: Add uprobe_multi usdt test code
In reply to Jiri OlsaAdding code in uprobe_multi test binary that defines 50k usdts and will serve as attach point for uprobe_multi usdt bench test in following patch.
--- tools/testing/selftests/bpf/uprobe_multi.c | 26 +++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/uprobe_multi.c +25 -1
@@ -2,6 +2,7 @@#include <stdio.h> #include <string.h> +#include <sdt.h> #define __PASTE (a, b) a##b#define PASTE (a, b) __PASTE(a, b)
@@ -53,6 +54,27 @@ static int bench(void)return 0 ; } +#define PROBE STAP_PROBE(test, usdt); + +#define PROBE10 PROBE PROBE PROBE PROBE PROBE \ + PROBE PROBE PROBE PROBE PROBE +#define PROBE100 PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \ + PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 +#define PROBE1000 PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \ + PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 +#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \ + PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 + +static int usdt (void ) +{ +PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 + PROBE10000 +return 0 ; +} +int main (int argc,char **argv) {if (argc !=2 )
@@ -60,8 +82,10 @@ int main(int argc, char **argv)if (!strcmp ("bench" , argv[1 ]))return bench (); +if (!strcmp ("usdt" , argv[1 ])) +return usdt (); error: -fprintf (stderr,"usage: %s <bench>\n" , argv[0 ]); +fprintf (stderr,"usage: %s <bench|usdt>\n" , argv[0 ]);return -1 ; }
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 25/28 selftests/bpf: Add uprobe_multi usdt bench test
In reply to Jiri OlsaAdding test that attaches 50k usdt probes in usdt_multi binary.
After the attach is done we run the binary and make sure we get proper amount of hits.
With current uprobes:
# perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
Performance counter stats for './test_progs -n 254/6':
1353.659680562 seconds time elapsed
With uprobe_multi link:
# perf stat --null ./test_progs -n 254/6 #254/6 uprobe_multi_test/bench_usdt:OK #254 uprobe_multi_test:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
Performance counter stats for './test_progs -n 254/6':
0.322046364 seconds time elapsed
--- .../bpf/prog_tests/uprobe_multi_test.c | 39 +++++++++++++++++++ .../selftests/bpf/progs/uprobe_multi_usdt.c | 16 ++++++++ 2 files changed, 55 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +39 -0
@@ -4,6 +4,7 @@#include <test_progs.h> #include "uprobe_multi.skel.h" #include "uprobe_multi_bench.skel.h" +#include "uprobe_multi_usdt.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h"
@@ -235,6 +236,42 @@ static void test_bench_attach_uprobe(void)printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); } +static void test_bench_attach_usdt (void ) +{ +long attach_start_ns =0 , attach_end_ns =0 ; +struct uprobe_multi_usdt *skel =NULL ; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; + + skel =uprobe_multi_usdt__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi__open" )) + goto cleanup; + + attach_start_ns =get_time_ns (); + + skel->links.usdt0 =bpf_program__attach_usdt (skel->progs.usdt0,-1 ,"./uprobe_multi" , +"test" ,"usdt" ,NULL ); +if (!ASSERT_OK_PTR (skel->links.usdt0,"bpf_program__attach_usdt" )) + goto cleanup; + + attach_end_ns =get_time_ns (); + +system ("./uprobe_multi usdt" ); + +ASSERT_EQ (skel->bss->count,50000 ,"usdt_count" ); + +cleanup: + detach_start_ns =get_time_ns (); +uprobe_multi_usdt__destroy (skel); + detach_end_ns =get_time_ns (); + + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +printf ("%s: attached in %7.3lfs\n" , __func__, attach_delta); +printf ("%s: detached in %7.3lfs\n" , __func__, detach_delta); +} +void test_uprobe_multi_test (void ) {if (test__start_subtest ("skel_api" ))
@@ -247,4 +284,6 @@ void test_uprobe_multi_test(void)test_link_api ();if (test__start_subtest ("bench_uprobe" ))test_bench_attach_uprobe (); +if (test__start_subtest ("bench_usdt" )) +test_bench_attach_usdt (); }
tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/usdt.bpf.h> + +char _license[] SEC("license") = "GPL"; + +int count; + +SEC("usdt") +int usdt0 (struct pt_regs *ctx) +{ + count++; +return 0 ; +}
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 26/28 selftests/bpf: Add uprobe_multi cookie test
In reply to Jiri OlsaAdding test for cookies setup/retrieval in uprobe_link uprobes and making sure bpf_get_attach_cookie works properly.
--- .../selftests/bpf/prog_tests/bpf_cookie.c | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+)
tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +78 -0
@@ -11,6 +11,7 @@#include <bpf/btf.h> #include "test_bpf_cookie.skel.h" #include "kprobe_multi.skel.h" +#include "uprobe_multi.skel.h" /* uprobe attach point */ static noinline void trigger_func(void)
@@ -239,6 +240,81 @@ static void kprobe_multi_attach_api_subtest(void)bpf_link__destroy (link1);kprobe_multi__destroy (skel); } + +/* defined in prog_tests/uprobe_multi_test.c */ +void uprobe_multi_func_1 (void ); +void uprobe_multi_func_2 (void ); +void uprobe_multi_func_3 (void ); + +static void uprobe_multi_test_run (struct uprobe_multi *skel) +{ + skel->bss->uprobe_multi_func_1_addr = (__u64 ) uprobe_multi_func_1; + skel->bss->uprobe_multi_func_2_addr = (__u64 ) uprobe_multi_func_2; + skel->bss->uprobe_multi_func_3_addr = (__u64 ) uprobe_multi_func_3; + + skel->bss->pid =getpid (); + skel->bss->test_cookie = true; + +uprobe_multi_func_1 (); +uprobe_multi_func_2 (); +uprobe_multi_func_3 (); + +ASSERT_EQ (skel->bss->uprobe_multi_func_1_result,1 ,"uprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_2_result,1 ,"uprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uprobe_multi_func_3_result,1 ,"uprobe_multi_func_3_result" ); + +ASSERT_EQ (skel->bss->uretprobe_multi_func_1_result,1 ,"uretprobe_multi_func_1_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_2_result,1 ,"uretprobe_multi_func_2_result" ); +ASSERT_EQ (skel->bss->uretprobe_multi_func_3_result,1 ,"uretprobe_multi_func_3_result" ); +} + +static void uprobe_multi_attach_api_subtest (void ) +{ +struct bpf_link *link1 =NULL , *link2 =NULL ; +struct uprobe_multi *skel =NULL ; +LIBBPF_OPTS (bpf_uprobe_multi_opts, opts); +const char *syms[3 ] = { +"uprobe_multi_func_1" , +"uprobe_multi_func_2" , +"uprobe_multi_func_3" , + }; +__u64 cookies[3 ]; + + cookies[0 ] =3 ;/* uprobe_multi_func_1 */ + cookies[1 ] =1 ;/* uprobe_multi_func_2 */ + cookies[2 ] =2 ;/* uprobe_multi_func_3 */ + + opts.syms = syms; + opts.cnt =ARRAY_SIZE (syms); + opts.cookies = &cookies[0 ]; + + skel =uprobe_multi__open_and_load (); +if (!ASSERT_OK_PTR (skel,"uprobe_multi" )) + goto cleanup; + + link1 =bpf_program__attach_uprobe_multi (skel->progs.uprobe,-1 , +"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link1,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + + cookies[0 ] =2 ;/* uprobe_multi_func_1 */ + cookies[1 ] =3 ;/* uprobe_multi_func_2 */ + cookies[2 ] =1 ;/* uprobe_multi_func_3 */ + + opts.retprobe = true; + link2 =bpf_program__attach_uprobe_multi (skel->progs.uretprobe,-1 , +"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link2,"bpf_program__attach_uprobe_multi_retprobe" )) + goto cleanup; + +uprobe_multi_test_run (skel); + +cleanup: +bpf_link__destroy (link2); +bpf_link__destroy (link1); +uprobe_multi__destroy (skel); +} +static void uprobe_subtest (struct test_bpf_cookie *skel) {DECLARE_LIBBPF_OPTS (bpf_uprobe_opts, opts);
@@ -515,6 +591,8 @@ void test_bpf_cookie(void)kprobe_multi_attach_api_subtest ();if (test__start_subtest ("uprobe" ))uprobe_subtest (skel); +if (test__start_subtest ("multi_uprobe_attach_api" )) +uprobe_multi_attach_api_subtest ();if (test__start_subtest ("tracepoint" ))tp_subtest (skel);if (test__start_subtest ("perf_event" ))
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 27/28 selftests/bpf: Add uprobe_multi pid filter tests
In reply to Jiri OlsaRunning api and link tests also with pid filter and checking the probe gets executed only for specific pid.
Spawning extra process to trigger attached uprobes and checking we get correct counts from executed programs.
--- .../bpf/prog_tests/uprobe_multi_test.c | 132 ++++++++++++++++-- .../selftests/bpf/progs/uprobe_multi.c | 6 +- 2 files changed, 126 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +121 -11
@@ -25,14 +25,87 @@ noinline void uprobe_multi_func_3(void) asmvolatile ("" ); } -static void uprobe_multi_test_run (struct uprobe_multi *skel) +struct child { +int go[2 ]; +int pid; +}; + +static void release_child (struct child *child) +{ +int child_status; + +if (!child) +return ; +close (child->go[1 ]); +close (child->go[0 ]); +if (child->pid >0 ) +waitpid (child->pid, &child_status,0 ); +} + +static void kick_child (struct child *child) +{ +char c =1 ; + +if (child) { +write (child->go[1 ], &c,1 ); +release_child (child); + } +fflush (NULL ); +} + +static struct child *spawn_child (void ) +{ +static struct child child; +int err; +int c; + +/* pipe to notify child to execute the trigger functions */ +if (pipe (child.go)) +return NULL ; + + child.pid =fork (); +if (child.pid <0 ) { +release_child (&child); + errno =EINVAL ; +return NULL ; + } + +/* child */ +if (child.pid ==0 ) { +close (child.go[1 ]); + +/* wait for parent's kick */ + err =read (child.go[0 ], &c,1 ); +if (err !=1 ) +exit (err); + +uprobe_multi_func_1 (); +uprobe_multi_func_2 (); +uprobe_multi_func_3 (); + +exit (errno); + } + +return &child; +} + +static void uprobe_multi_test_run (struct uprobe_multi *skel,struct child *child) { skel->bss->uprobe_multi_func_1_addr = (__u64 ) uprobe_multi_func_1; skel->bss->uprobe_multi_func_2_addr = (__u64 ) uprobe_multi_func_2; skel->bss->uprobe_multi_func_3_addr = (__u64 ) uprobe_multi_func_3; skel->bss->user_ptr = test_data; - skel->bss->pid =getpid (); + +/* +* Disable pid check in bpf program if we are pid filter test, +* because the probe should be executed only by child->pid +* passed at the probe attach. +*/ + skel->bss->pid = child ?0 :getpid (); + +if (child) +kick_child (child);/* trigger all probes */ uprobe_multi_func_1 ();
@@ -52,6 +125,9 @@ static void uprobe_multi_test_run(struct uprobe_multi *skel)ASSERT_EQ (skel->bss->uretprobe_multi_func_3_result,2 ,"uretprobe_multi_func_3_result" );ASSERT_EQ (skel->bss->uprobe_multi_sleep_result,6 ,"uprobe_multi_sleep_result" ); + +if (child) +ASSERT_EQ (skel->bss->child_pid, child->pid,"uprobe_multi_child_pid" ); }static void test_skel_api (void )
@@ -67,15 +143,17 @@ static void test_skel_api(void)if (!ASSERT_OK (err,"uprobe_multi__attach" )) goto cleanup; -uprobe_multi_test_run (skel); +uprobe_multi_test_run (skel,NULL ); cleanup:uprobe_multi__destroy (skel); }static void -test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +__test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts, +struct child *child) { +pid_t pid = child ? child->pid :-1 ;struct uprobe_multi *skel =NULL ; skel =uprobe_multi__open_and_load ();
@@ -83,35 +161,51 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi goto cleanup; opts->retprobe = false; - skel->links.uprobe =bpf_program__attach_uprobe_multi (skel->progs.uprobe,-1 , + skel->links.uprobe =bpf_program__attach_uprobe_multi (skel->progs.uprobe, pid, binary, pattern, opts);if (!ASSERT_OK_PTR (skel->links.uprobe,"bpf_program__attach_uprobe_multi" )) goto cleanup; opts->retprobe = true; - skel->links.uretprobe =bpf_program__attach_uprobe_multi (skel->progs.uretprobe,-1 , + skel->links.uretprobe =bpf_program__attach_uprobe_multi (skel->progs.uretprobe, pid, binary, pattern, opts);if (!ASSERT_OK_PTR (skel->links.uretprobe,"bpf_program__attach_uprobe_multi" )) goto cleanup; opts->retprobe = false; - skel->links.uprobe_sleep =bpf_program__attach_uprobe_multi (skel->progs.uprobe_sleep,-1 , + skel->links.uprobe_sleep =bpf_program__attach_uprobe_multi (skel->progs.uprobe_sleep, pid, binary, pattern, opts);if (!ASSERT_OK_PTR (skel->links.uprobe_sleep,"bpf_program__attach_uprobe_multi" )) goto cleanup; opts->retprobe = true; skel->links.uretprobe_sleep =bpf_program__attach_uprobe_multi (skel->progs.uretprobe_sleep, --1 , binary, pattern, opts); + pid, binary, pattern, opts);if (!ASSERT_OK_PTR (skel->links.uretprobe_sleep,"bpf_program__attach_uprobe_multi" )) goto cleanup; -uprobe_multi_test_run (skel); +uprobe_multi_test_run (skel, child); cleanup:uprobe_multi__destroy (skel); } +static void +test_attach_api (const char *binary,const char *pattern,struct bpf_uprobe_multi_opts *opts) +{ +struct child *child; + +/* no pid filter */ +__test_attach_api (binary, pattern, opts,NULL ); + +/* pid filter */ + child =spawn_child (); +if (!ASSERT_OK_PTR (child,"spawn_child" )) +return ; + +__test_attach_api (binary, pattern, opts, child); +} +static void test_attach_api_pattern (void ) {LIBBPF_OPTS (bpf_uprobe_multi_opts, opts);
@@ -134,7 +228,7 @@ static void test_attach_api_syms(void)test_attach_api ("/proc/self/exe" ,NULL , &opts); } -static void test_link_api (void ) +static void __test_link_api (struct child *child) {int prog_fd, link1_fd =-1 , link2_fd =-1 , link3_fd =-1 , link4_fd =-1 ;LIBBPF_OPTS (bpf_link_create_opts, opts);
@@ -155,6 +249,7 @@ static void test_link_api(void) opts.uprobe_multi.path = path; opts.uprobe_multi.offsets = offsets; opts.uprobe_multi.cnt =ARRAY_SIZE (syms); + opts.uprobe_multi.pid = child ? child->pid :0 ; skel =uprobe_multi__open_and_load ();if (!ASSERT_OK_PTR (skel,"uprobe_multi__open_and_load" ))
@@ -184,7 +279,7 @@ static void test_link_api(void)if (!ASSERT_GE (link4_fd,0 ,"link4_fd" )) goto cleanup; -uprobe_multi_test_run (skel); +uprobe_multi_test_run (skel, child); cleanup:if (link1_fd >=0 )
@@ -200,6 +295,21 @@ static void test_link_api(void)free (offsets); } +void test_link_api (void ) +{ +struct child *child; + +/* no pid filter */ +__test_link_api (NULL ); + +/* pid filter */ + child =spawn_child (); +if (!ASSERT_OK_PTR (child,"spawn_child" )) +return ; + +__test_link_api (child); +} +static void test_bench_attach_uprobe (void ) {long attach_start_ns =0 , attach_end_ns =0 ;
tools/testing/selftests/bpf/progs/uprobe_multi.c +5 -1
@@ -21,6 +21,8 @@ __u64 uretprobe_multi_func_3_result = 0;__u64 uprobe_multi_sleep_result =0 ;int pid =0 ; +int child_pid =0 ; +bool test_cookie = false;void *user_ptr =0 ;
@@ -34,7 +36,9 @@ static __always_inline bool verify_sleepable_user_copy(void)static void uprobe_multi_check (void *ctx,bool is_return,bool is_sleep) { -if (bpf_get_current_pid_tgid () >>32 != pid) + child_pid =bpf_get_current_pid_tgid () >>32 ; + +if (pid && child_pid != pid)return ;__u64 cookie = test_cookie ?bpf_get_attach_cookie (ctx) :0 ;
Signature · 1 line
2.41.0
Jiri Olsa
reviewerPATCH v7 28/28 selftests/bpf: Add extra link to uprobe_multi tests
In reply to Jiri OlsaAttaching extra program to same functions system wide for api and link tests.
This way we can test the pid filter works properly when there's extra system wide consumer on the same uprobe that will trigger the original uprobe handler.
We expect to have the same counts as before.
--- .../selftests/bpf/prog_tests/uprobe_multi_test.c | 16 ++++++++++++++++ tools/testing/selftests/bpf/progs/uprobe_multi.c | 6 ++++++ 2 files changed, 22 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +16 -0
@@ -184,6 +184,12 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mulif (!ASSERT_OK_PTR (skel->links.uretprobe_sleep,"bpf_program__attach_uprobe_multi" )) goto cleanup; + opts->retprobe = false; + skel->links.uprobe_extra =bpf_program__attach_uprobe_multi (skel->progs.uprobe_extra,-1 , + binary, pattern, opts); +if (!ASSERT_OK_PTR (skel->links.uprobe_extra,"bpf_program__attach_uprobe_multi" )) + goto cleanup; +uprobe_multi_test_run (skel, child); cleanup:
@@ -240,6 +246,7 @@ static void __test_link_api(struct child *child)"uprobe_multi_func_2" , "uprobe_multi_func_3", }; +int link_extra_fd =-1 ;int err; err =elf_resolve_syms_offsets (path,3 , syms, (unsigned long **) &offsets);
@@ -279,6 +286,13 @@ static void __test_link_api(struct child *child)if (!ASSERT_GE (link4_fd,0 ,"link4_fd" )) goto cleanup; + opts.kprobe_multi.flags =0 ; + opts.uprobe_multi.pid =0 ; + prog_fd =bpf_program__fd (skel->progs.uprobe_extra); + link_extra_fd =bpf_link_create (prog_fd,0 ,BPF_TRACE_UPROBE_MULTI , &opts); +if (!ASSERT_GE (link_extra_fd,0 ,"link_extra_fd" )) + goto cleanup; +uprobe_multi_test_run (skel, child); cleanup:
@@ -290,6 +304,8 @@ static void __test_link_api(struct child *child)close (link3_fd);if (link4_fd >=0 )close (link4_fd); +if (link_extra_fd >=0 ) +close (link_extra_fd);uprobe_multi__destroy (skel);free (offsets);
tools/testing/selftests/bpf/progs/uprobe_multi.c +6 -0
@@ -93,3 +93,9 @@ int uretprobe_sleep(struct pt_regs *ctx)uprobe_multi_check (ctx, true, true);return 0 ; } + +SEC("uprobe.multi//proc/self/exe:uprobe_multi_func_*") +int uprobe_extra (struct pt_regs *ctx) +{ +return 0 ; +}
Signature · 1 line
2.41.0
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Wed, 9 Aug 2023 10:34:12 +0200 you wrote:
hi,
this patchset is adding support to attach multiple uprobes and usdt probes through newuprobe_multilink.The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.
[...]
Here is the summary with links:
- [PATCHv7,bpf-next,01/28] bpf: Switch BPF_F_KPROBE_MULTI_RETURN macro to enum
https://git.kernel.org/bpf/bpf-next/c/c5487f8d9186 - [PATCHv7,bpf-next,02/28] bpf: Add attach_type checks under bpf_prog_attach_check_attach_type https://git.kernel.org/bpf/bpf-next/c/3505cb9fa26c - [PATCHv7,bpf-next,03/28] bpf: Add multi uprobe link https://git.kernel.org/bpf/bpf-next/c/89ae89f53d20 - [PATCHv7,bpf-next,04/28] bpf: Add cookies support for uprobe_multi link https://git.kernel.org/bpf/bpf-next/c/0b779b61f651 - [PATCHv7,bpf-next,05/28] bpf: Add pid filter support for uprobe_multi link https://git.kernel.org/bpf/bpf-next/c/b733eeade420 - [PATCHv7,bpf-next,06/28] bpf: Add bpf_get_func_ip helper support for uprobe link https://git.kernel.org/bpf/bpf-next/c/686328d80c43 - [PATCHv7,bpf-next,07/28] libbpf: Add uprobe_multi attach type and link names https://git.kernel.org/bpf/bpf-next/c/8097e460cabd - [PATCHv7,bpf-next,08/28] libbpf: Move elf_find_func_offset* functions to elf object https://git.kernel.org/bpf/bpf-next/c/5c742725045a - [PATCHv7,bpf-next,09/28] libbpf: Add elf_open/elf_close functions https://git.kernel.org/bpf/bpf-next/c/f90eb70d4489 - [PATCHv7,bpf-next,10/28] libbpf: Add elf symbol iterator https://git.kernel.org/bpf/bpf-next/c/3774705db171 - [PATCHv7,bpf-next,11/28] libbpf: Add elf_resolve_syms_offsets function https://git.kernel.org/bpf/bpf-next/c/7ace84c68929 - [PATCHv7,bpf-next,12/28] libbpf: Add elf_resolve_pattern_offsets function https://git.kernel.org/bpf/bpf-next/c/e613d1d0f7d4 - [PATCHv7,bpf-next,13/28] libbpf: Add bpf_link_create support for multi uprobes https://git.kernel.org/bpf/bpf-next/c/5054a303f896 - [PATCHv7,bpf-next,14/28] libbpf: Add bpf_program__attach_uprobe_multi function https://git.kernel.org/bpf/bpf-next/c/3140cf121c25 - [PATCHv7,bpf-next,15/28] libbpf: Add support for u[ret]probe.multi[.s] program sections https://git.kernel.org/bpf/bpf-next/c/5bfdd32dd575 - [PATCHv7,bpf-next,16/28] libbpf: Add uprobe multi link detection https://git.kernel.org/bpf/bpf-next/c/7e1b46812345 - [PATCHv7,bpf-next,17/28] libbpf: Add uprobe multi link support to bpf_program__attach_usdt https://git.kernel.org/bpf/bpf-next/c/5902da6d8a52 - [PATCHv7,bpf-next,18/28] selftests/bpf: Move get_time_ns to testing_helpers.h https://git.kernel.org/bpf/bpf-next/c/3830d04a7401 - [PATCHv7,bpf-next,19/28] selftests/bpf: Add uprobe_multi skel test https://git.kernel.org/bpf/bpf-next/c/75b3715720d7 - [PATCHv7,bpf-next,20/28] selftests/bpf: Add uprobe_multi api test https://git.kernel.org/bpf/bpf-next/c/ffc68903617a - [PATCHv7,bpf-next,21/28] selftests/bpf: Add uprobe_multi link test https://git.kernel.org/bpf/bpf-next/c/a93d22ea6092 - [PATCHv7,bpf-next,22/28] selftests/bpf: Add uprobe_multi test program https://git.kernel.org/bpf/bpf-next/c/519dfeaf5119 - [PATCHv7,bpf-next,23/28] selftests/bpf: Add uprobe_multi bench test https://git.kernel.org/bpf/bpf-next/c/3706919ee05f - [PATCHv7,bpf-next,24/28] selftests/bpf: Add uprobe_multi usdt test code https://git.kernel.org/bpf/bpf-next/c/4cde2d8aa7f7 - [PATCHv7,bpf-next,25/28] selftests/bpf: Add uprobe_multi usdt bench test https://git.kernel.org/bpf/bpf-next/c/85209e839fc2 - [PATCHv7,bpf-next,26/28] selftests/bpf: Add uprobe_multi cookie test https://git.kernel.org/bpf/bpf-next/c/e7cf9a48f8d6 - [PATCHv7,bpf-next,27/28] selftests/bpf: Add uprobe_multi pid filter tests https://git.kernel.org/bpf/bpf-next/c/d571efae0f1d - [PATCHv7,bpf-next,28/28] selftests/bpf: Add extra link to uprobe_multi tests https://git.kernel.org/bpf/bpf-next/c/8909a9392b41
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html