Skip to contentSkip to conversation

bpf: Add multi uprobe link

PATCH Superseded Jiri Olsa 5 participants 73 messages Jun 30, 2023 – Jul 11, 2023

Subsystems (8)
Kselftest BPF Tracing BPF selftests +4 more subsystems

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

v3 cover bpf: Add multi uprobe link

hi,
this patchset is adding support to attach multiple uprobes and usdt probes through new uprobe_multi link.

The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.

The main reason is that we need to install perf event for each probed function and profile shows perf event installation (perf_install_in_context) as culprit.

The new uprobe_multi link just creates raw uprobes and attaches the bpf program to them without perf event being involved.

In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to.

v3 changes:
- consolidate attach type checks in bpf_prog_attach_check_attach_type [Andrii]
- remove bpf_prog_active check [Alexei]
- change rcu locking [Andrii]
- allocate ref_ctr_offsets conditionally [Andrii]
- remove ctx check from bpf_uprobe_multi_cookie [Andrii]
- move some elf_* functions in elf.c object [Andrii]
- fix uprobe link detection code [Andrii]
- add usdt.s program section [Andrii]
- rename bpf_program__attach_uprobe_multi_opts to bpf_program__attach_uprobe_multi [Andrii]
- remove extra case from attach_uprobe_multi [Andrii]
- rework usdt_manager_attach_usdt [Andrii]
- rework/rename new elf_find_* functions [Andrii]
- elf iterator fixes [Andrii]
- renames
- elf_sym_iter_next loop restruct
- return directly GElf_Sym in elf_sym
- add elf_sym_offset helper
- add st_type arg elf_sym_iter_new
- get rid of '_' prefixed functions
- simplify offsets handling
- other smaller changes
- added acks
- todo:
- seems like more elf_* helpers could go in elf.c object,
I'll send that as follow up
- will send fill_info support when [3] is merged [Andrii]

There's support for bpftrace [2] and tetragon [1].

Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
uprobe_multi

thanks,
jirka

[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (26):
bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
bpf: Add multi uprobe link
bpf: Add cookies support for uprobe_multi link
bpf: Add pid filter support for uprobe_multi link
bpf: Add bpf_get_func_ip helper support for uprobe link
libbpf: Add uprobe_multi attach type and link names
libbpf: Move elf_find_func_offset* functions to elf object
libbpf: Add elf_open/elf_close functions
libbpf: Add elf symbol iterator
libbpf: Add elf_resolve_syms_offsets function
libbpf: Add elf_resolve_pattern_offsets function
libbpf: Add bpf_link_create support for multi uprobes
libbpf: Add bpf_program__attach_uprobe_multi function
libbpf: Add support for u[ret]probe.multi[.s] program sections
libbpf: Add uprobe multi link detection
libbpf: Add uprobe multi link support to bpf_program__attach_usdt selftests/bpf: Add uprobe_multi skel test
selftests/bpf: Add uprobe_multi api test
selftests/bpf: Add uprobe_multi link test
selftests/bpf: Add uprobe_multi test program
selftests/bpf: Add uprobe_multi bench test
selftests/bpf: Add usdt_multi test program
selftests/bpf: Add usdt_multi bench test
selftests/bpf: Add uprobe_multi cookie test
selftests/bpf: Add uprobe_multi pid filter tests
selftests/bpf: Add extra link to uprobe_multi tests

 include/linux/trace_events.h                               |   6 ++
 include/uapi/linux/bpf.h                                   |  16 ++++
 kernel/bpf/syscall.c                                       | 122 ++++++++++++-------------
 kernel/trace/bpf_trace.c                                   | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 tools/include/uapi/linux/bpf.h                             |  16 ++++
 tools/lib/bpf/Build                                        |   2 +-
 tools/lib/bpf/bpf.c                                        |  11 +++
 tools/lib/bpf/bpf.h                                        |  11 ++-
 tools/lib/bpf/elf.c                                        | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c                                     | 396 ++++++++++++++++++++++++++++++++++++++++++---------------------------------------
 tools/lib/bpf/libbpf.h                                     |  27 ++++++
 tools/lib/bpf/libbpf.map                                   |   1 +
 tools/lib/bpf/libbpf_elf.h                                 |  24 +++++
 tools/lib/bpf/libbpf_internal.h                            |   3 +
 tools/lib/bpf/usdt.c                                       | 109 +++++++++++++++--------
 tools/testing/selftests/bpf/Makefile                       |  10 +++
 tools/testing/selftests/bpf/prog_tests/bpf_cookie.c        |  78 ++++++++++++++++
 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi.c           | 110 +++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c      |  16 ++++
 tools/testing/selftests/bpf/uprobe_multi.c                 |  53 +++++++++++
 tools/testing/selftests/bpf/usdt_multi.c                   |  24 +++++
 22 files changed, 1969 insertions(+), 296 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
 create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
 create mode 100644 tools/testing/selftests/bpf/usdt_multi.c
author reviewer
#1 lore raw

PATCH v3 1/26 bpf: Add attach_type checks under bpf_prog_attach_check_attach_type

Add extra attach_type checks from link_create under bpf_prog_attach_check_attach_type.

---
 kernel/bpf/syscall.c | 108 +++++++++++++++++++------------------------
 1 file changed, 47 insertions(+), 61 deletions(-)
kernel/bpf/syscall.c +47 -61
index a2aef900519c..9046ad0f9b4e 100644
@@ -3502,34 +3502,6 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)
 	return fd;
 }
 
-static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
-					     enum bpf_attach_type attach_type)
-{
-	switch (prog->type) {
-	case BPF_PROG_TYPE_CGROUP_SOCK:
-	case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
-	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
-	case BPF_PROG_TYPE_SK_LOOKUP:
-		return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
-	case BPF_PROG_TYPE_CGROUP_SKB:
-		if (!capable(CAP_NET_ADMIN))
-			/* cg-skb progs can be loaded by unpriv user.
-			 * check permissions at attach time.
-			 */
-			return -EPERM;
-		return prog->enforce_expected_attach_type &&
-			prog->expected_attach_type != attach_type ?
-			-EINVAL : 0;
-	case BPF_PROG_TYPE_KPROBE:
-		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
-		    attach_type != BPF_TRACE_KPROBE_MULTI)
-			return -EINVAL;
-		return 0;
-	default:
-		return 0;
-	}
-}
-
 static enum bpf_prog_type
 attach_type_to_prog_type(enum bpf_attach_type attach_type)
 {
@@ -3593,6 +3565,53 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type)
 	}
 }
 
+static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
+					     enum bpf_attach_type attach_type)
+{
+	enum bpf_prog_type ptype;
+
+	switch (prog->type) {
+	case BPF_PROG_TYPE_CGROUP_SOCK:
+	case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
+	case BPF_PROG_TYPE_CGROUP_SOCKOPT:
+	case BPF_PROG_TYPE_SK_LOOKUP:
+		return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
+	case BPF_PROG_TYPE_CGROUP_SKB:
+		if (!capable(CAP_NET_ADMIN))
+			/* cg-skb progs can be loaded by unpriv user.
+			 * check permissions at attach time.
+			 */
+			return -EPERM;
+		return prog->enforce_expected_attach_type &&
+			prog->expected_attach_type != attach_type ?
+			-EINVAL : 0;
+	case BPF_PROG_TYPE_KPROBE:
+		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
+		    attach_type != BPF_TRACE_KPROBE_MULTI)
+			return -EINVAL;
+		if (attach_type != BPF_PERF_EVENT &&
+		    attach_type != BPF_TRACE_KPROBE_MULTI)
+			return -EINVAL;
+		return 0;
+	case BPF_PROG_TYPE_EXT:
+		return 0;
+	case BPF_PROG_TYPE_NETFILTER:
+		if (attach_type != BPF_NETFILTER)
+			return -EINVAL;
+		return 0;
+	case BPF_PROG_TYPE_PERF_EVENT:
+	case BPF_PROG_TYPE_TRACEPOINT:
+		if (attach_type != BPF_PERF_EVENT)
+			return -EINVAL;
+		return 0;
+	default:
+		ptype = attach_type_to_prog_type(attach_type);
+		if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type)
+			return -EINVAL;
+		return 0;
+	}
+}
+
 #define BPF_PROG_ATTACH_LAST_FIELD replace_bpf_fd
 
 #define BPF_F_ATTACH_MASK \
@@ -4658,7 +4677,6 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
 #define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
-	enum bpf_prog_type ptype;
 	struct bpf_prog *prog;
 	int ret;
 
@@ -4677,38 +4695,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 	if (ret)
 		goto out;
 
-	switch (prog->type) {
-	case BPF_PROG_TYPE_EXT:
-		break;
-	case BPF_PROG_TYPE_NETFILTER:
-		if (attr->link_create.attach_type != BPF_NETFILTER) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	case BPF_PROG_TYPE_PERF_EVENT:
-	case BPF_PROG_TYPE_TRACEPOINT:
-		if (attr->link_create.attach_type != BPF_PERF_EVENT) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	case BPF_PROG_TYPE_KPROBE:
-		if (attr->link_create.attach_type != BPF_PERF_EVENT &&
-		    attr->link_create.attach_type != BPF_TRACE_KPROBE_MULTI) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	default:
-		ptype = attach_type_to_prog_type(attr->link_create.attach_type);
-		if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type) {
-			ret = -EINVAL;
-			goto out;
-		}
-		break;
-	}
-
 	switch (prog->type) {
 	case BPF_PROG_TYPE_CGROUP_SKB:
 	case BPF_PROG_TYPE_CGROUP_SOCK:
Signature · 1 line
2.41.0
Acked-by on v3 1/26
#2 lore raw

Re: [PATCHv3 bpf-next 01/26] bpf: Add attach_type checks under bpf_prog_attach_check_attach_type

Quoted 64 lines · Jiri Olsa

Add extra attach_type checks from link_create under bpf_prog_attach_check_attach_type.

---
 kernel/bpf/syscall.c | 108 +++++++++++++++++++------------------------
 1 file changed, 47 insertions(+), 61 deletions(-)
kernel/bpf/syscall.c +11 -28
index a2aef900519c..9046ad0f9b4e 100644
@@ -3502,34 +3502,6 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)
        return fd;
 }
 
-static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
-                                            enum bpf_attach_type attach_type)
-{
-       switch (prog->type) {
-       case BPF_PROG_TYPE_CGROUP_SOCK:
-       case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
-       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
-       case BPF_PROG_TYPE_SK_LOOKUP:
-               return attach_type == prog->expected_attach_type ? 0 : -EINVAL;
-       case BPF_PROG_TYPE_CGROUP_SKB:
-               if (!capable(CAP_NET_ADMIN))
-                       /* cg-skb progs can be loaded by unpriv user.
-                        * check permissions at attach time.
-                        */
-                       return -EPERM;
-               return prog->enforce_expected_attach_type &&
-                       prog->expected_attach_type != attach_type ?
-                       -EINVAL : 0;
-       case BPF_PROG_TYPE_KPROBE:
-               if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
-                   attach_type != BPF_TRACE_KPROBE_MULTI)
-                       return -EINVAL;
-               return 0;
-       default:
-               return 0;
-       }
-}
-
 static enum bpf_prog_type
 attach_type_to_prog_type(enum bpf_attach_type attach_type)
 {
@@ -3593,6 +3565,53 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type)
        }
 }
 
+static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
+                                            enum bpf_attach_type attach_type)
+{
+       enum bpf_prog_type ptype;
+
+       switch (prog->type) {
+       case BPF_PROG_TYPE_CGROUP_SOCK:
+       case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
+       case BPF_PROG_TYPE_CGROUP_SOCKOPT:
+       case BPF_PROG_TYPE_SK_LOOKUP:
+               return attach_type == prog->expected_attach_type ? 0 : -EINVAL;

kernel/bpf/syscall.c attach_type_to_prog_type(enum bpf_attach_type attach_type) · line 3588

Quoted 10 lines
kernel/bpf/syscall.c +10 -0
+       case BPF_PROG_TYPE_CGROUP_SKB:
+               if (!capable(CAP_NET_ADMIN))
+                       /* cg-skb progs can be loaded by unpriv user.
+                        * check permissions at attach time.
+                        */
+                       return -EPERM;
+               return prog->enforce_expected_attach_type &&
+                       prog->expected_attach_type != attach_type ?
+                       -EINVAL : 0;
+       case BPF_PROG_TYPE_KPROBE:

nit, I'd keep KPROBE, TRACEPOINT and PERF_EVENT next to each other in this switch (that will just grow larger over time), as they are pretty closely related

otherwise lgtm:

Quoted 79 lines
kernel/bpf/syscall.c +26 -34
+               if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
+                   attach_type != BPF_TRACE_KPROBE_MULTI)
+                       return -EINVAL;
+               if (attach_type != BPF_PERF_EVENT &&
+                   attach_type != BPF_TRACE_KPROBE_MULTI)
+                       return -EINVAL;
+               return 0;
+       case BPF_PROG_TYPE_EXT:
+               return 0;
+       case BPF_PROG_TYPE_NETFILTER:
+               if (attach_type != BPF_NETFILTER)
+                       return -EINVAL;
+               return 0;
+       case BPF_PROG_TYPE_PERF_EVENT:
+       case BPF_PROG_TYPE_TRACEPOINT:
+               if (attach_type != BPF_PERF_EVENT)
+                       return -EINVAL;
+               return 0;
+       default:
+               ptype = attach_type_to_prog_type(attach_type);
+               if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type)
+                       return -EINVAL;
+               return 0;
+       }
+}
+
 #define BPF_PROG_ATTACH_LAST_FIELD replace_bpf_fd
 
 #define BPF_F_ATTACH_MASK \
@@ -4658,7 +4677,6 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
 #define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
-       enum bpf_prog_type ptype;
        struct bpf_prog *prog;
        int ret;
 
@@ -4677,38 +4695,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
        if (ret)
                goto out;
 
-       switch (prog->type) {
-       case BPF_PROG_TYPE_EXT:
-               break;
-       case BPF_PROG_TYPE_NETFILTER:
-               if (attr->link_create.attach_type != BPF_NETFILTER) {
-                       ret = -EINVAL;
-                       goto out;
-               }
-               break;
-       case BPF_PROG_TYPE_PERF_EVENT:
-       case BPF_PROG_TYPE_TRACEPOINT:
-               if (attr->link_create.attach_type != BPF_PERF_EVENT) {
-                       ret = -EINVAL;
-                       goto out;
-               }
-               break;
-       case BPF_PROG_TYPE_KPROBE:
-               if (attr->link_create.attach_type != BPF_PERF_EVENT &&
-                   attr->link_create.attach_type != BPF_TRACE_KPROBE_MULTI) {
-                       ret = -EINVAL;
-                       goto out;
-               }
-               break;
-       default:
-               ptype = attach_type_to_prog_type(attr->link_create.attach_type);
-               if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type) {
-                       ret = -EINVAL;
-                       goto out;
-               }
-               break;
-       }
-
        switch (prog->type) {
        case BPF_PROG_TYPE_CGROUP_SKB:
        case BPF_PROG_TYPE_CGROUP_SOCK:
--

2.41.0

author reviewer
#3 lore raw

PATCH v3 2/26 bpf: Add multi uprobe link

In reply to Jiri Olsa

Adding new multi uprobe link that allows to attach bpf program to multiple uprobes.

Uprobes to attach are specified via new link_create uprobe_multi union:

  struct {
          __u32           flags;
          __u32           cnt;
          __aligned_u64   path;
          __aligned_u64   offsets;
          __aligned_u64   ref_ctr_offsets;
  } uprobe_multi;

Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in ref_ctr_offsets array. All specified arrays have length of 'cnt'.

The 'flags' supports single bit for now that marks the uprobe as return probe.

---
 include/linux/trace_events.h   |   6 +
 include/uapi/linux/bpf.h       |  14 ++
 kernel/bpf/syscall.c           |  14 +-
 kernel/trace/bpf_trace.c       | 237 +++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  14 ++
 5 files changed, 282 insertions(+), 3 deletions(-)
include/linux/trace_events.h +6 -0
index 7c4a0b72334e..c71845e9d40a 100644
@@ -749,6 +749,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id,
 			    u32 *fd_type, const char **buf,
 			    u64 *probe_offset, u64 *probe_addr);
 int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
+int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
 #else
 static inline unsigned int trace_call_bpf(struct trace_event_call *call, void *ctx)
 {
@@ -795,6 +796,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
 {
 	return -EOPNOTSUPP;
 }
+static inline int
+bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 
 enum {
include/uapi/linux/bpf.h +14 -0
index 60a9d59beeab..a236139f08ce 100644
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {
 	BPF_LSM_CGROUP,
 	BPF_STRUCT_OPS,
 	BPF_NETFILTER,
+	BPF_TRACE_UPROBE_MULTI,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1053,6 +1054,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_KPROBE_MULTI = 8,
 	BPF_LINK_TYPE_STRUCT_OPS = 9,
 	BPF_LINK_TYPE_NETFILTER = 10,
+	BPF_LINK_TYPE_UPROBE_MULTI = 11,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1170,6 +1172,11 @@ enum bpf_link_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN	(1U << 0)
 
+/* link_create.uprobe_multi.flags used in LINK_CREATE command for
+ * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
+ */
+#define BPF_F_UPROBE_MULTI_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1579,6 +1586,13 @@ union bpf_attr {
 				__s32		priority;
 				__u32		flags;
 			} netfilter;
+			struct {
+				__u32		flags;
+				__u32		cnt;
+				__aligned_u64	path;
+				__aligned_u64	offsets;
+				__aligned_u64	ref_ctr_offsets;
+			} uprobe_multi;
 		};
 	} link_create;
 
kernel/bpf/syscall.c +11 -3
index 9046ad0f9b4e..3b0582a64ce4 100644
@@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)
 
 /* Clean up bpf_link and corresponding anon_inode file and FD. After
  * anon_inode is created, bpf_link can't be just kfree()'d due to deferred
- * anon_inode's release() call. This helper marksbpf_link as
+ * anon_inode's release() call. This helper marks bpf_link as
  * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt
  * is not decremented, it's the responsibility of a calling code that failed
  * to complete bpf_link initialization.
+ * This helper eventually calls link's dealloc callback, but does not call
+ * link's release callback.
  */
 void bpf_link_cleanup(struct bpf_link_primer *primer)
 {
@@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
 		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
 		    attach_type != BPF_TRACE_KPROBE_MULTI)
 			return -EINVAL;
+		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
+		    attach_type != BPF_TRACE_UPROBE_MULTI)
+			return -EINVAL;
 		if (attach_type != BPF_PERF_EVENT &&
-		    attach_type != BPF_TRACE_KPROBE_MULTI)
+		    attach_type != BPF_TRACE_KPROBE_MULTI &&
+		    attach_type != BPF_TRACE_UPROBE_MULTI)
 			return -EINVAL;
 		return 0;
 	case BPF_PROG_TYPE_EXT:
@@ -4748,8 +4754,10 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 	case BPF_PROG_TYPE_KPROBE:
 		if (attr->link_create.attach_type == BPF_PERF_EVENT)
 			ret = bpf_perf_link_attach(attr, prog);
-		else
+		else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI)
 			ret = bpf_kprobe_multi_link_attach(attr, prog);
+		else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI)
+			ret = bpf_uprobe_multi_link_attach(attr, prog);
 		break;
 	default:
 		ret = -EINVAL;
kernel/trace/bpf_trace.c +237 -0

246 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +14 -0
index 60a9d59beeab..a236139f08ce 100644
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {
 	BPF_LSM_CGROUP,
 	BPF_STRUCT_OPS,
 	BPF_NETFILTER,
+	BPF_TRACE_UPROBE_MULTI,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1053,6 +1054,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_KPROBE_MULTI = 8,
 	BPF_LINK_TYPE_STRUCT_OPS = 9,
 	BPF_LINK_TYPE_NETFILTER = 10,
+	BPF_LINK_TYPE_UPROBE_MULTI = 11,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1170,6 +1172,11 @@ enum bpf_link_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN	(1U << 0)
 
+/* link_create.uprobe_multi.flags used in LINK_CREATE command for
+ * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
+ */
+#define BPF_F_UPROBE_MULTI_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1579,6 +1586,13 @@ union bpf_attr {
 				__s32		priority;
 				__u32		flags;
 			} netfilter;
+			struct {
+				__u32		flags;
+				__u32		cnt;
+				__aligned_u64	path;
+				__aligned_u64	offsets;
+				__aligned_u64	ref_ctr_offsets;
+			} uprobe_multi;
 		};
 	} link_create;
 
Signature · 1 line
2.41.0
Acked-by on v3 2/26
#4 lore raw
Quoted 23 lines · Jiri Olsa

Adding new multi uprobe link that allows to attach bpf program to multiple uprobes.

Uprobes to attach are specified via new link_create uprobe_multi union:

  struct {
          __u32           flags;
          __u32           cnt;
          __aligned_u64   path;
          __aligned_u64   offsets;
          __aligned_u64   ref_ctr_offsets;
  } uprobe_multi;

Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in ref_ctr_offsets array. All specified arrays have length of 'cnt'.

The 'flags' supports single bit for now that marks the uprobe as return probe.

Quoted 9 lines
---
 include/linux/trace_events.h   |   6 +
 include/uapi/linux/bpf.h       |  14 ++
 kernel/bpf/syscall.c           |  14 +-
 kernel/trace/bpf_trace.c       | 237 +++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  14 ++
 5 files changed, 282 insertions(+), 3 deletions(-)

overall LGTM, but I think there is path leak, please fix that and add my ack

Quoted 45 lines
include/linux/trace_events.h +6 -0
index 7c4a0b72334e..c71845e9d40a 100644
@@ -749,6 +749,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id,
                            u32 *fd_type, const char **buf,
                            u64 *probe_offset, u64 *probe_addr);
 int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
+int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
 #else
 static inline unsigned int trace_call_bpf(struct trace_event_call *call, void *ctx)
 {
@@ -795,6 +796,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
 {
        return -EOPNOTSUPP;
 }
+static inline int
+bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
+{
+       return -EOPNOTSUPP;
+}
 #endif
 
 enum {
include/uapi/linux/bpf.h +2 -0
index 60a9d59beeab..a236139f08ce 100644
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {
        BPF_LSM_CGROUP,
        BPF_STRUCT_OPS,
        BPF_NETFILTER,
+       BPF_TRACE_UPROBE_MULTI,
        __MAX_BPF_ATTACH_TYPE
 };
 
@@ -1053,6 +1054,7 @@ enum bpf_link_type {
        BPF_LINK_TYPE_KPROBE_MULTI = 8,
        BPF_LINK_TYPE_STRUCT_OPS = 9,
        BPF_LINK_TYPE_NETFILTER = 10,
+       BPF_LINK_TYPE_UPROBE_MULTI = 11,
 
        MAX_BPF_LINK_TYPE,
 };

include/uapi/linux/bpf.h enum bpf_link_type { · line 1179

Quoted 8 lines
include/uapi/linux/bpf.h +5 -0
@@ -1170,6 +1172,11 @@ enum bpf_link_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN      (1U << 0)
 
+/* link_create.uprobe_multi.flags used in LINK_CREATE command for
+ * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
+ */
+#define BPF_F_UPROBE_MULTI_RETURN      (1U << 0)
+

any reason why we don't use anonymous ENUMs for all these UAPI constants? When we need to use these flags from BPF side (e.g., for BPF LSM), having them as #defines will be a PITA, as they won't be present in vmlinux.h

include/uapi/linux/bpf.h union bpf_attr { · line 1591

Quoted 10 lines
include/uapi/linux/bpf.h +3 -0
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1579,6 +1586,13 @@ union bpf_attr {
                                __s32           priority;
                                __u32           flags;
                        } netfilter;
+                       struct {
+                               __u32           flags;
+                               __u32           cnt;

total nit, but I'd move it after path/offsets/ref_ctr_offsets, and make the order cnt (as it applies to previous two offsets/ref_ctr_offsets) and then flags last. Seems like more logical order, but totally subjective

Quoted 12 lines
include/uapi/linux/bpf.h +4 -0
+                               __aligned_u64   path;
+                               __aligned_u64   offsets;
+                               __aligned_u64   ref_ctr_offsets;
+                       } uprobe_multi;
                };
        } link_create;
 
kernel/bpf/syscall.c +0 -0
index 9046ad0f9b4e..3b0582a64ce4 100644

kernel/bpf/syscall.c static void bpf_link_free_id(int id) · line 2821

Quoted 10 lines
kernel/bpf/syscall.c +3 -1
@@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)
 
 /* Clean up bpf_link and corresponding anon_inode file and FD. After
  * anon_inode is created, bpf_link can't be just kfree()'d due to deferred
- * anon_inode's release() call. This helper marksbpf_link as
+ * anon_inode's release() call. This helper marks bpf_link as
  * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt
  * is not decremented, it's the responsibility of a calling code that failed
  * to complete bpf_link initialization.
+ * This helper eventually calls link's dealloc callback, but does not call
+ * link's release callback.

Thanks for clarifying comments!

Quoted 4 lines
kernel/bpf/syscall.c +0 -0
  */
 void bpf_link_cleanup(struct bpf_link_primer *primer)
 {

kernel/bpf/syscall.c static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, · line 3599

Quoted 10 lines
kernel/bpf/syscall.c +5 -1
@@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
                if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
                    attach_type != BPF_TRACE_KPROBE_MULTI)
                        return -EINVAL;
+               if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
+                   attach_type != BPF_TRACE_UPROBE_MULTI)
+                       return -EINVAL;
                if (attach_type != BPF_PERF_EVENT &&
-                   attach_type != BPF_TRACE_KPROBE_MULTI)
+                   attach_type != BPF_TRACE_KPROBE_MULTI &&
+                   attach_type != BPF_TRACE_UPROBE_MULTI)

if this keeps growing, we should think about having a switch in a switch to not repeat BPF_TRACE_UPROBE_MULTI and BPF_TRACE_KPROBE_MULTI twice

Quoted 3 lines
kernel/bpf/syscall.c +0 -0
                        return -EINVAL;
                return 0;
        case BPF_PROG_TYPE_EXT:

[...]

Quoted 28 lines
kernel/bpf/syscall.c +28 -0
+       for (i = 0; i < cnt; i++) {
+               if (uref_ctr_offsets && __get_user(ref_ctr_offsets[i], uref_ctr_offsets + i)) {
+                       err = -EFAULT;
+                       goto error_free;
+               }
+               if (__get_user(uprobes[i].offset, uoffsets + i)) {
+                       err = -EFAULT;
+                       goto error_free;
+               }
+
+               uprobes[i].link = link;
+
+               if (flags & BPF_F_UPROBE_MULTI_RETURN)
+                       uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler;
+               else
+                       uprobes[i].consumer.handler = uprobe_multi_link_handler;
+       }
+
+       link->cnt = cnt;
+       link->uprobes = uprobes;
+       link->path = path;
+
+       bpf_link_init(&link->link, BPF_LINK_TYPE_UPROBE_MULTI,
+                     &bpf_uprobe_multi_link_lops, prog);
+
+       err = bpf_link_prime(&link->link, &link_primer);
+       if (err)
+               goto error_free;

kernel/bpf/syscall.c

Quoted 10 lines
kernel/bpf/syscall.c +10 -0
+
+       for (i = 0; i < cnt; i++) {
+               err = uprobe_register_refctr(d_real_inode(link->path.dentry),
+                                            uprobes[i].offset,
+                                            ref_ctr_offsets ? ref_ctr_offsets[i] : 0,
+                                            &uprobes[i].consumer);
+               if (err) {
+                       bpf_uprobe_unregister(&path, uprobes, i);
+                       bpf_link_cleanup(&link_primer);
+                       kvfree(ref_ctr_offsets);

are we missing path_put() in this error handling path? so maybe goto error_path_put here instead of return?

Quoted 15 lines
kernel/bpf/syscall.c +15 -0
+                       return err;
+               }
+       }
+
+       kvfree(ref_ctr_offsets);
+       return bpf_link_settle(&link_primer);
+
+error_free:
+       kvfree(ref_ctr_offsets);
+       kvfree(uprobes);
+       kfree(link);
+error_path_put:
+       path_put(&path);
+       return err;
+}

[...]

#5 lore raw

On Thu, Jul 06, 2023 at 03:34:10PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 24 lines
Quoted 24 lines
include/uapi/linux/bpf.h +2 -0
index 60a9d59beeab..a236139f08ce 100644
@@ -1036,6 +1036,7 @@ enum bpf_attach_type {
        BPF_LSM_CGROUP,
        BPF_STRUCT_OPS,
        BPF_NETFILTER,
+       BPF_TRACE_UPROBE_MULTI,
        __MAX_BPF_ATTACH_TYPE
 };
 
@@ -1053,6 +1054,7 @@ enum bpf_link_type {
        BPF_LINK_TYPE_KPROBE_MULTI = 8,
        BPF_LINK_TYPE_STRUCT_OPS = 9,
        BPF_LINK_TYPE_NETFILTER = 10,
+       BPF_LINK_TYPE_UPROBE_MULTI = 11,
 
        MAX_BPF_LINK_TYPE,
 };
@@ -1170,6 +1172,11 @@ enum bpf_link_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN      (1U << 0)
 
Quoted 10 lines

include/uapi/linux/bpf.h enum bpf_link_type { · line 1179

Quoted 5 lines
include/uapi/linux/bpf.h +5 -0
+/* link_create.uprobe_multi.flags used in LINK_CREATE command for
+ * BPF_TRACE_UPROBE_MULTI attach type to create return probe.
+ */
+#define BPF_F_UPROBE_MULTI_RETURN      (1U << 0)
+

any reason why we don't use anonymous ENUMs for all these UAPI constants? When we need to use these flags from BPF side (e.g., for BPF LSM), having them as #defines will be a PITA, as they won't be present in vmlinux.h

ugh right, we already did that before.. will change

Quoted 17 lines

include/uapi/linux/bpf.h union bpf_attr { · line 1591

Quoted 12 lines
include/uapi/linux/bpf.h +3 -0
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1579,6 +1586,13 @@ union bpf_attr {
                                __s32           priority;
                                __u32           flags;
                        } netfilter;
+                       struct {
+                               __u32           flags;
+                               __u32           cnt;

total nit, but I'd move it after path/offsets/ref_ctr_offsets, and make the order cnt (as it applies to previous two offsets/ref_ctr_offsets) and then flags last. Seems like more logical order, but totally subjective

ok

Quoted 36 lines
Quoted 13 lines
include/uapi/linux/bpf.h +4 -0
+                               __aligned_u64   path;
+                               __aligned_u64   offsets;
+                               __aligned_u64   ref_ctr_offsets;
+                       } uprobe_multi;
                };
        } link_create;
 
kernel/bpf/syscall.c +0 -0
index 9046ad0f9b4e..3b0582a64ce4 100644

kernel/bpf/syscall.c static void bpf_link_free_id(int id) · line 2821

Quoted 10 lines
kernel/bpf/syscall.c +3 -1
@@ -2813,10 +2813,12 @@ static void bpf_link_free_id(int id)
 
 /* Clean up bpf_link and corresponding anon_inode file and FD. After
  * anon_inode is created, bpf_link can't be just kfree()'d due to deferred
- * anon_inode's release() call. This helper marksbpf_link as
+ * anon_inode's release() call. This helper marks bpf_link as
  * defunct, releases anon_inode file and puts reserved FD. bpf_prog's refcnt
  * is not decremented, it's the responsibility of a calling code that failed
  * to complete bpf_link initialization.
+ * This helper eventually calls link's dealloc callback, but does not call
+ * link's release callback.

Thanks for clarifying comments!

Quoted 10 lines
kernel/bpf/syscall.c +3 -0
  */
 void bpf_link_cleanup(struct bpf_link_primer *primer)
 {
@@ -3589,8 +3591,12 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
                if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
                    attach_type != BPF_TRACE_KPROBE_MULTI)
                        return -EINVAL;
+               if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
+                   attach_type != BPF_TRACE_UPROBE_MULTI)
+                       return -EINVAL;
Quoted 8 lines

kernel/bpf/syscall.c static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog, · line 3599

Quoted 4 lines
kernel/bpf/syscall.c +2 -1
                if (attach_type != BPF_PERF_EVENT &&
-                   attach_type != BPF_TRACE_KPROBE_MULTI)
+                   attach_type != BPF_TRACE_KPROBE_MULTI &&
+                   attach_type != BPF_TRACE_UPROBE_MULTI)

if this keeps growing, we should think about having a switch in a switch to not repeat BPF_TRACE_UPROBE_MULTI and BPF_TRACE_KPROBE_MULTI twice

ok

SNIP

Quoted 10 lines

kernel/bpf/syscall.c static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,

Quoted 9 lines
kernel/bpf/syscall.c +9 -0
+       for (i = 0; i < cnt; i++) {
+               err = uprobe_register_refctr(d_real_inode(link->path.dentry),
+                                            uprobes[i].offset,
+                                            ref_ctr_offsets ? ref_ctr_offsets[i] : 0,
+                                            &uprobes[i].consumer);
+               if (err) {
+                       bpf_uprobe_unregister(&path, uprobes, i);
+                       bpf_link_cleanup(&link_primer);
+                       kvfree(ref_ctr_offsets);
Quoted 2 lines

are we missing path_put() in this error handling path? so maybe goto error_path_put here instead of return?

aaaah right path_put needs to go to dealloc callback :-\ will change, thanks

jirka

#6 lore raw
In reply to Jiri Olsa
Quoted 32 lines · Jiri Olsa

Adding new multi uprobe link that allows to attach bpf program to multiple uprobes.

Uprobes to attach are specified via new link_create uprobe_multi union:

  struct {
          __u32           flags;
          __u32           cnt;
          __aligned_u64   path;
          __aligned_u64   offsets;
          __aligned_u64   ref_ctr_offsets;
  } uprobe_multi;

Uprobes are defined for single binary specified in path and multiple calling sites specified in offsets array with optional reference counters specified in ref_ctr_offsets array. All specified arrays have length of 'cnt'.

The 'flags' supports single bit for now that marks the uprobe as return probe.

---
 include/linux/trace_events.h   |   6 +
 include/uapi/linux/bpf.h       |  14 ++
 kernel/bpf/syscall.c           |  14 +-
 kernel/trace/bpf_trace.c       | 237 +++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  14 ++
 5 files changed, 282 insertions(+), 3 deletions(-)

[...]

Quoted 4 lines
kernel/trace/bpf_trace.c +4 -0
+       flags = attr->link_create.uprobe_multi.flags;
+       if (flags & ~BPF_F_UPROBE_MULTI_RETURN)
+               return -EINVAL;
+

kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3069

Quoted 10 lines
kernel/trace/bpf_trace.c +10 -0
+       /*
+        * path, offsets and cnt are mandatory,
+        * ref_ctr_offsets is optional
+        */
+       upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
+       uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
+       cnt = attr->link_create.uprobe_multi.cnt;
+
+       if (!upath || !uoffsets || !cnt)
+               return -EINVAL;

see below for -EBADF, but we can also, additionally, return -EPROTO here, for example?

Quoted 6 lines
kernel/trace/bpf_trace.c +6 -0
+
+       uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
+
+       name = strndup_user(upath, PATH_MAX);
+       if (IS_ERR(name)) {
+               err = PTR_ERR(name);

kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3085

Quoted 10 lines
kernel/trace/bpf_trace.c +10 -0
+               return err;
+       }
+
+       err = kern_path(name, LOOKUP_FOLLOW, &path);
+       kfree(name);
+       if (err)
+               return err;
+
+       if (!d_is_reg(path.dentry)) {
+               err = -EINVAL;

as I mentioned in another patch, -EBADF here for feature detection (and it makes sense by itself, probably)

Quoted 17 lines
kernel/trace/bpf_trace.c +17 -0
+               goto error_path_put;
+       }
+
+       err = -ENOMEM;
+
+       link = kzalloc(sizeof(*link), GFP_KERNEL);
+       uprobes = kvcalloc(cnt, sizeof(*uprobes), GFP_KERNEL);
+
+       if (!uprobes || !link)
+               goto error_free;
+
+       if (uref_ctr_offsets) {
+               ref_ctr_offsets = kvcalloc(cnt, sizeof(*ref_ctr_offsets), GFP_KERNEL);
+               if (!ref_ctr_offsets)
+                       goto error_free;
+       }
+

[...]

#7 lore raw

On Thu, Jul 06, 2023 at 09:22:29PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 29 lines
Quoted 4 lines
kernel/trace/bpf_trace.c +4 -0
+       flags = attr->link_create.uprobe_multi.flags;
+       if (flags & ~BPF_F_UPROBE_MULTI_RETURN)
+               return -EINVAL;
+

kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3069

Quoted 10 lines
kernel/trace/bpf_trace.c +10 -0
+       /*
+        * path, offsets and cnt are mandatory,
+        * ref_ctr_offsets is optional
+        */
+       upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
+       uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
+       cnt = attr->link_create.uprobe_multi.cnt;
+
+       if (!upath || !uoffsets || !cnt)
+               return -EINVAL;

see below for -EBADF, but we can also, additionally, return -EPROTO here, for example?

Quoted 11 lines
kernel/trace/bpf_trace.c +11 -0
+
+       uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
+
+       name = strndup_user(upath, PATH_MAX);
+       if (IS_ERR(name)) {
+               err = PTR_ERR(name);
+               return err;
+       }
+
+       err = kern_path(name, LOOKUP_FOLLOW, &path);
+       kfree(name);
Quoted 8 lines

kernel/trace/bpf_trace.c static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) · line 3085

Quoted 5 lines
kernel/trace/bpf_trace.c +5 -0
+       if (err)
+               return err;
+
+       if (!d_is_reg(path.dentry)) {
+               err = -EINVAL;

as I mentioned in another patch, -EBADF here for feature detection (and it makes sense by itself, probably)

yes, I like this place better, also -EBADF error makes more sense in here

thanks,
jirka

Quoted 20 lines
Quoted 18 lines
kernel/trace/bpf_trace.c +17 -0
+               goto error_path_put;
+       }
+
+       err = -ENOMEM;
+
+       link = kzalloc(sizeof(*link), GFP_KERNEL);
+       uprobes = kvcalloc(cnt, sizeof(*uprobes), GFP_KERNEL);
+
+       if (!uprobes || !link)
+               goto error_free;
+
+       if (uref_ctr_offsets) {
+               ref_ctr_offsets = kvcalloc(cnt, sizeof(*ref_ctr_offsets), GFP_KERNEL);
+               if (!ref_ctr_offsets)
+                       goto error_free;
+       }
+

[...]

author reviewer
#8 lore raw

PATCH v3 3/26 bpf: Add cookies support for uprobe_multi link

In reply to Jiri Olsa

Adding support to specify cookies array for uprobe_multi link.

The cookies array share indexes and length with other uprobe_multi arrays (offsets/ref_ctr_offsets).

The cookies[i] value defines cookie for i-the uprobe and will be returned by bpf_get_attach_cookie helper when called from ebpf program hooked to that specific uprobe.

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  2 +-
 kernel/trace/bpf_trace.c       | 45 +++++++++++++++++++++++++++++++---
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 44 insertions(+), 5 deletions(-)
include/uapi/linux/bpf.h +1 -0
index a236139f08ce..4103f395593b 100644
@@ -1592,6 +1592,7 @@ union bpf_attr {
 				__aligned_u64	path;
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
+				__aligned_u64	cookies;
 			} uprobe_multi;
 		};
 	} link_create;
kernel/bpf/syscall.c +1 -1
index 3b0582a64ce4..affad356c603 100644
@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
 	return err;
 }
 
-#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
+#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
 	struct bpf_prog *prog;
kernel/trace/bpf_trace.c +41 -4

115 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +1 -0
index a236139f08ce..4103f395593b 100644
@@ -1592,6 +1592,7 @@ union bpf_attr {
 				__aligned_u64	path;
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
+				__aligned_u64	cookies;
 			} uprobe_multi;
 		};
 	} link_create;
Signature · 1 line
2.41.0
#9 lore raw

Re: [PATCHv3 bpf-next 03/26] bpf: Add cookies support for uprobe_multi link

Quoted 24 lines · Jiri Olsa

Adding support to specify cookies array for uprobe_multi link.

The cookies array share indexes and length with other uprobe_multi arrays (offsets/ref_ctr_offsets).

The cookies[i] value defines cookie for i-the uprobe and will be returned by bpf_get_attach_cookie helper when called from ebpf program hooked to that specific uprobe.

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  2 +-
 kernel/trace/bpf_trace.c       | 45 +++++++++++++++++++++++++++++++---
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 44 insertions(+), 5 deletions(-)
include/uapi/linux/bpf.h +0 -0
index a236139f08ce..4103f395593b 100644

include/uapi/linux/bpf.h union bpf_attr { · line 1595

Quoted 4 lines
include/uapi/linux/bpf.h +1 -0
@@ -1592,6 +1592,7 @@ union bpf_attr {
                                __aligned_u64   path;
                                __aligned_u64   offsets;
                                __aligned_u64   ref_ctr_offsets;
+                               __aligned_u64   cookies;

Could you pls. explain why the 'cookies' is defined as a pointer ? Are there real use cases to define different cookies for different probes in a single uprobe_multi ? Why can't we just use one cookie for all probes? Per my understanding, the cookie is used to identify the user or the application, rather than each probe.

We also defined it the same way in kprobe_multi...

Quoted 163 lines
include/uapi/linux/bpf.h +0 -0
                        } uprobe_multi;
                };
        } link_create;
kernel/bpf/syscall.c +1 -1
index 3b0582a64ce4..affad356c603 100644
@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
        return err;
 }
 
-#define BPF_LINK_CREATE_LAST_FIELD link_create.kprobe_multi.cookies
+#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
        struct bpf_prog *prog;
kernel/trace/bpf_trace.c +41 -4
index a0b9d034300f..4657df8f884e 100644
@@ -87,6 +87,8 @@ static int bpf_btf_printf_prepare(struct btf_ptr *ptr, u32 btf_ptr_size,
 static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 
+static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx);
+
 /**
  * trace_call_bpf - invoke BPF program
  * @call: tracepoint event
@@ -1099,6 +1101,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = {
        .arg1_type      = ARG_PTR_TO_CTX,
 };
 
+BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs)
+{
+       return bpf_uprobe_multi_cookie(current->bpf_ctx);
+}
+
+static const struct bpf_func_proto bpf_get_attach_cookie_proto_umulti = {
+       .func           = bpf_get_attach_cookie_uprobe_multi,
+       .gpl_only       = false,
+       .ret_type       = RET_INTEGER,
+       .arg1_type      = ARG_PTR_TO_CTX,
+};
+
 BPF_CALL_1(bpf_get_attach_cookie_trace, void *, ctx)
 {
        struct bpf_trace_run_ctx *run_ctx;
@@ -1545,9 +1559,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
                        &bpf_get_func_ip_proto_kprobe_multi :
                        &bpf_get_func_ip_proto_kprobe;
        case BPF_FUNC_get_attach_cookie:
-               return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ?
-                       &bpf_get_attach_cookie_proto_kmulti :
-                       &bpf_get_attach_cookie_proto_trace;
+               if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
+                       return &bpf_get_attach_cookie_proto_kmulti;
+               if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
+                       return &bpf_get_attach_cookie_proto_umulti;
+               return &bpf_get_attach_cookie_proto_trace;
        default:
                return bpf_tracing_func_proto(func_id, prog);
        }
@@ -2930,6 +2946,7 @@ struct bpf_uprobe_multi_link;
 struct bpf_uprobe {
        struct bpf_uprobe_multi_link *link;
        loff_t offset;
+       u64 cookie;
        struct uprobe_consumer consumer;
 };
 
@@ -2943,6 +2960,7 @@ struct bpf_uprobe_multi_link {
 struct bpf_uprobe_multi_run_ctx {
        struct bpf_run_ctx run_ctx;
        unsigned long entry_ip;
+       struct bpf_uprobe *uprobe;
 };
 
 static void bpf_uprobe_unregister(struct path *path, struct bpf_uprobe *uprobes,
@@ -2986,6 +3004,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
        struct bpf_uprobe_multi_link *link = uprobe->link;
        struct bpf_uprobe_multi_run_ctx run_ctx = {
                .entry_ip = entry_ip,
+               .uprobe = uprobe,
        };
        struct bpf_prog *prog = link->link.prog;
        bool sleepable = prog->aux->sleepable;
@@ -3032,6 +3051,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s
        return uprobe_prog_run(uprobe, func, regs);
 }
 
+static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
+{
+       struct bpf_uprobe_multi_run_ctx *run_ctx;
+
+       run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx);
+       return run_ctx->uprobe->cookie;
+}
+
 int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
 {
        struct bpf_uprobe_multi_link *link = NULL;
@@ -3040,6 +3067,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
        struct bpf_link_primer link_primer;
        struct bpf_uprobe *uprobes = NULL;
        unsigned long __user *uoffsets;
+       u64 __user *ucookies;
        void __user *upath;
        u32 flags, cnt, i;
        struct path path;
@@ -3059,7 +3087,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 
        /*
         * path, offsets and cnt are mandatory,
-        * ref_ctr_offsets is optional
+        * ref_ctr_offsets and cookies are optional
         */
        upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path);
        uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets);
@@ -3069,6 +3097,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
                return -EINVAL;
 
        uref_ctr_offsets = u64_to_user_ptr(attr->link_create.uprobe_multi.ref_ctr_offsets);
+       ucookies = u64_to_user_ptr(attr->link_create.uprobe_multi.cookies);
 
        name = strndup_user(upath, PATH_MAX);
        if (IS_ERR(name)) {
@@ -3101,6 +3130,10 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
        }
 
        for (i = 0; i < cnt; i++) {
+               if (ucookies && __get_user(uprobes[i].cookie, ucookies + i)) {
+                       err = -EFAULT;
+                       goto error_free;
+               }
                if (uref_ctr_offsets && __get_user(ref_ctr_offsets[i], uref_ctr_offsets + i)) {
                        err = -EFAULT;
                        goto error_free;
@@ -3158,4 +3191,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 {
        return -EOPNOTSUPP;
 }
+static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
+{
+       return 0;
+}
 #endif /* CONFIG_UPROBES */
tools/include/uapi/linux/bpf.h +1 -1
index a236139f08ce..4103f395593b 100644
@@ -1592,6 +1592,7 @@ union bpf_attr {
                                __aligned_u64   path;
                                __aligned_u64   offsets;
                                __aligned_u64   ref_ctr_offsets;
+                               __aligned_u64   cookies;
                        } uprobe_multi;
                };
        } link_create;
--

2.41.0

Signature · 2 lines
Regards
Yafang
#10 lore raw

Re: [PATCHv3 bpf-next 03/26] bpf: Add cookies support for uprobe_multi link

Quoted 36 lines · Yafang Shao
Quoted 24 lines · Jiri Olsa

Adding support to specify cookies array for uprobe_multi link.

The cookies array share indexes and length with other uprobe_multi arrays (offsets/ref_ctr_offsets).

The cookies[i] value defines cookie for i-the uprobe and will be returned by bpf_get_attach_cookie helper when called from ebpf program hooked to that specific uprobe.

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  2 +-
 kernel/trace/bpf_trace.c       | 45 +++++++++++++++++++++++++++++++---
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 44 insertions(+), 5 deletions(-)
include/uapi/linux/bpf.h +0 -0
index a236139f08ce..4103f395593b 100644

include/uapi/linux/bpf.h union bpf_attr { · line 1595

Quoted 4 lines
include/uapi/linux/bpf.h +1 -0
@@ -1592,6 +1592,7 @@ union bpf_attr {
                                __aligned_u64   path;
                                __aligned_u64   offsets;
                                __aligned_u64   ref_ctr_offsets;
+                               __aligned_u64   cookies;

Could you pls. explain why the 'cookies' is defined as a pointer ? Are there real use cases to define different cookies for different probes in a single uprobe_multi ? Why can't we just use one cookie for all probes? Per my understanding, the cookie is used to identify the user or the application, rather than each probe.

Quoted 1 line

We also defined it the same way in kprobe_multi...

yes, each probed function (or symbol/offset for uprobe) can get its own cookie and we use that to get probe-specific config in generic bpf program with the bpf_get_attach_cookie helper

jirka

author reviewer
#11 lore raw

PATCH v3 4/26 bpf: Add pid filter support for uprobe_multi link

In reply to Jiri Olsa

Adding support to specify pid for uprobe_multi link and the uprobes are created only for task with given pid value.

Using the consumer.filter filter callback for that, so the task gets filtered during the uprobe installation.

We still need to check the task during runtime in the uprobe handler, because the handler could get executed if there's another system wide consumer on the same uprobe (thanks Oleg for the insight).

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  2 +-
 kernel/trace/bpf_trace.c       | 33 +++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 36 insertions(+), 1 deletion(-)
include/uapi/linux/bpf.h +1 -0
index 4103f395593b..347c5ed6286b 100644
@@ -1593,6 +1593,7 @@ union bpf_attr {
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
 				__aligned_u64	cookies;
+				__u32		pid;
 			} uprobe_multi;
 		};
 	} link_create;
kernel/bpf/syscall.c +1 -1
index affad356c603..c16b9f80099d 100644
@@ -4680,7 +4680,7 @@ static int bpf_map_do_batch(const union bpf_attr *attr,
 	return err;
 }
 
-#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.cookies
+#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.pid
 static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 {
 	struct bpf_prog *prog;
kernel/trace/bpf_trace.c +33 -0
index 4657df8f884e..4ef51fd0497f 100644
@@ -2955,6 +2955,7 @@ struct bpf_uprobe_multi_link {
 	struct bpf_link link;
 	u32 cnt;
 	struct bpf_uprobe *uprobes;
+	struct task_struct *task;
 };
 
 struct bpf_uprobe_multi_run_ctx {
@@ -2981,6 +2982,8 @@ static void bpf_uprobe_multi_link_release(struct bpf_link *link)
 	umulti_link = container_of(link, struct bpf_uprobe_multi_link, link);
 	bpf_uprobe_unregister(&umulti_link->path, umulti_link->uprobes, umulti_link->cnt);
 	path_put(&umulti_link->path);
+	if (umulti_link->task)
+		put_task_struct(umulti_link->task);
 }
 
 static void bpf_uprobe_multi_link_dealloc(struct bpf_link *link)
@@ -3011,6 +3014,9 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
 	struct bpf_run_ctx *old_run_ctx;
 	int err = 0;
 
+	if (link->task && current != link->task)
+		return 0;
+
 	might_fault();
 
 	migrate_disable();
@@ -3033,6 +3039,16 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
 	return err;
 }
 
+static bool
+uprobe_multi_link_filter(struct uprobe_consumer *con, enum uprobe_filter_ctx ctx,
+			 struct mm_struct *mm)
+{
+	struct bpf_uprobe *uprobe;
+
+	uprobe = container_of(con, struct bpf_uprobe, consumer);
+	return uprobe->link->task->mm == mm;
+}
+
 static int
 uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs)
 {
@@ -3066,12 +3082,14 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	unsigned long *ref_ctr_offsets = NULL;
 	struct bpf_link_primer link_primer;
 	struct bpf_uprobe *uprobes = NULL;
+	struct task_struct *task = NULL;
 	unsigned long __user *uoffsets;
 	u64 __user *ucookies;
 	void __user *upath;
 	u32 flags, cnt, i;
 	struct path path;
 	char *name;
+	pid_t pid;
 	int err;
 
 	/* no support for 32bit archs yet */
@@ -3115,6 +3133,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 		goto error_path_put;
 	}
 
+	pid = attr->link_create.uprobe_multi.pid;
+	if (pid) {
+		rcu_read_lock();
+		task = get_pid_task(find_vpid(pid), PIDTYPE_PID);
+		rcu_read_unlock();
+		if (!task)
+			goto error_path_put;
+	}
+
 	err = -ENOMEM;
 
 	link = kzalloc(sizeof(*link), GFP_KERNEL);
@@ -3149,11 +3176,15 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 			uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler;
 		else
 			uprobes[i].consumer.handler = uprobe_multi_link_handler;
+
+		if (pid)
+			uprobes[i].consumer.filter = uprobe_multi_link_filter;
 	}
 
 	link->cnt = cnt;
 	link->uprobes = uprobes;
 	link->path = path;
+	link->task = task;
 
 	bpf_link_init(&link->link, BPF_LINK_TYPE_UPROBE_MULTI,
 		      &bpf_uprobe_multi_link_lops, prog);
@@ -3182,6 +3213,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	kvfree(ref_ctr_offsets);
 	kvfree(uprobes);
 	kfree(link);
+	if (task)
+		put_task_struct(task);
 error_path_put:
 	path_put(&path);
 	return err;
tools/include/uapi/linux/bpf.h +1 -0
index 4103f395593b..347c5ed6286b 100644
@@ -1593,6 +1593,7 @@ union bpf_attr {
 				__aligned_u64	offsets;
 				__aligned_u64	ref_ctr_offsets;
 				__aligned_u64	cookies;
+				__u32		pid;
 			} uprobe_multi;
 		};
 	} link_create;
Signature · 1 line
2.41.0
author reviewer
#12 lore raw

PATCH v3 5/26 bpf: Add bpf_get_func_ip helper support for uprobe link

In reply to Jiri Olsa

Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.

It returns the ip of the uprobe.

---
 kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)
kernel/trace/bpf_trace.c +30 -3
index 4ef51fd0497f..f5a41c1604b8 100644
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 
 static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx);
+static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 
 /**
  * trace_call_bpf - invoke BPF program
@@ -1101,6 +1102,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto_kmulti = {
 	.arg1_type	= ARG_PTR_TO_CTX,
 };
 
+BPF_CALL_1(bpf_get_func_ip_uprobe_multi, struct pt_regs *, regs)
+{
+	return bpf_uprobe_multi_entry_ip(current->bpf_ctx);
+}
+
+static const struct bpf_func_proto bpf_get_func_ip_proto_uprobe_multi = {
+	.func		= bpf_get_func_ip_uprobe_multi,
+	.gpl_only	= false,
+	.ret_type	= RET_INTEGER,
+	.arg1_type	= ARG_PTR_TO_CTX,
+};
+
 BPF_CALL_1(bpf_get_attach_cookie_uprobe_multi, struct pt_regs *, regs)
 {
 	return bpf_uprobe_multi_cookie(current->bpf_ctx);
@@ -1555,9 +1568,11 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_override_return_proto;
 #endif
 	case BPF_FUNC_get_func_ip:
-		return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ?
-			&bpf_get_func_ip_proto_kprobe_multi :
-			&bpf_get_func_ip_proto_kprobe;
+		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
+			return &bpf_get_func_ip_proto_kprobe_multi;
+		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
+			return &bpf_get_func_ip_proto_uprobe_multi;
+		return &bpf_get_func_ip_proto_kprobe;
 	case BPF_FUNC_get_attach_cookie:
 		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
 			return &bpf_get_attach_cookie_proto_kmulti;
@@ -3067,6 +3082,14 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s
 	return uprobe_prog_run(uprobe, func, regs);
 }
 
+static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
+{
+	struct bpf_uprobe_multi_run_ctx *run_ctx;
+
+	run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx);
+	return run_ctx->entry_ip;
+}
+
 static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
 {
 	struct bpf_uprobe_multi_run_ctx *run_ctx;
@@ -3228,4 +3251,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
 {
 	return 0;
 }
+static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
+{
+	return 0;
+}
 #endif /* CONFIG_UPROBES */
Signature · 1 line
2.41.0
#13 lore raw

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

Quoted 4 lines · Jiri Olsa

Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.

Quoted 8 lines

It returns the ip of the uprobe.

---
 kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

A slight aside related to bpf_get_func_ip() support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and using bpf_get_func_ip() there with uretprobes, and it seems like it doesn't work.

Is that intentional or we just missed that bpf_get_func_ip() doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them for bpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).

Thanks!

Quoted 7 lines
kernel/trace/bpf_trace.c +0 -0
index 4ef51fd0497f..f5a41c1604b8 100644
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 

[...]

#14 lore raw

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa

Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.

Quoted 8 lines

It returns the ip of the uprobe.

---
 kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

A slight aside related to bpf_get_func_ip() support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and using bpf_get_func_ip() there with uretprobes, and it seems like it doesn't work.

Quoted 5 lines

Is that intentional or we just missed that bpf_get_func_ip() doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them for bpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).

I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol

we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's get_func_ip helper version:

	BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) 
	{
		struct kprobe *kp = kprobe_running();

		if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
			return 0;

		return get_entry_ip((uintptr_t)kp->addr);
	}

I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel

I guess the app needs to store regs->ip and match it against symbol addresses in user space

jirka

Quoted 12 lines

Thanks!

Quoted 7 lines
kernel/trace/bpf_trace.c +0 -0
index 4ef51fd0497f..f5a41c1604b8 100644
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 

[...]

#15 lore raw

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

Quoted 44 lines · Jiri Olsa
Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa

Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.

Quoted 8 lines

It returns the ip of the uprobe.

---
 kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

A slight aside related to bpf_get_func_ip() support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and using bpf_get_func_ip() there with uretprobes, and it seems like it doesn't work.

Quoted 5 lines

Is that intentional or we just missed that bpf_get_func_ip() doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them for bpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).

I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol

we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's get_func_ip helper version:

        BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
        {
                struct kprobe *kp = kprobe_running();

                if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
                        return 0;

                return get_entry_ip((uintptr_t)kp->addr);
        }
Quoted 2 lines

I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel

what if we just return whatever was the IP where uprobe/uretprobe interrupt instruction was installed at? I haven't tried what does regs->ip contain for u*ret*probe, though, if it already has the IP where we installed uprobe itself, it might be ok as is. But still, feels like it would be nice to have bpf_get_func_ip() working for uprobe/uretprobe (even if semantics might differ for uprobes not at the entry of the function).

Quoted 19 lines

I guess the app needs to store regs->ip and match it against symbol addresses in user space

jirka

Quoted 12 lines

Thanks!

Quoted 7 lines
kernel/trace/bpf_trace.c +0 -0
index 4ef51fd0497f..f5a41c1604b8 100644
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 

[...]

#16 lore raw

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

Quoted 45 lines · Andrii Nakryiko
Quoted 44 lines · Jiri Olsa
Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa

Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.

Quoted 8 lines

It returns the ip of the uprobe.

---
 kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

A slight aside related to bpf_get_func_ip() support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and using bpf_get_func_ip() there with uretprobes, and it seems like it doesn't work.

Quoted 5 lines

Is that intentional or we just missed that bpf_get_func_ip() doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them for bpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).

I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol

we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's get_func_ip helper version:

        BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
        {
                struct kprobe *kp = kprobe_running();

                if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
                        return 0;

                return get_entry_ip((uintptr_t)kp->addr);
        }
Quoted 7 lines
Quoted 2 lines

I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel

what if we just return whatever was the IP where uprobe/uretprobe interrupt instruction was installed at? I haven't tried what does regs->ip contain for u*ret*probe, though, if it already has the IP where we installed uprobe itself, it might be ok as is. But still,

the entry uprobe gets the IP of the probed instruction, but the uretprobe gets IP of the next instruction after probed func retun - like 4c57f0 in the example below when uprobe is placed on trigger_func_get_func_ip

	  4c57eb:       e8 4b fe ff ff          call   4c563b <trigger_func_get_func_ip>
	  4c57f0:       eb 01                   jmp    4c57f3 <test_uprobe+0x1b1>
Quoted 3 lines

feels like it would be nice to have bpf_get_func_ip() working for uprobe/uretprobe (even if semantics might differ for uprobes not at the entry of the function).

but it looks like we actually have the original uprobe address stored in current->utask->vaddr before bpf program is executed (in uretprobe_dispatcher) so we should be able to get that address from there in uretprobe's get_func_ip helper function, I'll check

I don't mind the semantic change for uprobe's get_func_ip, because I think it's unlikely we will be able to change it in future to return the real function entry

jirka

Quoted 20 lines
Quoted 20 lines

I guess the app needs to store regs->ip and match it against symbol addresses in user space

jirka

Quoted 12 lines

Thanks!

Quoted 7 lines
kernel/trace/bpf_trace.c +0 -0
index 4ef51fd0497f..f5a41c1604b8 100644
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 

[...]

#17 lore raw

Re: [PATCHv3 bpf-next 05/26] bpf: Add bpf_get_func_ip helper support for uprobe link

Quoted 62 lines · Jiri Olsa
Quoted 45 lines · Andrii Nakryiko
Quoted 44 lines · Jiri Olsa
Quoted 19 lines · Andrii Nakryiko
Quoted 4 lines · Jiri Olsa

Adding support for bpf_get_func_ip helper being called from ebpf program attached by uprobe_multi link.

Quoted 8 lines

It returns the ip of the uprobe.

---
 kernel/trace/bpf_trace.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

A slight aside related to bpf_get_func_ip() support in uprobe/uretprobe. We just had a conversation with Alastair and Jordan (cc'ed) about bpftrace and using bpf_get_func_ip() there with uretprobes, and it seems like it doesn't work.

Quoted 5 lines

Is that intentional or we just missed that bpf_get_func_ip() doesn't work with uprobes/uretprobes? Do you think it would be hard to add support for them for bpf_get_func_ip()? It's a very useful helper, would be nice to have it working in all cases where it has meaningful behavior (and I think it does for uprobe and uretprobe).

I'm not sure we discussed at the time we added this helper, but I see same problem as we had with kprobes where we need to know if the probe is on the start of the symbol

we use KPROBE_FLAG_ON_FUNC_ENTRY flag for that in kprobe's get_func_ip helper version:

        BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
        {
                struct kprobe *kp = kprobe_running();

                if (!kp || !(kp->flags & KPROBE_FLAG_ON_FUNC_ENTRY))
                        return 0;

                return get_entry_ip((uintptr_t)kp->addr);
        }
Quoted 7 lines
Quoted 2 lines

I don't think we can have same flag for uprobe, we don't have the binary symbol data as we have for kernel

what if we just return whatever was the IP where uprobe/uretprobe interrupt instruction was installed at? I haven't tried what does regs->ip contain for u*ret*probe, though, if it already has the IP where we installed uprobe itself, it might be ok as is. But still,

the entry uprobe gets the IP of the probed instruction, but the uretprobe gets IP of the next instruction after probed func retun - like 4c57f0 in the example below when uprobe is placed on trigger_func_get_func_ip

          4c57eb:       e8 4b fe ff ff          call   4c563b <trigger_func_get_func_ip>
          4c57f0:       eb 01                   jmp    4c57f3 <test_uprobe+0x1b1>
Quoted 8 lines
Quoted 3 lines

feels like it would be nice to have bpf_get_func_ip() working for uprobe/uretprobe (even if semantics might differ for uprobes not at the entry of the function).

but it looks like we actually have the original uprobe address stored in current->utask->vaddr before bpf program is executed (in uretprobe_dispatcher) so we should be able to get that address from there in uretprobe's get_func_ip helper function, I'll check

ok, sounds great, seems like we have a pretty logical behavior for both uprobe and uretprobe.

Quoted 5 lines

I don't mind the semantic change for uprobe's get_func_ip, because I think it's unlikely we will be able to change it in future to return the real function entry

Yep, I don't think kernel can easily know where the function even begins. But that's ok. We can clarify helper's behavior in UAPI
comments: uprobe/uretprobe always work, but return not function IP, but the traced location. Thanks!

Quoted 22 lines

jirka

Quoted 20 lines
Quoted 20 lines

I guess the app needs to store regs->ip and match it against symbol addresses in user space

jirka

Quoted 12 lines

Thanks!

Quoted 7 lines
kernel/trace/bpf_trace.c +0 -0
index 4ef51fd0497f..f5a41c1604b8 100644
@@ -88,6 +88,7 @@ static u64 bpf_kprobe_multi_cookie(struct bpf_run_ctx *ctx);
 static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx);
 

[...]

author reviewer
#18 lore raw

PATCH v3 6/26 libbpf: Add uprobe_multi attach type and link names

In reply to Jiri Olsa

Adding new uprobe_multi attach type and link names, so the functions can resolve the new values.

---
 tools/lib/bpf/libbpf.c | 2 ++
 1 file changed, 2 insertions(+)
tools/lib/bpf/libbpf.c +2 -0
index 214f828ece6b..53dcf25c4878 100644
@@ -118,6 +118,7 @@ static const char * const attach_type_name[] = {
 	[BPF_TRACE_KPROBE_MULTI]	= "trace_kprobe_multi",
 	[BPF_STRUCT_OPS]		= "struct_ops",
 	[BPF_NETFILTER]			= "netfilter",
+	[BPF_TRACE_UPROBE_MULTI]	= "trace_uprobe_multi",
 };
 
 static const char * const link_type_name[] = {
@@ -132,6 +133,7 @@ static const char * const link_type_name[] = {
 	[BPF_LINK_TYPE_KPROBE_MULTI]		= "kprobe_multi",
 	[BPF_LINK_TYPE_STRUCT_OPS]		= "struct_ops",
 	[BPF_LINK_TYPE_NETFILTER]		= "netfilter",
+	[BPF_LINK_TYPE_UPROBE_MULTI]		= "uprobe_multi",
 };
 
 static const char * const map_type_name[] = {
Signature · 1 line
2.41.0
author reviewer
#19 lore raw

PATCH v3 7/26 libbpf: Move elf_find_func_offset* functions to elf object

In reply to Jiri Olsa

Adding new elf object that will contain elf related functions. There's no functional change.

---
 tools/lib/bpf/Build        |   2 +-
 tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c     | 186 +---------------------------------
 tools/lib/bpf/libbpf_elf.h |  11 +++
 4 files changed, 211 insertions(+), 186 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h
tools/lib/bpf/Build +1 -1
index b8b0a6369363..2d0c282c8588 100644
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
 	    netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
 	    btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-	    usdt.o zip.o
+	    usdt.o zip.o elf.o
tools/lib/bpf/elf.c new file +198 -0

198 lines not on this page · raw message

tools/lib/bpf/libbpf.c +1 -185

198 lines not on this page · raw message

tools/lib/bpf/libbpf_elf.h new file +11 -0
new file mode 100644
index 000000000000..1b652220fabf
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+
+#ifndef __LIBBPF_LIBBPF_ELF_H
+#define __LIBBPF_LIBBPF_ELF_H
+
+#include <libelf.h>
+
+long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
+long elf_find_func_offset_from_file(const char *binary_path, const char *name);
+
+#endif /* *__LIBBPF_LIBBPF_ELF_H */
Signature · 1 line
2.41.0
#20 lore raw

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

Quoted 15 lines · Jiri Olsa

Adding new elf object that will contain elf related functions. There's no functional change.

---
 tools/lib/bpf/Build        |   2 +-
 tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c     | 186 +---------------------------------
 tools/lib/bpf/libbpf_elf.h |  11 +++
 4 files changed, 211 insertions(+), 186 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h

[...]

Quoted 7 lines
tools/lib/bpf/libbpf_elf.h new file +1 -0
new file mode 100644
index 000000000000..1b652220fabf
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */

tools/lib/bpf/libbpf_elf.h line 11

Quoted 10 lines
tools/lib/bpf/libbpf_elf.h new file +10 -0
+
+#ifndef __LIBBPF_LIBBPF_ELF_H
+#define __LIBBPF_LIBBPF_ELF_H
+
+#include <libelf.h>
+
+long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
+long elf_find_func_offset_from_file(const char *binary_path, const char *name);
+
+#endif /* *__LIBBPF_LIBBPF_ELF_H */

we have libbpf_internal.h, let's put all this there for now, it's already all the internal stuff together, I don't know if separate header with few functions gives us much

Quoted 3 lines

--
2.41.0

#21 lore raw

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

Quoted 32 lines · Andrii Nakryiko
Quoted 15 lines · Jiri Olsa

Adding new elf object that will contain elf related functions. There's no functional change.

---
 tools/lib/bpf/Build        |   2 +-
 tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c     | 186 +---------------------------------
 tools/lib/bpf/libbpf_elf.h |  11 +++
 4 files changed, 211 insertions(+), 186 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h

[...]

Quoted 13 lines
tools/lib/bpf/libbpf_elf.h new file +7 -0
new file mode 100644
index 000000000000..1b652220fabf
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+
+#ifndef __LIBBPF_LIBBPF_ELF_H
+#define __LIBBPF_LIBBPF_ELF_H
+
+#include <libelf.h>
+
Quoted 8 lines

tools/lib/bpf/libbpf_elf.h line 11

Quoted 4 lines
tools/lib/bpf/libbpf_elf.h new file +4 -0
+long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
+long elf_find_func_offset_from_file(const char *binary_path, const char *name);
+
+#endif /* *__LIBBPF_LIBBPF_ELF_H */

we have libbpf_internal.h, let's put all this there for now, it's already all the internal stuff together, I don't know if separate header with few functions gives us much

there's more functions coming later in the patchset

	struct elf_fd {
		Elf *elf;
		int fd;
	};

	int elf_open(const char *binary_path, struct elf_fd *elf_fd);
	void elf_close(struct elf_fd *elf_fd);

	int elf_resolve_syms_offsets(const char *binary_path, int cnt,
				     const char **syms, unsigned long **poffsets);

	int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
					 unsigned long **poffsets, size_t *pcnt);

and there's probably more elf helpers to eventually move in:

	libbpf.c:static const char *elf_sym_str(const struct bpf_object *obj, size_t off);
	libbpf.c:static const char *elf_sec_str(const struct bpf_object *obj, size_t off);
	libbpf.c:static Elf_Scn *elf_sec_by_idx(const struct bpf_object *obj, size_t idx);
	libbpf.c:static Elf_Scn *elf_sec_by_name(const struct bpf_object *obj, const char *name);
	libbpf.c:static Elf64_Shdr *elf_sec_hdr(const struct bpf_object *obj, Elf_Scn *scn);
	libbpf.c:static const char *elf_sec_name(const struct bpf_object *obj, Elf_Scn *scn);
	libbpf.c:static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn);
	libbpf.c:static Elf64_Sym *elf_sym_by_idx(const struct bpf_object *obj, size_t idx);
	libbpf.c:static Elf64_Rel *elf_rel_by_idx(Elf_Data *data, size_t idx);

	usdt.c:static int find_elf_sec_by_name(Elf *elf, const char *sec_name, GElf_Shdr *shdr, Elf_Scn **scn)

	'struct elf_seg' stuff

	usdt.c:static int cmp_elf_segs(const void *_a, const void *_b)
	usdt.c:static int parse_elf_segs(Elf *elf, const char *path, struct elf_seg **segs, size_t *seg_cnt)
	usdt.c:static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs, size_t *seg_cnt)
	usdt.c:static struct elf_seg *find_elf_seg(struct elf_seg *segs, size_t seg_cnt, long virtaddr)
	usdt.c:static struct elf_seg *find_vma_seg(struct elf_seg *segs, size_t seg_cnt, long offset)

but I can add the new header file later in follow up changes when we have more elf functions in

jirka

#22 lore raw

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

Quoted 62 lines · Jiri Olsa
Quoted 32 lines · Andrii Nakryiko
Quoted 15 lines · Jiri Olsa

Adding new elf object that will contain elf related functions. There's no functional change.

---
 tools/lib/bpf/Build        |   2 +-
 tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c     | 186 +---------------------------------
 tools/lib/bpf/libbpf_elf.h |  11 +++
 4 files changed, 211 insertions(+), 186 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h

[...]

Quoted 13 lines
tools/lib/bpf/libbpf_elf.h new file +7 -0
new file mode 100644
index 000000000000..1b652220fabf
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+
+#ifndef __LIBBPF_LIBBPF_ELF_H
+#define __LIBBPF_LIBBPF_ELF_H
+
+#include <libelf.h>
+
Quoted 8 lines

tools/lib/bpf/libbpf_elf.h line 11

Quoted 4 lines
tools/lib/bpf/libbpf_elf.h new file +4 -0
+long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
+long elf_find_func_offset_from_file(const char *binary_path, const char *name);
+
+#endif /* *__LIBBPF_LIBBPF_ELF_H */

we have libbpf_internal.h, let's put all this there for now, it's already all the internal stuff together, I don't know if separate header with few functions gives us much

there's more functions coming later in the patchset

        struct elf_fd {
                Elf *elf;
                int fd;
        };

        int elf_open(const char *binary_path, struct elf_fd *elf_fd);
        void elf_close(struct elf_fd *elf_fd);

        int elf_resolve_syms_offsets(const char *binary_path, int cnt,
                                     const char **syms, unsigned long **poffsets);

        int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
                                         unsigned long **poffsets, size_t *pcnt);

and there's probably more elf helpers to eventually move in:

Quoted 10 lines
        libbpf.c:static const char *elf_sym_str(const struct bpf_object *obj, size_t off);
        libbpf.c:static const char *elf_sec_str(const struct bpf_object *obj, size_t off);
        libbpf.c:static Elf_Scn *elf_sec_by_idx(const struct bpf_object *obj, size_t idx);
        libbpf.c:static Elf_Scn *elf_sec_by_name(const struct bpf_object *obj, const char *name);
        libbpf.c:static Elf64_Shdr *elf_sec_hdr(const struct bpf_object *obj, Elf_Scn *scn);
        libbpf.c:static const char *elf_sec_name(const struct bpf_object *obj, Elf_Scn *scn);
        libbpf.c:static Elf_Data *elf_sec_data(const struct bpf_object *obj, Elf_Scn *scn);
        libbpf.c:static Elf64_Sym *elf_sym_by_idx(const struct bpf_object *obj, size_t idx);
        libbpf.c:static Elf64_Rel *elf_rel_by_idx(Elf_Data *data, size_t idx);

yep, I was anticipating that these will move as well. But I think it's fine if they all stay in libbpf_internal.h, IMO. I'd rather not have many small internal header for no good reason (like we have str_error.h right now, with a single func declaration)

Quoted 11 lines

usdt.c:static int find_elf_sec_by_name(Elf *elf, const char *sec_name, GElf_Shdr *shdr, Elf_Scn **scn)

'struct elf_seg' stuff

usdt.c:static int cmp_elf_segs(const void *_a, const void *_b)
usdt.c:static int parse_elf_segs(Elf *elf, const char *path, struct elf_seg **segs, size_t *seg_cnt) usdt.c:static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs, size_t *seg_cnt) usdt.c:static struct elf_seg *find_elf_seg(struct elf_seg *segs, size_t seg_cnt, long virtaddr) usdt.c:static struct elf_seg *find_vma_seg(struct elf_seg *segs, size_t seg_cnt, long offset)

Quoted 2 lines

but I can add the new header file later in follow up changes when we have more elf functions in

see above, I don't think we should, let's stick to libbpf_internal.h for now

Quoted 2 lines

jirka

#23 lore raw

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

In reply to Jiri Olsa
Quoted 31 lines · Jiri Olsa

Adding new elf object that will contain elf related functions. There's no functional change.

---
 tools/lib/bpf/Build        |   2 +-
 tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c     | 186 +---------------------------------
 tools/lib/bpf/libbpf_elf.h |  11 +++
 4 files changed, 211 insertions(+), 186 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h
tools/lib/bpf/Build +1 -1
index b8b0a6369363..2d0c282c8588 100644
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
            netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
            btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-           usdt.o zip.o
+           usdt.o zip.o elf.o
tools/lib/bpf/elf.c new file +0 -0
new file mode 100644
index 000000000000..2b62b4af28ce

tools/lib/bpf/elf.c line 6

Quoted 6 lines
tools/lib/bpf/elf.c new file +6 -0
@@ -0,0 +1,198 @@
+// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+#include <libelf.h>
+#include <gelf.h>
+#include <fcntl.h>
+#include <linux/kernel.h>

do you know why we need linux/kernel.h include? is it to get __u32 and other typedefs?

Quoted 7 lines
tools/lib/bpf/elf.c new file +7 -0
+
+#include "libbpf_elf.h"
+#include "libbpf_internal.h"
+#include "str_error.h"
+
+#define STRERR_BUFSIZE  128
+

[...]

#24 lore raw

Re: [PATCHv3 bpf-next 07/26] libbpf: Move elf_find_func_offset* functions to elf object

Quoted 33 lines · Andrii Nakryiko
Quoted 32 lines · Jiri Olsa

Adding new elf object that will contain elf related functions. There's no functional change.

---
 tools/lib/bpf/Build        |   2 +-
 tools/lib/bpf/elf.c        | 198 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c     | 186 +---------------------------------
 tools/lib/bpf/libbpf_elf.h |  11 +++
 4 files changed, 211 insertions(+), 186 deletions(-)
 create mode 100644 tools/lib/bpf/elf.c
 create mode 100644 tools/lib/bpf/libbpf_elf.h
tools/lib/bpf/Build +1 -1
index b8b0a6369363..2d0c282c8588 100644
@@ -1,4 +1,4 @@
 libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
            netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \
            btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \
-           usdt.o zip.o
+           usdt.o zip.o elf.o
tools/lib/bpf/elf.c new file +1 -0
new file mode 100644
index 000000000000..2b62b4af28ce
@@ -0,0 +1,198 @@
+// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
Quoted 8 lines

tools/lib/bpf/elf.c line 6

Quoted 5 lines
tools/lib/bpf/elf.c new file +5 -0
+
+#include <libelf.h>
+#include <gelf.h>
+#include <fcntl.h>
+#include <linux/kernel.h>

do you know why we need linux/kernel.h include? is it to get __u32 and other typedefs?

it's for the ARRAY_SIZE macro

jirka

Quoted 10 lines
Quoted 8 lines
tools/lib/bpf/elf.c new file +7 -0
+
+#include "libbpf_elf.h"
+#include "libbpf_internal.h"
+#include "str_error.h"
+
+#define STRERR_BUFSIZE  128
+

[...]

author reviewer
#25 lore raw

PATCH v3 8/26 libbpf: Add elf_open/elf_close functions

In reply to Jiri Olsa

Adding elf_open/elf_close functions and using it in elf_find_func_offset_from_file function. It will be used in following changes to save some common code.

---
 tools/lib/bpf/elf.c        | 59 +++++++++++++++++++++++++-------------
 tools/lib/bpf/libbpf_elf.h |  8 ++++++
 tools/lib/bpf/usdt.c       | 31 ++++++--------------
 3 files changed, 56 insertions(+), 42 deletions(-)
tools/lib/bpf/elf.c +39 -20
index 2b62b4af28ce..74e35071d22e 100644
@@ -11,6 +11,40 @@
 
 #define STRERR_BUFSIZE  128
 
+int elf_open(const char *binary_path, struct elf_fd *elf_fd)
+{
+	char errmsg[STRERR_BUFSIZE];
+	int fd, ret;
+	Elf *elf;
+
+	if (elf_version(EV_CURRENT) == EV_NONE) {
+		pr_warn("elf: failed to init libelf for %s\n", binary_path);
+		return -LIBBPF_ERRNO__LIBELF;
+	}
+	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
+	if (fd < 0) {
+		ret = -errno;
+		pr_warn("elf: failed to open %s: %s\n", binary_path,
+			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+		return ret;
+	}
+	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+	if (!elf) {
+		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+		close(fd);
+		return -LIBBPF_ERRNO__FORMAT;
+	}
+	elf_fd->fd = fd;
+	elf_fd->elf = elf;
+	return 0;
+}
+
+void elf_close(struct elf_fd *elf_fd)
+{
+	elf_end(elf_fd->elf);
+	close(elf_fd->fd);
+}
+
 /* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */
 static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
 {
@@ -171,28 +205,13 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
  */
 long elf_find_func_offset_from_file(const char *binary_path, const char *name)
 {
-	char errmsg[STRERR_BUFSIZE];
+	struct elf_fd elf_fd;
 	long ret = -ENOENT;
-	Elf *elf;
-	int fd;
 
-	fd = open(binary_path, O_RDONLY | O_CLOEXEC);
-	if (fd < 0) {
-		ret = -errno;
-		pr_warn("failed to open %s: %s\n", binary_path,
-			libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+	ret = elf_open(binary_path, &elf_fd);
+	if (ret)
 		return ret;
-	}
-	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-	if (!elf) {
-		pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
-		close(fd);
-		return -LIBBPF_ERRNO__FORMAT;
-	}
-
-	ret = elf_find_func_offset(elf, binary_path, name);
-	elf_end(elf);
-	close(fd);
+	ret = elf_find_func_offset(elf_fd.elf, binary_path, name);
+	elf_close(&elf_fd);
 	return ret;
 }
-
tools/lib/bpf/libbpf_elf.h +8 -0
index 1b652220fabf..c763ac35a85e 100644
@@ -5,6 +5,14 @@
 
 #include <libelf.h>
 
+struct elf_fd {
+	Elf *elf;
+	int fd;
+};
+
+int elf_open(const char *binary_path, struct elf_fd *elf_fd);
+void elf_close(struct elf_fd *elf_fd);
+
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
 long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
tools/lib/bpf/usdt.c +9 -22
index f1a141555f08..9fa883ebc0bd 100644
@@ -19,6 +19,7 @@
 #include "libbpf.h"
 #include "libbpf_common.h"
 #include "libbpf_internal.h"
+#include "libbpf_elf.h"
 #include "hashmap.h"
 
 /* libbpf's USDT support consists of BPF-side state/code and user-space
@@ -943,32 +944,22 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 					  const char *usdt_provider, const char *usdt_name,
 					  __u64 usdt_cookie)
 {
-	int i, fd, err, spec_map_fd, ip_map_fd;
+	int i, err, spec_map_fd, ip_map_fd;
 	LIBBPF_OPTS(bpf_uprobe_opts, opts);
 	struct hashmap *specs_hash = NULL;
 	struct bpf_link_usdt *link = NULL;
 	struct usdt_target *targets = NULL;
+	struct elf_fd elf_fd;
 	size_t target_cnt;
-	Elf *elf;
 
 	spec_map_fd = bpf_map__fd(man->specs_map);
 	ip_map_fd = bpf_map__fd(man->ip_to_spec_id_map);
 
-	fd = open(path, O_RDONLY | O_CLOEXEC);
-	if (fd < 0) {
-		err = -errno;
-		pr_warn("usdt: failed to open ELF binary '%s': %d\n", path, err);
+	err = elf_open(path, &elf_fd);
+	if (err)
 		return libbpf_err_ptr(err);
-	}
 
-	elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
-	if (!elf) {
-		err = -EBADF;
-		pr_warn("usdt: failed to parse ELF binary '%s': %s\n", path, elf_errmsg(-1));
-		goto err_out;
-	}
-
-	err = sanity_check_usdt_elf(elf, path);
+	err = sanity_check_usdt_elf(elf_fd.elf, path);
 	if (err)
 		goto err_out;
 
@@ -981,7 +972,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 	/* discover USDT in given binary, optionally limiting
 	 * activations to a given PID, if pid > 0
 	 */
-	err = collect_usdt_targets(man, elf, path, pid, usdt_provider, usdt_name,
+	err = collect_usdt_targets(man, elf_fd.elf, path, pid, usdt_provider, usdt_name,
 				   usdt_cookie, &targets, &target_cnt);
 	if (err <= 0) {
 		err = (err == 0) ? -ENOENT : err;
@@ -1066,9 +1057,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 
 	free(targets);
 	hashmap__free(specs_hash);
-	elf_end(elf);
-	close(fd);
-
+	elf_close(&elf_fd);
 	return &link->link;
 
 err_out:
@@ -1076,9 +1065,7 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
 		bpf_link__destroy(&link->link);
 	free(targets);
 	hashmap__free(specs_hash);
-	if (elf)
-		elf_end(elf);
-	close(fd);
+	elf_close(&elf_fd);
 	return libbpf_err_ptr(err);
 }
 
Signature · 1 line
2.41.0
Acked-by on v3 8/26
#26 lore raw

Re: [PATCHv3 bpf-next 08/26] libbpf: Add elf_open/elf_close functions

Quoted 5 lines · Jiri Olsa

Adding elf_open/elf_close functions and using it in elf_find_func_offset_from_file function. It will be used in following changes to save some common code.

Quoted 7 lines
---
 tools/lib/bpf/elf.c        | 59 +++++++++++++++++++++++++-------------
 tools/lib/bpf/libbpf_elf.h |  8 ++++++
 tools/lib/bpf/usdt.c       | 31 ++++++--------------
 3 files changed, 56 insertions(+), 42 deletions(-)

one nit below

Quoted 30 lines
tools/lib/bpf/elf.c +22 -0
index 2b62b4af28ce..74e35071d22e 100644
@@ -11,6 +11,40 @@
 
 #define STRERR_BUFSIZE  128
 
+int elf_open(const char *binary_path, struct elf_fd *elf_fd)
+{
+       char errmsg[STRERR_BUFSIZE];
+       int fd, ret;
+       Elf *elf;
+
+       if (elf_version(EV_CURRENT) == EV_NONE) {
+               pr_warn("elf: failed to init libelf for %s\n", binary_path);
+               return -LIBBPF_ERRNO__LIBELF;
+       }
+       fd = open(binary_path, O_RDONLY | O_CLOEXEC);
+       if (fd < 0) {
+               ret = -errno;
+               pr_warn("elf: failed to open %s: %s\n", binary_path,
+                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+               return ret;
+       }
+       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+       if (!elf) {
+               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+               close(fd);
+               return -LIBBPF_ERRNO__FORMAT;

tools/lib/bpf/elf.c line 45

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+       }
+       elf_fd->fd = fd;
+       elf_fd->elf = elf;
+       return 0;
+}
+
+void elf_close(struct elf_fd *elf_fd)
+{
+       elf_end(elf_fd->elf);
+       close(elf_fd->fd);

nit: I'd make elf_close() work correctly with a) NULL elf_fd and b) NULL elf_fd->elf, just to never have to think about this

Quoted 5 lines
tools/lib/bpf/elf.c +2 -0
+}
+
 /* Return next ELF section of sh_type after scn, or first of that type if scn is NULL. */
 static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
 {

[...]

#27 lore raw

Re: [PATCHv3 bpf-next 08/26] libbpf: Add elf_open/elf_close functions

Quoted 52 lines · Andrii Nakryiko
Quoted 5 lines · Jiri Olsa

Adding elf_open/elf_close functions and using it in elf_find_func_offset_from_file function. It will be used in following changes to save some common code.

Quoted 7 lines
---
 tools/lib/bpf/elf.c        | 59 +++++++++++++++++++++++++-------------
 tools/lib/bpf/libbpf_elf.h |  8 ++++++
 tools/lib/bpf/usdt.c       | 31 ++++++--------------
 3 files changed, 56 insertions(+), 42 deletions(-)

one nit below

Quoted 34 lines
tools/lib/bpf/elf.c +26 -0
index 2b62b4af28ce..74e35071d22e 100644
@@ -11,6 +11,40 @@
 
 #define STRERR_BUFSIZE  128
 
+int elf_open(const char *binary_path, struct elf_fd *elf_fd)
+{
+       char errmsg[STRERR_BUFSIZE];
+       int fd, ret;
+       Elf *elf;
+
+       if (elf_version(EV_CURRENT) == EV_NONE) {
+               pr_warn("elf: failed to init libelf for %s\n", binary_path);
+               return -LIBBPF_ERRNO__LIBELF;
+       }
+       fd = open(binary_path, O_RDONLY | O_CLOEXEC);
+       if (fd < 0) {
+               ret = -errno;
+               pr_warn("elf: failed to open %s: %s\n", binary_path,
+                       libbpf_strerror_r(ret, errmsg, sizeof(errmsg)));
+               return ret;
+       }
+       elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+       if (!elf) {
+               pr_warn("elf: could not read elf from %s: %s\n", binary_path, elf_errmsg(-1));
+               close(fd);
+               return -LIBBPF_ERRNO__FORMAT;
+       }
+       elf_fd->fd = fd;
+       elf_fd->elf = elf;
+       return 0;
Quoted 8 lines

tools/lib/bpf/elf.c line 45

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+}
+
+void elf_close(struct elf_fd *elf_fd)
+{
+       elf_end(elf_fd->elf);
+       close(elf_fd->fd);

nit: I'd make elf_close() work correctly with a) NULL elf_fd and b)

right, ok

Quoted 1 line

NULL elf_fd->elf, just to never have to think about this

there's NULL check in elf_end

thanks,
jirka

author reviewer
#28 lore raw

PATCH v3 9/26 libbpf: Add elf symbol iterator

In reply to Jiri Olsa

Adding elf symbol iterator object (and some functions) that follow open-coded iterator pattern and some functions to ease up iterating elf object symbols.

The idea is to iterate single symbol section with:

struct elf_sym_iter iter;
struct elf_sym *sym;

if (elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM))

        goto error;
  while ((sym = elf_sym_iter_next(&iter))) {
        ...
  }

I considered opening the elf inside the iterator and iterate all symbol sections, but then it gets more complicated wrt user checks for when the next section is processed.

Plus side is the we don't need 'exit' function, because caller/user is in charge of that.

The returned iterated symbol object from elf_sym_iter_next function is placed inside the struct elf_sym_iter, so no extra allocation or argument is needed.

---
 tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
 1 file changed, 117 insertions(+), 61 deletions(-)
tools/lib/bpf/elf.c +117 -61
index 74e35071d22e..fcce4bd2478f 100644
@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
 	return NULL;
 }
 
+struct elf_sym {
+	const char *name;
+	GElf_Sym sym;
+	GElf_Shdr sh;
+};
+
+struct elf_sym_iter {
+	Elf *elf;
+	Elf_Data *syms;
+	size_t nr_syms;
+	size_t strtabidx;
+	size_t next_sym_idx;
+	struct elf_sym sym;
+	int st_type;
+};
+
+static int elf_sym_iter_new(struct elf_sym_iter *iter,
+			    Elf *elf, const char *binary_path,
+			    int sh_type, int st_type)
+{
+	Elf_Scn *scn = NULL;
+	GElf_Ehdr ehdr;
+	GElf_Shdr sh;
+
+	memset(iter, 0, sizeof(*iter));
+
+	if (!gelf_getehdr(elf, &ehdr)) {
+		pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
+		return -EINVAL;
+	}
+
+	scn = elf_find_next_scn_by_type(elf, sh_type, NULL);
+	if (!scn) {
+		pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
+			 binary_path);
+		return -ENOENT;
+	}
+
+	if (!gelf_getshdr(scn, &sh))
+		return -EINVAL;
+
+	iter->strtabidx = sh.sh_link;
+	iter->syms = elf_getdata(scn, 0);
+	if (!iter->syms) {
+		pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
+			binary_path, elf_errmsg(-1));
+		return -EINVAL;
+	}
+	iter->nr_syms = iter->syms->d_size / sh.sh_entsize;
+	iter->elf = elf;
+	iter->st_type = st_type;
+	return 0;
+}
+
+static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
+{
+	struct elf_sym *ret = &iter->sym;
+	GElf_Sym *sym = &ret->sym;
+	const char *name = NULL;
+	Elf_Scn *sym_scn;
+	size_t idx;
+
+	for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
+		if (!gelf_getsym(iter->syms, idx, sym))
+			continue;
+		if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
+			continue;
+		name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
+		if (!name)
+			continue;
+
+		/* Transform symbol's virtual address (absolute for
+		 * binaries and relative for shared libs) into file
+		 * offset, which is what kernel is expecting for
+		 * uprobe/uretprobe attachment.
+		 * See Documentation/trace/uprobetracer.rst for more
+		 * details.
+		 * This is done by looking up symbol's containing
+		 * section's header and using iter's virtual address
+		 * (sh_addr) and corresponding file offset (sh_offset)
+		 * to transform sym.st_value (virtual address) into
+		 * desired final file offset.
+		 */
+		sym_scn = elf_getscn(iter->elf, sym->st_shndx);
+		if (!sym_scn)
+			continue;
+		if (!gelf_getshdr(sym_scn, &ret->sh))
+			continue;
+
+		iter->next_sym_idx = idx + 1;
+		ret->name = name;
+		return ret;
+	}
+
+	return NULL;
+}
+
+static unsigned long elf_sym_offset(struct elf_sym *sym)
+{
+	return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
+}
+
 /* Find offset of function name in the provided ELF object. "binary_path" is
  * the path to the ELF binary represented by "elf", and only used for error
  * reporting matters. "name" matches symbol name or name@@LIB for library
@@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 	 * reported as a warning/error.
 	 */
 	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
-		size_t nr_syms, strtabidx, idx;
-		Elf_Data *symbols = NULL;
-		Elf_Scn *scn = NULL;
+		struct elf_sym_iter iter;
+		struct elf_sym *sym;
 		int last_bind = -1;
-		const char *sname;
-		GElf_Shdr sh;
+		int curr_bind;
 
-		scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
-		if (!scn) {
-			pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
-				 binary_path);
-			continue;
-		}
-		if (!gelf_getshdr(scn, &sh))
-			continue;
-		strtabidx = sh.sh_link;
-		symbols = elf_getdata(scn, 0);
-		if (!symbols) {
-			pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
-				binary_path, elf_errmsg(-1));
-			ret = -LIBBPF_ERRNO__FORMAT;
+		ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
+		if (ret) {
+			if (ret == -ENOENT)
+				continue;
 			goto out;
 		}
-		nr_syms = symbols->d_size / sh.sh_entsize;
-
-		for (idx = 0; idx < nr_syms; idx++) {
-			int curr_bind;
-			GElf_Sym sym;
-			Elf_Scn *sym_scn;
-			GElf_Shdr sym_sh;
-
-			if (!gelf_getsym(symbols, idx, &sym))
-				continue;
-
-			if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
-				continue;
-
-			sname = elf_strptr(elf, strtabidx, sym.st_name);
-			if (!sname)
-				continue;
-
-			curr_bind = GELF_ST_BIND(sym.st_info);
 
+		while ((sym = elf_sym_iter_next(&iter))) {
 			/* User can specify func, func@@LIB or func@@LIB_VERSION. */
-			if (strncmp(sname, name, name_len) != 0)
+			if (strncmp(sym->name, name, name_len) != 0)
 				continue;
 			/* ...but we don't want a search for "foo" to match 'foo2" also, so any
 			 * additional characters in sname should be of the form "@@LIB".
 			 */
-			if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
+			if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
 				continue;
 
-			if (ret >= 0) {
+			curr_bind = GELF_ST_BIND(sym->sym.st_info);
+
+			if (ret > 0) {
 				/* handle multiple matches */
 				if (last_bind != STB_WEAK && curr_bind != STB_WEAK) {
 					/* Only accept one non-weak bind. */
 					pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n",
-						sname, name, binary_path);
+						sym->name, name, binary_path);
 					ret = -LIBBPF_ERRNO__FORMAT;
 					goto out;
 				} else if (curr_bind == STB_WEAK) {
@@ -158,25 +232,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
 				}
 			}
 
-			/* Transform symbol's virtual address (absolute for
-			 * binaries and relative for shared libs) into file
-			 * offset, which is what kernel is expecting for
-			 * uprobe/uretprobe attachment.
-			 * See Documentation/trace/uprobetracer.rst for more
-			 * details.
-			 * This is done by looking up symbol's containing
-			 * section's header and using it's virtual address
-			 * (sh_addr) and corresponding file offset (sh_offset)
-			 * to transform sym.st_value (virtual address) into
-			 * desired final file offset.
-			 */
-			sym_scn = elf_getscn(elf, sym.st_shndx);
-			if (!sym_scn)
-				continue;
-			if (!gelf_getshdr(sym_scn, &sym_sh))
-				continue;
-
-			ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset;
+			ret = elf_sym_offset(sym);
 			last_bind = curr_bind;
 		}
 		if (ret > 0)
Signature · 1 line
2.41.0
Acked-by on v3 9/26
#29 lore raw

Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator

Quoted 28 lines · Jiri Olsa

Adding elf symbol iterator object (and some functions) that follow open-coded iterator pattern and some functions to ease up iterating elf object symbols.

The idea is to iterate single symbol section with:

struct elf_sym_iter iter;
struct elf_sym *sym;

if (elf_sym_iter_new(&iter, elf, binary_path, SHT_DYNSYM))

        goto error;
  while ((sym = elf_sym_iter_next(&iter))) {
        ...
  }

I considered opening the elf inside the iterator and iterate all symbol sections, but then it gets more complicated wrt user checks for when the next section is processed.

Plus side is the we don't need 'exit' function, because caller/user is in charge of that.

The returned iterated symbol object from elf_sym_iter_next function is placed inside the struct elf_sym_iter, so no extra allocation or argument is needed.

Quoted 6 lines
---
 tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
 1 file changed, 117 insertions(+), 61 deletions(-)

A bunch of nits, but overall looks good. Please address nits, and add my ack

Quoted 5 lines
tools/lib/bpf/elf.c +0 -0
index 74e35071d22e..fcce4bd2478f 100644

tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 67

Quoted 9 lines
tools/lib/bpf/elf.c +6 -0
@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
        return NULL;
 }
 
+struct elf_sym {
+       const char *name;
+       GElf_Sym sym;
+       GElf_Shdr sh;
+};
+

if we want to use elf_sym_iter outside of elf.c, this should be in libbpf_internal.h?

Quoted 67 lines
tools/lib/bpf/elf.c +67 -0
+struct elf_sym_iter {
+       Elf *elf;
+       Elf_Data *syms;
+       size_t nr_syms;
+       size_t strtabidx;
+       size_t next_sym_idx;
+       struct elf_sym sym;
+       int st_type;
+};
+
+static int elf_sym_iter_new(struct elf_sym_iter *iter,
+                           Elf *elf, const char *binary_path,
+                           int sh_type, int st_type)
+{
+       Elf_Scn *scn = NULL;
+       GElf_Ehdr ehdr;
+       GElf_Shdr sh;
+
+       memset(iter, 0, sizeof(*iter));
+
+       if (!gelf_getehdr(elf, &ehdr)) {
+               pr_warn("elf: failed to get ehdr from %s: %s\n", binary_path, elf_errmsg(-1));
+               return -EINVAL;
+       }
+
+       scn = elf_find_next_scn_by_type(elf, sh_type, NULL);
+       if (!scn) {
+               pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
+                        binary_path);
+               return -ENOENT;
+       }
+
+       if (!gelf_getshdr(scn, &sh))
+               return -EINVAL;
+
+       iter->strtabidx = sh.sh_link;
+       iter->syms = elf_getdata(scn, 0);
+       if (!iter->syms) {
+               pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
+                       binary_path, elf_errmsg(-1));
+               return -EINVAL;
+       }
+       iter->nr_syms = iter->syms->d_size / sh.sh_entsize;
+       iter->elf = elf;
+       iter->st_type = st_type;
+       return 0;
+}
+
+static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
+{
+       struct elf_sym *ret = &iter->sym;
+       GElf_Sym *sym = &ret->sym;
+       const char *name = NULL;
+       Elf_Scn *sym_scn;
+       size_t idx;
+
+       for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
+               if (!gelf_getsym(iter->syms, idx, sym))
+                       continue;
+               if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
+                       continue;
+               name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
+               if (!name)
+                       continue;
+
+               /* Transform symbol's virtual address (absolute for
+                * binaries and relative for shared libs) into file

tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 144

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+                * offset, which is what kernel is expecting for
+                * uprobe/uretprobe attachment.
+                * See Documentation/trace/uprobetracer.rst for more
+                * details.
+                * This is done by looking up symbol's containing
+                * section's header and using iter's virtual address
+                * (sh_addr) and corresponding file offset (sh_offset)
+                * to transform sym.st_value (virtual address) into
+                * desired final file offset.
+                */

this comment is misplaced? we don't do the translation here

Quoted 25 lines
tools/lib/bpf/elf.c +19 -0
+               sym_scn = elf_getscn(iter->elf, sym->st_shndx);
+               if (!sym_scn)
+                       continue;
+               if (!gelf_getshdr(sym_scn, &ret->sh))
+                       continue;
+
+               iter->next_sym_idx = idx + 1;
+               ret->name = name;
+               return ret;
+       }
+
+       return NULL;
+}
+
+static unsigned long elf_sym_offset(struct elf_sym *sym)
+{
+       return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
+}
+
 /* Find offset of function name in the provided ELF object. "binary_path" is
  * the path to the ELF binary represented by "elf", and only used for error
  * reporting matters. "name" matches symbol name or name@@LIB for library
@@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
         * reported as a warning/error.
         */

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 198

Quoted 10 lines
tools/lib/bpf/elf.c +3 -5
        for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
-               size_t nr_syms, strtabidx, idx;
-               Elf_Data *symbols = NULL;
-               Elf_Scn *scn = NULL;
+               struct elf_sym_iter iter;
+               struct elf_sym *sym;
                int last_bind = -1;
-               const char *sname;
-               GElf_Shdr sh;
+               int curr_bind;

OCD nit:

$ rg 'curr(_|\b)' | wc -l
8
$ rg 'cur(_|\b)' | wc -l
148

and those 8 I consider an unfortunate accident ;) let's standardize on using "cur" consistently

Quoted 10 lines
tools/lib/bpf/elf.c +0 -9
-               scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
-               if (!scn) {
-                       pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
-                                binary_path);
-                       continue;
-               }
-               if (!gelf_getshdr(scn, &sh))
-                       continue;
-               strtabidx = sh.sh_link;

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 204

Quoted 10 lines
tools/lib/bpf/elf.c +4 -5
-               symbols = elf_getdata(scn, 0);
-               if (!symbols) {
-                       pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
-                               binary_path, elf_errmsg(-1));
-                       ret = -LIBBPF_ERRNO__FORMAT;
+               ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
+               if (ret) {
+                       if (ret == -ENOENT)
+                               continue;
                        goto out;

another styling nit: let's avoid unnecessary nesting of ifs:

if (ret == -ENOENT)

    continue;
if (ret)
    goto out;

simple and clean

Quoted 27 lines
tools/lib/bpf/elf.c +2 -20
                }
-               nr_syms = symbols->d_size / sh.sh_entsize;
-
-               for (idx = 0; idx < nr_syms; idx++) {
-                       int curr_bind;
-                       GElf_Sym sym;
-                       Elf_Scn *sym_scn;
-                       GElf_Shdr sym_sh;
-
-                       if (!gelf_getsym(symbols, idx, &sym))
-                               continue;
-
-                       if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
-                               continue;
-
-                       sname = elf_strptr(elf, strtabidx, sym.st_name);
-                       if (!sname)
-                               continue;
-
-                       curr_bind = GELF_ST_BIND(sym.st_info);
 
+               while ((sym = elf_sym_iter_next(&iter))) {
                        /* User can specify func, func@@LIB or func@@LIB_VERSION. */
-                       if (strncmp(sname, name, name_len) != 0)
+                       if (strncmp(sym->name, name, name_len) != 0)
                                continue;
                        /* ...but we don't want a search for "foo" to match 'foo2" also, so any

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 219

Quoted 10 lines
tools/lib/bpf/elf.c +4 -2
                         * additional characters in sname should be of the form "@@LIB".
                         */
-                       if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
+                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
                                continue;
 
-                       if (ret >= 0) {
+                       curr_bind = GELF_ST_BIND(sym->sym.st_info);
+
+                       if (ret > 0) {

used to be >=, why the change?

Quoted 39 lines
tools/lib/bpf/elf.c +2 -21
                                /* handle multiple matches */
                                if (last_bind != STB_WEAK && curr_bind != STB_WEAK) {
                                        /* Only accept one non-weak bind. */
                                        pr_warn("elf: ambiguous match for '%s', '%s' in '%s'\n",
-                                               sname, name, binary_path);
+                                               sym->name, name, binary_path);
                                        ret = -LIBBPF_ERRNO__FORMAT;
                                        goto out;
                                } else if (curr_bind == STB_WEAK) {
@@ -158,25 +232,7 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
                                }
                        }
 
-                       /* Transform symbol's virtual address (absolute for
-                        * binaries and relative for shared libs) into file
-                        * offset, which is what kernel is expecting for
-                        * uprobe/uretprobe attachment.
-                        * See Documentation/trace/uprobetracer.rst for more
-                        * details.
-                        * This is done by looking up symbol's containing
-                        * section's header and using it's virtual address
-                        * (sh_addr) and corresponding file offset (sh_offset)
-                        * to transform sym.st_value (virtual address) into
-                        * desired final file offset.
-                        */
-                       sym_scn = elf_getscn(elf, sym.st_shndx);
-                       if (!sym_scn)
-                               continue;
-                       if (!gelf_getshdr(sym_scn, &sym_sh))
-                               continue;
-
-                       ret = sym.st_value - sym_sh.sh_addr + sym_sh.sh_offset;
+                       ret = elf_sym_offset(sym);
                        last_bind = curr_bind;
                }
                if (ret > 0)
--

2.41.0

#30 lore raw

Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator

On Thu, Jul 06, 2023 at 04:24:48PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 19 lines
Quoted 6 lines
---
 tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
 1 file changed, 117 insertions(+), 61 deletions(-)

A bunch of nits, but overall looks good. Please address nits, and add my ack

Quoted 8 lines
tools/lib/bpf/elf.c +0 -0
index 74e35071d22e..fcce4bd2478f 100644
@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
        return NULL;
 }
 
Quoted 9 lines

tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 67

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+struct elf_sym {
+       const char *name;
+       GElf_Sym sym;
+       GElf_Shdr sh;
+};
+

if we want to use elf_sym_iter outside of elf.c, this should be in libbpf_internal.h?

yes eventually, but all the helper functions using elf_sym_iter that I added later are in elf.c, so there's no need atm

SNIP

Quoted 24 lines
Quoted 24 lines
tools/lib/bpf/elf.c +24 -0
+
+static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
+{
+       struct elf_sym *ret = &iter->sym;
+       GElf_Sym *sym = &ret->sym;
+       const char *name = NULL;
+       Elf_Scn *sym_scn;
+       size_t idx;
+
+       for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
+               if (!gelf_getsym(iter->syms, idx, sym))
+                       continue;
+               if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
+                       continue;
+               name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
+               if (!name)
+                       continue;
+
+               /* Transform symbol's virtual address (absolute for
+                * binaries and relative for shared libs) into file
+                * offset, which is what kernel is expecting for
+                * uprobe/uretprobe attachment.
+                * See Documentation/trace/uprobetracer.rst for more
+                * details.
Quoted 8 lines

tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 144

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+                * This is done by looking up symbol's containing
+                * section's header and using iter's virtual address
+                * (sh_addr) and corresponding file offset (sh_offset)
+                * to transform sym.st_value (virtual address) into
+                * desired final file offset.
+                */

this comment is misplaced? we don't do the translation here

right, should be placed at the elf_sym_offset function

Quoted 39 lines
Quoted 26 lines
tools/lib/bpf/elf.c +19 -0
+               sym_scn = elf_getscn(iter->elf, sym->st_shndx);
+               if (!sym_scn)
+                       continue;
+               if (!gelf_getshdr(sym_scn, &ret->sh))
+                       continue;
+
+               iter->next_sym_idx = idx + 1;
+               ret->name = name;
+               return ret;
+       }
+
+       return NULL;
+}
+
+static unsigned long elf_sym_offset(struct elf_sym *sym)
+{
+       return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
+}
+
 /* Find offset of function name in the provided ELF object. "binary_path" is
  * the path to the ELF binary represented by "elf", and only used for error
  * reporting matters. "name" matches symbol name or name@@LIB for library
@@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
         * reported as a warning/error.
         */

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 198

Quoted 10 lines
tools/lib/bpf/elf.c +3 -5
        for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
-               size_t nr_syms, strtabidx, idx;
-               Elf_Data *symbols = NULL;
-               Elf_Scn *scn = NULL;
+               struct elf_sym_iter iter;
+               struct elf_sym *sym;
                int last_bind = -1;
-               const char *sname;
-               GElf_Shdr sh;
+               int curr_bind;

OCD nit:

Quoted 7 lines
$ rg 'curr(_|\b)' | wc -l
8
$ rg 'cur(_|\b)' | wc -l
148

and those 8 I consider an unfortunate accident ;) let's standardize on using "cur" consistently

ok.. probably easier to make that change than treat ocd ;-)

Quoted 22 lines
Quoted 11 lines
tools/lib/bpf/elf.c +0 -9
-               scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
-               if (!scn) {
-                       pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
-                                binary_path);
-                       continue;
-               }
-               if (!gelf_getshdr(scn, &sh))
-                       continue;
-               strtabidx = sh.sh_link;

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 204

Quoted 10 lines
tools/lib/bpf/elf.c +4 -5
-               symbols = elf_getdata(scn, 0);
-               if (!symbols) {
-                       pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
-                               binary_path, elf_errmsg(-1));
-                       ret = -LIBBPF_ERRNO__FORMAT;
+               ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
+               if (ret) {
+                       if (ret == -ENOENT)
+                               continue;
                        goto out;
Quoted 8 lines

another styling nit: let's avoid unnecessary nesting of ifs:

if (ret == -ENOENT)

    continue;
if (ret)
    goto out;

simple and clean

ok

Quoted 35 lines
Quoted 35 lines
tools/lib/bpf/elf.c +3 -21
                }
-               nr_syms = symbols->d_size / sh.sh_entsize;
-
-               for (idx = 0; idx < nr_syms; idx++) {
-                       int curr_bind;
-                       GElf_Sym sym;
-                       Elf_Scn *sym_scn;
-                       GElf_Shdr sym_sh;
-
-                       if (!gelf_getsym(symbols, idx, &sym))
-                               continue;
-
-                       if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
-                               continue;
-
-                       sname = elf_strptr(elf, strtabidx, sym.st_name);
-                       if (!sname)
-                               continue;
-
-                       curr_bind = GELF_ST_BIND(sym.st_info);
 
+               while ((sym = elf_sym_iter_next(&iter))) {
                        /* User can specify func, func@@LIB or func@@LIB_VERSION. */
-                       if (strncmp(sname, name, name_len) != 0)
+                       if (strncmp(sym->name, name, name_len) != 0)
                                continue;
                        /* ...but we don't want a search for "foo" to match 'foo2" also, so any
                         * additional characters in sname should be of the form "@@LIB".
                         */
-                       if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
+                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
                                continue;
 
Quoted 6 lines

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 219

Quoted 4 lines
tools/lib/bpf/elf.c +3 -1
-                       if (ret >= 0) {
+                       curr_bind = GELF_ST_BIND(sym->sym.st_info);
+
+                       if (ret > 0) {

used to be >=, why the change?

the original code initialized ret with -ENOENT and did not change its value till this point, so the condition never triggered for the first loop, but it did for the new code because we now use ret earlier in:

	ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);

also the check makes sense to me only if ret > 0 .. when we find a duplicate value for symbol

jirka

#31 lore raw

Re: [PATCHv3 bpf-next 09/26] libbpf: Add elf symbol iterator

Quoted 118 lines · Jiri Olsa

On Thu, Jul 06, 2023 at 04:24:48PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 19 lines
Quoted 6 lines
---
 tools/lib/bpf/elf.c | 178 +++++++++++++++++++++++++++++---------------
 1 file changed, 117 insertions(+), 61 deletions(-)

A bunch of nits, but overall looks good. Please address nits, and add my ack

Quoted 8 lines
tools/lib/bpf/elf.c +0 -0
index 74e35071d22e..fcce4bd2478f 100644
@@ -59,6 +59,108 @@ static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn)
        return NULL;
 }
 
Quoted 9 lines

tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 67

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+struct elf_sym {
+       const char *name;
+       GElf_Sym sym;
+       GElf_Shdr sh;
+};
+

if we want to use elf_sym_iter outside of elf.c, this should be in libbpf_internal.h?

yes eventually, but all the helper functions using elf_sym_iter that I added later are in elf.c, so there's no need atm

SNIP

Quoted 24 lines
Quoted 24 lines
tools/lib/bpf/elf.c +24 -0
+
+static struct elf_sym *elf_sym_iter_next(struct elf_sym_iter *iter)
+{
+       struct elf_sym *ret = &iter->sym;
+       GElf_Sym *sym = &ret->sym;
+       const char *name = NULL;
+       Elf_Scn *sym_scn;
+       size_t idx;
+
+       for (idx = iter->next_sym_idx; idx < iter->nr_syms; idx++) {
+               if (!gelf_getsym(iter->syms, idx, sym))
+                       continue;
+               if (GELF_ST_TYPE(sym->st_info) != iter->st_type)
+                       continue;
+               name = elf_strptr(iter->elf, iter->strtabidx, sym->st_name);
+               if (!name)
+                       continue;
+
+               /* Transform symbol's virtual address (absolute for
+                * binaries and relative for shared libs) into file
+                * offset, which is what kernel is expecting for
+                * uprobe/uretprobe attachment.
+                * See Documentation/trace/uprobetracer.rst for more
+                * details.
Quoted 8 lines

tools/lib/bpf/elf.c static Elf_Scn *elf_find_next_scn_by_type(Elf *elf, int sh_type, Elf_Scn *scn) · line 144

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+                * This is done by looking up symbol's containing
+                * section's header and using iter's virtual address
+                * (sh_addr) and corresponding file offset (sh_offset)
+                * to transform sym.st_value (virtual address) into
+                * desired final file offset.
+                */

this comment is misplaced? we don't do the translation here

right, should be placed at the elf_sym_offset function

Quoted 44 lines
Quoted 26 lines
tools/lib/bpf/elf.c +19 -0
+               sym_scn = elf_getscn(iter->elf, sym->st_shndx);
+               if (!sym_scn)
+                       continue;
+               if (!gelf_getshdr(sym_scn, &ret->sh))
+                       continue;
+
+               iter->next_sym_idx = idx + 1;
+               ret->name = name;
+               return ret;
+       }
+
+       return NULL;
+}
+
+static unsigned long elf_sym_offset(struct elf_sym *sym)
+{
+       return sym->sym.st_value - sym->sh.sh_addr + sym->sh.sh_offset;
+}
+
 /* Find offset of function name in the provided ELF object. "binary_path" is
  * the path to the ELF binary represented by "elf", and only used for error
  * reporting matters. "name" matches symbol name or name@@LIB for library
@@ -90,64 +192,36 @@ long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name)
         * reported as a warning/error.
         */

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 198

Quoted 10 lines
tools/lib/bpf/elf.c +3 -5
        for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
-               size_t nr_syms, strtabidx, idx;
-               Elf_Data *symbols = NULL;
-               Elf_Scn *scn = NULL;
+               struct elf_sym_iter iter;
+               struct elf_sym *sym;
                int last_bind = -1;
-               const char *sname;
-               GElf_Shdr sh;
+               int curr_bind;

OCD nit:

$ rg 'curr(_|\b)' | wc -l
8
$ rg 'cur(_|\b)' | wc -l
148
Quoted 5 lines
Quoted 2 lines

and those 8 I consider an unfortunate accident ;) let's standardize on using "cur" consistently

ok.. probably easier to make that change than treat ocd ;-)

thanks :)

Quoted 80 lines
Quoted 22 lines
Quoted 11 lines
tools/lib/bpf/elf.c +0 -9
-               scn = elf_find_next_scn_by_type(elf, sh_types[i], NULL);
-               if (!scn) {
-                       pr_debug("elf: failed to find symbol table ELF sections in '%s'\n",
-                                binary_path);
-                       continue;
-               }
-               if (!gelf_getshdr(scn, &sh))
-                       continue;
-               strtabidx = sh.sh_link;

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 204

Quoted 10 lines
tools/lib/bpf/elf.c +4 -5
-               symbols = elf_getdata(scn, 0);
-               if (!symbols) {
-                       pr_warn("elf: failed to get symbols for symtab section in '%s': %s\n",
-                               binary_path, elf_errmsg(-1));
-                       ret = -LIBBPF_ERRNO__FORMAT;
+               ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);
+               if (ret) {
+                       if (ret == -ENOENT)
+                               continue;
                        goto out;
Quoted 8 lines

another styling nit: let's avoid unnecessary nesting of ifs:

if (ret == -ENOENT)

    continue;
if (ret)
    goto out;

simple and clean

ok

Quoted 35 lines
Quoted 35 lines
tools/lib/bpf/elf.c +3 -21
                }
-               nr_syms = symbols->d_size / sh.sh_entsize;
-
-               for (idx = 0; idx < nr_syms; idx++) {
-                       int curr_bind;
-                       GElf_Sym sym;
-                       Elf_Scn *sym_scn;
-                       GElf_Shdr sym_sh;
-
-                       if (!gelf_getsym(symbols, idx, &sym))
-                               continue;
-
-                       if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
-                               continue;
-
-                       sname = elf_strptr(elf, strtabidx, sym.st_name);
-                       if (!sname)
-                               continue;
-
-                       curr_bind = GELF_ST_BIND(sym.st_info);
 
+               while ((sym = elf_sym_iter_next(&iter))) {
                        /* User can specify func, func@@LIB or func@@LIB_VERSION. */
-                       if (strncmp(sname, name, name_len) != 0)
+                       if (strncmp(sym->name, name, name_len) != 0)
                                continue;
                        /* ...but we don't want a search for "foo" to match 'foo2" also, so any
                         * additional characters in sname should be of the form "@@LIB".
                         */
-                       if (!is_name_qualified && sname[name_len] != '\0' && sname[name_len] != '@')
+                       if (!is_name_qualified && sym->name[name_len] != '\0' && sym->name[name_len] != '@')
                                continue;
 
Quoted 6 lines

tools/lib/bpf/elf.c long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name) · line 219

Quoted 4 lines
tools/lib/bpf/elf.c +3 -1
-                       if (ret >= 0) {
+                       curr_bind = GELF_ST_BIND(sym->sym.st_info);
+
+                       if (ret > 0) {

used to be >=, why the change?

the original code initialized ret with -ENOENT and did not change its value till this point, so the condition never triggered for the first loop, but it did for the new code because we now use ret earlier in:

Quoted 4 lines
        ret = elf_sym_iter_new(&iter, elf, binary_path, sh_types[i], STT_FUNC);

also the check makes sense to me only if ret > 0 .. when we find a duplicate value for symbol

there is a subtle difference if we have some unresolved func with addr 0 or something. But you are right, it's probably not very sensical, we can keep it as > 0.

Quoted 2 lines

jirka

author reviewer
#32 lore raw

PATCH v3 10/26 libbpf: Add elf_resolve_syms_offsets function

In reply to Jiri Olsa

Adding elf_resolve_syms_offsets function that looks up offsets for symbols specified in syms array argument.

Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller.

---
 tools/lib/bpf/elf.c        | 105 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_elf.h |   2 +
 2 files changed, 107 insertions(+)
tools/lib/bpf/elf.c +105 -0
index fcce4bd2478f..7e2f3b2e1fb6 100644
@@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)
 	elf_close(&elf_fd);
 	return ret;
 }
+
+struct symbol {
+	const char *name;
+	int bind;
+	int idx;
+};
+
+static int symbol_cmp(const void *_a, const void *_b)
+{
+	const struct symbol *a = _a;
+	const struct symbol *b = _b;
+
+	return strcmp(a->name, b->name);
+}
+
+int elf_resolve_syms_offsets(const char *binary_path, int cnt,
+			     const char **syms, unsigned long **poffsets)
+{
+	int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+	int err = 0, i, cnt_done = 0;
+	unsigned long *offsets;
+	struct symbol *symbols;
+	struct elf_fd elf_fd;
+
+	err = elf_open(binary_path, &elf_fd);
+	if (err)
+		return err;
+
+	offsets = calloc(cnt, sizeof(*offsets));
+	symbols = calloc(cnt, sizeof(*symbols));
+
+	if (!offsets || !symbols) {
+		err = -ENOMEM;
+		goto out;
+	}
+
+	for (i = 0; i < cnt; i++) {
+		symbols[i].name = syms[i];
+		symbols[i].idx = i;
+	}
+
+	qsort(symbols, cnt, sizeof(*symbols), symbol_cmp);
+
+	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+		struct elf_sym_iter iter;
+		struct elf_sym *sym;
+
+		err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+		if (err) {
+			if (err == -ENOENT)
+				continue;
+			goto out;
+		}
+
+		while ((sym = elf_sym_iter_next(&iter))) {
+			int bind = GELF_ST_BIND(sym->sym.st_info);
+			struct symbol *found, tmp = {
+				.name = sym->name,
+			};
+			unsigned long *offset;
+
+			found = bsearch(&tmp, symbols, cnt, sizeof(*symbols), symbol_cmp);
+			if (!found)
+				continue;
+
+			offset = &offsets[found->idx];
+			if (*offset > 0) {
+				/* same offset, no problem */
+				if (*offset == elf_sym_offset(sym))
+					continue;
+				/* handle multiple matches */
+				if (found->bind != STB_WEAK && bind != STB_WEAK) {
+					/* Only accept one non-weak bind. */
+					pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",
+						sym->name, found->name, binary_path);
+					err = -LIBBPF_ERRNO__FORMAT;
+					goto out;
+				} else if (bind == STB_WEAK) {
+					/* already have a non-weak bind, and
+					 * this is a weak bind, so ignore.
+					 */
+					continue;
+				}
+			} else {
+				cnt_done++;
+			}
+			*offset = elf_sym_offset(sym);
+			found->bind = bind;
+		}
+	}
+
+	if (cnt != cnt_done) {
+		err = -ENOENT;
+		goto out;
+	}
+
+	*poffsets = offsets;
+
+out:
+	free(symbols);
+	if (err)
+		free(offsets);
+	elf_close(&elf_fd);
+	return err;
+}
tools/lib/bpf/libbpf_elf.h +2 -0
index c763ac35a85e..026c7b378727 100644
@@ -16,4 +16,6 @@ void elf_close(struct elf_fd *elf_fd);
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
 long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
+int elf_resolve_syms_offsets(const char *binary_path, int cnt,
+			     const char **syms, unsigned long **poffsets);
 #endif /* *__LIBBPF_LIBBPF_ELF_H */
Signature · 1 line
2.41.0
#33 lore raw

Re: [PATCHv3 bpf-next 10/26] libbpf: Add elf_resolve_syms_offsets function

Quoted 22 lines · Jiri Olsa

Adding elf_resolve_syms_offsets function that looks up offsets for symbols specified in syms array argument.

Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller.

---
 tools/lib/bpf/elf.c        | 105 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_elf.h |   2 +
 2 files changed, 107 insertions(+)
tools/lib/bpf/elf.c +1 -0
index fcce4bd2478f..7e2f3b2e1fb6 100644
@@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)
        elf_close(&elf_fd);
        return ret;
 }
+

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 284

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+struct symbol {
+       const char *name;
+       int bind;
+       int idx;
+};
+
+static int symbol_cmp(const void *_a, const void *_b)
+{
+       const struct symbol *a = _a;
+       const struct symbol *b = _b;

please, let's not (over)use leading underscores, x/y, s1/s2, whatever

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 288

Quoted 4 lines
tools/lib/bpf/elf.c +4 -0
+
+       return strcmp(a->name, b->name);
+}
+

probably worth leaving a comment that the caller should free offsets on success?

Quoted 28 lines
tools/lib/bpf/elf.c +28 -0
+int elf_resolve_syms_offsets(const char *binary_path, int cnt,
+                            const char **syms, unsigned long **poffsets)
+{
+       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+       int err = 0, i, cnt_done = 0;
+       unsigned long *offsets;
+       struct symbol *symbols;
+       struct elf_fd elf_fd;
+
+       err = elf_open(binary_path, &elf_fd);
+       if (err)
+               return err;
+
+       offsets = calloc(cnt, sizeof(*offsets));
+       symbols = calloc(cnt, sizeof(*symbols));
+
+       if (!offsets || !symbols) {
+               err = -ENOMEM;
+               goto out;
+       }
+
+       for (i = 0; i < cnt; i++) {
+               symbols[i].name = syms[i];
+               symbols[i].idx = i;
+       }
+
+       qsort(symbols, cnt, sizeof(*symbols), symbol_cmp);
+

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 326

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+               struct elf_sym_iter iter;
+               struct elf_sym *sym;
+
+               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+               if (err) {
+                       if (err == -ENOENT)
+                               continue;
+                       goto out;
+               }

same nit, no need for nested ifs

Quoted 11 lines
tools/lib/bpf/elf.c +11 -0
+
+               while ((sym = elf_sym_iter_next(&iter))) {
+                       int bind = GELF_ST_BIND(sym->sym.st_info);
+                       struct symbol *found, tmp = {
+                               .name = sym->name,
+                       };
+                       unsigned long *offset;
+
+                       found = bsearch(&tmp, symbols, cnt, sizeof(*symbols), symbol_cmp);
+                       if (!found)
+                               continue;

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 347

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+
+                       offset = &offsets[found->idx];
+                       if (*offset > 0) {
+                               /* same offset, no problem */
+                               if (*offset == elf_sym_offset(sym))
+                                       continue;
+                               /* handle multiple matches */
+                               if (found->bind != STB_WEAK && bind != STB_WEAK) {
+                                       /* Only accept one non-weak bind. */
+                                       pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",

typo: found

but also wouldn't sym->name and found->name be always the same? Maybe log sym->name, previous *offset and newly calculated elf_sym_offset(sym) instead?

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 349

Quoted 2 lines
tools/lib/bpf/elf.c +2 -0
+                                               sym->name, found->name, binary_path);
+                                       err = -LIBBPF_ERRNO__FORMAT;

I'd minimize using those custom libbpf-only errors, why not -ESRCH here?

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 360

Quoted 11 lines
tools/lib/bpf/elf.c +11 -0
+                                       goto out;
+                               } else if (bind == STB_WEAK) {
+                                       /* already have a non-weak bind, and
+                                        * this is a weak bind, so ignore.
+                                        */
+                                       continue;
+                               }
+                       } else {
+                               cnt_done++;
+                       }
+                       *offset = elf_sym_offset(sym);

maybe remember elf_sym_offset() result in a variable? you are using it in two (and with my suggestion above it will be three) places already

Quoted 32 lines
tools/lib/bpf/elf.c +18 -0
+                       found->bind = bind;
+               }
+       }
+
+       if (cnt != cnt_done) {
+               err = -ENOENT;
+               goto out;
+       }
+
+       *poffsets = offsets;
+
+out:
+       free(symbols);
+       if (err)
+               free(offsets);
+       elf_close(&elf_fd);
+       return err;
+}
tools/lib/bpf/libbpf_elf.h +2 -1
index c763ac35a85e..026c7b378727 100644
@@ -16,4 +16,6 @@ void elf_close(struct elf_fd *elf_fd);
 long elf_find_func_offset(Elf *elf, const char *binary_path, const char *name);
 long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
+int elf_resolve_syms_offsets(const char *binary_path, int cnt,
+                            const char **syms, unsigned long **poffsets);
 #endif /* *__LIBBPF_LIBBPF_ELF_H */
--

2.41.0

#34 lore raw

Re: [PATCHv3 bpf-next 10/26] libbpf: Add elf_resolve_syms_offsets function

Quoted 27 lines · Andrii Nakryiko
Quoted 26 lines · Jiri Olsa

Adding elf_resolve_syms_offsets function that looks up offsets for symbols specified in syms array argument.

Offsets are returned in allocated array with the 'cnt' size, that needs to be released by the caller.

---
 tools/lib/bpf/elf.c        | 105 +++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_elf.h |   2 +
 2 files changed, 107 insertions(+)
tools/lib/bpf/elf.c +5 -0
index fcce4bd2478f..7e2f3b2e1fb6 100644
@@ -271,3 +271,108 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name)
        elf_close(&elf_fd);
        return ret;
 }
+
+struct symbol {
+       const char *name;
+       int bind;
+       int idx;
Quoted 8 lines

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 284

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+};
+
+static int symbol_cmp(const void *_a, const void *_b)
+{
+       const struct symbol *a = _a;
+       const struct symbol *b = _b;

please, let's not (over)use leading underscores, x/y, s1/s2, whatever

ok

Quoted 7 lines

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 288

Quoted 5 lines
tools/lib/bpf/elf.c +4 -0
+
+       return strcmp(a->name, b->name);
+}
+

probably worth leaving a comment that the caller should free offsets on success?

ook

Quoted 33 lines
Quoted 33 lines
tools/lib/bpf/elf.c +32 -0
+int elf_resolve_syms_offsets(const char *binary_path, int cnt,
+                            const char **syms, unsigned long **poffsets)
+{
+       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+       int err = 0, i, cnt_done = 0;
+       unsigned long *offsets;
+       struct symbol *symbols;
+       struct elf_fd elf_fd;
+
+       err = elf_open(binary_path, &elf_fd);
+       if (err)
+               return err;
+
+       offsets = calloc(cnt, sizeof(*offsets));
+       symbols = calloc(cnt, sizeof(*symbols));
+
+       if (!offsets || !symbols) {
+               err = -ENOMEM;
+               goto out;
+       }
+
+       for (i = 0; i < cnt; i++) {
+               symbols[i].name = syms[i];
+               symbols[i].idx = i;
+       }
+
+       qsort(symbols, cnt, sizeof(*symbols), symbol_cmp);
+
+       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+               struct elf_sym_iter iter;
+               struct elf_sym *sym;
+
Quoted 8 lines

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 326

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+               if (err) {
+                       if (err == -ENOENT)
+                               continue;
+                       goto out;
+               }

same nit, no need for nested ifs

ok

Quoted 24 lines
Quoted 11 lines
tools/lib/bpf/elf.c +11 -0
+
+               while ((sym = elf_sym_iter_next(&iter))) {
+                       int bind = GELF_ST_BIND(sym->sym.st_info);
+                       struct symbol *found, tmp = {
+                               .name = sym->name,
+                       };
+                       unsigned long *offset;
+
+                       found = bsearch(&tmp, symbols, cnt, sizeof(*symbols), symbol_cmp);
+                       if (!found)
+                               continue;

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 347

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+
+                       offset = &offsets[found->idx];
+                       if (*offset > 0) {
+                               /* same offset, no problem */
+                               if (*offset == elf_sym_offset(sym))
+                                       continue;
+                               /* handle multiple matches */
+                               if (found->bind != STB_WEAK && bind != STB_WEAK) {
+                                       /* Only accept one non-weak bind. */
+                                       pr_warn("elf: ambiguous match foundr '%s', '%s' in '%s'\n",

typo: found

Quoted 3 lines

but also wouldn't sym->name and found->name be always the same? Maybe log sym->name, previous *offset and newly calculated elf_sym_offset(sym) instead?

ok

Quoted 5 lines

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 349

Quoted 3 lines
tools/lib/bpf/elf.c +2 -0
+                                               sym->name, found->name, binary_path);
+                                       err = -LIBBPF_ERRNO__FORMAT;

I'd minimize using those custom libbpf-only errors, why not -ESRCH here?

ok

Quoted 13 lines

tools/lib/bpf/elf.c long elf_find_func_offset_from_file(const char *binary_path, const char *name) · line 360

Quoted 12 lines
tools/lib/bpf/elf.c +11 -0
+                                       goto out;
+                               } else if (bind == STB_WEAK) {
+                                       /* already have a non-weak bind, and
+                                        * this is a weak bind, so ignore.
+                                        */
+                                       continue;
+                               }
+                       } else {
+                               cnt_done++;
+                       }
+                       *offset = elf_sym_offset(sym);
Quoted 2 lines

maybe remember elf_sym_offset() result in a variable? you are using it in two (and with my suggestion above it will be three) places already

ok

jirka

author reviewer
#35 lore raw

PATCH v3 11/26 libbpf: Add elf_resolve_pattern_offsets function

In reply to Jiri Olsa

Adding elf_resolve_pattern_offsets function that looks up offsets for symbols specified by pattern argument.

The 'pattern' argument allows wildcards (*?' supported).

Offsets are returned in allocated array together with its size and needs to be released by the caller.

---
 tools/lib/bpf/elf.c             | 57 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c          |  2 +-
 tools/lib/bpf/libbpf_elf.h      |  3 ++
 tools/lib/bpf/libbpf_internal.h |  1 +
 4 files changed, 62 insertions(+), 1 deletion(-)
tools/lib/bpf/elf.c +57 -0
index 7e2f3b2e1fb6..f2d1a8cc2f9d 100644
@@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,
 	elf_close(&elf_fd);
 	return err;
 }
+
+int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
+				unsigned long **poffsets, size_t *pcnt)
+{
+	int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+	unsigned long *offsets = NULL;
+	size_t cap = 0, cnt = 0;
+	struct elf_fd elf_fd;
+	int err = 0, i;
+
+	err = elf_open(binary_path, &elf_fd);
+	if (err)
+		return err;
+
+	for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+		struct elf_sym_iter iter;
+		struct elf_sym *sym;
+
+		err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+		if (err) {
+			if (err == -ENOENT)
+				continue;
+			goto out;
+		}
+
+		while ((sym = elf_sym_iter_next(&iter))) {
+			if (!glob_match(sym->name, pattern))
+				continue;
+
+			err = libbpf_ensure_mem((void **) &offsets, &cap, sizeof(*offsets),
+						cnt + 1);
+			if (err)
+				goto out;
+
+			offsets[cnt++] = elf_sym_offset(sym);
+		}
+
+		/* If we found anything in the first symbol section,
+		 * do not search others to avoid duplicates.
+		 */
+		if (cnt)
+			break;
+	}
+
+	if (cnt) {
+		*poffsets = offsets;
+		*pcnt = cnt;
+	} else {
+		err = -ENOENT;
+	}
+
+out:
+	if (err)
+		free(offsets);
+	elf_close(&elf_fd);
+	return err;
+}
tools/lib/bpf/libbpf.c +1 -1
index 093add8124d8..f33ef7cb1adc 100644
@@ -10509,7 +10509,7 @@ struct bpf_link *bpf_program__attach_ksyscall(const struct bpf_program *prog,
 }
 
 /* Adapted from perf/util/string.c */
-static bool glob_match(const char *str, const char *pat)
+bool glob_match(const char *str, const char *pat)
 {
 	while (*str && *pat && *pat != '*') {
 		if (*pat == '?') {      /* Matches any single character */
tools/lib/bpf/libbpf_elf.h +3 -0
index 026c7b378727..0c75d3b2398b 100644
@@ -18,4 +18,7 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
 int elf_resolve_syms_offsets(const char *binary_path, int cnt,
 			     const char **syms, unsigned long **poffsets);
+
+int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
+				 unsigned long **poffsets, size_t *pcnt);
 #endif /* *__LIBBPF_LIBBPF_ELF_H */
tools/lib/bpf/libbpf_internal.h +1 -0
index e4d05662a96c..7d75b92e531a 100644
@@ -577,4 +577,5 @@ static inline bool is_pow_of_2(size_t x)
 #define PROG_LOAD_ATTEMPTS 5
 int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts);
 
+bool glob_match(const char *str, const char *pat);
 #endif /* __LIBBPF_LIBBPF_INTERNAL_H */
Signature · 1 line
2.41.0
#36 lore raw

Re: [PATCHv3 bpf-next 11/26] libbpf: Add elf_resolve_pattern_offsets function

Quoted 22 lines · Jiri Olsa

Adding elf_resolve_pattern_offsets function that looks up offsets for symbols specified by pattern argument.

The 'pattern' argument allows wildcards (*?' supported).

Offsets are returned in allocated array together with its size and needs to be released by the caller.

---
 tools/lib/bpf/elf.c             | 57 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c          |  2 +-
 tools/lib/bpf/libbpf_elf.h      |  3 ++
 tools/lib/bpf/libbpf_internal.h |  1 +
 4 files changed, 62 insertions(+), 1 deletion(-)
tools/lib/bpf/elf.c +0 -0
index 7e2f3b2e1fb6..f2d1a8cc2f9d 100644

tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 379

Quoted 4 lines
tools/lib/bpf/elf.c +1 -0
@@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,
        elf_close(&elf_fd);
        return err;
 }
+

same, leave comment that caller should free offsets on success?

Quoted 13 lines
tools/lib/bpf/elf.c +13 -0
+int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
+                               unsigned long **poffsets, size_t *pcnt)
+{
+       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+       unsigned long *offsets = NULL;
+       size_t cap = 0, cnt = 0;
+       struct elf_fd elf_fd;
+       int err = 0, i;
+
+       err = elf_open(binary_path, &elf_fd);
+       if (err)
+               return err;
+

tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 402

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+               struct elf_sym_iter iter;
+               struct elf_sym *sym;
+
+               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+               if (err) {
+                       if (err == -ENOENT)
+                               continue;
+                       goto out;
+               }

ditto, minimize nesting, please

Quoted 5 lines
tools/lib/bpf/elf.c +5 -0
+
+               while ((sym = elf_sym_iter_next(&iter))) {
+                       if (!glob_match(sym->name, pattern))
+                               continue;
+

tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 417

Quoted 10 lines
tools/lib/bpf/elf.c +10 -0
+                       err = libbpf_ensure_mem((void **) &offsets, &cap, sizeof(*offsets),
+                                               cnt + 1);
+                       if (err)
+                               goto out;
+
+                       offsets[cnt++] = elf_sym_offset(sym);
+               }
+
+               /* If we found anything in the first symbol section,
+                * do not search others to avoid duplicates.

DYNSYM is going to have only exposed symbols, so for this pattern matching, maybe it's best to start with SYMTAB and only fallback to DYNSYM if we didn't find anything in SYMTAB (more realistically it would be that SYMTAB section is missing, so we fallback to DYNSYM; otherwise neither DYNSYM nor SYMTAB will have matching symbols, most probably, but that's minor)

other than that, LGTM

Quoted 56 lines
tools/lib/bpf/elf.c +18 -0
+                */
+               if (cnt)
+                       break;
+       }
+
+       if (cnt) {
+               *poffsets = offsets;
+               *pcnt = cnt;
+       } else {
+               err = -ENOENT;
+       }
+
+out:
+       if (err)
+               free(offsets);
+       elf_close(&elf_fd);
+       return err;
+}
tools/lib/bpf/libbpf.c +1 -1
index 093add8124d8..f33ef7cb1adc 100644
@@ -10509,7 +10509,7 @@ struct bpf_link *bpf_program__attach_ksyscall(const struct bpf_program *prog,
 }
 
 /* Adapted from perf/util/string.c */
-static bool glob_match(const char *str, const char *pat)
+bool glob_match(const char *str, const char *pat)
 {
        while (*str && *pat && *pat != '*') {
                if (*pat == '?') {      /* Matches any single character */
tools/lib/bpf/libbpf_elf.h +3 -0
index 026c7b378727..0c75d3b2398b 100644
@@ -18,4 +18,7 @@ long elf_find_func_offset_from_file(const char *binary_path, const char *name);
 
 int elf_resolve_syms_offsets(const char *binary_path, int cnt,
                             const char **syms, unsigned long **poffsets);
+
+int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
+                                unsigned long **poffsets, size_t *pcnt);
 #endif /* *__LIBBPF_LIBBPF_ELF_H */
tools/lib/bpf/libbpf_internal.h +1 -1
index e4d05662a96c..7d75b92e531a 100644
@@ -577,4 +577,5 @@ static inline bool is_pow_of_2(size_t x)
 #define PROG_LOAD_ATTEMPTS 5
 int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts);
 
+bool glob_match(const char *str, const char *pat);
 #endif /* __LIBBPF_LIBBPF_INTERNAL_H */
--

2.41.0

#37 lore raw

Re: [PATCHv3 bpf-next 11/26] libbpf: Add elf_resolve_pattern_offsets function

Quoted 18 lines · Andrii Nakryiko
Quoted 17 lines · Jiri Olsa

Adding elf_resolve_pattern_offsets function that looks up offsets for symbols specified by pattern argument.

The 'pattern' argument allows wildcards (*?' supported).

Offsets are returned in allocated array together with its size and needs to be released by the caller.

---
 tools/lib/bpf/elf.c             | 57 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c          |  2 +-
 tools/lib/bpf/libbpf_elf.h      |  3 ++
 tools/lib/bpf/libbpf_internal.h |  1 +
 4 files changed, 62 insertions(+), 1 deletion(-)
Quoted 11 lines

tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 379

Quoted 9 lines
tools/lib/bpf/elf.c +1 -0
index 7e2f3b2e1fb6..f2d1a8cc2f9d 100644
@@ -376,3 +376,60 @@ int elf_resolve_syms_offsets(const char *binary_path, int cnt,
        elf_close(&elf_fd);
        return err;
 }
+

same, leave comment that caller should free offsets on success?

yes, will add

Quoted 18 lines
Quoted 18 lines
tools/lib/bpf/elf.c +17 -0
+int elf_resolve_pattern_offsets(const char *binary_path, const char *pattern,
+                               unsigned long **poffsets, size_t *pcnt)
+{
+       int sh_types[2] = { SHT_DYNSYM, SHT_SYMTAB };
+       unsigned long *offsets = NULL;
+       size_t cap = 0, cnt = 0;
+       struct elf_fd elf_fd;
+       int err = 0, i;
+
+       err = elf_open(binary_path, &elf_fd);
+       if (err)
+               return err;
+
+       for (i = 0; i < ARRAY_SIZE(sh_types); i++) {
+               struct elf_sym_iter iter;
+               struct elf_sym *sym;
+
Quoted 8 lines

tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 402

Quoted 6 lines
tools/lib/bpf/elf.c +6 -0
+               err = elf_sym_iter_new(&iter, elf_fd.elf, binary_path, sh_types[i], STT_FUNC);
+               if (err) {
+                       if (err == -ENOENT)
+                               continue;
+                       goto out;
+               }

ditto, minimize nesting, please

ok

Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/elf.c +14 -0
+
+               while ((sym = elf_sym_iter_next(&iter))) {
+                       if (!glob_match(sym->name, pattern))
+                               continue;
+
+                       err = libbpf_ensure_mem((void **) &offsets, &cap, sizeof(*offsets),
+                                               cnt + 1);
+                       if (err)
+                               goto out;
+
+                       offsets[cnt++] = elf_sym_offset(sym);
+               }
+
+               /* If we found anything in the first symbol section,
Quoted 8 lines

tools/lib/bpf/elf.c int elf_resolve_syms_offsets(const char *binary_path, int cnt, · line 417

Quoted 1 line
tools/lib/bpf/elf.c +1 -0
+                * do not search others to avoid duplicates.

DYNSYM is going to have only exposed symbols, so for this pattern matching, maybe it's best to start with SYMTAB and only fallback to DYNSYM if we didn't find anything in SYMTAB (more realistically it would be that SYMTAB section is missing, so we fallback to DYNSYM; otherwise neither DYNSYM nor SYMTAB will have matching symbols, most probably, but that's minor)

makes sense, will switch

Quoted 2 lines

other than that, LGTM

thanks,
jirka

author reviewer
#38 lore raw

PATCH v3 12/26 libbpf: Add bpf_link_create support for multi uprobes

In reply to Jiri Olsa

Adding new uprobe_multi struct to bpf_link_create_opts object to pass multiple uprobe data to link_create attr uapi.

---
 tools/lib/bpf/bpf.c | 11 +++++++++++
 tools/lib/bpf/bpf.h | 11 ++++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)
tools/lib/bpf/bpf.c +11 -0
index ed86b37d8024..0fd35c91f50c 100644
@@ -733,6 +733,17 @@ int bpf_link_create(int prog_fd, int target_fd,
 		if (!OPTS_ZEROED(opts, kprobe_multi))
 			return libbpf_err(-EINVAL);
 		break;
+	case BPF_TRACE_UPROBE_MULTI:
+		attr.link_create.uprobe_multi.flags = OPTS_GET(opts, uprobe_multi.flags, 0);
+		attr.link_create.uprobe_multi.cnt = OPTS_GET(opts, uprobe_multi.cnt, 0);
+		attr.link_create.uprobe_multi.path = ptr_to_u64(OPTS_GET(opts, uprobe_multi.path, 0));
+		attr.link_create.uprobe_multi.offsets = ptr_to_u64(OPTS_GET(opts, uprobe_multi.offsets, 0));
+		attr.link_create.uprobe_multi.ref_ctr_offsets = ptr_to_u64(OPTS_GET(opts, uprobe_multi.ref_ctr_offsets, 0));
+		attr.link_create.uprobe_multi.cookies = ptr_to_u64(OPTS_GET(opts, uprobe_multi.cookies, 0));
+		attr.link_create.uprobe_multi.pid = OPTS_GET(opts, uprobe_multi.pid, 0);
+		if (!OPTS_ZEROED(opts, uprobe_multi))
+			return libbpf_err(-EINVAL);
+		break;
 	case BPF_TRACE_FENTRY:
 	case BPF_TRACE_FEXIT:
 	case BPF_MODIFY_RETURN:
tools/lib/bpf/bpf.h +10 -1
index 9aa0ee473754..82979b4f2769 100644
@@ -346,13 +346,22 @@ struct bpf_link_create_opts {
 			const unsigned long *addrs;
 			const __u64 *cookies;
 		} kprobe_multi;
+		struct {
+			__u32 flags;
+			__u32 cnt;
+			const char *path;
+			const unsigned long *offsets;
+			const unsigned long *ref_ctr_offsets;
+			const __u64 *cookies;
+			__u32 pid;
+		} uprobe_multi;
 		struct {
 			__u64 cookie;
 		} tracing;
 	};
 	size_t :0;
 };
-#define bpf_link_create_opts__last_field kprobe_multi.cookies
+#define bpf_link_create_opts__last_field uprobe_multi.pid
 
 LIBBPF_API int bpf_link_create(int prog_fd, int target_fd,
 			       enum bpf_attach_type attach_type,
Signature · 1 line
2.41.0
author reviewer
#39 lore raw

PATCH v3 13/26 libbpf: Add bpf_program__attach_uprobe_multi function

In reply to Jiri Olsa

Adding bpf_program__attach_uprobe_multi function that allows to attach multiple uprobes with uprobe_multi link.

The user can specify uprobes with direct arguments:

binary_path/func_pattern/pid

or with struct bpf_uprobe_multi_opts opts argument fields:

const char **syms;
const unsigned long *offsets;
const unsigned long *ref_ctr_offsets;
const __u64 *cookies;

User can specify 2 mutually exclusive set of inputs:

1) use only path/func_pattern/pid arguments

2) use path/pid with allowed combinations of:
syms/offsets/ref_ctr_offsets/cookies/cnt

- syms and offsets are mutually exclusive
- ref_ctr_offsets and cookies are optional

Any other usage results in error.

---
 tools/lib/bpf/libbpf.c   | 122 +++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h   |  27 +++++++++
 tools/lib/bpf/libbpf.map |   1 +
 3 files changed, 150 insertions(+)
tools/lib/bpf/libbpf.c +122 -0
index f33ef7cb1adc..b942f248038e 100644
@@ -10954,6 +10954,128 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz)
 	return -ENOENT;
 }
 
+struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+				 pid_t pid,
+				 const char *path,
+				 const char *func_pattern,
+				 const struct bpf_uprobe_multi_opts *opts)
+{
+	const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL;
+	LIBBPF_OPTS(bpf_link_create_opts, lopts);
+	unsigned long *resolved_offsets = NULL;
+	int err = 0, link_fd, prog_fd;
+	struct bpf_link *link = NULL;
+	char errmsg[STRERR_BUFSIZE];
+	char full_path[PATH_MAX];
+	const __u64 *cookies;
+	const char **syms;
+	bool has_pattern;
+	bool retprobe;
+	size_t cnt;
+
+	if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
+		return libbpf_err_ptr(-EINVAL);
+
+	syms = OPTS_GET(opts, syms, NULL);
+	offsets = OPTS_GET(opts, offsets, NULL);
+	ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
+	cookies = OPTS_GET(opts, cookies, NULL);
+	cnt = OPTS_GET(opts, cnt, 0);
+
+	/*
+	 * User can specify 2 mutually exclusive set of inputs:
+	 *
+	 * 1) use only path/func_pattern/pid arguments
+	 *
+	 * 2) use path/pid with allowed combinations of:
+	 *    syms/offsets/ref_ctr_offsets/cookies/cnt
+	 *
+	 *    - syms and offsets are mutually exclusive
+	 *    - ref_ctr_offsets and cookies are optional
+	 *
+	 * Any other usage results in error.
+	 */
+
+	if (!path && !func_pattern && !cnt)
+		return libbpf_err_ptr(-EINVAL);
+	if (func_pattern && !path)
+		return libbpf_err_ptr(-EINVAL);
+
+	has_pattern = path && func_pattern;
+
+	if (has_pattern) {
+		if (syms || offsets || ref_ctr_offsets || cookies || cnt)
+			return libbpf_err_ptr(-EINVAL);
+	} else {
+		if (!cnt)
+			return libbpf_err_ptr(-EINVAL);
+		if (!!syms == !!offsets)
+			return libbpf_err_ptr(-EINVAL);
+	}
+
+	if (has_pattern) {
+		if (!strchr(path, '/')) {
+			err = resolve_full_path(path, full_path, sizeof(full_path));
+			if (err) {
+				pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
+					prog->name, path, err);
+				return libbpf_err_ptr(err);
+			}
+			path = full_path;
+		}
+
+		err = elf_resolve_pattern_offsets(path, func_pattern,
+						  &resolved_offsets, &cnt);
+		if (err < 0)
+			return libbpf_err_ptr(err);
+		offsets = resolved_offsets;
+	} else if (syms) {
+		err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
+		if (err < 0)
+			return libbpf_err_ptr(err);
+		offsets = resolved_offsets;
+	}
+
+	retprobe = OPTS_GET(opts, retprobe, false);
+
+	lopts.uprobe_multi.path = path;
+	lopts.uprobe_multi.offsets = offsets;
+	lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
+	lopts.uprobe_multi.cookies = cookies;
+	lopts.uprobe_multi.cnt = cnt;
+	lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;
+
+	if (pid == 0)
+		pid = getpid();
+	if (pid > 0)
+		lopts.uprobe_multi.pid = pid;
+
+	link = calloc(1, sizeof(*link));
+	if (!link) {
+		err = -ENOMEM;
+		goto error;
+	}
+	link->detach = &bpf_link__detach_fd;
+
+	prog_fd = bpf_program__fd(prog);
+	link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
+	if (link_fd < 0) {
+		err = -errno;
+		pr_warn("prog '%s': failed to attach: %s\n",
+			prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
+		goto error;
+	}
+	link->fd = link_fd;
+	free(resolved_offsets);
+	return link;
+
+error:
+	free(resolved_offsets);
+	free(link);
+	return libbpf_err_ptr(err);
+}
+
 LIBBPF_API struct bpf_link *
 bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
 				const char *binary_path, size_t func_offset,
tools/lib/bpf/libbpf.h +27 -0
index 754da73c643b..7c218f610210 100644
@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
 				      const char *pattern,
 				      const struct bpf_kprobe_multi_opts *opts);
 
+struct bpf_uprobe_multi_opts {
+	/* size of this struct, for forward/backward compatibility */
+	size_t sz;
+	/* array of function symbols to attach */
+	const char **syms;
+	/* array of function addresses to attach */
+	const unsigned long *offsets;
+	/* array of refctr offsets to attach */
+	const unsigned long *ref_ctr_offsets;
+	/* array of user-provided values fetchable through bpf_get_attach_cookie */
+	const __u64 *cookies;
+	/* number of elements in syms/addrs/cookies arrays */
+	size_t cnt;
+	/* create return uprobes */
+	bool retprobe;
+	size_t :0;
+};
+
+#define bpf_uprobe_multi_opts__last_field retprobe
+
+LIBBPF_API struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+				 pid_t pid,
+				 const char *binary_path,
+				 const char *func_pattern,
+				 const struct bpf_uprobe_multi_opts *opts);
+
 struct bpf_ksyscall_opts {
 	/* size of this struct, for forward/backward compatibility */
 	size_t sz;
tools/lib/bpf/libbpf.map +1 -0
index 7521a2fb7626..d8d11ea6c35e 100644
@@ -395,4 +395,5 @@ LIBBPF_1.2.0 {
 LIBBPF_1.3.0 {
 	global:
 		bpf_obj_pin_opts;
+		bpf_program__attach_uprobe_multi;
 } LIBBPF_1.2.0;
Signature · 1 line
2.41.0
#40 lore raw

Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function

Quoted 76 lines · Jiri Olsa

Adding bpf_program__attach_uprobe_multi function that allows to attach multiple uprobes with uprobe_multi link.

The user can specify uprobes with direct arguments:

binary_path/func_pattern/pid

or with struct bpf_uprobe_multi_opts opts argument fields:

const char **syms;
const unsigned long *offsets;
const unsigned long *ref_ctr_offsets;
const __u64 *cookies;

User can specify 2 mutually exclusive set of inputs:

1) use only path/func_pattern/pid arguments

2) use path/pid with allowed combinations of:
syms/offsets/ref_ctr_offsets/cookies/cnt

- syms and offsets are mutually exclusive
- ref_ctr_offsets and cookies are optional

Any other usage results in error.

---
 tools/lib/bpf/libbpf.c   | 122 +++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h   |  27 +++++++++
 tools/lib/bpf/libbpf.map |   1 +
 3 files changed, 150 insertions(+)
tools/lib/bpf/libbpf.c +34 -0
index f33ef7cb1adc..b942f248038e 100644
@@ -10954,6 +10954,128 @@ static int resolve_full_path(const char *file, char *result, size_t result_sz)
        return -ENOENT;
 }
 
+struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+                                pid_t pid,
+                                const char *path,
+                                const char *func_pattern,
+                                const struct bpf_uprobe_multi_opts *opts)
+{
+       const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL;
+       LIBBPF_OPTS(bpf_link_create_opts, lopts);
+       unsigned long *resolved_offsets = NULL;
+       int err = 0, link_fd, prog_fd;
+       struct bpf_link *link = NULL;
+       char errmsg[STRERR_BUFSIZE];
+       char full_path[PATH_MAX];
+       const __u64 *cookies;
+       const char **syms;
+       bool has_pattern;
+       bool retprobe;
+       size_t cnt;
+
+       if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
+               return libbpf_err_ptr(-EINVAL);
+
+       syms = OPTS_GET(opts, syms, NULL);
+       offsets = OPTS_GET(opts, offsets, NULL);
+       ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
+       cookies = OPTS_GET(opts, cookies, NULL);
+       cnt = OPTS_GET(opts, cnt, 0);
+
+       /*
+        * User can specify 2 mutually exclusive set of inputs:
+        *
+        * 1) use only path/func_pattern/pid arguments
+        *

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11000

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+        * 2) use path/pid with allowed combinations of:
+        *    syms/offsets/ref_ctr_offsets/cookies/cnt
+        *
+        *    - syms and offsets are mutually exclusive
+        *    - ref_ctr_offsets and cookies are optional
+        *
+        * Any other usage results in error.
+        */
+
+       if (!path && !func_pattern && !cnt)

weird, I'd expect separate if (!path) return error (already bad, regardless of func_pattern or cnt)

then if (!func_pattern && cnt == 0) return error

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11005

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+               return libbpf_err_ptr(-EINVAL);
+       if (func_pattern && !path)
+               return libbpf_err_ptr(-EINVAL);
+
+       has_pattern = path && func_pattern;

this and above check must be some leftovers from previous version. path should always be present. and so you don't need has_pattern variable, just use "func_pattern" check

Quoted 22 lines
tools/lib/bpf/libbpf.c +22 -0
+
+       if (has_pattern) {
+               if (syms || offsets || ref_ctr_offsets || cookies || cnt)
+                       return libbpf_err_ptr(-EINVAL);
+       } else {
+               if (!cnt)
+                       return libbpf_err_ptr(-EINVAL);
+               if (!!syms == !!offsets)
+                       return libbpf_err_ptr(-EINVAL);
+       }
+
+       if (has_pattern) {
+               if (!strchr(path, '/')) {
+                       err = resolve_full_path(path, full_path, sizeof(full_path));
+                       if (err) {
+                               pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
+                                       prog->name, path, err);
+                               return libbpf_err_ptr(err);
+                       }
+                       path = full_path;
+               }
+

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11037

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+               err = elf_resolve_pattern_offsets(path, func_pattern,
+                                                 &resolved_offsets, &cnt);
+               if (err < 0)
+                       return libbpf_err_ptr(err);
+               offsets = resolved_offsets;
+       } else if (syms) {
+               err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
+               if (err < 0)
+                       return libbpf_err_ptr(err);
+               offsets = resolved_offsets;

you can extract this common error checking and offsets = resolved_offsets; to after if, it's common for both branches

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11047

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+       }
+
+       retprobe = OPTS_GET(opts, retprobe, false);
+
+       lopts.uprobe_multi.path = path;
+       lopts.uprobe_multi.offsets = offsets;
+       lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
+       lopts.uprobe_multi.cookies = cookies;
+       lopts.uprobe_multi.cnt = cnt;
+       lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;

retprobe is another unnecessary var, just inline check here to keep it simpler

Quoted 8 lines
tools/lib/bpf/libbpf.c +8 -0
+
+       if (pid == 0)
+               pid = getpid();
+       if (pid > 0)
+               lopts.uprobe_multi.pid = pid;
+
+       link = calloc(1, sizeof(*link));
+       if (!link) {

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11065

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+               err = -ENOMEM;
+               goto error;
+       }
+       link->detach = &bpf_link__detach_fd;
+
+       prog_fd = bpf_program__fd(prog);
+       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
+       if (link_fd < 0) {
+               err = -errno;
+               pr_warn("prog '%s': failed to attach: %s\n",

"failed to attach multi-uprobe"? We probably should have added "failed to attach multi-kprobe" in bpf_program__attach_kprobe_multi_opts as well?

Quoted 21 lines
tools/lib/bpf/libbpf.c +13 -0
+                       prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
+               goto error;
+       }
+       link->fd = link_fd;
+       free(resolved_offsets);
+       return link;
+
+error:
+       free(resolved_offsets);
+       free(link);
+       return libbpf_err_ptr(err);
+}
+
 LIBBPF_API struct bpf_link *
 bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
                                const char *binary_path, size_t func_offset,
tools/lib/bpf/libbpf.h +0 -0
index 754da73c643b..7c218f610210 100644

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 535

Quoted 7 lines
tools/lib/bpf/libbpf.h +4 -0
@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
                                      const char *pattern,
                                      const struct bpf_kprobe_multi_opts *opts);
 
+struct bpf_uprobe_multi_opts {
+       /* size of this struct, for forward/backward compatibility */
+       size_t sz;
+       /* array of function symbols to attach */

attach to?

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 537

Quoted 2 lines
tools/lib/bpf/libbpf.h +2 -0
+       const char **syms;
+       /* array of function addresses to attach */

attach to?

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 539

Quoted 2 lines
tools/lib/bpf/libbpf.h +2 -0
+       const unsigned long *offsets;
+       /* array of refctr offsets to attach */

we don't really attach to ref counters, so maybe "optional, array of associated ref counter offsets" or something along those lines ?

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 541

Quoted 2 lines
tools/lib/bpf/libbpf.h +2 -0
+       const unsigned long *ref_ctr_offsets;
+       /* array of user-provided values fetchable through bpf_get_attach_cookie */

"array of associated BPF cookies"? we can't keep explaining what BPF cookie is in every possible API :)

Quoted 7 lines
tools/lib/bpf/libbpf.h +7 -0
+       const __u64 *cookies;
+       /* number of elements in syms/addrs/cookies arrays */
+       size_t cnt;
+       /* create return uprobes */
+       bool retprobe;
+       size_t :0;
+};

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 558

Quoted 10 lines
tools/lib/bpf/libbpf.h +10 -0
+
+#define bpf_uprobe_multi_opts__last_field retprobe
+
+LIBBPF_API struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+                                pid_t pid,
+                                const char *binary_path,
+                                const char *func_pattern,
+                                const struct bpf_uprobe_multi_opts *opts);
+

ok, let's be good citizens and add documentation for this new API. Those comments about valid combinations belong here as well. Please take a look at existing doccomments for the format and conventions. Thanks!

Quoted 16 lines
tools/lib/bpf/libbpf.h +0 -0
 struct bpf_ksyscall_opts {
        /* size of this struct, for forward/backward compatibility */
        size_t sz;
tools/lib/bpf/libbpf.map +1 -1
index 7521a2fb7626..d8d11ea6c35e 100644
@@ -395,4 +395,5 @@ LIBBPF_1.2.0 {
 LIBBPF_1.3.0 {
        global:
                bpf_obj_pin_opts;
+               bpf_program__attach_uprobe_multi;
 } LIBBPF_1.2.0;
--

2.41.0

#41 lore raw

Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function

On Thu, Jul 06, 2023 at 09:05:23PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 36 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14 -0
+       if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
+               return libbpf_err_ptr(-EINVAL);
+
+       syms = OPTS_GET(opts, syms, NULL);
+       offsets = OPTS_GET(opts, offsets, NULL);
+       ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
+       cookies = OPTS_GET(opts, cookies, NULL);
+       cnt = OPTS_GET(opts, cnt, 0);
+
+       /*
+        * User can specify 2 mutually exclusive set of inputs:
+        *
+        * 1) use only path/func_pattern/pid arguments
+        *

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11000

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+        * 2) use path/pid with allowed combinations of:
+        *    syms/offsets/ref_ctr_offsets/cookies/cnt
+        *
+        *    - syms and offsets are mutually exclusive
+        *    - ref_ctr_offsets and cookies are optional
+        *
+        * Any other usage results in error.
+        */
+
+       if (!path && !func_pattern && !cnt)

weird, I'd expect separate if (!path) return error (already bad, regardless of func_pattern or cnt)

then if (!func_pattern && cnt == 0) return error

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11005

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+               return libbpf_err_ptr(-EINVAL);
+       if (func_pattern && !path)
+               return libbpf_err_ptr(-EINVAL);
+
+       has_pattern = path && func_pattern;
Quoted 3 lines

this and above check must be some leftovers from previous version. path should always be present. and so you don't need has_pattern variable, just use "func_pattern" check

hum, right, previous version had 2 paths, now there's just one, I'll change that together with the suggested change above

Quoted 28 lines
Quoted 28 lines
tools/lib/bpf/libbpf.c +27 -0
+
+       if (has_pattern) {
+               if (syms || offsets || ref_ctr_offsets || cookies || cnt)
+                       return libbpf_err_ptr(-EINVAL);
+       } else {
+               if (!cnt)
+                       return libbpf_err_ptr(-EINVAL);
+               if (!!syms == !!offsets)
+                       return libbpf_err_ptr(-EINVAL);
+       }
+
+       if (has_pattern) {
+               if (!strchr(path, '/')) {
+                       err = resolve_full_path(path, full_path, sizeof(full_path));
+                       if (err) {
+                               pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
+                                       prog->name, path, err);
+                               return libbpf_err_ptr(err);
+                       }
+                       path = full_path;
+               }
+
+               err = elf_resolve_pattern_offsets(path, func_pattern,
+                                                 &resolved_offsets, &cnt);
+               if (err < 0)
+                       return libbpf_err_ptr(err);
+               offsets = resolved_offsets;
Quoted 8 lines

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11037

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+       } else if (syms) {
+               err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
+               if (err < 0)
+                       return libbpf_err_ptr(err);
+               offsets = resolved_offsets;

you can extract this common error checking and offsets = resolved_offsets; to after if, it's common for both branches

not sure what you mean in here, offsets can be also provided by OPTS_GET(opts, offsets, NULL) earlier

Quoted 4 lines
Quoted 4 lines
tools/lib/bpf/libbpf.c +4 -0
+       }
+
+       retprobe = OPTS_GET(opts, retprobe, false);
+
Quoted 8 lines

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11047

Quoted 6 lines
tools/lib/bpf/libbpf.c +6 -0
+       lopts.uprobe_multi.path = path;
+       lopts.uprobe_multi.offsets = offsets;
+       lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
+       lopts.uprobe_multi.cookies = cookies;
+       lopts.uprobe_multi.cnt = cnt;
+       lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;

retprobe is another unnecessary var, just inline check here to keep it simpler

ok

Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.c +14 -0
+
+       if (pid == 0)
+               pid = getpid();
+       if (pid > 0)
+               lopts.uprobe_multi.pid = pid;
+
+       link = calloc(1, sizeof(*link));
+       if (!link) {
+               err = -ENOMEM;
+               goto error;
+       }
+       link->detach = &bpf_link__detach_fd;
+
+       prog_fd = bpf_program__fd(prog);
Quoted 8 lines

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11065

Quoted 4 lines
tools/lib/bpf/libbpf.c +4 -0
+       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
+       if (link_fd < 0) {
+               err = -errno;
+               pr_warn("prog '%s': failed to attach: %s\n",

"failed to attach multi-uprobe"? We probably should have added "failed to attach multi-kprobe" in bpf_program__attach_kprobe_multi_opts as well?

ook, will add

Quoted 25 lines
Quoted 25 lines
tools/lib/bpf/libbpf.c +13 -0
+                       prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
+               goto error;
+       }
+       link->fd = link_fd;
+       free(resolved_offsets);
+       return link;
+
+error:
+       free(resolved_offsets);
+       free(link);
+       return libbpf_err_ptr(err);
+}
+
 LIBBPF_API struct bpf_link *
 bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
                                const char *binary_path, size_t func_offset,
tools/lib/bpf/libbpf.h +0 -0
index 754da73c643b..7c218f610210 100644
@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
                                      const char *pattern,
                                      const struct bpf_kprobe_multi_opts *opts);
 
Quoted 6 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 535

Quoted 4 lines
tools/lib/bpf/libbpf.h +4 -0
+struct bpf_uprobe_multi_opts {
+       /* size of this struct, for forward/backward compatibility */
+       size_t sz;
+       /* array of function symbols to attach */

attach to?

ok

Quoted 5 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 537

Quoted 3 lines
tools/lib/bpf/libbpf.h +2 -0
+       const char **syms;
+       /* array of function addresses to attach */

attach to?

ook

Quoted 6 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 539

Quoted 3 lines
tools/lib/bpf/libbpf.h +2 -0
+       const unsigned long *offsets;
+       /* array of refctr offsets to attach */

we don't really attach to ref counters, so maybe "optional, array of associated ref counter offsets" or something along those lines ?

ok

Quoted 6 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 541

Quoted 3 lines
tools/lib/bpf/libbpf.h +2 -0
+       const unsigned long *ref_ctr_offsets;
+       /* array of user-provided values fetchable through bpf_get_attach_cookie */

"array of associated BPF cookies"? we can't keep explaining what BPF cookie is in every possible API :)

ook

Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.h +14 -0
+       const __u64 *cookies;
+       /* number of elements in syms/addrs/cookies arrays */
+       size_t cnt;
+       /* create return uprobes */
+       bool retprobe;
+       size_t :0;
+};
+
+#define bpf_uprobe_multi_opts__last_field retprobe
+
+LIBBPF_API struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+                                pid_t pid,
+                                const char *binary_path,
Quoted 8 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 558

Quoted 3 lines
tools/lib/bpf/libbpf.h +3 -0
+                                const char *func_pattern,
+                                const struct bpf_uprobe_multi_opts *opts);
+

ok, let's be good citizens and add documentation for this new API. Those comments about valid combinations belong here as well. Please take a look at existing doccomments for the format and conventions. Thanks!

ok, will add

thanks,
jirka

#42 lore raw

Re: [PATCHv3 bpf-next 13/26] libbpf: Add bpf_program__attach_uprobe_multi function

Quoted 85 lines · Jiri Olsa

On Thu, Jul 06, 2023 at 09:05:23PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 36 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14 -0
+       if (!OPTS_VALID(opts, bpf_uprobe_multi_opts))
+               return libbpf_err_ptr(-EINVAL);
+
+       syms = OPTS_GET(opts, syms, NULL);
+       offsets = OPTS_GET(opts, offsets, NULL);
+       ref_ctr_offsets = OPTS_GET(opts, ref_ctr_offsets, NULL);
+       cookies = OPTS_GET(opts, cookies, NULL);
+       cnt = OPTS_GET(opts, cnt, 0);
+
+       /*
+        * User can specify 2 mutually exclusive set of inputs:
+        *
+        * 1) use only path/func_pattern/pid arguments
+        *

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11000

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+        * 2) use path/pid with allowed combinations of:
+        *    syms/offsets/ref_ctr_offsets/cookies/cnt
+        *
+        *    - syms and offsets are mutually exclusive
+        *    - ref_ctr_offsets and cookies are optional
+        *
+        * Any other usage results in error.
+        */
+
+       if (!path && !func_pattern && !cnt)

weird, I'd expect separate if (!path) return error (already bad, regardless of func_pattern or cnt)

then if (!func_pattern && cnt == 0) return error

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11005

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+               return libbpf_err_ptr(-EINVAL);
+       if (func_pattern && !path)
+               return libbpf_err_ptr(-EINVAL);
+
+       has_pattern = path && func_pattern;
Quoted 3 lines

this and above check must be some leftovers from previous version. path should always be present. and so you don't need has_pattern variable, just use "func_pattern" check

hum, right, previous version had 2 paths, now there's just one, I'll change that together with the suggested change above

Quoted 28 lines
Quoted 28 lines
tools/lib/bpf/libbpf.c +27 -0
+
+       if (has_pattern) {
+               if (syms || offsets || ref_ctr_offsets || cookies || cnt)
+                       return libbpf_err_ptr(-EINVAL);
+       } else {
+               if (!cnt)
+                       return libbpf_err_ptr(-EINVAL);
+               if (!!syms == !!offsets)
+                       return libbpf_err_ptr(-EINVAL);
+       }
+
+       if (has_pattern) {
+               if (!strchr(path, '/')) {
+                       err = resolve_full_path(path, full_path, sizeof(full_path));
+                       if (err) {
+                               pr_warn("prog '%s': failed to resolve full path for '%s': %d\n",
+                                       prog->name, path, err);
+                               return libbpf_err_ptr(err);
+                       }
+                       path = full_path;
+               }
+
+               err = elf_resolve_pattern_offsets(path, func_pattern,
+                                                 &resolved_offsets, &cnt);
+               if (err < 0)
+                       return libbpf_err_ptr(err);
+               offsets = resolved_offsets;
Quoted 8 lines

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11037

Quoted 5 lines
tools/lib/bpf/libbpf.c +5 -0
+       } else if (syms) {
+               err = elf_resolve_syms_offsets(path, cnt, syms, &resolved_offsets);
+               if (err < 0)
+                       return libbpf_err_ptr(err);
+               offsets = resolved_offsets;

you can extract this common error checking and offsets = resolved_offsets; to after if, it's common for both branches

Quoted 2 lines

not sure what you mean in here, offsets can be also provided by OPTS_GET(opts, offsets, NULL) earlier

ah, I missed that it's else if, not just else. Never mind, it's good as is then.

Quoted 130 lines
Quoted 13 lines

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11047

Quoted 11 lines
tools/lib/bpf/libbpf.c +10 -0
+       }
+
+       retprobe = OPTS_GET(opts, retprobe, false);
+
+       lopts.uprobe_multi.path = path;
+       lopts.uprobe_multi.offsets = offsets;
+       lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
+       lopts.uprobe_multi.cookies = cookies;
+       lopts.uprobe_multi.cnt = cnt;
+       lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;

retprobe is another unnecessary var, just inline check here to keep it simpler

ok

Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.c +14 -0
+
+       if (pid == 0)
+               pid = getpid();
+       if (pid > 0)
+               lopts.uprobe_multi.pid = pid;
+
+       link = calloc(1, sizeof(*link));
+       if (!link) {
+               err = -ENOMEM;
+               goto error;
+       }
+       link->detach = &bpf_link__detach_fd;
+
+       prog_fd = bpf_program__fd(prog);
Quoted 8 lines

tools/lib/bpf/libbpf.c static int resolve_full_path(const char *file, char *result, size_t result_sz) · line 11065

Quoted 4 lines
tools/lib/bpf/libbpf.c +4 -0
+       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
+       if (link_fd < 0) {
+               err = -errno;
+               pr_warn("prog '%s': failed to attach: %s\n",

"failed to attach multi-uprobe"? We probably should have added "failed to attach multi-kprobe" in bpf_program__attach_kprobe_multi_opts as well?

ook, will add

Quoted 25 lines
Quoted 25 lines
tools/lib/bpf/libbpf.c +13 -0
+                       prog->name, libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
+               goto error;
+       }
+       link->fd = link_fd;
+       free(resolved_offsets);
+       return link;
+
+error:
+       free(resolved_offsets);
+       free(link);
+       return libbpf_err_ptr(err);
+}
+
 LIBBPF_API struct bpf_link *
 bpf_program__attach_uprobe_opts(const struct bpf_program *prog, pid_t pid,
                                const char *binary_path, size_t func_offset,
tools/lib/bpf/libbpf.h +0 -0
index 754da73c643b..7c218f610210 100644
@@ -529,6 +529,33 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
                                      const char *pattern,
                                      const struct bpf_kprobe_multi_opts *opts);
 
Quoted 6 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 535

Quoted 4 lines
tools/lib/bpf/libbpf.h +4 -0
+struct bpf_uprobe_multi_opts {
+       /* size of this struct, for forward/backward compatibility */
+       size_t sz;
+       /* array of function symbols to attach */

attach to?

ok

Quoted 5 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 537

Quoted 3 lines
tools/lib/bpf/libbpf.h +2 -0
+       const char **syms;
+       /* array of function addresses to attach */

attach to?

ook

Quoted 6 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 539

Quoted 3 lines
tools/lib/bpf/libbpf.h +2 -0
+       const unsigned long *offsets;
+       /* array of refctr offsets to attach */

we don't really attach to ref counters, so maybe "optional, array of associated ref counter offsets" or something along those lines ?

ok

Quoted 6 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 541

Quoted 3 lines
tools/lib/bpf/libbpf.h +2 -0
+       const unsigned long *ref_ctr_offsets;
+       /* array of user-provided values fetchable through bpf_get_attach_cookie */

"array of associated BPF cookies"? we can't keep explaining what BPF cookie is in every possible API :)

ook

Quoted 15 lines
Quoted 15 lines
tools/lib/bpf/libbpf.h +14 -0
+       const __u64 *cookies;
+       /* number of elements in syms/addrs/cookies arrays */
+       size_t cnt;
+       /* create return uprobes */
+       bool retprobe;
+       size_t :0;
+};
+
+#define bpf_uprobe_multi_opts__last_field retprobe
+
+LIBBPF_API struct bpf_link *
+bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
+                                pid_t pid,
+                                const char *binary_path,
Quoted 8 lines

tools/lib/bpf/libbpf.h bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 558

Quoted 3 lines
tools/lib/bpf/libbpf.h +3 -0
+                                const char *func_pattern,
+                                const struct bpf_uprobe_multi_opts *opts);
+

ok, let's be good citizens and add documentation for this new API. Those comments about valid combinations belong here as well. Please take a look at existing doccomments for the format and conventions. Thanks!

ok, will add

thanks,
jirka

author reviewer
#43 lore raw

PATCH v3 14/26 libbpf: Add support for u[ret]probe.multi[.s] program sections

In reply to Jiri Olsa

Adding support for several uprobe_multi program sections to allow auto attach of multi_uprobe programs.

---
 tools/lib/bpf/libbpf.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
tools/lib/bpf/libbpf.c +36 -0
index b942f248038e..06092b9752f1 100644
@@ -8654,6 +8654,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin
 static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
+static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 
@@ -8669,6 +8670,10 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("uretprobe.s+",		KPROBE, 0, SEC_SLEEPABLE, attach_uprobe),
 	SEC_DEF("kprobe.multi+",	KPROBE,	BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
 	SEC_DEF("kretprobe.multi+",	KPROBE,	BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
+	SEC_DEF("uprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
+	SEC_DEF("uretprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
+	SEC_DEF("uprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
+	SEC_DEF("uretprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
 	SEC_DEF("ksyscall+",		KPROBE,	0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("usdt+",		KPROBE,	0, SEC_NONE, attach_usdt),
@@ -10730,6 +10735,37 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru
 	return libbpf_get_error(*link);
 }
 
+static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link)
+{
+	char *probe_type = NULL, *binary_path = NULL, *func_name = NULL;
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+	int n, ret = -EINVAL;
+
+	*link = NULL;
+
+	n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%ms",
+		   &probe_type, &binary_path, &func_name);
+	switch (n) {
+	case 1:
+		/* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */
+		ret = 0;
+		break;
+	case 3:
+		opts.retprobe = strcmp(probe_type, "uretprobe.multi") == 0;
+		*link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts);
+		ret = libbpf_get_error(*link);
+		break;
+	default:
+		pr_warn("prog '%s': invalid format of section definition '%s'\n", prog->name,
+			prog->sec_name);
+		break;
+	}
+	free(probe_type);
+	free(binary_path);
+	free(func_name);
+	return ret;
+}
+
 static void gen_uprobe_legacy_event_name(char *buf, size_t buf_sz,
 					 const char *binary_path, uint64_t offset)
 {
Signature · 1 line
2.41.0
Acked-by on v3 14/26
#44 lore raw

Re: [PATCHv3 bpf-next 14/26] libbpf: Add support for u[ret]probe.multi[.s] program sections

Quoted 9 lines · Jiri Olsa

Adding support for several uprobe_multi program sections to allow auto attach of multi_uprobe programs.

---
 tools/lib/bpf/libbpf.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

LGTM

Quoted 64 lines
tools/lib/bpf/libbpf.c +36 -1
index b942f248038e..06092b9752f1 100644
@@ -8654,6 +8654,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin
 static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
+static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 
@@ -8669,6 +8670,10 @@ static const struct bpf_sec_def section_defs[] = {
        SEC_DEF("uretprobe.s+",         KPROBE, 0, SEC_SLEEPABLE, attach_uprobe),
        SEC_DEF("kprobe.multi+",        KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
        SEC_DEF("kretprobe.multi+",     KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
+       SEC_DEF("uprobe.multi+",        KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
+       SEC_DEF("uretprobe.multi+",     KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
+       SEC_DEF("uprobe.multi.s+",      KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
+       SEC_DEF("uretprobe.multi.s+",   KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
        SEC_DEF("ksyscall+",            KPROBE, 0, SEC_NONE, attach_ksyscall),
        SEC_DEF("kretsyscall+",         KPROBE, 0, SEC_NONE, attach_ksyscall),
        SEC_DEF("usdt+",                KPROBE, 0, SEC_NONE, attach_usdt),
@@ -10730,6 +10735,37 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru
        return libbpf_get_error(*link);
 }
 
+static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link)
+{
+       char *probe_type = NULL, *binary_path = NULL, *func_name = NULL;
+       LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+       int n, ret = -EINVAL;
+
+       *link = NULL;
+
+       n = sscanf(prog->sec_name, "%m[^/]/%m[^:]:%ms",
+                  &probe_type, &binary_path, &func_name);
+       switch (n) {
+       case 1:
+               /* handle SEC("u[ret]probe") - format is valid, but auto-attach is impossible. */
+               ret = 0;
+               break;
+       case 3:
+               opts.retprobe = strcmp(probe_type, "uretprobe.multi") == 0;
+               *link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts);
+               ret = libbpf_get_error(*link);
+               break;
+       default:
+               pr_warn("prog '%s': invalid format of section definition '%s'\n", prog->name,
+                       prog->sec_name);
+               break;
+       }
+       free(probe_type);
+       free(binary_path);
+       free(func_name);
+       return ret;
+}
+
 static void gen_uprobe_legacy_event_name(char *buf, size_t buf_sz,
                                         const char *binary_path, uint64_t offset)
 {
--

2.41.0

author reviewer
#45 lore raw

PATCH v3 15/26 libbpf: Add uprobe multi link detection

In reply to Jiri Olsa

Adding uprobe-multi link detection. It will be used later in bpf_program__attach_usdt function to check and use uprobe_multi link over standard uprobe links.

---
 tools/lib/bpf/libbpf.c          | 35 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_internal.h |  2 ++
 2 files changed, 37 insertions(+)
tools/lib/bpf/libbpf.c +35 -0
index 06092b9752f1..4f61f9dc1748 100644
@@ -4817,6 +4817,38 @@ static int probe_perf_link(void)
 	return link_fd < 0 && err == -EBADF;
 }
 
+static int probe_uprobe_multi_link(void)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
+		.expected_attach_type = BPF_TRACE_UPROBE_MULTI,
+	);
+	LIBBPF_OPTS(bpf_link_create_opts, link_opts);
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	unsigned long offset = 0;
+	int prog_fd, link_fd;
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
+				insns, ARRAY_SIZE(insns), &load_opts);
+	if (prog_fd < 0)
+		return -errno;
+
+	/* create single uprobe on offset 0 in current process */
+	link_opts.uprobe_multi.path = "/proc/self/exe";
+	link_opts.uprobe_multi.offsets = &offset;
+	link_opts.uprobe_multi.cnt = 1;
+
+	link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts);
+
+	if (link_fd >= 0)
+		close(link_fd);
+	close(prog_fd);
+
+	return link_fd >= 0;
+}
+
 static int probe_kern_bpf_cookie(void)
 {
 	struct bpf_insn insns[] = {
@@ -4913,6 +4945,9 @@ static struct kern_feature_desc {
 	[FEAT_SYSCALL_WRAPPER] = {
 		"Kernel using syscall wrapper", probe_kern_syscall_wrapper,
 	},
+	[FEAT_UPROBE_MULTI_LINK] = {
+		"BPF uprobe multi link support", probe_uprobe_multi_link,
+	},
 };
 
 bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
tools/lib/bpf/libbpf_internal.h +2 -0
index 7d75b92e531a..9c04b3fe1207 100644
@@ -354,6 +354,8 @@ enum kern_feature_id {
 	FEAT_BTF_ENUM64,
 	/* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */
 	FEAT_SYSCALL_WRAPPER,
+	/* BPF uprobe_multi link support */
+	FEAT_UPROBE_MULTI_LINK,
 	__FEAT_CNT,
 };
 
Signature · 1 line
2.41.0
#46 lore raw

Re: [PATCHv3 bpf-next 15/26] libbpf: Add uprobe multi link detection

Quoted 34 lines · Jiri Olsa

Adding uprobe-multi link detection. It will be used later in bpf_program__attach_usdt function to check and use uprobe_multi link over standard uprobe links.

---
 tools/lib/bpf/libbpf.c          | 35 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_internal.h |  2 ++
 2 files changed, 37 insertions(+)
tools/lib/bpf/libbpf.c +15 -0
index 06092b9752f1..4f61f9dc1748 100644
@@ -4817,6 +4817,38 @@ static int probe_perf_link(void)
        return link_fd < 0 && err == -EBADF;
 }
 
+static int probe_uprobe_multi_link(void)
+{
+       LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
+               .expected_attach_type = BPF_TRACE_UPROBE_MULTI,
+       );
+       LIBBPF_OPTS(bpf_link_create_opts, link_opts);
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       unsigned long offset = 0;
+       int prog_fd, link_fd;
+
+       prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
+                               insns, ARRAY_SIZE(insns), &load_opts);

tools/lib/bpf/libbpf.c static int probe_perf_link(void) · line 4844

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+       if (prog_fd < 0)
+               return -errno;
+
+       /* create single uprobe on offset 0 in current process */
+       link_opts.uprobe_multi.path = "/proc/self/exe";
+       link_opts.uprobe_multi.offsets = &offset;
+       link_opts.uprobe_multi.cnt = 1;
+
+       link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts);
+

so I'd like us to avoid successfully attaching anything. This might have unintended consequences (e.g., unintentionally breaking backing huge pages into normal pages, just because we happen to successfully attach briefly). So let's work on feature detection that fails to create a link, but does it in a way that we know that the feature itself is supported by the kernel

Some ideas we could do:

1. Pass invalid file (e.g., just root, "/" as path), but modify kernel-side logic to return not -EINVAL, but -EBADF (and I think it would be good to do this anyway). Then expect -EBADF as a signal that the feature is supported.

2. Also, we can return -EPROTO instead of -EINVAL on invalid combination of paramers or something like that

I'd start with -EBADF change.

In general, we should write kernel-side code in such a way that allows simple and efficient feature-detection. We shouldn't repeat the nightmare of memcg-based mem accounting :(

Quoted 11 lines
tools/lib/bpf/libbpf.c +7 -0
+       if (link_fd >= 0)
+               close(link_fd);
+       close(prog_fd);
+
+       return link_fd >= 0;
+}
+
 static int probe_kern_bpf_cookie(void)
 {
        struct bpf_insn insns[] = {

tools/lib/bpf/libbpf.c static struct kern_feature_desc { · line 4949

Quoted 5 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -4913,6 +4945,9 @@ static struct kern_feature_desc {
        [FEAT_SYSCALL_WRAPPER] = {
                "Kernel using syscall wrapper", probe_kern_syscall_wrapper,
        },
+       [FEAT_UPROBE_MULTI_LINK] = {
+               "BPF uprobe multi link support", probe_uprobe_multi_link,

nit: BPF multi-uprobe link support

tools/lib/bpf/libbpf_internal.h enum kern_feature_id { · line 357

Quoted 13 lines
tools/lib/bpf/libbpf.c +1 -0
+       },
 };
 
 bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
tools/lib/bpf/libbpf_internal.h +1 -0
index 7d75b92e531a..9c04b3fe1207 100644
@@ -354,6 +354,8 @@ enum kern_feature_id {
        FEAT_BTF_ENUM64,
        /* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */
        FEAT_SYSCALL_WRAPPER,
+       /* BPF uprobe_multi link support */

same, multi-uprobe link support

Quoted 7 lines
tools/lib/bpf/libbpf_internal.h +1 -1
+       FEAT_UPROBE_MULTI_LINK,
        __FEAT_CNT,
 };
 
--

2.41.0

#47 lore raw

Re: [PATCHv3 bpf-next 15/26] libbpf: Add uprobe multi link detection

Quoted 53 lines · Andrii Nakryiko
Quoted 34 lines · Jiri Olsa

Adding uprobe-multi link detection. It will be used later in bpf_program__attach_usdt function to check and use uprobe_multi link over standard uprobe links.

---
 tools/lib/bpf/libbpf.c          | 35 +++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf_internal.h |  2 ++
 2 files changed, 37 insertions(+)
tools/lib/bpf/libbpf.c +15 -0
index 06092b9752f1..4f61f9dc1748 100644
@@ -4817,6 +4817,38 @@ static int probe_perf_link(void)
        return link_fd < 0 && err == -EBADF;
 }
 
+static int probe_uprobe_multi_link(void)
+{
+       LIBBPF_OPTS(bpf_prog_load_opts, load_opts,
+               .expected_attach_type = BPF_TRACE_UPROBE_MULTI,
+       );
+       LIBBPF_OPTS(bpf_link_create_opts, link_opts);
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       unsigned long offset = 0;
+       int prog_fd, link_fd;
+
+       prog_fd = bpf_prog_load(BPF_PROG_TYPE_KPROBE, NULL, "GPL",
+                               insns, ARRAY_SIZE(insns), &load_opts);

tools/lib/bpf/libbpf.c static int probe_perf_link(void) · line 4844

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+       if (prog_fd < 0)
+               return -errno;
+
+       /* create single uprobe on offset 0 in current process */
+       link_opts.uprobe_multi.path = "/proc/self/exe";
+       link_opts.uprobe_multi.offsets = &offset;
+       link_opts.uprobe_multi.cnt = 1;
+
+       link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts);
+

so I'd like us to avoid successfully attaching anything. This might have unintended consequences (e.g., unintentionally breaking backing huge pages into normal pages, just because we happen to successfully attach briefly). So let's work on feature detection that fails to create a link, but does it in a way that we know that the feature itself is supported by the kernel

Quoted 6 lines

Some ideas we could do:

1. Pass invalid file (e.g., just root, "/" as path), but modify kernel-side logic to return not -EINVAL, but -EBADF (and I think it would be good to do this anyway). Then expect -EBADF as a signal that the feature is supported.

ah ok, so -EBADF from inside uprobe_multi link setup code would mean it's supported, anything else means it's not.. should work also for old kernels, I don't think we have -EBADF in related paths, will check

Quoted 32 lines

2. Also, we can return -EPROTO instead of -EINVAL on invalid combination of paramers or something like that

I'd start with -EBADF change.

In general, we should write kernel-side code in such a way that allows simple and efficient feature-detection. We shouldn't repeat the nightmare of memcg-based mem accounting :(

Quoted 11 lines
tools/lib/bpf/libbpf.c +7 -0
+       if (link_fd >= 0)
+               close(link_fd);
+       close(prog_fd);
+
+       return link_fd >= 0;
+}
+
 static int probe_kern_bpf_cookie(void)
 {
        struct bpf_insn insns[] = {

tools/lib/bpf/libbpf.c static struct kern_feature_desc { · line 4949

Quoted 5 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -4913,6 +4945,9 @@ static struct kern_feature_desc {
        [FEAT_SYSCALL_WRAPPER] = {
                "Kernel using syscall wrapper", probe_kern_syscall_wrapper,
        },
+       [FEAT_UPROBE_MULTI_LINK] = {
+               "BPF uprobe multi link support", probe_uprobe_multi_link,

nit: BPF multi-uprobe link support

Quoted 3 lines
tools/lib/bpf/libbpf.c +1 -0
+       },
 };
 
Quoted 12 lines

tools/lib/bpf/libbpf_internal.h enum kern_feature_id { · line 357

Quoted 10 lines
tools/lib/bpf/libbpf.c +0 -0
 bool kernel_supports(const struct bpf_object *obj, enum kern_feature_id feat_id)
tools/lib/bpf/libbpf_internal.h +1 -0
index 7d75b92e531a..9c04b3fe1207 100644
@@ -354,6 +354,8 @@ enum kern_feature_id {
        FEAT_BTF_ENUM64,
        /* Kernel uses syscall wrapper (CONFIG_ARCH_HAS_SYSCALL_WRAPPER) */
        FEAT_SYSCALL_WRAPPER,
+       /* BPF uprobe_multi link support */

same, multi-uprobe link support

ok, thanks

jirka

Quoted 10 lines
Quoted 10 lines
tools/lib/bpf/libbpf_internal.h +1 -1
+       FEAT_UPROBE_MULTI_LINK,
        __FEAT_CNT,
 };
 
--

2.41.0

author reviewer
#48 lore raw

PATCH v3 16/26 libbpf: Add uprobe multi link support to bpf_program__attach_usdt

In reply to Jiri Olsa

Adding support for usdt_manager_attach_usdt to use uprobe_multi link to attach to usdt probes.

The uprobe_multi support is detected before the usdt program is loaded and its expected_attach_type is set accordingly.

If uprobe_multi support is detected the usdt_manager_attach_usdt gathers uprobes info and calls bpf_program__attach_uprobe to create all needed uprobes.

If uprobe_multi support is not detected the old behaviour stays.

Also adding usdt.s program section for sleepable usdt probes.

---
 tools/lib/bpf/libbpf.c | 13 +++++--
 tools/lib/bpf/usdt.c   | 78 ++++++++++++++++++++++++++++++++++--------
 2 files changed, 75 insertions(+), 16 deletions(-)
tools/lib/bpf/libbpf.c +11 -2
index 4f61f9dc1748..e234c2e860f9 100644
@@ -365,6 +365,8 @@ enum sec_def_flags {
 	SEC_SLEEPABLE = 8,
 	/* BPF program support non-linear XDP buffer */
 	SEC_XDP_FRAGS = 16,
+	/* Setup proper attach type for usdt probes. */
+	SEC_USDT = 32,
 };
 
 struct bpf_sec_def {
@@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,
 	if (prog->type == BPF_PROG_TYPE_XDP && (def & SEC_XDP_FRAGS))
 		opts->prog_flags |= BPF_F_XDP_HAS_FRAGS;
 
+	/* special check for usdt to use uprobe_multi link */
+	if ((def & SEC_USDT) && kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK))
+		prog->expected_attach_type = BPF_TRACE_UPROBE_MULTI;
+
 	if ((def & SEC_ATTACH_BTF) && !prog->attach_btf_id) {
 		int btf_obj_fd = 0, btf_type_id = 0, err;
 		const char *attach_name;
@@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
 	if (!insns || !insns_cnt)
 		return -EINVAL;
 
-	load_attr.expected_attach_type = prog->expected_attach_type;
 	if (kernel_supports(obj, FEAT_PROG_NAME))
 		prog_name = prog->name;
 	load_attr.attach_prog_fd = prog->attach_prog_fd;
@@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
 		insns_cnt = prog->insns_cnt;
 	}
 
+	/* allow prog_prepare_load_fn to change expected_attach_type */
+	load_attr.expected_attach_type = prog->expected_attach_type;
+
 	if (obj->gen_loader) {
 		bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name,
 				   license, insns, insns_cnt, &load_attr,
@@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("uretprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
 	SEC_DEF("ksyscall+",		KPROBE,	0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
-	SEC_DEF("usdt+",		KPROBE,	0, SEC_NONE, attach_usdt),
+	SEC_DEF("usdt+",		KPROBE,	0, SEC_USDT, attach_usdt),
+	SEC_DEF("usdt.s+",		KPROBE,	0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),
 	SEC_DEF("tc",			SCHED_CLS, 0, SEC_NONE),
 	SEC_DEF("classifier",		SCHED_CLS, 0, SEC_NONE),
 	SEC_DEF("action",		SCHED_ACT, 0, SEC_NONE),
tools/lib/bpf/usdt.c +64 -14

123 lines not on this page · raw message

Signature · 1 line
2.41.0
#49 lore raw

Re: [PATCHv3 bpf-next 16/26] libbpf: Add uprobe multi link support to bpf_program__attach_usdt

Quoted 35 lines · Jiri Olsa

Adding support for usdt_manager_attach_usdt to use uprobe_multi link to attach to usdt probes.

The uprobe_multi support is detected before the usdt program is loaded and its expected_attach_type is set accordingly.

If uprobe_multi support is detected the usdt_manager_attach_usdt gathers uprobes info and calls bpf_program__attach_uprobe to create all needed uprobes.

If uprobe_multi support is not detected the old behaviour stays.

Also adding usdt.s program section for sleepable usdt probes.

---
 tools/lib/bpf/libbpf.c | 13 +++++--
 tools/lib/bpf/usdt.c   | 78 ++++++++++++++++++++++++++++++++++--------
 2 files changed, 75 insertions(+), 16 deletions(-)
tools/lib/bpf/libbpf.c +2 -0
index 4f61f9dc1748..e234c2e860f9 100644
@@ -365,6 +365,8 @@ enum sec_def_flags {
        SEC_SLEEPABLE = 8,
        /* BPF program support non-linear XDP buffer */
        SEC_XDP_FRAGS = 16,
+       /* Setup proper attach type for usdt probes. */
+       SEC_USDT = 32,
 };
 
 struct bpf_sec_def {

tools/lib/bpf/libbpf.c static int libbpf_prepare_prog_load(struct bpf_program *prog, · line 6813

Quoted 5 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,
        if (prog->type == BPF_PROG_TYPE_XDP && (def & SEC_XDP_FRAGS))
                opts->prog_flags |= BPF_F_XDP_HAS_FRAGS;
 
+       /* special check for usdt to use uprobe_multi link */
+       if ((def & SEC_USDT) && kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK))

please pass prog->obj to kernel_supports(), it will be especially important later with BPF token stuff

Quoted 24 lines
tools/lib/bpf/libbpf.c +5 -1
+               prog->expected_attach_type = BPF_TRACE_UPROBE_MULTI;
+
        if ((def & SEC_ATTACH_BTF) && !prog->attach_btf_id) {
                int btf_obj_fd = 0, btf_type_id = 0, err;
                const char *attach_name;
@@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
        if (!insns || !insns_cnt)
                return -EINVAL;
 
-       load_attr.expected_attach_type = prog->expected_attach_type;
        if (kernel_supports(obj, FEAT_PROG_NAME))
                prog_name = prog->name;
        load_attr.attach_prog_fd = prog->attach_prog_fd;
@@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
                insns_cnt = prog->insns_cnt;
        }
 
+       /* allow prog_prepare_load_fn to change expected_attach_type */
+       load_attr.expected_attach_type = prog->expected_attach_type;
+
        if (obj->gen_loader) {
                bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name,
                                   license, insns, insns_cnt, &load_attr,

tools/lib/bpf/libbpf.c static const struct bpf_sec_def section_defs[] = { · line 8723

Quoted 6 lines
tools/lib/bpf/libbpf.c +2 -1
@@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {
        SEC_DEF("uretprobe.multi.s+",   KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
        SEC_DEF("ksyscall+",            KPROBE, 0, SEC_NONE, attach_ksyscall),
        SEC_DEF("kretsyscall+",         KPROBE, 0, SEC_NONE, attach_ksyscall),
-       SEC_DEF("usdt+",                KPROBE, 0, SEC_NONE, attach_usdt),
+       SEC_DEF("usdt+",                KPROBE, 0, SEC_USDT, attach_usdt),
+       SEC_DEF("usdt.s+",              KPROBE, 0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),

spaces around |

Quoted 17 lines
tools/lib/bpf/libbpf.c +0 -0
        SEC_DEF("tc",                   SCHED_CLS, 0, SEC_NONE),
        SEC_DEF("classifier",           SCHED_CLS, 0, SEC_NONE),
        SEC_DEF("action",               SCHED_ACT, 0, SEC_NONE),
tools/lib/bpf/usdt.c +2 -0
index 9fa883ebc0bd..6ff66a8eaf85 100644
@@ -809,6 +809,8 @@ struct bpf_link_usdt {
                long abs_ip;
                struct bpf_link *link;
        } *uprobes;
+
+       struct bpf_link *multi_link;
 };
 
 static int bpf_link_usdt_detach(struct bpf_link *link)

tools/lib/bpf/usdt.c static int bpf_link_usdt_detach(struct bpf_link *link) · line 822

Quoted 4 lines
tools/lib/bpf/usdt.c +1 -0
@@ -817,6 +819,9 @@ static int bpf_link_usdt_detach(struct bpf_link *link)
        struct usdt_manager *man = usdt_link->usdt_man;
        int i;
 
+       /* When having multi_link, uprobe_cnt is 0 */

misplaced comment, move down to for() loop?

Quoted 20 lines
tools/lib/bpf/usdt.c +4 -0
+       bpf_link__destroy(usdt_link->multi_link);
+
        for (i = 0; i < usdt_link->uprobe_cnt; i++) {
                /* detach underlying uprobe link */
                bpf_link__destroy(usdt_link->uprobes[i].link);
@@ -944,11 +949,13 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
                                          const char *usdt_provider, const char *usdt_name,
                                          __u64 usdt_cookie)
 {
+       unsigned long *offsets = NULL, *ref_ctr_offsets = NULL;
        int i, err, spec_map_fd, ip_map_fd;
        LIBBPF_OPTS(bpf_uprobe_opts, opts);
        struct hashmap *specs_hash = NULL;
        struct bpf_link_usdt *link = NULL;
        struct usdt_target *targets = NULL;
+       __u64 *cookies = NULL;
        struct elf_fd elf_fd;
        size_t target_cnt;
 

tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1005

Quoted 8 lines
tools/lib/bpf/usdt.c +1 -4
@@ -995,10 +1002,21 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
        link->link.detach = &bpf_link_usdt_detach;
        link->link.dealloc = &bpf_link_usdt_dealloc;
 
-       link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
-       if (!link->uprobes) {
-               err = -ENOMEM;
-               goto err_out;
+       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {

see how we feature-detect has_sema_refcnt and has_bpf_cookie, let's do the same with UPROBE_MULTI_LINK, detect once, remember, consistently use it (it also matters later for BPF token)

Quoted 39 lines
tools/lib/bpf/usdt.c +27 -5
+               offsets = calloc(target_cnt, sizeof(*offsets));
+               cookies = calloc(target_cnt, sizeof(*cookies));
+               ref_ctr_offsets = calloc(target_cnt, sizeof(*ref_ctr_offsets));
+
+               if (!offsets || !ref_ctr_offsets || !cookies) {
+                       err = -ENOMEM;
+                       goto err_out;
+               }
+       } else {
+               link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
+               if (!link->uprobes) {
+                       err = -ENOMEM;
+                       goto err_out;
+               }
        }
 
        for (i = 0; i < target_cnt; i++) {
@@ -1039,20 +1057,48 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
                        goto err_out;
                }
 
-               opts.ref_ctr_offset = target->sema_off;
-               opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
-               uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
-                                                             target->rel_ip, &opts);
-               err = libbpf_get_error(uprobe_link);
+               if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
+                       offsets[i] = target->rel_ip;
+                       ref_ctr_offsets[i] = target->sema_off;
+                       cookies[i] = spec_id;
+               } else {
+                       opts.ref_ctr_offset = target->sema_off;
+                       opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
+                       uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
+                                                                     target->rel_ip, &opts);
+                       err = libbpf_get_error(uprobe_link);
+                       if (err) {
+                               pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
+                                       i, usdt_provider, usdt_name, path, err);

tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1082

Quoted 10 lines
tools/lib/bpf/usdt.c +10 -0
+                               goto err_out;
+                       }
+
+                       link->uprobes[i].link = uprobe_link;
+                       link->uprobes[i].abs_ip = target->abs_ip;
+                       link->uprobe_cnt++;
+               }
+       }
+
+       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {

same as above, we should feature-detect once per usdt_manager while we have associated bpf_object

tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1092

Quoted 10 lines
tools/lib/bpf/usdt.c +10 -0
+               LIBBPF_OPTS(bpf_uprobe_multi_opts, opts_multi,
+                       .cnt = target_cnt,
+                       .offsets = offsets,
+                       .ref_ctr_offsets = ref_ctr_offsets,
+                       .cookies = cookies,
+               );
+
+               link->multi_link = bpf_program__attach_uprobe_multi(prog, pid, path,
+                                                                   NULL, &opts_multi);
+               err = libbpf_get_error(link->multi_link);

let's not use libbpf_get_error() in new code, there is no need, just err = -errno and if (!link->multi_link)

Quoted 31 lines
tools/lib/bpf/usdt.c +9 -6
                if (err) {
-                       pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
-                               i, usdt_provider, usdt_name, path, err);
+                       pr_warn("usdt: failed to attach uprobe multi for '%s:%s' in '%s': %d\n",
+                               usdt_provider, usdt_name, path, err);
                        goto err_out;
                }
 
-               link->uprobes[i].link = uprobe_link;
-               link->uprobes[i].abs_ip = target->abs_ip;
-               link->uprobe_cnt++;
+               free(offsets);
+               free(ref_ctr_offsets);
+               free(cookies);
        }
 
        free(targets);
@@ -1061,6 +1107,10 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
        return &link->link;
 
 err_out:
+       free(offsets);
+       free(ref_ctr_offsets);
+       free(cookies);
+
        if (link)
                bpf_link__destroy(&link->link);
        free(targets);
--

2.41.0

#50 lore raw

Re: [PATCHv3 bpf-next 16/26] libbpf: Add uprobe multi link support to bpf_program__attach_usdt

Quoted 39 lines · Andrii Nakryiko
Quoted 38 lines · Jiri Olsa

Adding support for usdt_manager_attach_usdt to use uprobe_multi link to attach to usdt probes.

The uprobe_multi support is detected before the usdt program is loaded and its expected_attach_type is set accordingly.

If uprobe_multi support is detected the usdt_manager_attach_usdt gathers uprobes info and calls bpf_program__attach_uprobe to create all needed uprobes.

If uprobe_multi support is not detected the old behaviour stays.

Also adding usdt.s program section for sleepable usdt probes.

---
 tools/lib/bpf/libbpf.c | 13 +++++--
 tools/lib/bpf/usdt.c   | 78 ++++++++++++++++++++++++++++++++++--------
 2 files changed, 75 insertions(+), 16 deletions(-)
tools/lib/bpf/libbpf.c +2 -0
index 4f61f9dc1748..e234c2e860f9 100644
@@ -365,6 +365,8 @@ enum sec_def_flags {
        SEC_SLEEPABLE = 8,
        /* BPF program support non-linear XDP buffer */
        SEC_XDP_FRAGS = 16,
+       /* Setup proper attach type for usdt probes. */
+       SEC_USDT = 32,
 };
 
 struct bpf_sec_def {
@@ -6807,6 +6809,10 @@ static int libbpf_prepare_prog_load(struct bpf_program *prog,
        if (prog->type == BPF_PROG_TYPE_XDP && (def & SEC_XDP_FRAGS))
                opts->prog_flags |= BPF_F_XDP_HAS_FRAGS;
 
Quoted 5 lines

tools/lib/bpf/libbpf.c static int libbpf_prepare_prog_load(struct bpf_program *prog, · line 6813

Quoted 2 lines
tools/lib/bpf/libbpf.c +2 -0
+       /* special check for usdt to use uprobe_multi link */
+       if ((def & SEC_USDT) && kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK))

please pass prog->obj to kernel_supports(), it will be especially important later with BPF token stuff

did not realize I have it in prog->obj ;-) ok

Quoted 18 lines
Quoted 18 lines
tools/lib/bpf/libbpf.c +2 -1
+               prog->expected_attach_type = BPF_TRACE_UPROBE_MULTI;
+
        if ((def & SEC_ATTACH_BTF) && !prog->attach_btf_id) {
                int btf_obj_fd = 0, btf_type_id = 0, err;
                const char *attach_name;
@@ -6875,7 +6881,6 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
        if (!insns || !insns_cnt)
                return -EINVAL;
 
-       load_attr.expected_attach_type = prog->expected_attach_type;
        if (kernel_supports(obj, FEAT_PROG_NAME))
                prog_name = prog->name;
        load_attr.attach_prog_fd = prog->attach_prog_fd;
@@ -6911,6 +6916,9 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
                insns_cnt = prog->insns_cnt;
        }
 
Quoted 15 lines

tools/lib/bpf/libbpf.c static const struct bpf_sec_def section_defs[] = { · line 8723

Quoted 13 lines
tools/lib/bpf/libbpf.c +5 -1
+       /* allow prog_prepare_load_fn to change expected_attach_type */
+       load_attr.expected_attach_type = prog->expected_attach_type;
+
        if (obj->gen_loader) {
                bpf_gen__prog_load(obj->gen_loader, prog->type, prog->name,
                                   license, insns, insns_cnt, &load_attr,
@@ -8711,7 +8719,8 @@ static const struct bpf_sec_def section_defs[] = {
        SEC_DEF("uretprobe.multi.s+",   KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
        SEC_DEF("ksyscall+",            KPROBE, 0, SEC_NONE, attach_ksyscall),
        SEC_DEF("kretsyscall+",         KPROBE, 0, SEC_NONE, attach_ksyscall),
-       SEC_DEF("usdt+",                KPROBE, 0, SEC_NONE, attach_usdt),
+       SEC_DEF("usdt+",                KPROBE, 0, SEC_USDT, attach_usdt),
+       SEC_DEF("usdt.s+",              KPROBE, 0, SEC_USDT|SEC_SLEEPABLE, attach_usdt),

spaces around |

ook

Quoted 16 lines
Quoted 16 lines
tools/lib/bpf/libbpf.c +0 -0
        SEC_DEF("tc",                   SCHED_CLS, 0, SEC_NONE),
        SEC_DEF("classifier",           SCHED_CLS, 0, SEC_NONE),
        SEC_DEF("action",               SCHED_ACT, 0, SEC_NONE),
tools/lib/bpf/usdt.c +2 -0
index 9fa883ebc0bd..6ff66a8eaf85 100644
@@ -809,6 +809,8 @@ struct bpf_link_usdt {
                long abs_ip;
                struct bpf_link *link;
        } *uprobes;
+
+       struct bpf_link *multi_link;
 };
 
Quoted 8 lines

tools/lib/bpf/usdt.c static int bpf_link_usdt_detach(struct bpf_link *link) · line 822

Quoted 6 lines
tools/lib/bpf/usdt.c +1 -0
 static int bpf_link_usdt_detach(struct bpf_link *link)
@@ -817,6 +819,9 @@ static int bpf_link_usdt_detach(struct bpf_link *link)
        struct usdt_manager *man = usdt_link->usdt_man;
        int i;
 
+       /* When having multi_link, uprobe_cnt is 0 */

misplaced comment, move down to for() loop?

right, will move

Quoted 24 lines
Quoted 24 lines
tools/lib/bpf/usdt.c +4 -0
+       bpf_link__destroy(usdt_link->multi_link);
+
        for (i = 0; i < usdt_link->uprobe_cnt; i++) {
                /* detach underlying uprobe link */
                bpf_link__destroy(usdt_link->uprobes[i].link);
@@ -944,11 +949,13 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
                                          const char *usdt_provider, const char *usdt_name,
                                          __u64 usdt_cookie)
 {
+       unsigned long *offsets = NULL, *ref_ctr_offsets = NULL;
        int i, err, spec_map_fd, ip_map_fd;
        LIBBPF_OPTS(bpf_uprobe_opts, opts);
        struct hashmap *specs_hash = NULL;
        struct bpf_link_usdt *link = NULL;
        struct usdt_target *targets = NULL;
+       __u64 *cookies = NULL;
        struct elf_fd elf_fd;
        size_t target_cnt;
 
@@ -995,10 +1002,21 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
        link->link.detach = &bpf_link_usdt_detach;
        link->link.dealloc = &bpf_link_usdt_dealloc;
 
Quoted 9 lines

tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1005

Quoted 5 lines
tools/lib/bpf/usdt.c +1 -4
-       link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
-       if (!link->uprobes) {
-               err = -ENOMEM;
-               goto err_out;
+       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {

see how we feature-detect has_sema_refcnt and has_bpf_cookie, let's do the same with UPROBE_MULTI_LINK, detect once, remember, consistently use it (it also matters later for BPF token)

ok

Quoted 45 lines
Quoted 45 lines
tools/lib/bpf/usdt.c +32 -5
+               offsets = calloc(target_cnt, sizeof(*offsets));
+               cookies = calloc(target_cnt, sizeof(*cookies));
+               ref_ctr_offsets = calloc(target_cnt, sizeof(*ref_ctr_offsets));
+
+               if (!offsets || !ref_ctr_offsets || !cookies) {
+                       err = -ENOMEM;
+                       goto err_out;
+               }
+       } else {
+               link->uprobes = calloc(target_cnt, sizeof(*link->uprobes));
+               if (!link->uprobes) {
+                       err = -ENOMEM;
+                       goto err_out;
+               }
        }
 
        for (i = 0; i < target_cnt; i++) {
@@ -1039,20 +1057,48 @@ struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct
                        goto err_out;
                }
 
-               opts.ref_ctr_offset = target->sema_off;
-               opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
-               uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
-                                                             target->rel_ip, &opts);
-               err = libbpf_get_error(uprobe_link);
+               if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {
+                       offsets[i] = target->rel_ip;
+                       ref_ctr_offsets[i] = target->sema_off;
+                       cookies[i] = spec_id;
+               } else {
+                       opts.ref_ctr_offset = target->sema_off;
+                       opts.bpf_cookie = man->has_bpf_cookie ? spec_id : 0;
+                       uprobe_link = bpf_program__attach_uprobe_opts(prog, pid, path,
+                                                                     target->rel_ip, &opts);
+                       err = libbpf_get_error(uprobe_link);
+                       if (err) {
+                               pr_warn("usdt: failed to attach uprobe #%d for '%s:%s' in '%s': %d\n",
+                                       i, usdt_provider, usdt_name, path, err);
+                               goto err_out;
+                       }
+
+                       link->uprobes[i].link = uprobe_link;
+                       link->uprobes[i].abs_ip = target->abs_ip;
Quoted 8 lines

tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1082

Quoted 5 lines
tools/lib/bpf/usdt.c +5 -0
+                       link->uprobe_cnt++;
+               }
+       }
+
+       if (kernel_supports(NULL, FEAT_UPROBE_MULTI_LINK)) {

same as above, we should feature-detect once per usdt_manager while we have associated bpf_object

ook

Quoted 12 lines

tools/lib/bpf/usdt.c struct bpf_link *usdt_manager_attach_usdt(struct usdt_manager *man, const struct · line 1092

Quoted 11 lines
tools/lib/bpf/usdt.c +10 -0
+               LIBBPF_OPTS(bpf_uprobe_multi_opts, opts_multi,
+                       .cnt = target_cnt,
+                       .offsets = offsets,
+                       .ref_ctr_offsets = ref_ctr_offsets,
+                       .cookies = cookies,
+               );
+
+               link->multi_link = bpf_program__attach_uprobe_multi(prog, pid, path,
+                                                                   NULL, &opts_multi);
+               err = libbpf_get_error(link->multi_link);
Quoted 2 lines

let's not use libbpf_get_error() in new code, there is no need, just err = -errno and if (!link->multi_link)

ah right, did not see its comment that says it's no longer recomended, will change

thanks,
jirka

author reviewer
#51 lore raw

PATCH v3 17/26 selftests/bpf: Add uprobe_multi skel test

In reply to Jiri Olsa

Adding uprobe_multi test for skeleton load/attach functions, to test skeleton auto attach for uprobe_multi link.

Test that bpf_get_func_ip works properly for uprobe_multi attachment.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 76 ++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        | 91 +++++++++++++++++++
 2 files changed, 167 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c new file +76 -0
new file mode 100644
index 000000000000..5cd1116bbb62
@@ -0,0 +1,76 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <unistd.h>
+#include <test_progs.h>
+#include "uprobe_multi.skel.h"
+
+static char test_data[] = "test_data";
+
+noinline void uprobe_multi_func_1(void)
+{
+	asm volatile ("");
+}
+
+noinline void uprobe_multi_func_2(void)
+{
+	asm volatile ("");
+}
+
+noinline void uprobe_multi_func_3(void)
+{
+	asm volatile ("");
+}
+
+static void uprobe_multi_test_run(struct uprobe_multi *skel)
+{
+	skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1;
+	skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2;
+	skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3;
+
+	skel->bss->user_ptr = test_data;
+	skel->bss->pid = getpid();
+
+	/* trigger all probes */
+	uprobe_multi_func_1();
+	uprobe_multi_func_2();
+	uprobe_multi_func_3();
+
+	/*
+	 * There are 2 entry and 2 exit probe called for each uprobe_multi_func_[123]
+	 * function and each slepable probe (6) increments uprobe_multi_sleep_result.
+	 */
+	ASSERT_EQ(skel->bss->uprobe_multi_func_1_result, 2, "uprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_2_result, 2, "uprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_3_result, 2, "uprobe_multi_func_3_result");
+
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_1_result, 2, "uretprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_2_result, 2, "uretprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 2, "uretprobe_multi_func_3_result");
+
+	ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 6, "uprobe_multi_sleep_result");
+}
+
+static void test_skel_api(void)
+{
+	struct uprobe_multi *skel = NULL;
+	int err;
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+		goto cleanup;
+
+	err = uprobe_multi__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi__attach"))
+		goto cleanup;
+
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	uprobe_multi__destroy(skel);
+}
+
+void test_uprobe_multi_test(void)
+{
+	if (test__start_subtest("skel_api"))
+		test_skel_api();
+}
tools/testing/selftests/bpf/progs/uprobe_multi.c new file +91 -0

91 lines not on this page · raw message

Signature · 1 line
2.41.0
author reviewer
#52 lore raw

PATCH v3 18/26 selftests/bpf: Add uprobe_multi api test

In reply to Jiri Olsa

Adding uprobe_multi test for bpf_program__attach_uprobe_multi attach function.

Testing attachment using glob patterns and via bpf_uprobe_multi_opts paths/syms fields.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 71 +++++++++++++++++++
 1 file changed, 71 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +71 -0
index 5cd1116bbb62..f97a68871e73 100644
@@ -69,8 +69,79 @@ static void test_skel_api(void)
 	uprobe_multi__destroy(skel);
 }
 
+static void
+test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+{
+	struct bpf_link *link1 = NULL, *link2 = NULL;
+	struct bpf_link *link3 = NULL, *link4 = NULL;
+	struct uprobe_multi *skel = NULL;
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+		goto cleanup;
+
+	opts->retprobe = false;
+	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
+	opts->retprobe = true;
+	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+		goto cleanup;
+
+	opts->retprobe = false;
+	link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
+	opts->retprobe = true;
+	link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+		goto cleanup;
+
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	bpf_link__destroy(link4);
+	bpf_link__destroy(link3);
+	bpf_link__destroy(link2);
+	bpf_link__destroy(link1);
+	uprobe_multi__destroy(skel);
+}
+
+static void test_attach_api_pattern(void)
+{
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+
+	test_attach_api("/proc/self/exe", "uprobe_multi_func_*", &opts);
+	test_attach_api("/proc/self/exe", "uprobe_multi_func_?", &opts);
+}
+
+static void test_attach_api_syms(void)
+{
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+	const char *syms[3] = {
+		"uprobe_multi_func_1",
+		"uprobe_multi_func_2",
+		"uprobe_multi_func_3",
+	};
+
+	opts.syms = syms;
+	opts.cnt = ARRAY_SIZE(syms);
+	test_attach_api("/proc/self/exe", NULL, &opts);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
 		test_skel_api();
+	if (test__start_subtest("attach_api_pattern"))
+		test_attach_api_pattern();
+	if (test__start_subtest("attach_api_syms"))
+		test_attach_api_syms();
 }
Signature · 1 line
2.41.0
#53 lore raw

Re: [PATCHv3 bpf-next 18/26] selftests/bpf: Add uprobe_multi api test

Quoted 37 lines · Jiri Olsa

Adding uprobe_multi test for bpf_program__attach_uprobe_multi attach function.

Testing attachment using glob patterns and via bpf_uprobe_multi_opts paths/syms fields.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 71 +++++++++++++++++++
 1 file changed, 71 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +17 -0
index 5cd1116bbb62..f97a68871e73 100644
@@ -69,8 +69,79 @@ static void test_skel_api(void)
        uprobe_multi__destroy(skel);
 }
 
+static void
+test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+{
+       struct bpf_link *link1 = NULL, *link2 = NULL;
+       struct bpf_link *link3 = NULL, *link4 = NULL;
+       struct uprobe_multi *skel = NULL;
+
+       skel = uprobe_multi__open_and_load();
+       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+               goto cleanup;
+
+       opts->retprobe = false;
+       link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+               goto cleanup;
+

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 98

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10 -0
+       opts->retprobe = true;
+       link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+               goto cleanup;
+
+       opts->retprobe = false;
+       link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))

link3

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 104

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6 -0
+               goto cleanup;
+
+       opts->retprobe = true;
+       link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))

link4

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 113

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +9 -0
+               goto cleanup;
+
+       uprobe_multi_test_run(skel);
+
+cleanup:
+       bpf_link__destroy(link4);
+       bpf_link__destroy(link3);
+       bpf_link__destroy(link2);
+       bpf_link__destroy(link1);

you could have used skel->links.{test_uprobe,test_uretprobe,test_uprobe_sleep,test_uretprobe_sleep} "containers" to not manage destruction of these links manually

Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +11 -0
+       uprobe_multi__destroy(skel);
+}
+
+static void test_attach_api_pattern(void)
+{
+       LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+
+       test_attach_api("/proc/self/exe", "uprobe_multi_func_*", &opts);
+       test_attach_api("/proc/self/exe", "uprobe_multi_func_?", &opts);
+}
+

[...]

#54 lore raw

Re: [PATCHv3 bpf-next 18/26] selftests/bpf: Add uprobe_multi api test

Quoted 60 lines · Andrii Nakryiko
Quoted 37 lines · Jiri Olsa

Adding uprobe_multi test for bpf_program__attach_uprobe_multi attach function.

Testing attachment using glob patterns and via bpf_uprobe_multi_opts paths/syms fields.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 71 +++++++++++++++++++
 1 file changed, 71 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +17 -0
index 5cd1116bbb62..f97a68871e73 100644
@@ -69,8 +69,79 @@ static void test_skel_api(void)
        uprobe_multi__destroy(skel);
 }
 
+static void
+test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+{
+       struct bpf_link *link1 = NULL, *link2 = NULL;
+       struct bpf_link *link3 = NULL, *link4 = NULL;
+       struct uprobe_multi *skel = NULL;
+
+       skel = uprobe_multi__open_and_load();
+       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+               goto cleanup;
+
+       opts->retprobe = false;
+       link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+               goto cleanup;
+

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 98

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10 -0
+       opts->retprobe = true;
+       link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+               goto cleanup;
+
+       opts->retprobe = false;
+       link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))

link3

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 104

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6 -0
+               goto cleanup;
+
+       opts->retprobe = true;
+       link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
+                                                     binary, pattern, opts);
+       if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))

link4

Quoted 13 lines

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_skel_api(void) · line 113

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +9 -0
+               goto cleanup;
+
+       uprobe_multi_test_run(skel);
+
+cleanup:
+       bpf_link__destroy(link4);
+       bpf_link__destroy(link3);
+       bpf_link__destroy(link2);
+       bpf_link__destroy(link1);

you could have used skel->links.{test_uprobe,test_uretprobe,test_uprobe_sleep,test_uretprobe_sleep} "containers" to not manage destruction of these links manually

ah right, I keep forgetting we have those variables ready in there :-\
will change

thanks,
jirka

Quoted 14 lines
Quoted 12 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +11 -0
+       uprobe_multi__destroy(skel);
+}
+
+static void test_attach_api_pattern(void)
+{
+       LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+
+       test_attach_api("/proc/self/exe", "uprobe_multi_func_*", &opts);
+       test_attach_api("/proc/self/exe", "uprobe_multi_func_?", &opts);
+}
+

[...]

author reviewer
#55 lore raw

PATCH v3 19/26 selftests/bpf: Add uprobe_multi link test

In reply to Jiri Olsa

Adding uprobe_multi test for bpf_link_create attach function.

Testing attachment using the struct bpf_link_create_opts.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 68 +++++++++++++++++++
 1 file changed, 68 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +68 -0
index f97a68871e73..fd858636b8b0 100644
@@ -3,6 +3,7 @@
 #include <unistd.h>
 #include <test_progs.h>
 #include "uprobe_multi.skel.h"
+#include "bpf/libbpf_elf.h"
 
 static char test_data[] = "test_data";
 
@@ -136,6 +137,71 @@ static void test_attach_api_syms(void)
 	test_attach_api("/proc/self/exe", NULL, &opts);
 }
 
+static void test_link_api(void)
+{
+	int prog_fd, link1_fd = -1, link2_fd = -1, link3_fd = -1, link4_fd = -1;
+	LIBBPF_OPTS(bpf_link_create_opts, opts);
+	const char *path = "/proc/self/exe";
+	struct uprobe_multi *skel = NULL;
+	unsigned long *offsets = NULL;
+	const char *syms[3] = {
+		"uprobe_multi_func_1",
+		"uprobe_multi_func_2",
+		"uprobe_multi_func_3",
+	};
+	int err;
+
+	err = elf_resolve_syms_offsets(path, 3, syms, (unsigned long **) &offsets);
+	if (!ASSERT_OK(err, "elf_resolve_syms_offsets"))
+		return;
+
+	opts.uprobe_multi.path = path;
+	opts.uprobe_multi.offsets = offsets;
+	opts.uprobe_multi.cnt = ARRAY_SIZE(syms);
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = 0;
+	prog_fd = bpf_program__fd(skel->progs.test_uprobe);
+	link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link1_fd, 0, "link1_fd"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+	prog_fd = bpf_program__fd(skel->progs.test_uretprobe);
+	link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link2_fd, 0, "link2_fd"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = 0;
+	prog_fd = bpf_program__fd(skel->progs.test_uprobe_sleep);
+	link3_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link1_fd, 0, "link3_fd"))
+		goto cleanup;
+
+	opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+	prog_fd = bpf_program__fd(skel->progs.test_uretprobe_sleep);
+	link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link2_fd, 0, "link4_fd"))
+		goto cleanup;
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	if (link1_fd >= 0)
+		close(link1_fd);
+	if (link2_fd >= 0)
+		close(link2_fd);
+	if (link3_fd >= 0)
+		close(link3_fd);
+	if (link4_fd >= 0)
+		close(link4_fd);
+
+	uprobe_multi__destroy(skel);
+	free(offsets);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
@@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)
 		test_attach_api_pattern();
 	if (test__start_subtest("attach_api_syms"))
 		test_attach_api_syms();
+	if (test__start_subtest("link_api"))
+		test_link_api();
 }
Signature · 1 line
2.41.0
#56 lore raw

Re: [PATCHv3 bpf-next 19/26] selftests/bpf: Add uprobe_multi link test

Quoted 10 lines · Jiri Olsa

Adding uprobe_multi test for bpf_link_create attach function.

Testing attachment using the struct bpf_link_create_opts.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 68 +++++++++++++++++++
 1 file changed, 68 insertions(+)

[...]

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 181

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10 -0
+       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+       prog_fd = bpf_program__fd(skel->progs.test_uretprobe);
+       link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+       if (!ASSERT_GE(link2_fd, 0, "link2_fd"))
+               goto cleanup;
+
+       opts.kprobe_multi.flags = 0;
+       prog_fd = bpf_program__fd(skel->progs.test_uprobe_sleep);
+       link3_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+       if (!ASSERT_GE(link1_fd, 0, "link3_fd"))

link3_fd

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 187

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6 -0
+               goto cleanup;
+
+       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+       prog_fd = bpf_program__fd(skel->progs.test_uretprobe_sleep);
+       link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+       if (!ASSERT_GE(link2_fd, 0, "link4_fd"))

link4_fd

Quoted 30 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +19 -1
+               goto cleanup;
+       uprobe_multi_test_run(skel);
+
+cleanup:
+       if (link1_fd >= 0)
+               close(link1_fd);
+       if (link2_fd >= 0)
+               close(link2_fd);
+       if (link3_fd >= 0)
+               close(link3_fd);
+       if (link4_fd >= 0)
+               close(link4_fd);
+
+       uprobe_multi__destroy(skel);
+       free(offsets);
+}
+
 void test_uprobe_multi_test(void)
 {
        if (test__start_subtest("skel_api"))
@@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)
                test_attach_api_pattern();
        if (test__start_subtest("attach_api_syms"))
                test_attach_api_syms();
+       if (test__start_subtest("link_api"))
+               test_link_api();
 }
--

2.41.0

#57 lore raw

Re: [PATCHv3 bpf-next 19/26] selftests/bpf: Add uprobe_multi link test

Quoted 27 lines · Andrii Nakryiko
Quoted 10 lines · Jiri Olsa

Adding uprobe_multi test for bpf_link_create attach function.

Testing attachment using the struct bpf_link_create_opts.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 68 +++++++++++++++++++
 1 file changed, 68 insertions(+)

[...]

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 181

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10 -0
+       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+       prog_fd = bpf_program__fd(skel->progs.test_uretprobe);
+       link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+       if (!ASSERT_GE(link2_fd, 0, "link2_fd"))
+               goto cleanup;
+
+       opts.kprobe_multi.flags = 0;
+       prog_fd = bpf_program__fd(skel->progs.test_uprobe_sleep);
+       link3_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+       if (!ASSERT_GE(link1_fd, 0, "link3_fd"))

link3_fd

Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_attach_api_syms(void) · line 187

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6 -0
+               goto cleanup;
+
+       opts.kprobe_multi.flags = BPF_F_UPROBE_MULTI_RETURN;
+       prog_fd = bpf_program__fd(skel->progs.test_uretprobe_sleep);
+       link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+       if (!ASSERT_GE(link2_fd, 0, "link4_fd"))

link4_fd

right, will change

thanks,
jirka

Quoted 31 lines
Quoted 31 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +19 -1
+               goto cleanup;
+       uprobe_multi_test_run(skel);
+
+cleanup:
+       if (link1_fd >= 0)
+               close(link1_fd);
+       if (link2_fd >= 0)
+               close(link2_fd);
+       if (link3_fd >= 0)
+               close(link3_fd);
+       if (link4_fd >= 0)
+               close(link4_fd);
+
+       uprobe_multi__destroy(skel);
+       free(offsets);
+}
+
 void test_uprobe_multi_test(void)
 {
        if (test__start_subtest("skel_api"))
@@ -144,4 +210,6 @@ void test_uprobe_multi_test(void)
                test_attach_api_pattern();
        if (test__start_subtest("attach_api_syms"))
                test_attach_api_syms();
+       if (test__start_subtest("link_api"))
+               test_link_api();
 }
--

2.41.0

author reviewer
#58 lore raw

PATCH v3 20/26 selftests/bpf: Add uprobe_multi test program

In reply to Jiri Olsa

Adding uprobe_multi test program that defines 50k uprobe_multi_func_* functions and will serve as attach point for uprobe_multi bench test in following patch.

---
 tools/testing/selftests/bpf/Makefile       |  5 ++
 tools/testing/selftests/bpf/uprobe_multi.c | 53 ++++++++++++++++++++++
 2 files changed, 58 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
tools/testing/selftests/bpf/Makefile +5 -0
index ad6b585e0d7c..acf7c9a29082 100644
@@ -567,6 +567,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko	\
 		       $(OUTPUT)/liburandom_read.so			\
 		       $(OUTPUT)/xdp_synproxy				\
 		       $(OUTPUT)/sign-file				\
+		       $(OUTPUT)/uprobe_multi				\
 		       ima_setup.sh 					\
 		       verify_sig_setup.sh				\
 		       $(wildcard progs/btf_dump_test_case_*.c)		\
@@ -670,6 +671,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o
 	$(call msg,BINARY,,$@)
 	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@
 
+$(OUTPUT)/uprobe_multi: uprobe_multi.c
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+
 EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)	\
 	prog_tests/tests.h map_tests/tests.h verifier/tests.h		\
 	feature bpftool							\
tools/testing/selftests/bpf/uprobe_multi.c new file +53 -0

53 lines not on this page · raw message

Signature · 1 line
2.41.0
#59 lore raw

Re: [PATCHv3 bpf-next 20/26] selftests/bpf: Add uprobe_multi test program

Quoted 17 lines · Jiri Olsa

Adding uprobe_multi test program that defines 50k uprobe_multi_func_* functions and will serve as attach point for uprobe_multi bench test in following patch.

---
 tools/testing/selftests/bpf/Makefile       |  5 ++
 tools/testing/selftests/bpf/uprobe_multi.c | 53 ++++++++++++++++++++++
 2 files changed, 58 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c
tools/testing/selftests/bpf/Makefile +0 -0
index ad6b585e0d7c..acf7c9a29082 100644

tools/testing/selftests/bpf/Makefile TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ · line 570

Quoted 4 lines
tools/testing/selftests/bpf/Makefile +1 -0
@@ -567,6 +567,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko      \
                       $(OUTPUT)/liburandom_read.so                     \
                       $(OUTPUT)/xdp_synproxy                           \
                       $(OUTPUT)/sign-file                              \
+                      $(OUTPUT)/uprobe_multi                           \

would adding all this to urandom_read be too bad in terms of size and performance? I'm just not sure we won't yet more custom binaries and rules in Makefile

Quoted 76 lines
tools/testing/selftests/bpf/Makefile +4 -0
                       ima_setup.sh                                     \
                       verify_sig_setup.sh                              \
                       $(wildcard progs/btf_dump_test_case_*.c)         \
@@ -670,6 +671,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o
        $(call msg,BINARY,,$@)
        $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@
 
+$(OUTPUT)/uprobe_multi: uprobe_multi.c
+       $(call msg,BINARY,,$@)
+       $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+
 EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool                                                 \
tools/testing/selftests/bpf/uprobe_multi.c new file +53 -1
new file mode 100644
index 000000000000..115a7f6cebfa
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <stdio.h>
+
+#define __PASTE(a, b) a##b
+#define PASTE(a, b) __PASTE(a, b)
+
+#define NAME(name, idx) PASTE(name, idx)
+
+#define DEF(name, idx)  int NAME(name, idx)(void) { return 0; }
+#define CALL(name, idx) NAME(name, idx)();
+
+#define F(body, name, idx) body(name, idx)
+
+#define F10(body, name, idx) \
+       F(body, PASTE(name, idx), 0) F(body, PASTE(name, idx), 1) F(body, PASTE(name, idx), 2) \
+       F(body, PASTE(name, idx), 3) F(body, PASTE(name, idx), 4) F(body, PASTE(name, idx), 5) \
+       F(body, PASTE(name, idx), 6) F(body, PASTE(name, idx), 7) F(body, PASTE(name, idx), 8) \
+       F(body, PASTE(name, idx), 9)
+
+#define F100(body, name, idx) \
+       F10(body, PASTE(name, idx), 0) F10(body, PASTE(name, idx), 1) F10(body, PASTE(name, idx), 2) \
+       F10(body, PASTE(name, idx), 3) F10(body, PASTE(name, idx), 4) F10(body, PASTE(name, idx), 5) \
+       F10(body, PASTE(name, idx), 6) F10(body, PASTE(name, idx), 7) F10(body, PASTE(name, idx), 8) \
+       F10(body, PASTE(name, idx), 9)
+
+#define F1000(body, name, idx) \
+       F100(body, PASTE(name, idx), 0) F100(body, PASTE(name, idx), 1) F100(body, PASTE(name, idx), 2) \
+       F100(body, PASTE(name, idx), 3) F100(body, PASTE(name, idx), 4) F100(body, PASTE(name, idx), 5) \
+       F100(body, PASTE(name, idx), 6) F100(body, PASTE(name, idx), 7) F100(body, PASTE(name, idx), 8) \
+       F100(body, PASTE(name, idx), 9)
+
+#define F10000(body, name, idx) \
+       F1000(body, PASTE(name, idx), 0) F1000(body, PASTE(name, idx), 1) F1000(body, PASTE(name, idx), 2) \
+       F1000(body, PASTE(name, idx), 3) F1000(body, PASTE(name, idx), 4) F1000(body, PASTE(name, idx), 5) \
+       F1000(body, PASTE(name, idx), 6) F1000(body, PASTE(name, idx), 7) F1000(body, PASTE(name, idx), 8) \
+       F1000(body, PASTE(name, idx), 9)
+
+F10000(DEF, uprobe_multi_func_, 0)
+F10000(DEF, uprobe_multi_func_, 1)
+F10000(DEF, uprobe_multi_func_, 2)
+F10000(DEF, uprobe_multi_func_, 3)
+F10000(DEF, uprobe_multi_func_, 4)
+
+int main(void)
+{
+       F10000(CALL, uprobe_multi_func_, 0)
+       F10000(CALL, uprobe_multi_func_, 1)
+       F10000(CALL, uprobe_multi_func_, 2)
+       F10000(CALL, uprobe_multi_func_, 3)
+       F10000(CALL, uprobe_multi_func_, 4)
+       return 0;
+}
--

2.41.0

author reviewer
#60 lore raw

PATCH v3 21/26 selftests/bpf: Add uprobe_multi bench test

In reply to Jiri Olsa

Adding test that attaches 50k uprobes in uprobe_multi binary.

After the attach is done we run the binary and make sure we get proper amount of hits.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 56 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        |  9 +++
 2 files changed, 65 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +56 -0
index fd858636b8b0..547d46965d70 100644
@@ -202,6 +202,60 @@ static void test_link_api(void)
 	free(offsets);
 }
 
+static inline __u64 get_time_ns(void)
+{
+	struct timespec t;
+
+	clock_gettime(CLOCK_MONOTONIC, &t);
+	return (__u64) t.tv_sec * 1000000000 + t.tv_nsec;
+}
+
+static void test_bench_attach_uprobe(void)
+{
+	long attach_start_ns, attach_end_ns;
+	long detach_start_ns, detach_end_ns;
+	double attach_delta, detach_delta;
+	struct uprobe_multi *skel = NULL;
+	struct bpf_program *prog;
+	int err;
+
+	skel = uprobe_multi__open();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
+		goto cleanup;
+
+	bpf_object__for_each_program(prog, skel->obj)
+		bpf_program__set_autoload(prog, false);
+
+	bpf_program__set_autoload(skel->progs.test_uprobe_bench, true);
+
+	err = uprobe_multi__load(skel);
+	if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
+		goto cleanup;
+
+	attach_start_ns = get_time_ns();
+
+	err = uprobe_multi__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi__attach"))
+		goto cleanup;
+
+	attach_end_ns = get_time_ns();
+
+	system("./uprobe_multi");
+
+	ASSERT_EQ(skel->bss->count, 50000, "uprobes_count");
+
+cleanup:
+	detach_start_ns = get_time_ns();
+	uprobe_multi__destroy(skel);
+	detach_end_ns = get_time_ns();
+
+	attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
+	detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
+
+	printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
+	printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
@@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)
 		test_attach_api_syms();
 	if (test__start_subtest("link_api"))
 		test_link_api();
+	if (test__start_subtest("bench_uprobe"))
+		test_bench_attach_uprobe();
 }
tools/testing/selftests/bpf/progs/uprobe_multi.c +9 -0
index 1eeb9b7b9cad..cd73139dc881 100644
@@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)
 	uprobe_multi_check(ctx, true, true);
 	return 0;
 }
+
+int count;
+
+SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*")
+int test_uprobe_bench(struct pt_regs *ctx)
+{
+	count++;
+	return 0;
+}
Signature · 1 line
2.41.0
#61 lore raw

Re: [PATCHv3 bpf-next 21/26] selftests/bpf: Add uprobe_multi bench test

Quoted 18 lines · Jiri Olsa

Adding test that attaches 50k uprobes in uprobe_multi binary.

After the attach is done we run the binary and make sure we get proper amount of hits.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 56 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        |  9 +++
 2 files changed, 65 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +0 -0
index fd858636b8b0..547d46965d70 100644
@@ -202,6 +202,60 @@ static void test_link_api(void)
        free(offsets);

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 212

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +8 -0
 }
 
+static inline __u64 get_time_ns(void)
+{
+       struct timespec t;
+
+       clock_gettime(CLOCK_MONOTONIC, &t);
+       return (__u64) t.tv_sec * 1000000000 + t.tv_nsec;
+}
+

hmm.. I would expect we have this helper somewhere in common headers. If not, we should probably move all such helpers into one header and use it everywhere

Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +7 -0
+static void test_bench_attach_uprobe(void)
+{
+       long attach_start_ns, attach_end_ns;
+       long detach_start_ns, detach_end_ns;
+       double attach_delta, detach_delta;
+       struct uprobe_multi *skel = NULL;
+       struct bpf_program *prog;

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 229

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10 -0
+       int err;
+
+       skel = uprobe_multi__open();
+       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
+               goto cleanup;
+
+       bpf_object__for_each_program(prog, skel->obj)
+               bpf_program__set_autoload(prog, false);
+
+       bpf_program__set_autoload(skel->progs.test_uprobe_bench, true);

I don't get why you bothered adding this test_uprobe_bench into progs/uprobe_multi and go through this manual auto-load setting/resetting, instead of just having test_uprobe_bench in a separate skeleton?...

Quoted 17 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +17 -0
+
+       err = uprobe_multi__load(skel);
+       if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
+               goto cleanup;
+
+       attach_start_ns = get_time_ns();
+
+       err = uprobe_multi__attach(skel);
+       if (!ASSERT_OK(err, "uprobe_multi__attach"))
+               goto cleanup;
+
+       attach_end_ns = get_time_ns();
+
+       system("./uprobe_multi");
+
+       ASSERT_EQ(skel->bss->count, 50000, "uprobes_count");
+

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 256

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10 -0
+cleanup:
+       detach_start_ns = get_time_ns();
+       uprobe_multi__destroy(skel);
+       detach_end_ns = get_time_ns();
+
+       attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
+       detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
+
+       printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
+       printf("%s: detached in %7.3lfs\n", __func__, detach_delta);

and for us lazy folks, what are the numbers you see on your machine?

Quoted 32 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +4 -0
+}
+
 void test_uprobe_multi_test(void)
 {
        if (test__start_subtest("skel_api"))
@@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)
                test_attach_api_syms();
        if (test__start_subtest("link_api"))
                test_link_api();
+       if (test__start_subtest("bench_uprobe"))
+               test_bench_attach_uprobe();
 }
tools/testing/selftests/bpf/progs/uprobe_multi.c +9 -1
index 1eeb9b7b9cad..cd73139dc881 100644
@@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)
        uprobe_multi_check(ctx, true, true);
        return 0;
 }
+
+int count;
+
+SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*")
+int test_uprobe_bench(struct pt_regs *ctx)
+{
+       count++;
+       return 0;
+}
--

2.41.0

#62 lore raw

Re: [PATCHv3 bpf-next 21/26] selftests/bpf: Add uprobe_multi bench test

Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Jiri Olsa

Adding test that attaches 50k uprobes in uprobe_multi binary.

After the attach is done we run the binary and make sure we get proper amount of hits.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 56 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        |  9 +++
 2 files changed, 65 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +0 -0
index fd858636b8b0..547d46965d70 100644
@@ -202,6 +202,60 @@ static void test_link_api(void)
        free(offsets);
 }
 
Quoted 12 lines

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 212

Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +8 -0
+static inline __u64 get_time_ns(void)
+{
+       struct timespec t;
+
+       clock_gettime(CLOCK_MONOTONIC, &t);
+       return (__u64) t.tv_sec * 1000000000 + t.tv_nsec;
+}
+

hmm.. I would expect we have this helper somewhere in common headers. If not, we should probably move all such helpers into one header and use it everywhere

hum, now I see it's also defined in bench.h, I'll try to use it from there ;-) also perhaps in kprobe multi test

Quoted 15 lines
Quoted 15 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +14 -0
+static void test_bench_attach_uprobe(void)
+{
+       long attach_start_ns, attach_end_ns;
+       long detach_start_ns, detach_end_ns;
+       double attach_delta, detach_delta;
+       struct uprobe_multi *skel = NULL;
+       struct bpf_program *prog;
+       int err;
+
+       skel = uprobe_multi__open();
+       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
+               goto cleanup;
+
+       bpf_object__for_each_program(prog, skel->obj)
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 229

Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +3 -0
+               bpf_program__set_autoload(prog, false);
+
+       bpf_program__set_autoload(skel->progs.test_uprobe_bench, true);

I don't get why you bothered adding this test_uprobe_bench into progs/uprobe_multi and go through this manual auto-load setting/resetting, instead of just having test_uprobe_bench in a separate skeleton?...

ok, will add separate file for that.. will be simpler

Quoted 22 lines
Quoted 22 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +21 -0
+
+       err = uprobe_multi__load(skel);
+       if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
+               goto cleanup;
+
+       attach_start_ns = get_time_ns();
+
+       err = uprobe_multi__attach(skel);
+       if (!ASSERT_OK(err, "uprobe_multi__attach"))
+               goto cleanup;
+
+       attach_end_ns = get_time_ns();
+
+       system("./uprobe_multi");
+
+       ASSERT_EQ(skel->bss->count, 50000, "uprobes_count");
+
+cleanup:
+       detach_start_ns = get_time_ns();
+       uprobe_multi__destroy(skel);
+       detach_end_ns = get_time_ns();
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_link_api(void) · line 256

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6 -0
+
+       attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
+       detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
+
+       printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
+       printf("%s: detached in %7.3lfs\n", __func__, detach_delta);

and for us lazy folks, what are the numbers you see on your machine?

right, will put it to the changelog

	# ./test_progs -n 260/5 -v
	bpf_testmod.ko is already unloaded.
	Loading bpf_testmod.ko...
	Successfully loaded bpf_testmod.ko.
	test_bench_attach_uprobe:PASS:uprobe_multi__open 0 nsec
	test_bench_attach_uprobe:PASS:uprobe_multi__load 0 nsec
	test_bench_attach_uprobe:PASS:uprobe_multi__attach 0 nsec
	test_bench_attach_uprobe:PASS:uprobes_count 0 nsec
	test_bench_attach_uprobe: attached in   0.398s
	test_bench_attach_uprobe: detached in   0.443s
	#260/5   uprobe_multi_test/bench_uprobe:OK
	#260     uprobe_multi_test:OK
	Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
	Successfully unloaded bpf_testmod.ko.

thanks,
jirka

Quoted 33 lines
Quoted 33 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +4 -0
+}
+
 void test_uprobe_multi_test(void)
 {
        if (test__start_subtest("skel_api"))
@@ -212,4 +266,6 @@ void test_uprobe_multi_test(void)
                test_attach_api_syms();
        if (test__start_subtest("link_api"))
                test_link_api();
+       if (test__start_subtest("bench_uprobe"))
+               test_bench_attach_uprobe();
 }
tools/testing/selftests/bpf/progs/uprobe_multi.c +9 -1
index 1eeb9b7b9cad..cd73139dc881 100644
@@ -89,3 +89,12 @@ int test_uretprobe_sleep(struct pt_regs *ctx)
        uprobe_multi_check(ctx, true, true);
        return 0;
 }
+
+int count;
+
+SEC("?uprobe.multi/./uprobe_multi:uprobe_multi_func_*")
+int test_uprobe_bench(struct pt_regs *ctx)
+{
+       count++;
+       return 0;
+}
--

2.41.0

author reviewer
#63 lore raw

PATCH v3 22/26 selftests/bpf: Add usdt_multi test program

In reply to Jiri Olsa

Adding usdt_multi test program that defines 50k usdts and will serve as attach point for uprobe_multi usdt bench test in following patch.

---
 tools/testing/selftests/bpf/Makefile     |  5 +++++
 tools/testing/selftests/bpf/usdt_multi.c | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/usdt_multi.c
tools/testing/selftests/bpf/Makefile +5 -0
index acf7c9a29082..9762467cf0ba 100644
@@ -568,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko	\
 		       $(OUTPUT)/xdp_synproxy				\
 		       $(OUTPUT)/sign-file				\
 		       $(OUTPUT)/uprobe_multi				\
+		       $(OUTPUT)/usdt_multi				\
 		       ima_setup.sh 					\
 		       verify_sig_setup.sh				\
 		       $(wildcard progs/btf_dump_test_case_*.c)		\
@@ -675,6 +676,10 @@ $(OUTPUT)/uprobe_multi: uprobe_multi.c
 	$(call msg,BINARY,,$@)
 	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
 
+$(OUTPUT)/usdt_multi: usdt_multi.c
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+
 EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)	\
 	prog_tests/tests.h map_tests/tests.h verifier/tests.h		\
 	feature bpftool							\
tools/testing/selftests/bpf/usdt_multi.c new file +24 -0
new file mode 100644
index 000000000000..fedf856bad2b
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <sdt.h>
+
+#define PROBE STAP_PROBE(test, usdt);
+
+#define PROBE10    PROBE PROBE PROBE PROBE PROBE \
+		   PROBE PROBE PROBE PROBE PROBE
+#define PROBE100   PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \
+		   PROBE10 PROBE10 PROBE10 PROBE10 PROBE10
+#define PROBE1000  PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \
+		   PROBE100 PROBE100 PROBE100 PROBE100 PROBE100
+#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \
+		   PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000
+
+int main(void)
+{
+	PROBE10000
+	PROBE10000
+	PROBE10000
+	PROBE10000
+	PROBE10000
+	return 0;
+}
Signature · 1 line
2.41.0
#64 lore raw

Re: [PATCHv3 bpf-next 22/26] selftests/bpf: Add usdt_multi test program

Quoted 17 lines · Jiri Olsa

Adding usdt_multi test program that defines 50k usdts and will serve as attach point for uprobe_multi usdt bench test in following patch.

---
 tools/testing/selftests/bpf/Makefile     |  5 +++++
 tools/testing/selftests/bpf/usdt_multi.c | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/usdt_multi.c
tools/testing/selftests/bpf/Makefile +0 -0
index acf7c9a29082..9762467cf0ba 100644

tools/testing/selftests/bpf/Makefile TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ · line 571

Quoted 4 lines
tools/testing/selftests/bpf/Makefile +1 -0
@@ -568,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko      \
                       $(OUTPUT)/xdp_synproxy                           \
                       $(OUTPUT)/sign-file                              \
                       $(OUTPUT)/uprobe_multi                           \
+                      $(OUTPUT)/usdt_multi                             \

that's a bit too much, if you can't put everything into urandom_read, let's at least combine uprobe_multi and usdt_multi

Quoted 47 lines
tools/testing/selftests/bpf/Makefile +4 -0
                       ima_setup.sh                                     \
                       verify_sig_setup.sh                              \
                       $(wildcard progs/btf_dump_test_case_*.c)         \
@@ -675,6 +676,10 @@ $(OUTPUT)/uprobe_multi: uprobe_multi.c
        $(call msg,BINARY,,$@)
        $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
 
+$(OUTPUT)/usdt_multi: usdt_multi.c
+       $(call msg,BINARY,,$@)
+       $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $^ $(LDLIBS) -o $@
+
 EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool                                                 \
tools/testing/selftests/bpf/usdt_multi.c new file +24 -1
new file mode 100644
index 000000000000..fedf856bad2b
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <sdt.h>
+
+#define PROBE STAP_PROBE(test, usdt);
+
+#define PROBE10    PROBE PROBE PROBE PROBE PROBE \
+                  PROBE PROBE PROBE PROBE PROBE
+#define PROBE100   PROBE10 PROBE10 PROBE10 PROBE10 PROBE10 \
+                  PROBE10 PROBE10 PROBE10 PROBE10 PROBE10
+#define PROBE1000  PROBE100 PROBE100 PROBE100 PROBE100 PROBE100 \
+                  PROBE100 PROBE100 PROBE100 PROBE100 PROBE100
+#define PROBE10000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000 \
+                  PROBE1000 PROBE1000 PROBE1000 PROBE1000 PROBE1000
+
+int main(void)
+{
+       PROBE10000
+       PROBE10000
+       PROBE10000
+       PROBE10000
+       PROBE10000
+       return 0;
+}
--

2.41.0

author reviewer
#65 lore raw

PATCH v3 23/26 selftests/bpf: Add usdt_multi bench test

In reply to Jiri Olsa

Adding test that attaches 50k usdt probes in usdt_multi binary.

After the attach is done we run the binary and make sure we get proper amount of hits.

With current uprobes:

  # perf stat --null ./test_progs -n 254/6
  #254/6   uprobe_multi_test/bench_usdt:OK
  #254     uprobe_multi_test:OK
  Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

Performance counter stats for './test_progs -n 254/6':

      1353.659680562 seconds time elapsed

With uprobe_multi link:

  # perf stat --null ./test_progs -n 254/6
  #254/6   uprobe_multi_test/bench_usdt:OK
  #254     uprobe_multi_test:OK
  Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

Performance counter stats for './test_progs -n 254/6':

         0.322046364 seconds time elapsed
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 50 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi_usdt.c   | 16 ++++++
 2 files changed, 66 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +50 -0
index 547d46965d70..b12dc1f992e5 100644
@@ -4,6 +4,7 @@
 #include <test_progs.h>
 #include "uprobe_multi.skel.h"
 #include "bpf/libbpf_elf.h"
+#include "uprobe_multi_usdt.skel.h"
 
 static char test_data[] = "test_data";
 
@@ -256,6 +257,53 @@ static void test_bench_attach_uprobe(void)
 	printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
 }
 
+static void test_bench_attach_usdt(void)
+{
+	struct uprobe_multi_usdt *skel = NULL;
+	long attach_start_ns, attach_end_ns;
+	long detach_start_ns, detach_end_ns;
+	double attach_delta, detach_delta;
+	struct bpf_program *prog;
+	int err;
+
+	skel = uprobe_multi_usdt__open();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
+		goto cleanup;
+
+	bpf_object__for_each_program(prog, skel->obj)
+		bpf_program__set_autoload(prog, false);
+
+	bpf_program__set_autoload(skel->progs.usdt0, true);
+
+	err = uprobe_multi_usdt__load(skel);
+	if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
+		goto cleanup;
+
+	attach_start_ns = get_time_ns();
+
+	skel->links.usdt0 = bpf_program__attach_usdt(skel->progs.usdt0, -1, "./usdt_multi",
+						     "test", "usdt", NULL);
+	if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
+		goto cleanup;
+
+	attach_end_ns = get_time_ns();
+
+	system("./usdt_multi");
+
+	ASSERT_EQ(skel->bss->count, 50000, "usdt_count");
+
+cleanup:
+	detach_start_ns = get_time_ns();
+	uprobe_multi_usdt__destroy(skel);
+	detach_end_ns = get_time_ns();
+
+	attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
+	detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
+
+	printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
+	printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
@@ -268,4 +316,6 @@ void test_uprobe_multi_test(void)
 		test_link_api();
 	if (test__start_subtest("bench_uprobe"))
 		test_bench_attach_uprobe();
+	if (test__start_subtest("bench_usdt"))
+		test_bench_attach_usdt();
 }
tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c new file +16 -0
new file mode 100644
index 000000000000..9e1c33d0bd2f
@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/usdt.bpf.h>
+
+char _license[] SEC("license") = "GPL";
+
+int count;
+
+SEC("usdt")
+int usdt0(struct pt_regs *ctx)
+{
+	count++;
+	return 0;
+}
Signature · 1 line
2.41.0
#66 lore raw

Re: [PATCHv3 bpf-next 23/26] selftests/bpf: Add usdt_multi bench test

Quoted 8 lines · Jiri Olsa

Adding test that attaches 50k usdt probes in usdt_multi binary.

After the attach is done we run the binary and make sure we get proper amount of hits.

With current uprobes:

Quoted 9 lines
  # perf stat --null ./test_progs -n 254/6
  #254/6   uprobe_multi_test/bench_usdt:OK
  #254     uprobe_multi_test:OK
  Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

Performance counter stats for './test_progs -n 254/6':

      1353.659680562 seconds time elapsed

20 minutes, I commend your patience!

Quoted 10 lines

With uprobe_multi link:

  # perf stat --null ./test_progs -n 254/6
  #254/6   uprobe_multi_test/bench_usdt:OK
  #254     uprobe_multi_test:OK
  Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

Performance counter stats for './test_progs -n 254/6':

         0.322046364 seconds time elapsed

This is an impressive speed up, especially taking into account that there is no batch attachment magic that we had to do for kprobes. Very nice!

Quoted 31 lines
---
 .../bpf/prog_tests/uprobe_multi_test.c        | 50 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi_usdt.c   | 16 ++++++
 2 files changed, 66 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +8 -0
index 547d46965d70..b12dc1f992e5 100644
@@ -4,6 +4,7 @@
 #include <test_progs.h>
 #include "uprobe_multi.skel.h"
 #include "bpf/libbpf_elf.h"
+#include "uprobe_multi_usdt.skel.h"
 
 static char test_data[] = "test_data";
 
@@ -256,6 +257,53 @@ static void test_bench_attach_uprobe(void)
        printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
 }
 
+static void test_bench_attach_usdt(void)
+{
+       struct uprobe_multi_usdt *skel = NULL;
+       long attach_start_ns, attach_end_ns;
+       long detach_start_ns, detach_end_ns;
+       double attach_delta, detach_delta;
+       struct bpf_program *prog;

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_bench_attach_uprobe(void) · line 276

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +10 -0
+       int err;
+
+       skel = uprobe_multi_usdt__open();
+       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
+               goto cleanup;
+
+       bpf_object__for_each_program(prog, skel->obj)
+               bpf_program__set_autoload(prog, false);
+
+       bpf_program__set_autoload(skel->progs.usdt0, true);

there is nothing else in that skeleton, why this set_autoload() business?

Quoted 65 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +32 -0
+
+       err = uprobe_multi_usdt__load(skel);
+       if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
+               goto cleanup;
+
+       attach_start_ns = get_time_ns();
+
+       skel->links.usdt0 = bpf_program__attach_usdt(skel->progs.usdt0, -1, "./usdt_multi",
+                                                    "test", "usdt", NULL);
+       if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
+               goto cleanup;
+
+       attach_end_ns = get_time_ns();
+
+       system("./usdt_multi");
+
+       ASSERT_EQ(skel->bss->count, 50000, "usdt_count");
+
+cleanup:
+       detach_start_ns = get_time_ns();
+       uprobe_multi_usdt__destroy(skel);
+       detach_end_ns = get_time_ns();
+
+       attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
+       detach_delta = (detach_end_ns - detach_start_ns) / 1000000000.0;
+
+       printf("%s: attached in %7.3lfs\n", __func__, attach_delta);
+       printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
+}
+
 void test_uprobe_multi_test(void)
 {
        if (test__start_subtest("skel_api"))
@@ -268,4 +316,6 @@ void test_uprobe_multi_test(void)
                test_link_api();
        if (test__start_subtest("bench_uprobe"))
                test_bench_attach_uprobe();
+       if (test__start_subtest("bench_usdt"))
+               test_bench_attach_usdt();
 }
tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c new file +16 -1
new file mode 100644
index 000000000000..9e1c33d0bd2f
@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/usdt.bpf.h>
+
+char _license[] SEC("license") = "GPL";
+
+int count;
+
+SEC("usdt")
+int usdt0(struct pt_regs *ctx)
+{
+       count++;
+       return 0;
+}
--

2.41.0

#67 lore raw

Re: [PATCHv3 bpf-next 23/26] selftests/bpf: Add usdt_multi bench test

On Thu, Jul 06, 2023 at 09:42:05PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +11 -0
+static void test_bench_attach_usdt(void)
+{
+       struct uprobe_multi_usdt *skel = NULL;
+       long attach_start_ns, attach_end_ns;
+       long detach_start_ns, detach_end_ns;
+       double attach_delta, detach_delta;
+       struct bpf_program *prog;
+       int err;
+
+       skel = uprobe_multi_usdt__open();
+       if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c static void test_bench_attach_uprobe(void) · line 276

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +6 -0
+               goto cleanup;
+
+       bpf_object__for_each_program(prog, skel->obj)
+               bpf_program__set_autoload(prog, false);
+
+       bpf_program__set_autoload(skel->progs.usdt0, true);

there is nothing else in that skeleton, why this set_autoload() business?

nah copy&paste error from the uprobe bench test.. will change ;-)

jirka

SNIP

author reviewer
#68 lore raw

PATCH v3 24/26 selftests/bpf: Add uprobe_multi cookie test

In reply to Jiri Olsa

Adding test for cookies setup/retrieval in uprobe_link uprobes and making sure bpf_get_attach_cookie works properly.

---
 .../selftests/bpf/prog_tests/bpf_cookie.c     | 78 +++++++++++++++++++
 1 file changed, 78 insertions(+)
tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +78 -0
index 26b2d1bffdfd..4162d1724b79 100644
@@ -11,6 +11,7 @@
 #include <bpf/btf.h>
 #include "test_bpf_cookie.skel.h"
 #include "kprobe_multi.skel.h"
+#include "uprobe_multi.skel.h"
 
 /* uprobe attach point */
 static noinline void trigger_func(void)
@@ -239,6 +240,81 @@ static void kprobe_multi_attach_api_subtest(void)
 	bpf_link__destroy(link1);
 	kprobe_multi__destroy(skel);
 }
+
+/* defined in prog_tests/uprobe_multi_test.c */
+void uprobe_multi_func_1(void);
+void uprobe_multi_func_2(void);
+void uprobe_multi_func_3(void);
+
+static void uprobe_multi_test_run(struct uprobe_multi *skel)
+{
+	skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1;
+	skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2;
+	skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3;
+
+	skel->bss->pid = getpid();
+	skel->bss->test_cookie = true;
+
+	uprobe_multi_func_1();
+	uprobe_multi_func_2();
+	uprobe_multi_func_3();
+
+	ASSERT_EQ(skel->bss->uprobe_multi_func_1_result, 1, "uprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_2_result, 1, "uprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uprobe_multi_func_3_result, 1, "uprobe_multi_func_3_result");
+
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_1_result, 1, "uretprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_2_result, 1, "uretprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 1, "uretprobe_multi_func_3_result");
+}
+
+static void uprobe_multi_attach_api_subtest(void)
+{
+	struct bpf_link *link1 = NULL, *link2 = NULL;
+	struct uprobe_multi *skel = NULL;
+	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+	const char *syms[3] = {
+		"uprobe_multi_func_1",
+		"uprobe_multi_func_2",
+		"uprobe_multi_func_3",
+	};
+	__u64 cookies[3];
+
+	cookies[0] = 3; /* uprobe_multi_func_1 */
+	cookies[1] = 1; /* uprobe_multi_func_2 */
+	cookies[2] = 2; /* uprobe_multi_func_3 */
+
+	opts.syms = syms;
+	opts.cnt = ARRAY_SIZE(syms);
+	opts.cookies = &cookies[0];
+
+	skel = uprobe_multi__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi"))
+		goto cleanup;
+
+	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+						 "/proc/self/exe", NULL, &opts);
+	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
+	cookies[0] = 2; /* uprobe_multi_func_1 */
+	cookies[1] = 3; /* uprobe_multi_func_2 */
+	cookies[2] = 1; /* uprobe_multi_func_3 */
+
+	opts.retprobe = true;
+	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+						      "/proc/self/exe", NULL, &opts);
+	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
+		goto cleanup;
+
+	uprobe_multi_test_run(skel);
+
+cleanup:
+	bpf_link__destroy(link2);
+	bpf_link__destroy(link1);
+	uprobe_multi__destroy(skel);
+}
+
 static void uprobe_subtest(struct test_bpf_cookie *skel)
 {
 	DECLARE_LIBBPF_OPTS(bpf_uprobe_opts, opts);
@@ -515,6 +591,8 @@ void test_bpf_cookie(void)
 		kprobe_multi_attach_api_subtest();
 	if (test__start_subtest("uprobe"))
 		uprobe_subtest(skel);
+	if (test__start_subtest("multi_uprobe_attach_api"))
+		uprobe_multi_attach_api_subtest();
 	if (test__start_subtest("tracepoint"))
 		tp_subtest(skel);
 	if (test__start_subtest("perf_event"))
Signature · 1 line
2.41.0
author reviewer
#69 lore raw

PATCH v3 25/26 selftests/bpf: Add uprobe_multi pid filter tests

In reply to Jiri Olsa

Running api and link tests also with pid filter and checking the probe gets executed only for specific pid.

Spawning extra process to trigger attached uprobes and checking we get correct counts from executed programs.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 131 ++++++++++++++++--
 .../selftests/bpf/progs/uprobe_multi.c        |   6 +-
 2 files changed, 125 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +120 -11
index b12dc1f992e5..8ca7a45e21e6 100644
@@ -23,14 +23,86 @@ noinline void uprobe_multi_func_3(void)
 	asm volatile ("");
 }
 
-static void uprobe_multi_test_run(struct uprobe_multi *skel)
+struct child {
+	int go[2];
+	int pid;
+};
+
+static void release_child(struct child *child)
+{
+	int child_status;
+
+	if (!child)
+		return;
+	close(child->go[1]);
+	close(child->go[0]);
+	if (child->pid > 0)
+		waitpid(child->pid, &child_status, 0);
+}
+
+static void kick_child(struct child *child)
+{
+	char c = 1;
+
+	if (child) {
+		write(child->go[1], &c, 1);
+		release_child(child);
+	}
+	fflush(NULL);
+}
+
+static struct child *spawn_child(void)
+{
+	static struct child child;
+	int err;
+	int c;
+
+	/* pid filter */
+	if (!ASSERT_OK(pipe(child.go), "pipe"))
+		return NULL;
+
+	child.pid = fork();
+	if (child.pid < 0) {
+		release_child(&child);
+		return NULL;
+	}
+
+	/* child */
+	if (child.pid == 0) {
+		close(child.go[1]);
+		fflush(NULL);
+		/* wait for parent's kick */
+		err = read(child.go[0], &c, 1);
+		if (!ASSERT_EQ(err, 1, "child_read_pipe"))
+			exit(err);
+
+		uprobe_multi_func_1();
+		uprobe_multi_func_2();
+		uprobe_multi_func_3();
+
+		exit(errno);
+	}
+
+	return &child;
+}
+
+static void uprobe_multi_test_run(struct uprobe_multi *skel, struct child *child)
 {
 	skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1;
 	skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2;
 	skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3;
 
 	skel->bss->user_ptr = test_data;
-	skel->bss->pid = getpid();
+
+	/*
+	 * Disable pid check in bpf program if we are pid filter test,
+	 * because the probe should be executed only by child->pid
+	 * passed at the probe attach.
+	 */
+	skel->bss->pid = child ? 0 : getpid();
+
+	if (child)
+		kick_child(child);
 
 	/* trigger all probes */
 	uprobe_multi_func_1();
@@ -50,6 +122,9 @@ static void uprobe_multi_test_run(struct uprobe_multi *skel)
 	ASSERT_EQ(skel->bss->uretprobe_multi_func_3_result, 2, "uretprobe_multi_func_3_result");
 
 	ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 6, "uprobe_multi_sleep_result");
+
+	if (child)
+		ASSERT_EQ(skel->bss->child_pid, child->pid, "uprobe_multi_child_pid");
 }
 
 static void test_skel_api(void)
@@ -65,17 +140,19 @@ static void test_skel_api(void)
 	if (!ASSERT_OK(err, "uprobe_multi__attach"))
 		goto cleanup;
 
-	uprobe_multi_test_run(skel);
+	uprobe_multi_test_run(skel, NULL);
 
 cleanup:
 	uprobe_multi__destroy(skel);
 }
 
 static void
-test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+__test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts,
+		  struct child *child)
 {
 	struct bpf_link *link1 = NULL, *link2 = NULL;
 	struct bpf_link *link3 = NULL, *link4 = NULL;
+	pid_t pid = child ? child->pid : -1;
 	struct uprobe_multi *skel = NULL;
 
 	skel = uprobe_multi__open_and_load();
@@ -83,30 +160,30 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi
 		goto cleanup;
 
 	opts->retprobe = false;
-	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, -1,
+	link1 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
 		goto cleanup;
 
 	opts->retprobe = true;
-	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, -1,
+	link2 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
 		goto cleanup;
 
 	opts->retprobe = false;
-	link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, -1,
+	link3 = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_sleep, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link1, "bpf_program__attach_uprobe_multi"))
 		goto cleanup;
 
 	opts->retprobe = true;
-	link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, -1,
+	link4 = bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_sleep, pid,
 						      binary, pattern, opts);
 	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
 		goto cleanup;
 
-	uprobe_multi_test_run(skel);
+	uprobe_multi_test_run(skel, child);
 
 cleanup:
 	bpf_link__destroy(link4);
@@ -116,6 +193,22 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi
 	uprobe_multi__destroy(skel);
 }
 
+static void
+test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi_opts *opts)
+{
+	struct child *child;
+
+	/* no pid filter */
+	__test_attach_api(binary, pattern, opts, NULL);
+
+	/* pid filter */
+	child = spawn_child();
+	if (!child)
+		return;
+
+	__test_attach_api(binary, pattern, opts, child);
+}
+
 static void test_attach_api_pattern(void)
 {
 	LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
@@ -138,7 +231,7 @@ static void test_attach_api_syms(void)
 	test_attach_api("/proc/self/exe", NULL, &opts);
 }
 
-static void test_link_api(void)
+static void __test_link_api(struct child *child)
 {
 	int prog_fd, link1_fd = -1, link2_fd = -1, link3_fd = -1, link4_fd = -1;
 	LIBBPF_OPTS(bpf_link_create_opts, opts);
@@ -159,6 +252,7 @@ static void test_link_api(void)
 	opts.uprobe_multi.path = path;
 	opts.uprobe_multi.offsets = offsets;
 	opts.uprobe_multi.cnt = ARRAY_SIZE(syms);
+	opts.uprobe_multi.pid = child ? child->pid : 0;
 
 	skel = uprobe_multi__open_and_load();
 	if (!ASSERT_OK_PTR(skel, "uprobe_multi__open_and_load"))
@@ -187,7 +281,7 @@ static void test_link_api(void)
 	link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
 	if (!ASSERT_GE(link2_fd, 0, "link4_fd"))
 		goto cleanup;
-	uprobe_multi_test_run(skel);
+	uprobe_multi_test_run(skel, child);
 
 cleanup:
 	if (link1_fd >= 0)
@@ -203,6 +297,21 @@ static void test_link_api(void)
 	free(offsets);
 }
 
+void test_link_api(void)
+{
+	struct child *child;
+
+	/* no pid filter */
+	__test_link_api(NULL);
+
+	/* pid filter */
+	child = spawn_child();
+	if (!child)
+		return;
+
+	__test_link_api(child);
+}
+
 static inline __u64 get_time_ns(void)
 {
 	struct timespec t;
tools/testing/selftests/bpf/progs/uprobe_multi.c +5 -1
index cd73139dc881..1086312b5d1e 100644
@@ -21,6 +21,8 @@ __u64 uretprobe_multi_func_3_result = 0;
 __u64 uprobe_multi_sleep_result = 0;
 
 int pid = 0;
+int child_pid = 0;
+
 bool test_cookie = false;
 void *user_ptr = 0;
 
@@ -34,7 +36,9 @@ static __always_inline bool verify_sleepable_user_copy(void)
 
 static void uprobe_multi_check(void *ctx, bool is_return, bool is_sleep)
 {
-	if (bpf_get_current_pid_tgid() >> 32 != pid)
+	child_pid = bpf_get_current_pid_tgid() >> 32;
+
+	if (pid && child_pid != pid)
 		return;
 
 	__u64 cookie = test_cookie ? bpf_get_attach_cookie(ctx) : 0;
Signature · 1 line
2.41.0
author reviewer
#70 lore raw

PATCH v3 26/26 selftests/bpf: Add extra link to uprobe_multi tests

In reply to Jiri Olsa

Attaching extra program to same functions system wide for api and link tests.

This way we can test the pid filter works properly when there's extra system wide consumer on the same uprobe that will trigger the original uprobe handler.

We expect to have the same counts as before.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 19 +++++++++++++++++++
 .../selftests/bpf/progs/uprobe_multi.c        |  6 ++++++
 2 files changed, 25 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +19 -0
index 8ca7a45e21e6..39bd16786cd0 100644
@@ -153,6 +153,7 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mul
 	struct bpf_link *link1 = NULL, *link2 = NULL;
 	struct bpf_link *link3 = NULL, *link4 = NULL;
 	pid_t pid = child ? child->pid : -1;
+	struct bpf_link *link_extra = NULL;
 	struct uprobe_multi *skel = NULL;
 
 	skel = uprobe_multi__open_and_load();
@@ -183,9 +184,16 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mul
 	if (!ASSERT_OK_PTR(link2, "bpf_program__attach_uprobe_multi_retprobe"))
 		goto cleanup;
 
+	opts->retprobe = false;
+	link_extra = bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_extra, -1,
+						      binary, pattern, opts);
+	if (!ASSERT_OK_PTR(link_extra, "bpf_program__attach_uprobe_multi"))
+		goto cleanup;
+
 	uprobe_multi_test_run(skel, child);
 
 cleanup:
+	bpf_link__destroy(link_extra);
 	bpf_link__destroy(link4);
 	bpf_link__destroy(link3);
 	bpf_link__destroy(link2);
@@ -243,6 +251,7 @@ static void __test_link_api(struct child *child)
 		"uprobe_multi_func_2",
 		"uprobe_multi_func_3",
 	};
+	int link_extra_fd = -1;
 	int err;
 
 	err = elf_resolve_syms_offsets(path, 3, syms, (unsigned long **) &offsets);
@@ -281,6 +290,14 @@ static void __test_link_api(struct child *child)
 	link4_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
 	if (!ASSERT_GE(link2_fd, 0, "link4_fd"))
 		goto cleanup;
+
+	opts.kprobe_multi.flags = 0;
+	opts.uprobe_multi.pid = 0;
+	prog_fd = bpf_program__fd(skel->progs.test_uprobe_extra);
+	link_extra_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts);
+	if (!ASSERT_GE(link_extra_fd, 0, "link_extra_fd"))
+		goto cleanup;
+
 	uprobe_multi_test_run(skel, child);
 
 cleanup:
@@ -292,6 +309,8 @@ static void __test_link_api(struct child *child)
 		close(link3_fd);
 	if (link4_fd >= 0)
 		close(link4_fd);
+	if (link_extra_fd >= 0)
+		close(link_extra_fd);
 
 	uprobe_multi__destroy(skel);
 	free(offsets);
tools/testing/selftests/bpf/progs/uprobe_multi.c +6 -0
index 1086312b5d1e..8ee957670303 100644
@@ -102,3 +102,9 @@ int test_uprobe_bench(struct pt_regs *ctx)
 	count++;
 	return 0;
 }
+
+SEC("uprobe.multi//proc/self/exe:uprobe_multi_func_*")
+int test_uprobe_extra(struct pt_regs *ctx)
+{
+	return 0;
+}
Signature · 1 line
2.41.0
maintainer
#71 lore raw
In reply to Jiri Olsa
Quoted 111 lines · Jiri Olsa

hi,
this patchset is adding support to attach multiple uprobes and usdt probes through new uprobe_multi link.

The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.

The main reason is that we need to install perf event for each probed function and profile shows perf event installation (perf_install_in_context) as culprit.

The new uprobe_multi link just creates raw uprobes and attaches the bpf program to them without perf event being involved.

In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to.

v3 changes:
- consolidate attach type checks in bpf_prog_attach_check_attach_type [Andrii]
- remove bpf_prog_active check [Alexei]
- change rcu locking [Andrii]
- allocate ref_ctr_offsets conditionally [Andrii]
- remove ctx check from bpf_uprobe_multi_cookie [Andrii]
- move some elf_* functions in elf.c object [Andrii]
- fix uprobe link detection code [Andrii]
- add usdt.s program section [Andrii]
- rename bpf_program__attach_uprobe_multi_opts to bpf_program__attach_uprobe_multi [Andrii]
- remove extra case from attach_uprobe_multi [Andrii]
- rework usdt_manager_attach_usdt [Andrii]
- rework/rename new elf_find_* functions [Andrii]
- elf iterator fixes [Andrii]
- renames
- elf_sym_iter_next loop restruct
- return directly GElf_Sym in elf_sym
- add elf_sym_offset helper
- add st_type arg elf_sym_iter_new
- get rid of '_' prefixed functions
- simplify offsets handling
- other smaller changes
- added acks
- todo:
- seems like more elf_* helpers could go in elf.c object,
I'll send that as follow up
- will send fill_info support when [3] is merged [Andrii]

There's support for bpftrace [2] and tetragon [1].

Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
uprobe_multi

thanks,
jirka

[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (26):
bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
bpf: Add multi uprobe link
bpf: Add cookies support for uprobe_multi link
bpf: Add pid filter support for uprobe_multi link
bpf: Add bpf_get_func_ip helper support for uprobe link
libbpf: Add uprobe_multi attach type and link names
libbpf: Move elf_find_func_offset* functions to elf object
libbpf: Add elf_open/elf_close functions
libbpf: Add elf symbol iterator
libbpf: Add elf_resolve_syms_offsets function
libbpf: Add elf_resolve_pattern_offsets function
libbpf: Add bpf_link_create support for multi uprobes
libbpf: Add bpf_program__attach_uprobe_multi function
libbpf: Add support for u[ret]probe.multi[.s] program sections
libbpf: Add uprobe multi link detection
libbpf: Add uprobe multi link support to bpf_program__attach_usdt selftests/bpf: Add uprobe_multi skel test
selftests/bpf: Add uprobe_multi api test
selftests/bpf: Add uprobe_multi link test
selftests/bpf: Add uprobe_multi test program
selftests/bpf: Add uprobe_multi bench test
selftests/bpf: Add usdt_multi test program
selftests/bpf: Add usdt_multi bench test
selftests/bpf: Add uprobe_multi cookie test
selftests/bpf: Add uprobe_multi pid filter tests
selftests/bpf: Add extra link to uprobe_multi tests

  include/linux/trace_events.h                               |   6 ++
  include/uapi/linux/bpf.h                                   |  16 ++++
  kernel/bpf/syscall.c                                       | 122 ++++++++++++-------------
  kernel/trace/bpf_trace.c                                   | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
  tools/include/uapi/linux/bpf.h                             |  16 ++++
  tools/lib/bpf/Build                                        |   2 +-
  tools/lib/bpf/bpf.c                                        |  11 +++
  tools/lib/bpf/bpf.h                                        |  11 ++-
  tools/lib/bpf/elf.c                                        | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  tools/lib/bpf/libbpf.c                                     | 396 ++++++++++++++++++++++++++++++++++++++++++---------------------------------------
  tools/lib/bpf/libbpf.h                                     |  27 ++++++
  tools/lib/bpf/libbpf.map                                   |   1 +
  tools/lib/bpf/libbpf_elf.h                                 |  24 +++++
  tools/lib/bpf/libbpf_internal.h                            |   3 +
  tools/lib/bpf/usdt.c                                       | 109 +++++++++++++++--------
  tools/testing/selftests/bpf/Makefile                       |  10 +++
  tools/testing/selftests/bpf/prog_tests/bpf_cookie.c        |  78 ++++++++++++++++
  tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  tools/testing/selftests/bpf/progs/uprobe_multi.c           | 110 +++++++++++++++++++++++
  tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c      |  16 ++++
  tools/testing/selftests/bpf/uprobe_multi.c                 |  53 +++++++++++
  tools/testing/selftests/bpf/usdt_multi.c                   |  24 +++++
  22 files changed, 1969 insertions(+), 296 deletions(-)
Quoted 7 lines

create mode 100644 tools/lib/bpf/elf.c
create mode 100644 tools/lib/bpf/libbpf_elf.h
create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c create mode 100644 tools/testing/selftests/bpf/usdt_multi.c

Looks like BPF CI did not trigger because of build error with LLVM:

https://github.com/kernel-patches/bpf/actions/runs/5423873866/jobs/9908376097

[...]
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]

          if (!ASSERT_OK(err, "uprobe_multi__attach"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK(err, "uprobe_multi__attach"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:37: note: initialize the variable 'attach_end_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                                             ^
                                              = 0
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:22: note: initialize the variable 'attach_start_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                              ^
                               = 0
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:37: note: initialize the variable 'attach_end_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                                             ^
                                              = 0
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:22: note: initialize the variable 'attach_start_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                              ^
                               = 0

10 errors generated.
TEST-OBJ [test_progs] lookup_key.test.o
make: *** [Makefile:578: /tmp/work/bpf/bpf/tools/testing/selftests/bpf/uprobe_multi_test.test.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/tmp/work/bpf/bpf/tools/testing/selftests/bpf'
Error: Process completed with exit code 2.

#72 lore raw
Quoted 120 lines · Daniel Borkmann
Quoted 111 lines · Jiri Olsa

hi,
this patchset is adding support to attach multiple uprobes and usdt probes through new uprobe_multi link.

The current uprobe is attached through the perf event and attaching many uprobes takes a lot of time because of that.

The main reason is that we need to install perf event for each probed function and profile shows perf event installation (perf_install_in_context) as culprit.

The new uprobe_multi link just creates raw uprobes and attaches the bpf program to them without perf event being involved.

In addition to being faster we also save file descriptors. For the current uprobe attach we use extra perf event fd for each probed function. The new link just need one fd that covers all the functions we are attaching to.

v3 changes:
- consolidate attach type checks in bpf_prog_attach_check_attach_type [Andrii]
- remove bpf_prog_active check [Alexei]
- change rcu locking [Andrii]
- allocate ref_ctr_offsets conditionally [Andrii]
- remove ctx check from bpf_uprobe_multi_cookie [Andrii]
- move some elf_* functions in elf.c object [Andrii]
- fix uprobe link detection code [Andrii]
- add usdt.s program section [Andrii]
- rename bpf_program__attach_uprobe_multi_opts to bpf_program__attach_uprobe_multi [Andrii]
- remove extra case from attach_uprobe_multi [Andrii]
- rework usdt_manager_attach_usdt [Andrii]
- rework/rename new elf_find_* functions [Andrii]
- elf iterator fixes [Andrii]
- renames
- elf_sym_iter_next loop restruct
- return directly GElf_Sym in elf_sym
- add elf_sym_offset helper
- add st_type arg elf_sym_iter_new
- get rid of '_' prefixed functions
- simplify offsets handling
- other smaller changes
- added acks
- todo:
- seems like more elf_* helpers could go in elf.c object,
I'll send that as follow up
- will send fill_info support when [3] is merged [Andrii]

There's support for bpftrace [2] and tetragon [1].

Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
uprobe_multi

thanks,
jirka

[1] https://github.com/cilium/tetragon/pull/936
[2] https://github.com/iovisor/bpftrace/compare/master...olsajiri:bpftrace:uprobe_multi
[3] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (26):
bpf: Add attach_type checks under bpf_prog_attach_check_attach_type
bpf: Add multi uprobe link
bpf: Add cookies support for uprobe_multi link
bpf: Add pid filter support for uprobe_multi link
bpf: Add bpf_get_func_ip helper support for uprobe link
libbpf: Add uprobe_multi attach type and link names
libbpf: Move elf_find_func_offset* functions to elf object
libbpf: Add elf_open/elf_close functions
libbpf: Add elf symbol iterator
libbpf: Add elf_resolve_syms_offsets function
libbpf: Add elf_resolve_pattern_offsets function
libbpf: Add bpf_link_create support for multi uprobes
libbpf: Add bpf_program__attach_uprobe_multi function
libbpf: Add support for u[ret]probe.multi[.s] program sections
libbpf: Add uprobe multi link detection
libbpf: Add uprobe multi link support to bpf_program__attach_usdt selftests/bpf: Add uprobe_multi skel test
selftests/bpf: Add uprobe_multi api test
selftests/bpf: Add uprobe_multi link test
selftests/bpf: Add uprobe_multi test program
selftests/bpf: Add uprobe_multi bench test
selftests/bpf: Add usdt_multi test program
selftests/bpf: Add usdt_multi bench test
selftests/bpf: Add uprobe_multi cookie test
selftests/bpf: Add uprobe_multi pid filter tests
selftests/bpf: Add extra link to uprobe_multi tests

  include/linux/trace_events.h                               |   6 ++
  include/uapi/linux/bpf.h                                   |  16 ++++
  kernel/bpf/syscall.c                                       | 122 ++++++++++++-------------
  kernel/trace/bpf_trace.c                                   | 346 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
  tools/include/uapi/linux/bpf.h                             |  16 ++++
  tools/lib/bpf/Build                                        |   2 +-
  tools/lib/bpf/bpf.c                                        |  11 +++
  tools/lib/bpf/bpf.h                                        |  11 ++-
  tools/lib/bpf/elf.c                                        | 435 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  tools/lib/bpf/libbpf.c                                     | 396 ++++++++++++++++++++++++++++++++++++++++++---------------------------------------
  tools/lib/bpf/libbpf.h                                     |  27 ++++++
  tools/lib/bpf/libbpf.map                                   |   1 +
  tools/lib/bpf/libbpf_elf.h                                 |  24 +++++
  tools/lib/bpf/libbpf_internal.h                            |   3 +
  tools/lib/bpf/usdt.c                                       | 109 +++++++++++++++--------
  tools/testing/selftests/bpf/Makefile                       |  10 +++
  tools/testing/selftests/bpf/prog_tests/bpf_cookie.c        |  78 ++++++++++++++++
  tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c | 449 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  tools/testing/selftests/bpf/progs/uprobe_multi.c           | 110 +++++++++++++++++++++++
  tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c      |  16 ++++
  tools/testing/selftests/bpf/uprobe_multi.c                 |  53 +++++++++++
  tools/testing/selftests/bpf/usdt_multi.c                   |  24 +++++
  22 files changed, 1969 insertions(+), 296 deletions(-)
Quoted 7 lines

create mode 100644 tools/lib/bpf/elf.c
create mode 100644 tools/lib/bpf/libbpf_elf.h
create mode 100644 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi.c create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_usdt.c create mode 100644 tools/testing/selftests/bpf/uprobe_multi.c create mode 100644 tools/testing/selftests/bpf/usdt_multi.c

Quoted 3 lines

Looks like BPF CI did not trigger because of build error with LLVM:

https://github.com/kernel-patches/bpf/actions/runs/5423873866/jobs/9908376097

will check, thanks

jirka

Quoted 114 lines

[...]
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]

          if (!ASSERT_OK(err, "uprobe_multi__attach"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:367:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK(err, "uprobe_multi__attach"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:37: note: initialize the variable 'attach_end_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                                             ^
                                              = 0
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:361:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:381:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:352:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:344:22: note: initialize the variable 'attach_start_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                              ^
                               = 0
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:414:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel->links.usdt0, "bpf_program__attach_usdt"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_end_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:18: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                          ^~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:37: note: initialize the variable 'attach_end_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                                             ^
                                              = 0
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:407:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_EQ(err, 0, "uprobe_multi_usdt__load"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:6: error: variable 'attach_start_ns' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:428:34: note: uninitialized use occurs here
          attach_delta = (attach_end_ns - attach_start_ns) / 1000000000.0;
                                          ^~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:398:2: note: remove the 'if' if its condition is always false
          if (!ASSERT_OK_PTR(skel, "uprobe_multi__open"))
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/work/bpf/bpf/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c:391:22: note: initialize the variable 'attach_start_ns' to silence this warning
          long attach_start_ns, attach_end_ns;
                              ^
                               = 0

10 errors generated.
TEST-OBJ [test_progs] lookup_key.test.o
make: *** [Makefile:578: /tmp/work/bpf/bpf/tools/testing/selftests/bpf/uprobe_multi_test.test.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/tmp/work/bpf/bpf/tools/testing/selftests/bpf'
Error: Process completed with exit code 2.