Skip to contentSkip to conversation

bpf: tracing_multi link info support

PATCH Applied Jiri Olsa 5 participants 10 messages Jun 29 – Jul 1

Subsystems (7)
Kselftest BPF Tracing BPF selftests +3 more subsystems
lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

v4 cover bpf: tracing_multi link info support

hi,
adding link info support for tracing_multi link.

Adding the kernel support with related selftests and bpftool support to display the tracing_multi link info, the output looks like:

  # bpftool link
  ...
  61: tracing_multi  prog 167
          attach_type trace_fentry_multi  btf_obj_id 1  count 3
          btf_id           addr             cookie           func [module]
          92598            ffffffff825017c4 10               bpf_fentry_test1
          92600            ffffffff82503814 30               bpf_fentry_test2
          92601            ffffffff82503824 20               bpf_fentry_test3
          pids test_progs(1540)

thanks,
jirka

v1: https://lore.kernel.org/bpf/[email protected]/
v2: https://lore.kernel.org/bpf/[email protected]/
v3: https://lore.kernel.org/bpf/[email protected]/

v4 changes:
- fix wrong allocation check [Alexei]

v3 changes:
- added acks [Andrii, Leon]
- display cookies as decimal numbers [Andrii]
- renamed obj_id to btf_obj_id [Andrii]
- several fixes in bpftool change [Andrii]

v2 changes:
- fix kernel symbol search logic [sashiko]
- typo fixes [bots]

---
Jiri Olsa (3):
bpf: Add tracing_multi link info support selftests/bpf: Add tracing_multi link info tests
bpftool: Add tracing_multi link info output

 include/uapi/linux/bpf.h                                |   9 +++
 kernel/trace/bpf_trace.c                                |  55 +++++++++++++++++
 tools/bpf/bpftool/link.c                                | 133 ++++++++++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h                          |   9 +++
 tools/testing/selftests/bpf/prog_tests/fill_link_info.c | 242 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_fill_link_info.c |   6 ++
 6 files changed, 454 insertions(+)
author reviewer
#1 lore raw

PATCH v4 1/3 bpf: Add tracing_multi link info support

Adding BPF_OBJ_GET_INFO_BY_FD support for tracing_multi links.

We expose following tracing_multi link data:
- attach_type of the program
- number of ids
- array of BTF ids
- array of its related kernel addresses
- array of cookies

The change follows the kprobe_multi and uprobe_multi link-info convention of optional output arrays with an in/out count,

On top of standard tracing link data we also expose addresses, because they are useful info for user (especially when the attachment was done via pattern). This data is hidden when kallsyms does not allow exposing kernel pointer values.

---
 include/uapi/linux/bpf.h       |  9 ++++++
 kernel/trace/bpf_trace.c       | 55 ++++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |  9 ++++++
 3 files changed, 73 insertions(+)
include/uapi/linux/bpf.h +9 -0
index 89b36de5fdbb..4180262814e7 100644
@@ -6839,6 +6839,15 @@ struct bpf_link_info {
 			__u32 flags;
 			__u32 pid;
 		} uprobe_multi;
+		struct {
+			__u32 attach_type;
+			__u32 count; /* in/out: tracing_multi target count */
+			__u32 btf_obj_id;
+			__u32 :32;
+			__aligned_u64 ids;
+			__aligned_u64 addrs;
+			__aligned_u64 cookies;
+		} tracing_multi;
 		struct {
 			__u32 type; /* enum bpf_perf_event_type */
 			__u32 :32;
kernel/trace/bpf_trace.c +55 -0

67 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +9 -0
index 89b36de5fdbb..4180262814e7 100644
@@ -6839,6 +6839,15 @@ struct bpf_link_info {
 			__u32 flags;
 			__u32 pid;
 		} uprobe_multi;
+		struct {
+			__u32 attach_type;
+			__u32 count; /* in/out: tracing_multi target count */
+			__u32 btf_obj_id;
+			__u32 :32;
+			__aligned_u64 ids;
+			__aligned_u64 addrs;
+			__aligned_u64 cookies;
+		} tracing_multi;
 		struct {
 			__u32 type; /* enum bpf_perf_event_type */
 			__u32 :32;
Signature · 1 line
2.54.0
author reviewer
#2 lore raw

PATCH v4 2/3 selftests/bpf: Add tracing_multi link info tests

In reply to Jiri Olsa

Adding tracing_multi link info tests that follow the kprobe_multi and uprobe_multi tests logic.

Assisted-by Codex:GPT-5
---
 .../selftests/bpf/prog_tests/fill_link_info.c | 242 ++++++++++++++++++
 .../selftests/bpf/progs/test_fill_link_info.c |   6 +
 2 files changed, 248 insertions(+)
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +242 -0
index f589eefbf9fb..0918321c8e63 100644
@@ -4,6 +4,7 @@
 #include <string.h>
 #include <linux/bpf.h>
 #include <linux/limits.h>
+#include <bpf/btf.h>
 #include <test_progs.h>
 #include "trace_helpers.h"
 #include "test_fill_link_info.skel.h"
@@ -24,6 +25,22 @@ static __u64 kmulti_cookies[] = { 3, 1, 2 };
 #define KPROBE_FUNC "bpf_fentry_test1"
 static __u64 kprobe_addr;
 
+static const char * const tmulti_syms[] = {
+	"bpf_fentry_test2",
+	"bpf_fentry_test1",
+	"bpf_fentry_test3",
+};
+
+static __u64 tmulti_cookies[] = { 30, 10, 20 };
+#define TRACING_MULTI_CNT ARRAY_SIZE(tmulti_syms)
+
+struct tmulti_target {
+	const char *name;
+	__u64 addr;
+	__u64 cookie;
+	__u32 id;
+};
+
 #define UPROBE_FILE "/proc/self/exe"
 static ssize_t uprobe_offset;
 /* uprobe attach point */
@@ -396,6 +413,224 @@ static void test_kprobe_multi_fill_link_info(struct test_fill_link_info *skel,
 	bpf_link__destroy(link);
 }
 
+static int tmulti_target_cmp(const void *a, const void *b)
+{
+	const struct tmulti_target *ta = a;
+	const struct tmulti_target *tb = b;
+
+	return (ta->id > tb->id) - (ta->id < tb->id);
+}
+
+static int setup_tmulti_targets(const struct bpf_program *prog,
+				struct tmulti_target *targets,
+				__u32 *btf_obj_id)
+{
+	struct bpf_prog_info prog_info;
+	__u32 len = sizeof(prog_info);
+	struct btf *btf;
+	int err, i;
+	__s32 id;
+
+	btf = btf__load_vmlinux_btf();
+	if (!ASSERT_OK_PTR(btf, "btf__load_vmlinux_btf"))
+		return -1;
+
+	for (i = 0; i < TRACING_MULTI_CNT; i++) {
+		id = btf__find_by_name_kind(btf, tmulti_syms[i], BTF_KIND_FUNC);
+		if (!ASSERT_GT(id, 0, "btf__find_by_name_kind"))
+			goto error;
+
+		targets[i].name = tmulti_syms[i];
+		targets[i].addr = ksym_get_addr(tmulti_syms[i]);
+		targets[i].cookie = tmulti_cookies[i];
+		targets[i].id = id;
+	}
+
+	memset(&prog_info, 0, len);
+	err = bpf_prog_get_info_by_fd(bpf_program__fd(prog), &prog_info, &len);
+	if (!ASSERT_OK(err, "bpf_prog_get_info_by_fd"))
+		goto error;
+	if (!ASSERT_GT(prog_info.attach_btf_obj_id, 0, "attach_btf_obj_id"))
+		goto error;
+	*btf_obj_id = prog_info.attach_btf_obj_id;
+
+	/*
+	 * The kernel tracing multi attach sorts ids. We sort as well,
+	 * so we can easily compare ids and cookies later.
+	 */
+	qsort(targets, TRACING_MULTI_CNT, sizeof(targets[0]), tmulti_target_cmp);
+	btf__free(btf);
+	return 0;
+
+error:
+	btf__free(btf);
+	return -1;
+}
+
+static int verify_tracing_multi_link_info(int fd, const struct bpf_program *prog,
+					  const struct tmulti_target *targets,
+					  __u32 btf_obj_id, bool has_cookies)
+{
+	enum bpf_attach_type attach_type = bpf_program__expected_attach_type(prog);
+	__u64 addrs[TRACING_MULTI_CNT], cookies[TRACING_MULTI_CNT];
+	__u32 ids[TRACING_MULTI_CNT];
+	struct bpf_link_info info;
+	__u32 len = sizeof(info);
+	int err, i;
+
+	memset(&info, 0, sizeof(info));
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	if (!ASSERT_OK(err, "bpf_link_get_info_by_fd"))
+		return -1;
+
+	if (!ASSERT_EQ(info.type, BPF_LINK_TYPE_TRACING_MULTI, "info.type"))
+		return -1;
+
+	ASSERT_EQ(info.tracing_multi.attach_type, attach_type, "info.tracing_multi.attach_type");
+	ASSERT_EQ(info.tracing_multi.count, TRACING_MULTI_CNT, "info.tracing_multi.count");
+
+	memset(ids, 0, sizeof(ids));
+	memset(cookies, 0, sizeof(cookies));
+	memset(addrs, 0, sizeof(addrs));
+
+	info.tracing_multi.ids = ptr_to_u64(ids);
+	info.tracing_multi.addrs = ptr_to_u64(addrs);
+	info.tracing_multi.cookies = has_cookies ? ptr_to_u64(cookies) : 0;
+	info.tracing_multi.count = TRACING_MULTI_CNT;
+
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	if (!ASSERT_OK(err, "bpf_link_get_info_by_fd"))
+		return -1;
+
+	if (!ASSERT_EQ(info.type, BPF_LINK_TYPE_TRACING_MULTI, "info.type"))
+		return -1;
+
+	ASSERT_EQ(info.tracing_multi.attach_type, attach_type, "info.tracing_multi.attach_type");
+	ASSERT_EQ(info.tracing_multi.count, TRACING_MULTI_CNT, "info.tracing_multi.count");
+	ASSERT_EQ(info.tracing_multi.btf_obj_id, btf_obj_id, "tracing_multi.btf_obj_id");
+
+	for (i = 0; i < TRACING_MULTI_CNT; i++) {
+		ASSERT_EQ(ids[i], targets[i].id, "tracing_multi.ids");
+		ASSERT_EQ(cookies[i], has_cookies ? targets[i].cookie : 0, "tracing_multi.cookies");
+
+		if (targets[i].addr) {
+			struct ksym *ksym;
+
+			if (!ASSERT_NEQ(addrs[i], 0, "tracing_multi.addrs"))
+				return -1;
+			ksym = ksym_search(addrs[i]);
+			if (!ASSERT_OK_PTR(ksym, "ksym_search"))
+				return -1;
+			ASSERT_STREQ(ksym->name, targets[i].name, "tracing_multi.addr_name");
+		} else {
+			ASSERT_EQ(addrs[i], 0, "tracing_multi.addrs");
+		}
+	}
+
+	return 0;
+}
+
+static void verify_tracing_multi_invalid_user_buffer(int fd, const struct tmulti_target *targets)
+{
+	__u32 ids[TRACING_MULTI_CNT] = {};
+	struct bpf_link_info info;
+	__u32 len = sizeof(info);
+	int err, i;
+
+	/* Wrong info setup (ids != NULL and cnt == 0) -> EINVAL */
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.ids = ptr_to_u64(ids);
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_EQ(err, -EINVAL, "tracing_multi.invalid_count");
+
+	/* Smaller than actual count provided -> ENOSPC */
+	memset(ids, 0, sizeof(ids));
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.ids = ptr_to_u64(ids);
+	info.tracing_multi.count = TRACING_MULTI_CNT - 1;
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_EQ(err, -ENOSPC, "tracing_multi.small_count");
+	for (i = 0; i < TRACING_MULTI_CNT - 1; i++)
+		ASSERT_EQ(ids[i], targets[i].id, "tracing_multi.partial_ids");
+	/* check that the last entry is not populated */
+	ASSERT_EQ(ids[i], 0, "tracing_multi.partial_ids");
+
+	/* Bigger than actual count provided -> OK */
+	memset(ids, 0, sizeof(ids));
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.ids = ptr_to_u64(ids);
+	info.tracing_multi.count = TRACING_MULTI_CNT + 1;
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_OK(err, "tracing_multi.big_count");
+	for (i = 0; i < TRACING_MULTI_CNT; i++)
+		ASSERT_EQ(ids[i], targets[i].id, "tracing_multi.ids");
+
+	/* Invalid ids pointer -> EFAULT */
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.ids = 0x1;
+	info.tracing_multi.count = TRACING_MULTI_CNT;
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_EQ(err, -EFAULT, "tracing_multi.bad_btf_ids");
+
+	/* Invalid cookies pointer -> EFAULT */
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.cookies = 0x1;
+	info.tracing_multi.count = TRACING_MULTI_CNT;
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_EQ(err, -EFAULT, "tracing_multi.bad_cookies");
+
+	/* Invalid addrs pointer -> EFAULT */
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.addrs = 0x1;
+	info.tracing_multi.count = TRACING_MULTI_CNT;
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_EQ(err, -EFAULT, "tracing_multi.bad_addrs");
+}
+
+static void test_tracing_multi_fill_link_info(struct test_fill_link_info *skel,
+					      bool has_cookies, bool invalid)
+{
+	LIBBPF_OPTS(bpf_tracing_multi_opts, opts);
+	struct tmulti_target targets[TRACING_MULTI_CNT];
+	__u32 ids[TRACING_MULTI_CNT], btf_obj_id;
+	__u64 cookies[TRACING_MULTI_CNT];
+	struct bpf_link *link;
+	int link_fd, err, i;
+
+#ifndef __x86_64__
+	test__skip();
+	return;
+#endif
+
+	if (setup_tmulti_targets(skel->progs.tmulti_run, targets, &btf_obj_id))
+		return;
+
+	for (i = 0; i < TRACING_MULTI_CNT; i++) {
+		ids[i] = targets[i].id;
+		cookies[i] = targets[i].cookie;
+	}
+
+	opts.ids = ids;
+	opts.cnt = TRACING_MULTI_CNT;
+	if (has_cookies)
+		opts.cookies = cookies;
+
+	link = bpf_program__attach_tracing_multi(skel->progs.tmulti_run, NULL, &opts);
+	if (!ASSERT_OK_PTR(link, "bpf_program__attach_tracing_multi"))
+		return;
+
+	link_fd = bpf_link__fd(link);
+	if (invalid) {
+		verify_tracing_multi_invalid_user_buffer(link_fd, targets);
+	} else {
+		err = verify_tracing_multi_link_info(link_fd, skel->progs.tmulti_run,
+						     targets, btf_obj_id, has_cookies);
+		ASSERT_OK(err, "verify_tracing_multi_link_info");
+	}
+
+	bpf_link__destroy(link);
+}
+
 #define SEC(name) __attribute__((section(name), used))
 
 static short uprobe_link_info_sema_1 SEC(".probes");
@@ -640,6 +875,13 @@ void test_fill_link_info(void)
 	if (test__start_subtest("kprobe_multi_invalid_ubuff"))
 		test_kprobe_multi_fill_link_info(skel, true, true, true);
 
+	if (test__start_subtest("tracing_multi_link_info")) {
+		test_tracing_multi_fill_link_info(skel, false, false);
+		test_tracing_multi_fill_link_info(skel, true, false);
+	}
+	if (test__start_subtest("tracing_multi_invalid_ubuff"))
+		test_tracing_multi_fill_link_info(skel, true, true);
+
 	if (test__start_subtest("uprobe_multi_link_info"))
 		test_uprobe_multi_fill_link_info(skel, false, false);
 	if (test__start_subtest("uretprobe_multi_link_info"))
tools/testing/selftests/bpf/progs/test_fill_link_info.c +6 -0
index 137bd6292163..c85081538e93 100644
@@ -58,4 +58,10 @@ int BPF_PROG(umulti_run)
 	return 0;
 }
 
+SEC("fentry.multi")
+int BPF_PROG(tmulti_run)
+{
+	return 0;
+}
+
 char _license[] SEC("license") = "GPL";
Signature · 1 line
2.54.0
#3 lore raw

Re: [PATCHv4 bpf-next 2/3] selftests/bpf: Add tracing_multi link info tests

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] Missing UAPI and kernel implementation for 'tracing_multi' link info. - [Medium] Flawed negative…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] Missing UAPI and kernel implementation for 'tracing_multi' link info.
- [Medium] Flawed negative testing logic for 'tracing_multi' invalid user buffers. --

commit 10d5275538071b94b5cfb747b2b37b5cb2e5d158
Author: Jiri Olsa <[email protected]>

selftests/bpf: Add tracing_multi link info tests

Adding tracing_multi link info tests that follow the kprobe_multi and uprobe_multi tests logic.

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +0 -0
index f589eefbf9fbd..0918321c8e63d 100644

[ ... ]

Quoted 1 line
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +0 -0
@@ -396,6 +413,224 @@ static void test_kprobe_multi_fill_link_info(struct test_fill_link_info *skel,

[ ... ]

Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +11 -0
+static int verify_tracing_multi_link_info(int fd, const struct bpf_program *prog,
+					  const struct tmulti_target *targets,
+					  __u32 btf_obj_id, bool has_cookies)
+{
+	enum bpf_attach_type attach_type = bpf_program__expected_attach_type(prog);
+	__u64 addrs[TRACING_MULTI_CNT], cookies[TRACING_MULTI_CNT];
+	__u32 ids[TRACING_MULTI_CNT];
+	struct bpf_link_info info;
+	__u32 len = sizeof(info);
+	int err, i;
+

tools/testing/selftests/bpf/prog_tests/fill_link_info.c static void test_kprobe_multi_fill_link_info(struct test_fill_link_info *skel, · line 490

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +10 -0
+	memset(&info, 0, sizeof(info));
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	if (!ASSERT_OK(err, "bpf_link_get_info_by_fd"))
+		return -1;
+
+	if (!ASSERT_EQ(info.type, BPF_LINK_TYPE_TRACING_MULTI, "info.type"))
+		return -1;
+
+	ASSERT_EQ(info.tracing_multi.attach_type, attach_type, "info.tracing_multi.attach_type");
+	ASSERT_EQ(info.tracing_multi.count, TRACING_MULTI_CNT, "info.tracing_multi.count");

[Severity: Medium]
Does struct bpf_link_info currently have a tracing_multi member? It appears this might cause a compilation failure because the UAPI header doesn't include it. Also, does the kernel side implement the .fill_link_info callback in bpf_tracing_multi_link_lops?

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +4 -0
+
+	memset(ids, 0, sizeof(ids));
+	memset(cookies, 0, sizeof(cookies));
+	memset(addrs, 0, sizeof(addrs));

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +6 -0
+static void verify_tracing_multi_invalid_user_buffer(int fd, const struct tmulti_target *targets)
+{
+	__u32 ids[TRACING_MULTI_CNT] = {};
+	struct bpf_link_info info;
+	__u32 len = sizeof(info);
+	int err, i;

[ ... ]

tools/testing/selftests/bpf/prog_tests/fill_link_info.c static void test_kprobe_multi_fill_link_info(struct test_fill_link_info *skel, · line 587

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +6 -0
+	/* Invalid addrs pointer -> EFAULT */
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.addrs = 0x1;
+	info.tracing_multi.count = TRACING_MULTI_CNT;
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_EQ(err, -EFAULT, "tracing_multi.bad_addrs");

[Severity: Medium]
Will this test actually return -EFAULT?

By zeroing out the info struct before these checks, the mandatory ids pointer is left as NULL while count is non-zero. If the kernel validation logic eagerly rejects the missing ids array with -EINVAL before reaching the -EFAULT paths, won't this assertion fail?

Quoted 1 line

+}

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#4 lore raw

Re: [PATCHv4 bpf-next 2/3] selftests/bpf: Add tracing_multi link info tests

On Mon, Jun 29, 2026 at 09:34:06PM +0000, [email protected] wrote:

SNIP

Quoted 17 lines
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +6 -0
+static void verify_tracing_multi_invalid_user_buffer(int fd, const struct tmulti_target *targets)
+{
+	__u32 ids[TRACING_MULTI_CNT] = {};
+	struct bpf_link_info info;
+	__u32 len = sizeof(info);
+	int err, i;

[ ... ]

tools/testing/selftests/bpf/prog_tests/fill_link_info.c static void test_kprobe_multi_fill_link_info(struct test_fill_link_info *skel, · line 587

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/fill_link_info.c +6 -0
+	/* Invalid addrs pointer -> EFAULT */
+	memset(&info, 0, sizeof(info));
+	info.tracing_multi.addrs = 0x1;
+	info.tracing_multi.count = TRACING_MULTI_CNT;
+	err = bpf_link_get_info_by_fd(fd, &info, &len);
+	ASSERT_EQ(err, -EFAULT, "tracing_multi.bad_addrs");

[Severity: Medium]
Will this test actually return -EFAULT?

Quoted 4 lines

By zeroing out the info struct before these checks, the mandatory ids pointer is left as NULL while count is non-zero. If the kernel validation logic eagerly rejects the missing ids array with -EINVAL before reaching the -EFAULT paths, won't this assertion fail?

there's no such eager reject in bpf_tracing_multi_link_fill_link_info, the test passes

jirka

author reviewer
#5 lore raw

PATCH v4 3/3 bpftool: Add tracing_multi link info output

In reply to Jiri Olsa

Adding bpftool support to show tracing_multi link details, the new output looks like:

  # bpftool link
  ...
  61: tracing_multi  prog 167
          attach_type trace_fentry_multi  btf_obj_id 1  count 3
          btf_id           addr             cookie           func [module]
          92598            ffffffff825017c4 10               bpf_fentry_test1
          92600            ffffffff82503814 30               bpf_fentry_test2
          92601            ffffffff82503824 20               bpf_fentry_test3
          pids test_progs(1540)
Assisted-by Codex:GPT-5
---
 tools/bpf/bpftool/link.c | 133 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 133 insertions(+)
tools/bpf/bpftool/link.c +133 -0
index bdcd717b0348..088d1d206065 100644
@@ -377,6 +377,25 @@ static __u64 *u64_to_arr(__u64 val)
 	return (__u64 *) u64_to_ptr(val);
 }
 
+static __u32 *u64_to_u32_arr(__u64 val)
+{
+	return (__u32 *)u64_to_ptr(val);
+}
+
+static struct kernel_sym *find_kernel_sym_by_addr(__u64 addr, bool is_ibt_enabled)
+{
+	struct kernel_sym *sym;
+
+	if (!addr)
+		return NULL;
+
+	sym = kernel_syms_search(&dd, addr);
+	if (!sym && is_ibt_enabled && addr >= 4)
+		sym = kernel_syms_search(&dd, addr - 4);
+
+	return sym;
+}
+
 static void
 show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr)
 {
@@ -403,6 +422,52 @@ show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr)
 	jsonw_end_array(json_wtr);
 }
 
+static void
+show_tracing_multi_json(struct bpf_link_info *info, json_writer_t *wtr)
+{
+	bool is_ibt_enabled = is_x86_ibt_enabled(), show_symbol;
+	__u64 *addrs, *cookies;
+	__u32 i, *ids;
+
+	if (!dd.sym_count)
+		kernel_syms_load(&dd);
+	show_symbol = !!dd.sym_count;
+
+	show_link_attach_type_json(info->tracing_multi.attach_type, wtr);
+	jsonw_uint_field(wtr, "func_cnt", info->tracing_multi.count);
+	jsonw_uint_field(wtr, "btf_obj_id", info->tracing_multi.btf_obj_id);
+	jsonw_name(wtr, "funcs");
+
+	jsonw_start_array(wtr);
+
+	ids = u64_to_u32_arr(info->tracing_multi.ids);
+	addrs = u64_to_arr(info->tracing_multi.addrs);
+	cookies = u64_to_arr(info->tracing_multi.cookies);
+
+	for (i = 0; i < info->tracing_multi.count; i++) {
+		struct kernel_sym *sym;
+		__u64 addr = addrs[i];
+
+		sym = show_symbol ? find_kernel_sym_by_addr(addr, is_ibt_enabled) : NULL;
+
+		jsonw_start_object(wtr);
+		jsonw_uint_field(wtr, "id", ids[i]);
+		jsonw_uint_field(wtr, "addr", addr);
+		if (sym) {
+			jsonw_string_field(wtr, "func", sym->name);
+			if (sym->module[0] == '\0') {
+				jsonw_name(wtr, "module");
+				jsonw_null(wtr);
+			} else {
+				jsonw_string_field(wtr, "module", sym->module);
+			}
+		}
+		jsonw_uint_field(wtr, "cookie", cookies[i]);
+		jsonw_end_object(wtr);
+	}
+	jsonw_end_array(wtr);
+}
+
 static void
 show_perf_event_kprobe_json(struct bpf_link_info *info, json_writer_t *wtr)
 {
@@ -589,6 +654,9 @@ static int show_link_close_json(int fd, struct bpf_link_info *info)
 	case BPF_LINK_TYPE_UPROBE_MULTI:
 		show_uprobe_multi_json(info, json_wtr);
 		break;
+	case BPF_LINK_TYPE_TRACING_MULTI:
+		show_tracing_multi_json(info, json_wtr);
+		break;
 	case BPF_LINK_TYPE_PERF_EVENT:
 		switch (info->perf_event.type) {
 		case BPF_PERF_EVENT_EVENT:
@@ -833,6 +901,46 @@ static void show_uprobe_multi_plain(struct bpf_link_info *info)
 	}
 }
 
+static void show_tracing_multi_plain(struct bpf_link_info *info)
+{
+	bool is_ibt_enabled = is_x86_ibt_enabled(), show_symbol;
+	__u64 *addrs, *cookies;
+	__u32 i, *ids;
+
+	if (!info->tracing_multi.count)
+		return;
+
+	if (!dd.sym_count)
+		kernel_syms_load(&dd);
+	show_symbol = !!dd.sym_count;
+
+	printf("\n\t");
+	show_link_attach_type_plain(info->tracing_multi.attach_type);
+	printf("btf_obj_id %u  ", info->tracing_multi.btf_obj_id);
+	printf("count %u  ", info->tracing_multi.count);
+
+	printf("\n\t%-16s %-16s %-16s %s",
+	       "btf_id", "addr", "cookie", "func [module]");
+
+	ids = u64_to_u32_arr(info->tracing_multi.ids);
+	addrs = u64_to_arr(info->tracing_multi.addrs);
+	cookies = u64_to_arr(info->tracing_multi.cookies);
+
+	for (i = 0; i < info->tracing_multi.count; i++) {
+		__u64 addr = addrs[i];
+		struct kernel_sym *sym;
+
+		sym = show_symbol ? find_kernel_sym_by_addr(addr, is_ibt_enabled) : NULL;
+
+		printf("\n\t%-16u %016llx %-16llu", ids[i], addr, cookies[i]);
+		if (sym) {
+			printf(" %s", sym->name);
+			if (sym->module[0] != '\0')
+				printf(" [%s]", sym->module);
+		}
+	}
+}
+
 static void show_perf_event_kprobe_plain(struct bpf_link_info *info)
 {
 	const char *buf;
@@ -989,6 +1097,9 @@ static int show_link_close_plain(int fd, struct bpf_link_info *info)
 	case BPF_LINK_TYPE_UPROBE_MULTI:
 		show_uprobe_multi_plain(info);
 		break;
+	case BPF_LINK_TYPE_TRACING_MULTI:
+		show_tracing_multi_plain(info);
+		break;
 	case BPF_LINK_TYPE_PERF_EVENT:
 		switch (info->perf_event.type) {
 		case BPF_PERF_EVENT_EVENT:
@@ -1029,6 +1140,7 @@ static int show_link_close_plain(int fd, struct bpf_link_info *info)
 static int do_show_link(int fd)
 {
 	__u64 *ref_ctr_offsets = NULL, *offsets = NULL, *cookies = NULL;
+	__u32 *ids = NULL;
 	struct bpf_link_info info;
 	__u32 len = sizeof(info);
 	char path_buf[PATH_MAX];
@@ -1114,6 +1226,26 @@ static int do_show_link(int fd)
 			goto again;
 		}
 	}
+	if (info.type == BPF_LINK_TYPE_TRACING_MULTI && !info.tracing_multi.ids) {
+		count = info.tracing_multi.count;
+		if (count) {
+			ids = calloc(count, sizeof(__u32));
+			addrs = calloc(count, sizeof(__u64));
+			cookies = calloc(count, sizeof(__u64));
+			if (!ids || !addrs || !cookies) {
+				p_err("mem alloc failed");
+				close(fd);
+				free(cookies);
+				free(addrs);
+				free(ids);
+				return -ENOMEM;
+			}
+			info.tracing_multi.ids = ptr_to_u64(ids);
+			info.tracing_multi.addrs = ptr_to_u64(addrs);
+			info.tracing_multi.cookies = ptr_to_u64(cookies);
+			goto again;
+		}
+	}
 	if (info.type == BPF_LINK_TYPE_PERF_EVENT) {
 		switch (info.perf_event.type) {
 		case BPF_PERF_EVENT_TRACEPOINT:
@@ -1153,6 +1285,7 @@ static int do_show_link(int fd)
 	free(cookies);
 	free(offsets);
 	free(addrs);
+	free(ids);
 	close(fd);
 	return 0;
 }
Signature · 1 line
2.54.0
#6 lore raw

Re: [PATCHv4 bpf-next 3/3] bpftool: Add tracing_multi link info output

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] bpftool uses a non-existent `tracing_multi` union member in `struct bpf_link_info`. - [Medium]…

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] bpftool uses a non-existent tracing_multi union member in struct bpf_link_info.
- [Medium] The kernel-side bpf_tracing_multi_link_lops is missing the .fill_link_info callback.
- [Low] Memory allocated for ids, addrs, and cookies is leaked on the error path of the bpf_link_get_info_by_fd syscall. --

commit 3c4bad76286f49adf87a7b01dc7042ac6368c5ce
Author: Jiri Olsa <[email protected]>

bpftool: Add tracing_multi link info output

Adding bpftool support to show tracing_multi link details. The new output will display attach types, counts, BTF IDs, addresses, cookies, and functions.

Quoted 4 lines
tools/bpf/bpftool/link.c +0 -0
index bdcd717b0348f..088d1d206065f 100644

[ ... ]

Quoted 6 lines
tools/bpf/bpftool/link.c +2 -0
@@ -403,6 +422,52 @@ show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr)
 	jsonw_end_array(json_wtr);
 }
 
+static void
+show_tracing_multi_json(struct bpf_link_info *info, json_writer_t *wtr)

tools/bpf/bpftool/link.c show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr) · line 436

Quoted 10 lines
tools/bpf/bpftool/link.c +10 -0
+{
+	bool is_ibt_enabled = is_x86_ibt_enabled(), show_symbol;
+	__u64 *addrs, *cookies;
+	__u32 i, *ids;
+
+	if (!dd.sym_count)
+		kernel_syms_load(&dd);
+	show_symbol = !!dd.sym_count;
+
+	show_link_attach_type_json(info->tracing_multi.attach_type, wtr);

[Severity: Medium]
Does struct bpf_link_info in include/uapi/linux/bpf.h include a tracing_multi member?

It looks like the tracing_multi struct is missing from struct bpf_link_info in the UAPI, which causes a compile error here when trying to access info->tracing_multi.attach_type.

Additionally, the kernel-side implementation in kernel/trace/bpf_trace.c appears to be missing the .fill_link_info callback in bpf_tracing_multi_link_lops.

Without it, will the kernel silently return unpopulated link details when queried via bpf_obj_get_info_by_fd()?

[ ... ]

Quoted 12 lines
tools/bpf/bpftool/link.c +8 -0
@@ -1114,6 +1226,26 @@ static int do_show_link(int fd)
 			goto again;
 		}
 	}
+	if (info.type == BPF_LINK_TYPE_TRACING_MULTI && !info.tracing_multi.ids) {
+		count = info.tracing_multi.count;
+		if (count) {
+			ids = calloc(count, sizeof(__u32));
+			addrs = calloc(count, sizeof(__u64));
+			cookies = calloc(count, sizeof(__u64));
+			if (!ids || !addrs || !cookies) {
+				p_err("mem alloc failed");

tools/bpf/bpftool/link.c static int do_show_link(int fd) · line 1246

Quoted 10 lines
tools/bpf/bpftool/link.c +10 -0
+				close(fd);
+				free(cookies);
+				free(addrs);
+				free(ids);
+				return -ENOMEM;
+			}
+			info.tracing_multi.ids = ptr_to_u64(ids);
+			info.tracing_multi.addrs = ptr_to_u64(addrs);
+			info.tracing_multi.cookies = ptr_to_u64(cookies);
+			goto again;

[Severity: Low]
Do we leak the memory allocated for ids, addrs, and cookies if the subsequent bpf_link_get_info_by_fd() syscall fails?

When jumping to again:, if bpf_link_get_info_by_fd() returns an error, the function returns early without freeing these arrays:

tools/bpf/bpftool/link.c:do_show_link() {
    ...
    again:
	err = bpf_link_get_info_by_fd(fd, &info, &len);
	if (err) {
		p_err("can't get link info: %s",
		      strerror(errno));
		close(fd);
		return err;
	}
    ...
}

Should the error path free these arrays before returning?

Quoted 2 lines
tools/bpf/bpftool/link.c +2 -0
+		}
+	}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#7 lore raw

Re: [PATCHv4 bpf-next 3/3] bpftool: Add tracing_multi link info output

Quoted 98 lines · On Mon, Jun 29, 2026 at 09:33:29PM +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] bpftool uses a non-existent tracing_multi union member in struct bpf_link_info.
- [Medium] The kernel-side bpf_tracing_multi_link_lops is missing the .fill_link_info callback.
- [Low] Memory allocated for ids, addrs, and cookies is leaked on the error path of the bpf_link_get_info_by_fd syscall. --

commit 3c4bad76286f49adf87a7b01dc7042ac6368c5ce
Author: Jiri Olsa <[email protected]>

bpftool: Add tracing_multi link info output

Adding bpftool support to show tracing_multi link details. The new output will display attach types, counts, BTF IDs, addresses, cookies, and functions.

Quoted 4 lines
tools/bpf/bpftool/link.c +0 -0
index bdcd717b0348f..088d1d206065f 100644

[ ... ]

Quoted 6 lines
tools/bpf/bpftool/link.c +2 -0
@@ -403,6 +422,52 @@ show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr)
 	jsonw_end_array(json_wtr);
 }
 
+static void
+show_tracing_multi_json(struct bpf_link_info *info, json_writer_t *wtr)

tools/bpf/bpftool/link.c show_uprobe_multi_json(struct bpf_link_info *info, json_writer_t *wtr) · line 436

Quoted 10 lines
tools/bpf/bpftool/link.c +10 -0
+{
+	bool is_ibt_enabled = is_x86_ibt_enabled(), show_symbol;
+	__u64 *addrs, *cookies;
+	__u32 i, *ids;
+
+	if (!dd.sym_count)
+		kernel_syms_load(&dd);
+	show_symbol = !!dd.sym_count;
+
+	show_link_attach_type_json(info->tracing_multi.attach_type, wtr);

[Severity: Medium]
Does struct bpf_link_info in include/uapi/linux/bpf.h include a tracing_multi member?

It looks like the tracing_multi struct is missing from struct bpf_link_info in the UAPI, which causes a compile error here when trying to access info->tracing_multi.attach_type.

Additionally, the kernel-side implementation in kernel/trace/bpf_trace.c appears to be missing the .fill_link_info callback in bpf_tracing_multi_link_lops.

Without it, will the kernel silently return unpopulated link details when queried via bpf_obj_get_info_by_fd()?

[ ... ]

Quoted 12 lines
tools/bpf/bpftool/link.c +8 -0
@@ -1114,6 +1226,26 @@ static int do_show_link(int fd)
 			goto again;
 		}
 	}
+	if (info.type == BPF_LINK_TYPE_TRACING_MULTI && !info.tracing_multi.ids) {
+		count = info.tracing_multi.count;
+		if (count) {
+			ids = calloc(count, sizeof(__u32));
+			addrs = calloc(count, sizeof(__u64));
+			cookies = calloc(count, sizeof(__u64));
+			if (!ids || !addrs || !cookies) {
+				p_err("mem alloc failed");

tools/bpf/bpftool/link.c static int do_show_link(int fd) · line 1246

Quoted 10 lines
tools/bpf/bpftool/link.c +10 -0
+				close(fd);
+				free(cookies);
+				free(addrs);
+				free(ids);
+				return -ENOMEM;
+			}
+			info.tracing_multi.ids = ptr_to_u64(ids);
+			info.tracing_multi.addrs = ptr_to_u64(addrs);
+			info.tracing_multi.cookies = ptr_to_u64(cookies);
+			goto again;

[Severity: Low]
Do we leak the memory allocated for ids, addrs, and cookies if the subsequent bpf_link_get_info_by_fd() syscall fails?

When jumping to again:, if bpf_link_get_info_by_fd() returns an error, the function returns early without freeing these arrays:

tools/bpf/bpftool/link.c:do_show_link() {
    ...
    again:
	err = bpf_link_get_info_by_fd(fd, &info, &len);
	if (err) {
		p_err("can't get link info: %s",
		      strerror(errno));
		close(fd);
		return err;
	}
    ...
}
Quoted 1 line

Should the error path free these arrays before returning?

right, that's pre-existing issue, I think we could do something like below.. will send a separate fix

jirka

---

tools/bpf/bpftool/link.c +3 -3
index 088d1d206065..3f5b3b12a404 100644
@@ -1156,8 +1156,7 @@ static int do_show_link(int fd)
 	if (err) {
 		p_err("can't get link info: %s",
 		      strerror(errno));
-		close(fd);
-		return err;
+		goto out;
 	}
 	if (info.type == BPF_LINK_TYPE_RAW_TRACEPOINT &&
 	    !info.raw_tracepoint.tp_name) {
@@ -1281,13 +1280,14 @@ static int do_show_link(int fd)
 	else
 		show_link_close_plain(fd, &info);
 
+out:
 	free(ref_ctr_offsets);
 	free(cookies);
 	free(offsets);
 	free(addrs);
 	free(ids);
 	close(fd);
-	return 0;
+	return err;
 }
 
 static int do_show(int argc, char **argv)
#8 lore raw

Re: [PATCHv4 bpf-next 3/3] bpftool: Add tracing_multi link info output

In reply to Jiri Olsa
Quoted 20 lines · Jiri Olsa

Adding bpftool support to show tracing_multi link details, the new output looks like:

  # bpftool link
  ...
  61: tracing_multi  prog 167
          attach_type trace_fentry_multi  btf_obj_id 1  count 3
          btf_id           addr             cookie           func [module]
          92598            ffffffff825017c4 10               bpf_fentry_test1
          92600            ffffffff82503814 30               bpf_fentry_test2
          92601            ffffffff82503824 20               bpf_fentry_test3
          pids test_progs(1540)
Assisted-by Codex:GPT-5
---
 tools/bpf/bpftool/link.c | 133 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 133 insertions(+)

[...]

Quoted 22 lines

static void show_perf_event_kprobe_plain(struct bpf_link_info *info)

 {
        const char *buf;
tools/bpf/bpftool/link.c +4 -0
@@ -989,6 +1097,9 @@ static int show_link_close_plain(int fd, struct bpf_link_info *info)
        case BPF_LINK_TYPE_UPROBE_MULTI:
                show_uprobe_multi_plain(info);
                break;
+       case BPF_LINK_TYPE_TRACING_MULTI:
+               show_tracing_multi_plain(info);
+               break;
        case BPF_LINK_TYPE_PERF_EVENT:
                switch (info->perf_event.type) {
                case BPF_PERF_EVENT_EVENT:
@@ -1029,6 +1140,7 @@ static int show_link_close_plain(int fd, struct bpf_link_info *info)
 static int do_show_link(int fd)
 {
        __u64 *ref_ctr_offsets = NULL, *offsets = NULL, *cookies = NULL;
+       __u32 *ids = NULL;
        struct bpf_link_info info;
        __u32 len = sizeof(info);
        char path_buf[PATH_MAX];

tools/bpf/bpftool/link.c static int do_show_link(int fd) · line 1229

Quoted 4 lines
tools/bpf/bpftool/link.c +1 -0
@@ -1114,6 +1226,26 @@ static int do_show_link(int fd)
                        goto again;
                }
        }
+       if (info.type == BPF_LINK_TYPE_TRACING_MULTI && !info.tracing_multi.ids) {

&& info.tracing_multi.count and avoid that nested if (just use info.tracing_multi.count directly for callocs())

just a nit for maybe a follow up, if you care

Quoted 33 lines
tools/bpf/bpftool/link.c +20 -1
+               count = info.tracing_multi.count;
+               if (count) {
+                       ids = calloc(count, sizeof(__u32));
+                       addrs = calloc(count, sizeof(__u64));
+                       cookies = calloc(count, sizeof(__u64));
+                       if (!ids || !addrs || !cookies) {
+                               p_err("mem alloc failed");
+                               close(fd);
+                               free(cookies);
+                               free(addrs);
+                               free(ids);
+                               return -ENOMEM;
+                       }
+                       info.tracing_multi.ids = ptr_to_u64(ids);
+                       info.tracing_multi.addrs = ptr_to_u64(addrs);
+                       info.tracing_multi.cookies = ptr_to_u64(cookies);
+                       goto again;
+               }
+       }
        if (info.type == BPF_LINK_TYPE_PERF_EVENT) {
                switch (info.perf_event.type) {
                case BPF_PERF_EVENT_TRACEPOINT:
@@ -1153,6 +1285,7 @@ static int do_show_link(int fd)
        free(cookies);
        free(offsets);
        free(addrs);
+       free(ids);
        close(fd);
        return 0;
 }
--

2.54.0

#9 lore raw
In reply to Jiri Olsa
This series was applied to bpf/bpf-next.git (master) by Andrii Nakryiko <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Andrii Nakryiko <[email protected]>:

Quoted 16 lines · On Mon, 29 Jun 2026 23:22:05 +0200 you wrote:

hi,
adding link info support for tracing_multi link.

Adding the kernel support with related selftests and bpftool support to display the tracing_multi link info, the output looks like:

  # bpftool link
  ...
  61: tracing_multi  prog 167
          attach_type trace_fentry_multi  btf_obj_id 1  count 3
          btf_id           addr             cookie           func [module]
          92598            ffffffff825017c4 10               bpf_fentry_test1
          92600            ffffffff82503814 30               bpf_fentry_test2
          92601            ffffffff82503824 20               bpf_fentry_test3
          pids test_progs(1540)
Quoted 1 line

[...]

Here is the summary with links:
- [PATCHv4,bpf-next,1/3] bpf: Add tracing_multi link info support

    https://git.kernel.org/bpf/bpf-next/c/859055e07697
  - [PATCHv4,bpf-next,2/3] selftests/bpf: Add tracing_multi link info tests
    https://git.kernel.org/bpf/bpf-next/c/d36e4dd547bb
  - [PATCHv4,bpf-next,3/3] bpftool: Add tracing_multi link info output
    https://git.kernel.org/bpf/bpf-next/c/37c1e353c9fd

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html