Skip to contentSkip to conversation

bpf: Introduce kprobe_multi session attach

PATCH Superseded Jiri Olsa 5 participants 28 messages Apr 22, 2024 – Apr 30, 2024

Subsystems (8)
Kselftest BPF Tracing BPF selftests +4 more subsystems

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

cover bpf: Introduce kprobe_multi session attach

hi,
adding support to attach kprobe program through kprobe_multi link in a session mode, which means:
- program is attached to both function entry and return
- entry program can decided if the return program gets executed
- entry program can share u64 cookie value with return program

The initial RFC for this was posted in [0] and later discussed more and which ended up with the session idea [1]

Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.

At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.

In addition the possibility to control the return program execution and sharing data within entry and return probe allows for other use cases.

Changes from last RFC version [1]:
- changed wrapper name to session
- changed flag to adding new attach type for session:
BPF_TRACE_KPROBE_MULTI_SESSION
it's more convenient wrt filtering on kfuncs setup and seems to make more sense alltogether
- renamed bpf_kprobe_multi_is_return to bpf_session_is_return
- added bpf_session_cookie kfunc, which actually already works on current fprobe implementation (not just fprobe-on-fgraph) and it provides the shared data between entry/return probes [Andrii]

we could actually make the cookie size configurable.. thoughts? (it's 8 bytes atm)

- better attach setup conditions changes [Andrii]
- I'm not including uprobes change atm, because it needs extra uprobe change so I'll post it separately

Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
bpf/session_data

thanks,
jirka

[0] https://lore.kernel.org/bpf/[email protected]/
[1] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (7):
bpf: Add support for kprobe multi session attach
bpf: Add support for kprobe multi session context
bpf: Add support for kprobe multi session cookie
libbpf: Add support for kprobe multi session attach
libbpf: Add kprobe session attach type name to attach_type_name selftests/bpf: Add kprobe multi session test selftests/bpf: Add kprobe multi wrapper cookie test

 include/uapi/linux/bpf.h                                        |   1 +
 kernel/bpf/btf.c                                                |   3 +++
 kernel/bpf/syscall.c                                            |   7 +++++-
 kernel/bpf/verifier.c                                           |   7 ++++++
 kernel/trace/bpf_trace.c                                        | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------
 tools/include/uapi/linux/bpf.h                                  |   1 +
 tools/lib/bpf/bpf.c                                             |   1 +
 tools/lib/bpf/libbpf.c                                          |  41 ++++++++++++++++++++++++++++++++--
 tools/lib/bpf/libbpf.h                                          |   4 +++-
 tools/testing/selftests/bpf/bpf_kfuncs.h                        |   3 +++
 tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c      |  84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/kprobe_multi_session.c        | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c |  56 ++++++++++++++++++++++++++++++++++++++++++++++
 13 files changed, 396 insertions(+), 18 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session.c
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c
author reviewer
#1 lore raw

PATCH 1/7 bpf: Add support for kprobe multi session attach

Adding support to attach bpf program for entry and return probe of the same function. This is common use case which at the moment requires to create two kprobe multi links.

Adding new BPF_TRACE_KPROBE_MULTI_SESSION attach type that instructs kernel to attach single link program to both entry and exit probe.

It's possible to control execution of the bpf program on return probe simply by returning zero or non zero from the entry bpf program execution to execute or not the bpf program on return probe respectively.

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  7 ++++++-
 kernel/trace/bpf_trace.c       | 28 ++++++++++++++++++++--------
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 28 insertions(+), 9 deletions(-)
include/uapi/linux/bpf.h +1 -0
index cee0a7915c08..fb8ecb199273 100644
@@ -1115,6 +1115,7 @@ enum bpf_attach_type {
 	BPF_CGROUP_UNIX_GETSOCKNAME,
 	BPF_NETKIT_PRIMARY,
 	BPF_NETKIT_PEER,
+	BPF_TRACE_KPROBE_MULTI_SESSION,
 	__MAX_BPF_ATTACH_TYPE
 };
 
kernel/bpf/syscall.c +6 -1
index 7d392ec83655..7c0cb1800b9f 100644
@@ -3974,11 +3974,15 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
 		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI &&
 		    attach_type != BPF_TRACE_KPROBE_MULTI)
 			return -EINVAL;
+		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI_SESSION &&
+		    attach_type != BPF_TRACE_KPROBE_MULTI_SESSION)
+			return -EINVAL;
 		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
 		    attach_type != BPF_TRACE_UPROBE_MULTI)
 			return -EINVAL;
 		if (attach_type != BPF_PERF_EVENT &&
 		    attach_type != BPF_TRACE_KPROBE_MULTI &&
+		    attach_type != BPF_TRACE_KPROBE_MULTI_SESSION &&
 		    attach_type != BPF_TRACE_UPROBE_MULTI)
 			return -EINVAL;
 		return 0;
@@ -5239,7 +5243,8 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 	case BPF_PROG_TYPE_KPROBE:
 		if (attr->link_create.attach_type == BPF_PERF_EVENT)
 			ret = bpf_perf_link_attach(attr, prog);
-		else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI)
+		else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI ||
+			 attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI_SESSION)
 			ret = bpf_kprobe_multi_link_attach(attr, prog);
 		else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI)
 			ret = bpf_uprobe_multi_link_attach(attr, prog);
kernel/trace/bpf_trace.c +20 -8
index afb232b1d7c2..3b15a40f425f 100644
@@ -1631,6 +1631,17 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	}
 }
 
+static bool is_kprobe_multi(const struct bpf_prog *prog)
+{
+	return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ||
+	       prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI_SESSION;
+}
+
+static inline bool is_kprobe_multi_session(const struct bpf_prog *prog)
+{
+	return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI_SESSION;
+}
+
 static const struct bpf_func_proto *
 kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 {
@@ -1646,13 +1657,13 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_override_return_proto;
 #endif
 	case BPF_FUNC_get_func_ip:
-		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
+		if (is_kprobe_multi(prog))
 			return &bpf_get_func_ip_proto_kprobe_multi;
 		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
 			return &bpf_get_func_ip_proto_uprobe_multi;
 		return &bpf_get_func_ip_proto_kprobe;
 	case BPF_FUNC_get_attach_cookie:
-		if (prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI)
+		if (is_kprobe_multi(prog))
 			return &bpf_get_attach_cookie_proto_kmulti;
 		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
 			return &bpf_get_attach_cookie_proto_umulti;
@@ -2834,10 +2845,11 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip,
 			  void *data)
 {
 	struct bpf_kprobe_multi_link *link;
+	int err;
 
 	link = container_of(fp, struct bpf_kprobe_multi_link, fp);
-	kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs);
-	return 0;
+	err = kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs);
+	return is_kprobe_multi_session(link->link.prog) ? err : 0;
 }
 
 static void
@@ -2981,7 +2993,7 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	if (sizeof(u64) != sizeof(void *))
 		return -EOPNOTSUPP;
 
-	if (prog->expected_attach_type != BPF_TRACE_KPROBE_MULTI)
+	if (!is_kprobe_multi(prog))
 		return -EINVAL;
 
 	flags = attr->link_create.kprobe_multi.flags;
@@ -3062,10 +3074,10 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	if (err)
 		goto error;
 
-	if (flags & BPF_F_KPROBE_MULTI_RETURN)
-		link->fp.exit_handler = kprobe_multi_link_exit_handler;
-	else
+	if (!(flags & BPF_F_KPROBE_MULTI_RETURN))
 		link->fp.entry_handler = kprobe_multi_link_handler;
+	if ((flags & BPF_F_KPROBE_MULTI_RETURN) || is_kprobe_multi_session(prog))
+		link->fp.exit_handler = kprobe_multi_link_exit_handler;
 
 	link->addrs = addrs;
 	link->cookies = cookies;
tools/include/uapi/linux/bpf.h +1 -0
index cee0a7915c08..fb8ecb199273 100644
@@ -1115,6 +1115,7 @@ enum bpf_attach_type {
 	BPF_CGROUP_UNIX_GETSOCKNAME,
 	BPF_NETKIT_PRIMARY,
 	BPF_NETKIT_PEER,
+	BPF_TRACE_KPROBE_MULTI_SESSION,
 	__MAX_BPF_ATTACH_TYPE
 };
 
Signature · 1 line
2.44.0
#2 lore raw

Re: [PATCH bpf-next 1/7] bpf: Add support for kprobe multi session attach

Quoted 26 lines · Jiri Olsa

Adding support to attach bpf program for entry and return probe of the same function. This is common use case which at the moment requires to create two kprobe multi links.

Adding new BPF_TRACE_KPROBE_MULTI_SESSION attach type that instructs kernel to attach single link program to both entry and exit probe.

It's possible to control execution of the bpf program on return probe simply by returning zero or non zero from the entry bpf program execution to execute or not the bpf program on return probe respectively.

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  7 ++++++-
 kernel/trace/bpf_trace.c       | 28 ++++++++++++++++++++--------
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 28 insertions(+), 9 deletions(-)
include/uapi/linux/bpf.h +0 -0
index cee0a7915c08..fb8ecb199273 100644

include/uapi/linux/bpf.h enum bpf_attach_type { · line 1118

Quoted 4 lines
include/uapi/linux/bpf.h +1 -0
@@ -1115,6 +1115,7 @@ enum bpf_attach_type {
        BPF_CGROUP_UNIX_GETSOCKNAME,
        BPF_NETKIT_PRIMARY,
        BPF_NETKIT_PEER,
+       BPF_TRACE_KPROBE_MULTI_SESSION,

let's use a shorter BPF_TRACE_KPROBE_SESSION? we'll just know that it's multi-variant (there is no point in adding non-multi kprobes going forward anyways, it's a new default)

Quoted 3 lines
include/uapi/linux/bpf.h +0 -0
        __MAX_BPF_ATTACH_TYPE
 };
 

[...]

Quoted 5 lines
kernel/trace/bpf_trace.c +0 -0
index afb232b1d7c2..3b15a40f425f 100644

kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1640

Quoted 10 lines
kernel/trace/bpf_trace.c +7 -0
@@ -1631,6 +1631,17 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
        }
 }
 
+static bool is_kprobe_multi(const struct bpf_prog *prog)
+{
+       return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ||
+              prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI_SESSION;
+}
+
+static inline bool is_kprobe_multi_session(const struct bpf_prog *prog)

ditto, this multi is just a distraction at this point, IMO

Quoted 7 lines
kernel/trace/bpf_trace.c +4 -0
+{
+       return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI_SESSION;
+}
+
 static const struct bpf_func_proto *
 kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 {

[...]

#3 lore raw

Re: [PATCH bpf-next 1/7] bpf: Add support for kprobe multi session attach

Quoted 50 lines · Andrii Nakryiko
Quoted 26 lines · Jiri Olsa

Adding support to attach bpf program for entry and return probe of the same function. This is common use case which at the moment requires to create two kprobe multi links.

Adding new BPF_TRACE_KPROBE_MULTI_SESSION attach type that instructs kernel to attach single link program to both entry and exit probe.

It's possible to control execution of the bpf program on return probe simply by returning zero or non zero from the entry bpf program execution to execute or not the bpf program on return probe respectively.

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  7 ++++++-
 kernel/trace/bpf_trace.c       | 28 ++++++++++++++++++++--------
 tools/include/uapi/linux/bpf.h |  1 +
 4 files changed, 28 insertions(+), 9 deletions(-)
include/uapi/linux/bpf.h +0 -0
index cee0a7915c08..fb8ecb199273 100644

include/uapi/linux/bpf.h enum bpf_attach_type { · line 1118

Quoted 4 lines
include/uapi/linux/bpf.h +1 -0
@@ -1115,6 +1115,7 @@ enum bpf_attach_type {
        BPF_CGROUP_UNIX_GETSOCKNAME,
        BPF_NETKIT_PRIMARY,
        BPF_NETKIT_PEER,
+       BPF_TRACE_KPROBE_MULTI_SESSION,

let's use a shorter BPF_TRACE_KPROBE_SESSION? we'll just know that it's multi-variant (there is no point in adding non-multi kprobes going forward anyways, it's a new default)

Quoted 3 lines
include/uapi/linux/bpf.h +0 -0
        __MAX_BPF_ATTACH_TYPE
 };
 

[...]

Quoted 8 lines
kernel/trace/bpf_trace.c +0 -0
index afb232b1d7c2..3b15a40f425f 100644
@@ -1631,6 +1631,17 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
        }
 }
 
Quoted 9 lines

kernel/trace/bpf_trace.c bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 1640

Quoted 7 lines
kernel/trace/bpf_trace.c +7 -0
+static bool is_kprobe_multi(const struct bpf_prog *prog)
+{
+       return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI ||
+              prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI_SESSION;
+}
+
+static inline bool is_kprobe_multi_session(const struct bpf_prog *prog)

ditto, this multi is just a distraction at this point, IMO

ok, sounds good, will drop multi for session stuff

jirka

Quoted 10 lines
Quoted 8 lines
kernel/trace/bpf_trace.c +4 -0
+{
+       return prog->expected_attach_type == BPF_TRACE_KPROBE_MULTI_SESSION;
+}
+
 static const struct bpf_func_proto *
 kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 {

[...]

author reviewer
#4 lore raw

PATCH 2/7 bpf: Add support for kprobe multi session context

In reply to Jiri Olsa

Adding struct bpf_session_run_ctx object to hold session related data, which is atm is_return bool and data pointer coming in following changes.

Placing bpf_session_run_ctx layer in between bpf_run_ctx and bpf_kprobe_multi_run_ctx so the session data can be retrieved regardless of if it's kprobe_multi or uprobe_multi link, which support is coming in future. This way both kprobe_multi and uprobe_multi can use same kfuncs to access the session data.

Adding bpf_session_is_return kfunc that returns true if the bpf program is executed from the exit probe of the kprobe multi link attached in wrapper mode. It returns false otherwise.

Adding new kprobe hook for kprobe program type.

---
 kernel/bpf/btf.c         |  3 ++
 kernel/trace/bpf_trace.c | 67 +++++++++++++++++++++++++++++++++++-----
 2 files changed, 63 insertions(+), 7 deletions(-)
kernel/bpf/btf.c +3 -0
index 6d46cee47ae3..7431230b1e3a 100644
@@ -218,6 +218,7 @@ enum btf_kfunc_hook {
 	BTF_KFUNC_HOOK_SOCKET_FILTER,
 	BTF_KFUNC_HOOK_LWT,
 	BTF_KFUNC_HOOK_NETFILTER,
+	BTF_KFUNC_HOOK_KPROBE,
 	BTF_KFUNC_HOOK_MAX,
 };
 
@@ -8140,6 +8141,8 @@ static int bpf_prog_type_to_kfunc_hook(enum bpf_prog_type prog_type)
 		return BTF_KFUNC_HOOK_LWT;
 	case BPF_PROG_TYPE_NETFILTER:
 		return BTF_KFUNC_HOOK_NETFILTER;
+	case BPF_PROG_TYPE_KPROBE:
+		return BTF_KFUNC_HOOK_KPROBE;
 	default:
 		return BTF_KFUNC_HOOK_MAX;
 	}
kernel/trace/bpf_trace.c +60 -7

119 lines not on this page · raw message

Signature · 1 line
2.44.0
Acked-by on v1 2/7
#5 lore raw

Re: [PATCH bpf-next 2/7] bpf: Add support for kprobe multi session context

Quoted 15 lines · Jiri Olsa

Adding struct bpf_session_run_ctx object to hold session related data, which is atm is_return bool and data pointer coming in following changes.

Placing bpf_session_run_ctx layer in between bpf_run_ctx and bpf_kprobe_multi_run_ctx so the session data can be retrieved regardless of if it's kprobe_multi or uprobe_multi link, which support is coming in future. This way both kprobe_multi and uprobe_multi can use same kfuncs to access the session data.

Adding bpf_session_is_return kfunc that returns true if the bpf program is executed from the exit probe of the kprobe multi link attached in wrapper mode. It returns false otherwise.

Quoted 8 lines

Adding new kprobe hook for kprobe program type.

---
 kernel/bpf/btf.c         |  3 ++
 kernel/trace/bpf_trace.c | 67 +++++++++++++++++++++++++++++++++++-----
 2 files changed, 63 insertions(+), 7 deletions(-)

LGTM, but see the question below

[...]

Quoted 10 lines
kernel/trace/bpf_trace.c +1 -1
@@ -2848,7 +2859,7 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip,
        int err;
 
        link = container_of(fp, struct bpf_kprobe_multi_link, fp);
-       err = kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs);
+       err = kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, false);
        return is_kprobe_multi_session(link->link.prog) ? err : 0;
 }
 

kernel/trace/bpf_trace.c kprobe_multi_link_exit_handler(struct fprobe *fp, unsigned long fentry_ip, · line 2874

Quoted 5 lines
kernel/trace/bpf_trace.c +1 -1
@@ -2860,7 +2871,7 @@ kprobe_multi_link_exit_handler(struct fprobe *fp, unsigned long fentry_ip,
        struct bpf_kprobe_multi_link *link;
 
        link = container_of(fp, struct bpf_kprobe_multi_link, fp);
-       kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs);
+       kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, true);

Is there some way to figure out whether we are an entry or return probe from struct fprobe itself? I was hoping to have a single callback for both entry and exit handler in fprobe to keep callback call chain a bit simpler

Quoted 3 lines
kernel/trace/bpf_trace.c +0 -0
 }
 
 static int symbols_cmp_r(const void *a, const void *b, const void *priv)

[...]

#6 lore raw

Re: [PATCH bpf-next 2/7] bpf: Add support for kprobe multi session context

Quoted 43 lines · Andrii Nakryiko
Quoted 15 lines · Jiri Olsa

Adding struct bpf_session_run_ctx object to hold session related data, which is atm is_return bool and data pointer coming in following changes.

Placing bpf_session_run_ctx layer in between bpf_run_ctx and bpf_kprobe_multi_run_ctx so the session data can be retrieved regardless of if it's kprobe_multi or uprobe_multi link, which support is coming in future. This way both kprobe_multi and uprobe_multi can use same kfuncs to access the session data.

Adding bpf_session_is_return kfunc that returns true if the bpf program is executed from the exit probe of the kprobe multi link attached in wrapper mode. It returns false otherwise.

Quoted 8 lines

Adding new kprobe hook for kprobe program type.

---
 kernel/bpf/btf.c         |  3 ++
 kernel/trace/bpf_trace.c | 67 +++++++++++++++++++++++++++++++++++-----
 2 files changed, 63 insertions(+), 7 deletions(-)

LGTM, but see the question below

[...]

Quoted 12 lines
kernel/trace/bpf_trace.c +1 -1
@@ -2848,7 +2859,7 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip,
        int err;
 
        link = container_of(fp, struct bpf_kprobe_multi_link, fp);
-       err = kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs);
+       err = kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, false);
        return is_kprobe_multi_session(link->link.prog) ? err : 0;
 }
 
@@ -2860,7 +2871,7 @@ kprobe_multi_link_exit_handler(struct fprobe *fp, unsigned long fentry_ip,
        struct bpf_kprobe_multi_link *link;
 
Quoted 8 lines

kernel/trace/bpf_trace.c kprobe_multi_link_exit_handler(struct fprobe *fp, unsigned long fentry_ip, · line 2874

Quoted 3 lines
kernel/trace/bpf_trace.c +1 -1
        link = container_of(fp, struct bpf_kprobe_multi_link, fp);
-       kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs);
+       kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, true);

Is there some way to figure out whether we are an entry or return probe from struct fprobe itself? I was hoping to have a single callback for both entry and exit handler in fprobe to keep callback call chain a bit simpler

AFAICS not at the moment.. also both callbacks have same arguments, just the entry handler returns int and exit handler void

jirka

author reviewer
#7 lore raw

PATCH 3/7 bpf: Add support for kprobe multi session cookie

In reply to Jiri Olsa

Adding support for cookie within the session of kprobe multi entry and return program.

The session cookie is u64 value and can be retrieved be new kfunc bpf_session_cookie, which returns pointer to the cookie value. The bpf program can use the pointer to store (on entry) and load (on return) the value.

The cookie value is implemented via fprobe feature that allows to share values between entry and return ftrace fprobe callbacks.

---
 kernel/bpf/verifier.c    |  7 +++++++
 kernel/trace/bpf_trace.c | 19 ++++++++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +7 -0
index 68cfd6fc6ad4..baaca451aebc 100644
@@ -10987,6 +10987,7 @@ enum special_kfunc_type {
 	KF_bpf_percpu_obj_drop_impl,
 	KF_bpf_throw,
 	KF_bpf_iter_css_task_new,
+	KF_bpf_session_cookie,
 };
 
 BTF_SET_START(special_kfunc_set)
@@ -11013,6 +11014,7 @@ BTF_ID(func, bpf_throw)
 #ifdef CONFIG_CGROUPS
 BTF_ID(func, bpf_iter_css_task_new)
 #endif
+BTF_ID(func, bpf_session_cookie)
 BTF_SET_END(special_kfunc_set)
 
 BTF_ID_LIST(special_kfunc_list)
@@ -11043,6 +11045,7 @@ BTF_ID(func, bpf_iter_css_task_new)
 #else
 BTF_ID_UNUSED
 #endif
+BTF_ID(func, bpf_session_cookie)
 
 static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta)
 {
@@ -12409,6 +12412,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 				 * because packet slices are not refcounted (see
 				 * dynptr_type_refcounted)
 				 */
+			} else if (meta.func_id == special_kfunc_list[KF_bpf_session_cookie]) {
+				mark_reg_known_zero(env, regs, BPF_REG_0);
+				regs[BPF_REG_0].type = PTR_TO_MEM;
+				regs[BPF_REG_0].mem_size = sizeof(u64);
 			} else {
 				verbose(env, "kernel function %s unhandled dynamic return type\n",
 					meta.func_name);
kernel/trace/bpf_trace.c +16 -3
index d82402316d84..a6863c1905ca 100644
@@ -2599,6 +2599,7 @@ fs_initcall(bpf_event_init);
 struct bpf_session_run_ctx {
 	struct bpf_run_ctx run_ctx;
 	bool is_return;
+	void *data;
 };
 
 #ifdef CONFIG_FPROBE
@@ -2819,11 +2820,12 @@ static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
 static int
 kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link,
 			   unsigned long entry_ip, struct pt_regs *regs,
-			   bool is_return)
+			   bool is_return, void *data)
 {
 	struct bpf_kprobe_multi_run_ctx run_ctx = {
 		.session_ctx = {
 			.is_return = is_return,
+			.data = data,
 		},
 		.link = link,
 		.entry_ip = entry_ip,
@@ -2859,7 +2861,7 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip,
 	int err;
 
 	link = container_of(fp, struct bpf_kprobe_multi_link, fp);
-	err = kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, false);
+	err = kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, false, data);
 	return is_kprobe_multi_session(link->link.prog) ? err : 0;
 }
 
@@ -2871,7 +2873,7 @@ kprobe_multi_link_exit_handler(struct fprobe *fp, unsigned long fentry_ip,
 	struct bpf_kprobe_multi_link *link;
 
 	link = container_of(fp, struct bpf_kprobe_multi_link, fp);
-	kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, true);
+	kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs, true, data);
 }
 
 static int symbols_cmp_r(const void *a, const void *b, const void *priv)
@@ -3089,6 +3091,8 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 		link->fp.entry_handler = kprobe_multi_link_handler;
 	if ((flags & BPF_F_KPROBE_MULTI_RETURN) || is_kprobe_multi_session(prog))
 		link->fp.exit_handler = kprobe_multi_link_exit_handler;
+	if (is_kprobe_multi_session(prog))
+		link->fp.entry_data_size = sizeof(u64);
 
 	link->addrs = addrs;
 	link->cookies = cookies;
@@ -3526,10 +3530,19 @@ __bpf_kfunc bool bpf_session_is_return(void)
 	return session_ctx->is_return;
 }
 
+__bpf_kfunc __u64 *bpf_session_cookie(void)
+{
+	struct bpf_session_run_ctx *session_ctx;
+
+	session_ctx = container_of(current->bpf_ctx, struct bpf_session_run_ctx, run_ctx);
+	return session_ctx->data;
+}
+
 __bpf_kfunc_end_defs();
 
 BTF_KFUNCS_START(kprobe_multi_kfunc_set_ids)
 BTF_ID_FLAGS(func, bpf_session_is_return)
+BTF_ID_FLAGS(func, bpf_session_cookie)
 BTF_KFUNCS_END(kprobe_multi_kfunc_set_ids)
 
 static int bpf_kprobe_multi_filter(const struct bpf_prog *prog, u32 kfunc_id)
Signature · 1 line
2.44.0
#8 lore raw

Re: [PATCH bpf-next 3/7] bpf: Add support for kprobe multi session cookie

Quoted 47 lines · Jiri Olsa

Adding support for cookie within the session of kprobe multi entry and return program.

The session cookie is u64 value and can be retrieved be new kfunc bpf_session_cookie, which returns pointer to the cookie value. The bpf program can use the pointer to store (on entry) and load (on return) the value.

The cookie value is implemented via fprobe feature that allows to share values between entry and return ftrace fprobe callbacks.

---
 kernel/bpf/verifier.c    |  7 +++++++
 kernel/trace/bpf_trace.c | 19 ++++++++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +3 -0
index 68cfd6fc6ad4..baaca451aebc 100644
@@ -10987,6 +10987,7 @@ enum special_kfunc_type {
        KF_bpf_percpu_obj_drop_impl,
        KF_bpf_throw,
        KF_bpf_iter_css_task_new,
+       KF_bpf_session_cookie,
 };
 
 BTF_SET_START(special_kfunc_set)
@@ -11013,6 +11014,7 @@ BTF_ID(func, bpf_throw)
 #ifdef CONFIG_CGROUPS
 BTF_ID(func, bpf_iter_css_task_new)
 #endif
+BTF_ID(func, bpf_session_cookie)
 BTF_SET_END(special_kfunc_set)
 
 BTF_ID_LIST(special_kfunc_list)
@@ -11043,6 +11045,7 @@ BTF_ID(func, bpf_iter_css_task_new)
 #else
 BTF_ID_UNUSED
 #endif
+BTF_ID(func, bpf_session_cookie)
 
 static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta)
 {

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 12418

Quoted 7 lines
kernel/bpf/verifier.c +4 -0
@@ -12409,6 +12412,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
                                 * because packet slices are not refcounted (see
                                 * dynptr_type_refcounted)
                                 */
+                       } else if (meta.func_id == special_kfunc_list[KF_bpf_session_cookie]) {
+                               mark_reg_known_zero(env, regs, BPF_REG_0);
+                               regs[BPF_REG_0].type = PTR_TO_MEM;
+                               regs[BPF_REG_0].mem_size = sizeof(u64);

Are you sure you need this?

} else if (!__btf_type_is_struct(ptr_type)) {

block should have handled it automatically.

Quoted 2 lines
kernel/bpf/verifier.c +2 -0
+__bpf_kfunc __u64 *bpf_session_cookie(void)
+{

...

#9 lore raw

Re: [PATCH bpf-next 3/7] bpf: Add support for kprobe multi session cookie

Quoted 58 lines · Alexei Starovoitov
Quoted 47 lines · Jiri Olsa

Adding support for cookie within the session of kprobe multi entry and return program.

The session cookie is u64 value and can be retrieved be new kfunc bpf_session_cookie, which returns pointer to the cookie value. The bpf program can use the pointer to store (on entry) and load (on return) the value.

The cookie value is implemented via fprobe feature that allows to share values between entry and return ftrace fprobe callbacks.

---
 kernel/bpf/verifier.c    |  7 +++++++
 kernel/trace/bpf_trace.c | 19 ++++++++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +3 -0
index 68cfd6fc6ad4..baaca451aebc 100644
@@ -10987,6 +10987,7 @@ enum special_kfunc_type {
        KF_bpf_percpu_obj_drop_impl,
        KF_bpf_throw,
        KF_bpf_iter_css_task_new,
+       KF_bpf_session_cookie,
 };
 
 BTF_SET_START(special_kfunc_set)
@@ -11013,6 +11014,7 @@ BTF_ID(func, bpf_throw)
 #ifdef CONFIG_CGROUPS
 BTF_ID(func, bpf_iter_css_task_new)
 #endif
+BTF_ID(func, bpf_session_cookie)
 BTF_SET_END(special_kfunc_set)
 
 BTF_ID_LIST(special_kfunc_list)
@@ -11043,6 +11045,7 @@ BTF_ID(func, bpf_iter_css_task_new)
 #else
 BTF_ID_UNUSED
 #endif
+BTF_ID(func, bpf_session_cookie)
 
 static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta)
 {

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 12418

Quoted 7 lines
kernel/bpf/verifier.c +4 -0
@@ -12409,6 +12412,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
                                 * because packet slices are not refcounted (see
                                 * dynptr_type_refcounted)
                                 */
+                       } else if (meta.func_id == special_kfunc_list[KF_bpf_session_cookie]) {
+                               mark_reg_known_zero(env, regs, BPF_REG_0);
+                               regs[BPF_REG_0].type = PTR_TO_MEM;
+                               regs[BPF_REG_0].mem_size = sizeof(u64);

Are you sure you need this?

Quoted 3 lines

} else if (!__btf_type_is_struct(ptr_type)) {

block should have handled it automatically.

yes, but only as read-only memory and we need the bpf program to be able to write to it

I'll double check that, but AFAICS we can't set r0_size/!r0_rdonly before we reach that if block, because bpf_session_cookie has no arguments

jirka

Quoted 5 lines
Quoted 3 lines
kernel/bpf/verifier.c +2 -0
+__bpf_kfunc __u64 *bpf_session_cookie(void)
+{

...

Acked-by on v1 3/7
#10 lore raw

Re: [PATCH bpf-next 3/7] bpf: Add support for kprobe multi session cookie

In reply to Jiri Olsa
Quoted 9 lines · Jiri Olsa

Adding support for cookie within the session of kprobe multi entry and return program.

The session cookie is u64 value and can be retrieved be new kfunc bpf_session_cookie, which returns pointer to the cookie value. The bpf program can use the pointer to store (on entry) and load (on return) the value.

Quoted 9 lines

The cookie value is implemented via fprobe feature that allows to share values between entry and return ftrace fprobe callbacks.

---
 kernel/bpf/verifier.c    |  7 +++++++
 kernel/trace/bpf_trace.c | 19 ++++++++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)

Had the same question as Alexei, but this read-write semantics quirk makes sense. But it's probably a bit more reliable and cleaner to handle it by special casing this kfunc a bit earlier (see KF_bpf_rbtree_add_impl) and setting r0_size = 8, r0_rdonly = false. And then let generic PTR -> INT logic kick in. You'll be futzing with register state much less.

Other than that looks good:

Quoted 10 lines
kernel/bpf/verifier.c +1 -0
index 68cfd6fc6ad4..baaca451aebc 100644
@@ -10987,6 +10987,7 @@ enum special_kfunc_type {
        KF_bpf_percpu_obj_drop_impl,
        KF_bpf_throw,
        KF_bpf_iter_css_task_new,
+       KF_bpf_session_cookie,
 };

[...]

#11 lore raw

Re: [PATCH bpf-next 3/7] bpf: Add support for kprobe multi session cookie

Quoted 20 lines · Andrii Nakryiko
Quoted 9 lines · Jiri Olsa

Adding support for cookie within the session of kprobe multi entry and return program.

The session cookie is u64 value and can be retrieved be new kfunc bpf_session_cookie, which returns pointer to the cookie value. The bpf program can use the pointer to store (on entry) and load (on return) the value.

Quoted 9 lines

The cookie value is implemented via fprobe feature that allows to share values between entry and return ftrace fprobe callbacks.

---
 kernel/bpf/verifier.c    |  7 +++++++
 kernel/trace/bpf_trace.c | 19 ++++++++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)
Quoted 6 lines

Had the same question as Alexei, but this read-write semantics quirk makes sense. But it's probably a bit more reliable and cleaner to handle it by special casing this kfunc a bit earlier (see KF_bpf_rbtree_add_impl) and setting r0_size = 8, r0_rdonly = false. And then let generic PTR -> INT logic kick in. You'll be futzing with register state much less.

ok, will try it this way

thanks,
jirka

Quoted 17 lines

Other than that looks good:

Quoted 10 lines
kernel/bpf/verifier.c +1 -0
index 68cfd6fc6ad4..baaca451aebc 100644
@@ -10987,6 +10987,7 @@ enum special_kfunc_type {
        KF_bpf_percpu_obj_drop_impl,
        KF_bpf_throw,
        KF_bpf_iter_css_task_new,
+       KF_bpf_session_cookie,
 };

[...]

author reviewer
#12 lore raw

PATCH 4/7 libbpf: Add support for kprobe multi session attach

In reply to Jiri Olsa

Adding support to attach program in kprobe multi session mode with bpf_program__attach_kprobe_multi_opts function.

Adding session bool to bpf_kprobe_multi_opts struct that allows to load and attach the bpf program via kprobe multi session. the attachment to create kprobe multi session.

Also adding new program loader section that allows:
SEC("kprobe.session/bpf_fentry_test*")

and loads/attaches kprobe program as kprobe multi session.

---
 tools/lib/bpf/bpf.c    |  1 +
 tools/lib/bpf/libbpf.c | 40 ++++++++++++++++++++++++++++++++++++++--
 tools/lib/bpf/libbpf.h |  4 +++-
 3 files changed, 42 insertions(+), 3 deletions(-)
tools/lib/bpf/bpf.c +1 -0
index c9f4e04f38fe..5f556e38910f 100644
@@ -766,6 +766,7 @@ int bpf_link_create(int prog_fd, int target_fd,
 			return libbpf_err(-EINVAL);
 		break;
 	case BPF_TRACE_KPROBE_MULTI:
+	case BPF_TRACE_KPROBE_MULTI_SESSION:
 		attr.link_create.kprobe_multi.flags = OPTS_GET(opts, kprobe_multi.flags, 0);
 		attr.link_create.kprobe_multi.cnt = OPTS_GET(opts, kprobe_multi.cnt, 0);
 		attr.link_create.kprobe_multi.syms = ptr_to_u64(OPTS_GET(opts, kprobe_multi.syms, 0));
tools/lib/bpf/libbpf.c +38 -2
index 97eb6e5dd7c8..ca605240205f 100644
@@ -9272,6 +9272,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin
 static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
+static int attach_kprobe_session(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link);
@@ -9288,6 +9289,7 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("uretprobe.s+",		KPROBE, 0, SEC_SLEEPABLE, attach_uprobe),
 	SEC_DEF("kprobe.multi+",	KPROBE,	BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
 	SEC_DEF("kretprobe.multi+",	KPROBE,	BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
+	SEC_DEF("kprobe.session+",	KPROBE,	BPF_TRACE_KPROBE_MULTI_SESSION, SEC_NONE, attach_kprobe_session),
 	SEC_DEF("uprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
 	SEC_DEF("uretprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
 	SEC_DEF("uprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
@@ -11380,13 +11382,14 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
 	struct kprobe_multi_resolve res = {
 		.pattern = pattern,
 	};
+	enum bpf_attach_type attach_type;
 	struct bpf_link *link = NULL;
 	char errmsg[STRERR_BUFSIZE];
 	const unsigned long *addrs;
 	int err, link_fd, prog_fd;
+	bool retprobe, session;
 	const __u64 *cookies;
 	const char **syms;
-	bool retprobe;
 	size_t cnt;
 
 	if (!OPTS_VALID(opts, bpf_kprobe_multi_opts))
@@ -11425,6 +11428,13 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
 	}
 
 	retprobe = OPTS_GET(opts, retprobe, false);
+	session  = OPTS_GET(opts, session, false);
+
+	if (retprobe && session)
+		return libbpf_err_ptr(-EINVAL);
+
+	attach_type = session ? BPF_TRACE_KPROBE_MULTI_SESSION :
+				BPF_TRACE_KPROBE_MULTI;
 
 	lopts.kprobe_multi.syms = syms;
 	lopts.kprobe_multi.addrs = addrs;
@@ -11439,7 +11449,7 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
 	}
 	link->detach = &bpf_link__detach_fd;
 
-	link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_KPROBE_MULTI, &lopts);
+	link_fd = bpf_link_create(prog_fd, 0, attach_type, &lopts);
 	if (link_fd < 0) {
 		err = -errno;
 		pr_warn("prog '%s': failed to attach: %s\n",
@@ -11545,6 +11555,32 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru
 	return libbpf_get_error(*link);
 }
 
+static int attach_kprobe_session(const struct bpf_program *prog, long cookie,
+				 struct bpf_link **link)
+{
+	LIBBPF_OPTS(bpf_kprobe_multi_opts, opts, .session = true);
+	const char *spec;
+	char *pattern;
+	int n;
+
+	*link = NULL;
+
+	/* no auto-attach for SEC("kprobe.session") */
+	if (strcmp(prog->sec_name, "kprobe.session") == 0)
+		return 0;
+
+	spec = prog->sec_name + sizeof("kprobe.session/") - 1;
+	n = sscanf(spec, "%m[a-zA-Z0-9_.*?]", &pattern);
+	if (n < 1) {
+		pr_warn("kprobe session pattern is invalid: %s\n", pattern);
+		return -EINVAL;
+	}
+
+	*link = bpf_program__attach_kprobe_multi_opts(prog, pattern, &opts);
+	free(pattern);
+	return libbpf_get_error(*link);
+}
+
 static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link)
 {
 	char *probe_type = NULL, *binary_path = NULL, *func_name = NULL;
tools/lib/bpf/libbpf.h +3 -1
index 1333ae20ebe6..c3f77d9260fe 100644
@@ -539,10 +539,12 @@ struct bpf_kprobe_multi_opts {
 	size_t cnt;
 	/* create return kprobes */
 	bool retprobe;
+	/* create session kprobes */
+	bool session;
 	size_t :0;
 };
 
-#define bpf_kprobe_multi_opts__last_field retprobe
+#define bpf_kprobe_multi_opts__last_field session
 
 LIBBPF_API struct bpf_link *
 bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
Signature · 1 line
2.44.0
Acked-by on v1 4/7
#13 lore raw

Re: [PATCH bpf-next 4/7] libbpf: Add support for kprobe multi session attach

Quoted 11 lines · Jiri Olsa

Adding support to attach program in kprobe multi session mode with bpf_program__attach_kprobe_multi_opts function.

Adding session bool to bpf_kprobe_multi_opts struct that allows to load and attach the bpf program via kprobe multi session. the attachment to create kprobe multi session.

Also adding new program loader section that allows:
SEC("kprobe.session/bpf_fentry_test*")

Quoted 9 lines

and loads/attaches kprobe program as kprobe multi session.

---
 tools/lib/bpf/bpf.c    |  1 +
 tools/lib/bpf/libbpf.c | 40 ++++++++++++++++++++++++++++++++++++++--
 tools/lib/bpf/libbpf.h |  4 +++-
 3 files changed, 42 insertions(+), 3 deletions(-)

Minor nits below, but LGTM overall:

Quoted 49 lines
tools/lib/bpf/bpf.c +1 -0
index c9f4e04f38fe..5f556e38910f 100644
@@ -766,6 +766,7 @@ int bpf_link_create(int prog_fd, int target_fd,
                        return libbpf_err(-EINVAL);
                break;
        case BPF_TRACE_KPROBE_MULTI:
+       case BPF_TRACE_KPROBE_MULTI_SESSION:
                attr.link_create.kprobe_multi.flags = OPTS_GET(opts, kprobe_multi.flags, 0);
                attr.link_create.kprobe_multi.cnt = OPTS_GET(opts, kprobe_multi.cnt, 0);
                attr.link_create.kprobe_multi.syms = ptr_to_u64(OPTS_GET(opts, kprobe_multi.syms, 0));
tools/lib/bpf/libbpf.c +4 -1
index 97eb6e5dd7c8..ca605240205f 100644
@@ -9272,6 +9272,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin
 static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
+static int attach_kprobe_session(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link);
@@ -9288,6 +9289,7 @@ static const struct bpf_sec_def section_defs[] = {
        SEC_DEF("uretprobe.s+",         KPROBE, 0, SEC_SLEEPABLE, attach_uprobe),
        SEC_DEF("kprobe.multi+",        KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
        SEC_DEF("kretprobe.multi+",     KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
+       SEC_DEF("kprobe.session+",      KPROBE, BPF_TRACE_KPROBE_MULTI_SESSION, SEC_NONE, attach_kprobe_session),
        SEC_DEF("uprobe.multi+",        KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
        SEC_DEF("uretprobe.multi+",     KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
        SEC_DEF("uprobe.multi.s+",      KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
@@ -11380,13 +11382,14 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
        struct kprobe_multi_resolve res = {
                .pattern = pattern,
        };
+       enum bpf_attach_type attach_type;
        struct bpf_link *link = NULL;
        char errmsg[STRERR_BUFSIZE];
        const unsigned long *addrs;
        int err, link_fd, prog_fd;
+       bool retprobe, session;
        const __u64 *cookies;
        const char **syms;
-       bool retprobe;
        size_t cnt;
 
        if (!OPTS_VALID(opts, bpf_kprobe_multi_opts))

tools/lib/bpf/libbpf.c bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 11437

Quoted 10 lines
tools/lib/bpf/libbpf.c +7 -0
@@ -11425,6 +11428,13 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
        }
 
        retprobe = OPTS_GET(opts, retprobe, false);
+       session  = OPTS_GET(opts, session, false);
+
+       if (retprobe && session)
+               return libbpf_err_ptr(-EINVAL);
+
+       attach_type = session ? BPF_TRACE_KPROBE_MULTI_SESSION :
+                               BPF_TRACE_KPROBE_MULTI;

doesn't fit under 100?

Quoted 30 lines
tools/lib/bpf/libbpf.c +15 -1
        lopts.kprobe_multi.syms = syms;
        lopts.kprobe_multi.addrs = addrs;
@@ -11439,7 +11449,7 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
        }
        link->detach = &bpf_link__detach_fd;
 
-       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_KPROBE_MULTI, &lopts);
+       link_fd = bpf_link_create(prog_fd, 0, attach_type, &lopts);
        if (link_fd < 0) {
                err = -errno;
                pr_warn("prog '%s': failed to attach: %s\n",
@@ -11545,6 +11555,32 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru
        return libbpf_get_error(*link);
 }
 
+static int attach_kprobe_session(const struct bpf_program *prog, long cookie,
+                                struct bpf_link **link)
+{
+       LIBBPF_OPTS(bpf_kprobe_multi_opts, opts, .session = true);
+       const char *spec;
+       char *pattern;
+       int n;
+
+       *link = NULL;
+
+       /* no auto-attach for SEC("kprobe.session") */
+       if (strcmp(prog->sec_name, "kprobe.session") == 0)
+               return 0;
+

tools/lib/bpf/libbpf.c static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru · line 11581

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+       spec = prog->sec_name + sizeof("kprobe.session/") - 1;
+       n = sscanf(spec, "%m[a-zA-Z0-9_.*?]", &pattern);
+       if (n < 1) {
+               pr_warn("kprobe session pattern is invalid: %s\n", pattern);
+               return -EINVAL;
+       }
+
+       *link = bpf_program__attach_kprobe_multi_opts(prog, pattern, &opts);
+       free(pattern);
+       return libbpf_get_error(*link);

let's try not to add new uses of libbpf_get_error? Would this work:

return *link ? 0 : -errno;

?

Quoted 26 lines
tools/lib/bpf/libbpf.c +2 -0
+}
+
 static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link)
 {
        char *probe_type = NULL, *binary_path = NULL, *func_name = NULL;
tools/lib/bpf/libbpf.h +3 -2
index 1333ae20ebe6..c3f77d9260fe 100644
@@ -539,10 +539,12 @@ struct bpf_kprobe_multi_opts {
        size_t cnt;
        /* create return kprobes */
        bool retprobe;
+       /* create session kprobes */
+       bool session;
        size_t :0;
 };
 
-#define bpf_kprobe_multi_opts__last_field retprobe
+#define bpf_kprobe_multi_opts__last_field session
 
 LIBBPF_API struct bpf_link *
 bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
--

2.44.0

#14 lore raw

Re: [PATCH bpf-next 4/7] libbpf: Add support for kprobe multi session attach

On Tue, Apr 23, 2024 at 05:26:59PM -0700, Andrii Nakryiko wrote:

SNIP

Quoted 39 lines
Quoted 39 lines
tools/lib/bpf/libbpf.c +4 -1
index 97eb6e5dd7c8..ca605240205f 100644
@@ -9272,6 +9272,7 @@ static int attach_tp(const struct bpf_program *prog, long cookie, struct bpf_lin
 static int attach_raw_tp(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_trace(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
+static int attach_kprobe_session(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_lsm(const struct bpf_program *prog, long cookie, struct bpf_link **link);
 static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_link **link);
@@ -9288,6 +9289,7 @@ static const struct bpf_sec_def section_defs[] = {
        SEC_DEF("uretprobe.s+",         KPROBE, 0, SEC_SLEEPABLE, attach_uprobe),
        SEC_DEF("kprobe.multi+",        KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
        SEC_DEF("kretprobe.multi+",     KPROBE, BPF_TRACE_KPROBE_MULTI, SEC_NONE, attach_kprobe_multi),
+       SEC_DEF("kprobe.session+",      KPROBE, BPF_TRACE_KPROBE_MULTI_SESSION, SEC_NONE, attach_kprobe_session),
        SEC_DEF("uprobe.multi+",        KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
        SEC_DEF("uretprobe.multi+",     KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
        SEC_DEF("uprobe.multi.s+",      KPROBE, BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
@@ -11380,13 +11382,14 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
        struct kprobe_multi_resolve res = {
                .pattern = pattern,
        };
+       enum bpf_attach_type attach_type;
        struct bpf_link *link = NULL;
        char errmsg[STRERR_BUFSIZE];
        const unsigned long *addrs;
        int err, link_fd, prog_fd;
+       bool retprobe, session;
        const __u64 *cookies;
        const char **syms;
-       bool retprobe;
        size_t cnt;
 
        if (!OPTS_VALID(opts, bpf_kprobe_multi_opts))
@@ -11425,6 +11428,13 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
        }
 
Quoted 10 lines

tools/lib/bpf/libbpf.c bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog, · line 11437

Quoted 8 lines
tools/lib/bpf/libbpf.c +7 -0
        retprobe = OPTS_GET(opts, retprobe, false);
+       session  = OPTS_GET(opts, session, false);
+
+       if (retprobe && session)
+               return libbpf_err_ptr(-EINVAL);
+
+       attach_type = session ? BPF_TRACE_KPROBE_MULTI_SESSION :
+                               BPF_TRACE_KPROBE_MULTI;

doesn't fit under 100?

88 ;-) ok

Quoted 44 lines
Quoted 31 lines
tools/lib/bpf/libbpf.c +15 -1
        lopts.kprobe_multi.syms = syms;
        lopts.kprobe_multi.addrs = addrs;
@@ -11439,7 +11449,7 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
        }
        link->detach = &bpf_link__detach_fd;
 
-       link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_KPROBE_MULTI, &lopts);
+       link_fd = bpf_link_create(prog_fd, 0, attach_type, &lopts);
        if (link_fd < 0) {
                err = -errno;
                pr_warn("prog '%s': failed to attach: %s\n",
@@ -11545,6 +11555,32 @@ static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru
        return libbpf_get_error(*link);
 }
 
+static int attach_kprobe_session(const struct bpf_program *prog, long cookie,
+                                struct bpf_link **link)
+{
+       LIBBPF_OPTS(bpf_kprobe_multi_opts, opts, .session = true);
+       const char *spec;
+       char *pattern;
+       int n;
+
+       *link = NULL;
+
+       /* no auto-attach for SEC("kprobe.session") */
+       if (strcmp(prog->sec_name, "kprobe.session") == 0)
+               return 0;
+

tools/lib/bpf/libbpf.c static int attach_kprobe_multi(const struct bpf_program *prog, long cookie, stru · line 11581

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+       spec = prog->sec_name + sizeof("kprobe.session/") - 1;
+       n = sscanf(spec, "%m[a-zA-Z0-9_.*?]", &pattern);
+       if (n < 1) {
+               pr_warn("kprobe session pattern is invalid: %s\n", pattern);
+               return -EINVAL;
+       }
+
+       *link = bpf_program__attach_kprobe_multi_opts(prog, pattern, &opts);
+       free(pattern);
+       return libbpf_get_error(*link);

let's try not to add new uses of libbpf_get_error? Would this work:

Quoted 1 line

return *link ? 0 : -errno;

ok

jirka

author reviewer
#15 lore raw

PATCH 5/7 libbpf: Add kprobe session attach type name to attach_type_name

In reply to Jiri Olsa

Adding kprobe session attach type name to attach_type_name, so libbpf_bpf_attach_type_str returns proper string name for BPF_TRACE_KPROBE_MULTI_SESSION attach type.

---
 tools/lib/bpf/libbpf.c | 1 +
 1 file changed, 1 insertion(+)
tools/lib/bpf/libbpf.c +1 -0
index ca605240205f..9bf6cccb3443 100644
@@ -132,6 +132,7 @@ static const char * const attach_type_name[] = {
 	[BPF_TRACE_UPROBE_MULTI]	= "trace_uprobe_multi",
 	[BPF_NETKIT_PRIMARY]		= "netkit_primary",
 	[BPF_NETKIT_PEER]		= "netkit_peer",
+	[BPF_TRACE_KPROBE_MULTI_SESSION]	= "trace_kprobe_multi_session",
 };
 
 static const char * const link_type_name[] = {
Signature · 1 line
2.44.0
#16 lore raw

Re: [PATCH bpf-next 5/7] libbpf: Add kprobe session attach type name to attach_type_name

Quoted 15 lines · Jiri Olsa

Adding kprobe session attach type name to attach_type_name, so libbpf_bpf_attach_type_str returns proper string name for BPF_TRACE_KPROBE_MULTI_SESSION attach type.

---
 tools/lib/bpf/libbpf.c | 1 +
 1 file changed, 1 insertion(+)
tools/lib/bpf/libbpf.c +0 -0
index ca605240205f..9bf6cccb3443 100644

tools/lib/bpf/libbpf.c static const char * const attach_type_name[] = { · line 135

Quoted 4 lines
tools/lib/bpf/libbpf.c +1 -0
@@ -132,6 +132,7 @@ static const char * const attach_type_name[] = {
        [BPF_TRACE_UPROBE_MULTI]        = "trace_uprobe_multi",
        [BPF_NETKIT_PRIMARY]            = "netkit_primary",
        [BPF_NETKIT_PEER]               = "netkit_peer",
+       [BPF_TRACE_KPROBE_MULTI_SESSION]        = "trace_kprobe_multi_session",

we got to shorten this to just BPF_TRACE_KPROBE_SESSION :)

Quoted 6 lines
tools/lib/bpf/libbpf.c +0 -1
 };
 
 static const char * const link_type_name[] = {
--

2.44.0

#17 lore raw

Re: [PATCH bpf-next 5/7] libbpf: Add kprobe session attach type name to attach_type_name

Quoted 11 lines · Andrii Nakryiko
Quoted 10 lines · Jiri Olsa

Adding kprobe session attach type name to attach_type_name, so libbpf_bpf_attach_type_str returns proper string name for BPF_TRACE_KPROBE_MULTI_SESSION attach type.

---
 tools/lib/bpf/libbpf.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 11 lines

tools/lib/bpf/libbpf.c static const char * const attach_type_name[] = { · line 135

Quoted 9 lines
tools/lib/bpf/libbpf.c +1 -0
index ca605240205f..9bf6cccb3443 100644
@@ -132,6 +132,7 @@ static const char * const attach_type_name[] = {
        [BPF_TRACE_UPROBE_MULTI]        = "trace_uprobe_multi",
        [BPF_NETKIT_PRIMARY]            = "netkit_primary",
        [BPF_NETKIT_PEER]               = "netkit_peer",
+       [BPF_TRACE_KPROBE_MULTI_SESSION]        = "trace_kprobe_multi_session",

we got to shorten this to just BPF_TRACE_KPROBE_SESSION :)

ook :)

jirka

Quoted 8 lines
Quoted 8 lines
tools/lib/bpf/libbpf.c +0 -1
 };
 
 static const char * const link_type_name[] = {
--

2.44.0

author reviewer
#18 lore raw

PATCH 6/7 selftests/bpf: Add kprobe multi session test

In reply to Jiri Olsa

Adding kprobe multi session test and testing that the entry program return value controls execution of the return probe program.

---
 tools/testing/selftests/bpf/bpf_kfuncs.h      |   2 +
 .../bpf/prog_tests/kprobe_multi_test.c        |  49 +++++++++
 .../bpf/progs/kprobe_multi_session.c          | 100 ++++++++++++++++++
 3 files changed, 151 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session.c
tools/testing/selftests/bpf/bpf_kfuncs.h +2 -0
index 14ebe7d9e1a3..180030b5d828 100644
@@ -75,4 +75,6 @@ extern void bpf_key_put(struct bpf_key *key) __ksym;
 extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr,
 				      struct bpf_dynptr *sig_ptr,
 				      struct bpf_key *trusted_keyring) __ksym;
+
+extern bool bpf_session_is_return(void) __ksym;
 #endif
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +49 -0

65 lines not on this page · raw message

tools/testing/selftests/bpf/progs/kprobe_multi_session.c new file +100 -0

100 lines not on this page · raw message

Signature · 1 line
2.44.0
#19 lore raw

Re: [PATCH bpf-next 6/7] selftests/bpf: Add kprobe multi session test

Quoted 40 lines · Jiri Olsa

Adding kprobe multi session test and testing that the entry program return value controls execution of the return probe program.

---
 tools/testing/selftests/bpf/bpf_kfuncs.h      |   2 +
 .../bpf/prog_tests/kprobe_multi_test.c        |  49 +++++++++
 .../bpf/progs/kprobe_multi_session.c          | 100 ++++++++++++++++++
 3 files changed, 151 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session.c
tools/testing/selftests/bpf/bpf_kfuncs.h +2 -0
index 14ebe7d9e1a3..180030b5d828 100644
@@ -75,4 +75,6 @@ extern void bpf_key_put(struct bpf_key *key) __ksym;
 extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr,
                                      struct bpf_dynptr *sig_ptr,
                                      struct bpf_key *trusted_keyring) __ksym;
+
+extern bool bpf_session_is_return(void) __ksym;
 #endif
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +2 -0
index 51628455b6f5..d1f116665551 100644
@@ -4,6 +4,7 @@
 #include "trace_helpers.h"
 #include "kprobe_multi_empty.skel.h"
 #include "kprobe_multi_override.skel.h"
+#include "kprobe_multi_session.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "bpf/hashmap.h"
 
@@ -326,6 +327,52 @@ static void test_attach_api_fails(void)
        kprobe_multi__destroy(skel);
 }
 
+static void test_session_skel_api(void)

tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 340

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +10 -0
+{
+       struct kprobe_multi_session *skel = NULL;
+       LIBBPF_OPTS(bpf_kprobe_multi_opts, opts);
+       LIBBPF_OPTS(bpf_test_run_opts, topts);
+       struct bpf_link *link = NULL;
+       int err, prog_fd;
+
+       skel = kprobe_multi_session__open_and_load();
+       if (!ASSERT_OK_PTR(skel, "kprobe_multi_session__open_and_load"))
+               goto cleanup;

return?

tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 344

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +4 -0
+
+       skel->bss->pid = getpid();
+
+       err =  kprobe_multi_session__attach(skel);

nit: extra space

Quoted 15 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +15 -0
+       if (!ASSERT_OK(err, " kprobe_multi_session__attach"))
+               goto cleanup;
+
+       prog_fd = bpf_program__fd(skel->progs.trigger);
+       err = bpf_prog_test_run_opts(prog_fd, &topts);
+       ASSERT_OK(err, "test_run");
+       ASSERT_EQ(topts.retval, 0, "test_run");
+
+       ASSERT_EQ(skel->bss->kprobe_test1_result, 1, "kprobe_test1_result");
+       ASSERT_EQ(skel->bss->kprobe_test2_result, 1, "kprobe_test2_result");
+       ASSERT_EQ(skel->bss->kprobe_test3_result, 1, "kprobe_test3_result");
+       ASSERT_EQ(skel->bss->kprobe_test4_result, 1, "kprobe_test4_result");
+       ASSERT_EQ(skel->bss->kprobe_test5_result, 1, "kprobe_test5_result");
+       ASSERT_EQ(skel->bss->kprobe_test6_result, 1, "kprobe_test6_result");
+       ASSERT_EQ(skel->bss->kprobe_test7_result, 1, "kprobe_test7_result");

tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 369

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +10 -0
+       ASSERT_EQ(skel->bss->kprobe_test8_result, 1, "kprobe_test8_result");
+
+       ASSERT_EQ(skel->bss->kretprobe_test1_result, 0, "kretprobe_test1_result");
+       ASSERT_EQ(skel->bss->kretprobe_test2_result, 1, "kretprobe_test2_result");
+       ASSERT_EQ(skel->bss->kretprobe_test3_result, 0, "kretprobe_test3_result");
+       ASSERT_EQ(skel->bss->kretprobe_test4_result, 1, "kretprobe_test4_result");
+       ASSERT_EQ(skel->bss->kretprobe_test5_result, 0, "kretprobe_test5_result");
+       ASSERT_EQ(skel->bss->kretprobe_test6_result, 1, "kretprobe_test6_result");
+       ASSERT_EQ(skel->bss->kretprobe_test7_result, 0, "kretprobe_test7_result");
+       ASSERT_EQ(skel->bss->kretprobe_test8_result, 1, "kretprobe_test8_result");

see below, even if array of ksym ptrs idea doesn't work out, at least results can be an array (which is cleaner to work with both on BPF and user space sides)

Quoted 83 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +8 -0
+
+cleanup:
+       bpf_link__destroy(link);
+       kprobe_multi_session__destroy(skel);
+}
+
 static size_t symbol_hash(long key, void *ctx __maybe_unused)
 {
        return str_hash((const char *) key);
@@ -690,4 +737,6 @@ void test_kprobe_multi_test(void)
                test_attach_api_fails();
        if (test__start_subtest("attach_override"))
                test_attach_override();
+       if (test__start_subtest("session"))
+               test_session_skel_api();
 }
tools/testing/selftests/bpf/progs/kprobe_multi_session.c new file +61 -0
new file mode 100644
index 000000000000..9fb87f7f69da
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <stdbool.h>
+#include "bpf_kfuncs.h"
+
+char _license[] SEC("license") = "GPL";
+
+extern const void bpf_fentry_test1 __ksym;
+extern const void bpf_fentry_test2 __ksym;
+extern const void bpf_fentry_test3 __ksym;
+extern const void bpf_fentry_test4 __ksym;
+extern const void bpf_fentry_test5 __ksym;
+extern const void bpf_fentry_test6 __ksym;
+extern const void bpf_fentry_test7 __ksym;
+extern const void bpf_fentry_test8 __ksym;
+
+int pid = 0;
+
+__u64 kprobe_test1_result = 0;
+__u64 kprobe_test2_result = 0;
+__u64 kprobe_test3_result = 0;
+__u64 kprobe_test4_result = 0;
+__u64 kprobe_test5_result = 0;
+__u64 kprobe_test6_result = 0;
+__u64 kprobe_test7_result = 0;
+__u64 kprobe_test8_result = 0;
+
+__u64 kretprobe_test1_result = 0;
+__u64 kretprobe_test2_result = 0;
+__u64 kretprobe_test3_result = 0;
+__u64 kretprobe_test4_result = 0;
+__u64 kretprobe_test5_result = 0;
+__u64 kretprobe_test6_result = 0;
+__u64 kretprobe_test7_result = 0;
+__u64 kretprobe_test8_result = 0;
+
+static int session_check(void *ctx, bool is_return)
+{
+       if (bpf_get_current_pid_tgid() >> 32 != pid)
+               return 1;
+
+       __u64 addr = bpf_get_func_ip(ctx);
+
+#define SET(__var, __addr) ({                  \
+       if ((const void *) addr == __addr)      \
+               __var = 1;                      \
+})
+
+       if (is_return) {
+               SET(kretprobe_test1_result, &bpf_fentry_test1);
+               SET(kretprobe_test2_result, &bpf_fentry_test2);
+               SET(kretprobe_test3_result, &bpf_fentry_test3);
+               SET(kretprobe_test4_result, &bpf_fentry_test4);
+               SET(kretprobe_test5_result, &bpf_fentry_test5);
+               SET(kretprobe_test6_result, &bpf_fentry_test6);
+               SET(kretprobe_test7_result, &bpf_fentry_test7);
+               SET(kretprobe_test8_result, &bpf_fentry_test8);
+       } else {
+               SET(kprobe_test1_result, &bpf_fentry_test1);

tools/testing/selftests/bpf/progs/kprobe_multi_session.c line 71

Quoted 10 lines
tools/testing/selftests/bpf/progs/kprobe_multi_session.c new file +10 -0
+               SET(kprobe_test2_result, &bpf_fentry_test2);
+               SET(kprobe_test3_result, &bpf_fentry_test3);
+               SET(kprobe_test4_result, &bpf_fentry_test4);
+               SET(kprobe_test5_result, &bpf_fentry_test5);
+               SET(kprobe_test6_result, &bpf_fentry_test6);
+               SET(kprobe_test7_result, &bpf_fentry_test7);
+               SET(kprobe_test8_result, &bpf_fentry_test8);
+       }
+
+#undef SET

curious, have you tried implementing this through a proper for loop? I wonder if something like

void *kfuncs[] = { &bpf_fentry_test1, ..., &bpf_fentry_test8 };

and then generic loop over this array would work. Can you please try?

Quoted 32 lines
tools/testing/selftests/bpf/progs/kprobe_multi_session.c new file +29 -1
+
+       /*
+        * Force probes for function bpf_fentry_test[1357] not to
+        * install and execute the return probe
+        */
+       if (((const void *) addr == &bpf_fentry_test1) ||
+           ((const void *) addr == &bpf_fentry_test3) ||
+           ((const void *) addr == &bpf_fentry_test5) ||
+           ((const void *) addr == &bpf_fentry_test7))
+               return 1;
+
+       return 0;
+}
+
+/*
+ * No tests in here, just to trigger 'bpf_fentry_test*'
+ * through tracing test_run
+ */
+SEC("fentry/bpf_modify_return_test")
+int BPF_PROG(trigger)
+{
+       return 0;
+}
+
+SEC("kprobe.session/bpf_fentry_test*")
+int test_kprobe(struct pt_regs *ctx)
+{
+       return session_check(ctx, bpf_session_is_return());
+}
--

2.44.0

#20 lore raw

Re: [PATCH bpf-next 6/7] selftests/bpf: Add kprobe multi session test

Quoted 1 line · Andrii Nakryiko

On Mon, Apr 22, 2024 at 5:13 AM Jiri Olsa <[email protected]> wrote:

SNIP

Quoted 16 lines
Quoted 16 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +1 -0
index 51628455b6f5..d1f116665551 100644
@@ -4,6 +4,7 @@
 #include "trace_helpers.h"
 #include "kprobe_multi_empty.skel.h"
 #include "kprobe_multi_override.skel.h"
+#include "kprobe_multi_session.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "bpf/hashmap.h"
 
@@ -326,6 +327,52 @@ static void test_attach_api_fails(void)
        kprobe_multi__destroy(skel);
 }
 
Quoted 13 lines

tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 340

Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +11 -0
+static void test_session_skel_api(void)
+{
+       struct kprobe_multi_session *skel = NULL;
+       LIBBPF_OPTS(bpf_kprobe_multi_opts, opts);
+       LIBBPF_OPTS(bpf_test_run_opts, topts);
+       struct bpf_link *link = NULL;
+       int err, prog_fd;
+
+       skel = kprobe_multi_session__open_and_load();
+       if (!ASSERT_OK_PTR(skel, "kprobe_multi_session__open_and_load"))
+               goto cleanup;

return?

ok

Quoted 29 lines

tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 344

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +4 -0
+
+       skel->bss->pid = getpid();
+
+       err =  kprobe_multi_session__attach(skel);

nit: extra space

Quoted 21 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +21 -0
+       if (!ASSERT_OK(err, " kprobe_multi_session__attach"))
+               goto cleanup;
+
+       prog_fd = bpf_program__fd(skel->progs.trigger);
+       err = bpf_prog_test_run_opts(prog_fd, &topts);
+       ASSERT_OK(err, "test_run");
+       ASSERT_EQ(topts.retval, 0, "test_run");
+
+       ASSERT_EQ(skel->bss->kprobe_test1_result, 1, "kprobe_test1_result");
+       ASSERT_EQ(skel->bss->kprobe_test2_result, 1, "kprobe_test2_result");
+       ASSERT_EQ(skel->bss->kprobe_test3_result, 1, "kprobe_test3_result");
+       ASSERT_EQ(skel->bss->kprobe_test4_result, 1, "kprobe_test4_result");
+       ASSERT_EQ(skel->bss->kprobe_test5_result, 1, "kprobe_test5_result");
+       ASSERT_EQ(skel->bss->kprobe_test6_result, 1, "kprobe_test6_result");
+       ASSERT_EQ(skel->bss->kprobe_test7_result, 1, "kprobe_test7_result");
+       ASSERT_EQ(skel->bss->kprobe_test8_result, 1, "kprobe_test8_result");
+
+       ASSERT_EQ(skel->bss->kretprobe_test1_result, 0, "kretprobe_test1_result");
+       ASSERT_EQ(skel->bss->kretprobe_test2_result, 1, "kretprobe_test2_result");
+       ASSERT_EQ(skel->bss->kretprobe_test3_result, 0, "kretprobe_test3_result");
+       ASSERT_EQ(skel->bss->kretprobe_test4_result, 1, "kretprobe_test4_result");
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 369

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +4 -0
+       ASSERT_EQ(skel->bss->kretprobe_test5_result, 0, "kretprobe_test5_result");
+       ASSERT_EQ(skel->bss->kretprobe_test6_result, 1, "kretprobe_test6_result");
+       ASSERT_EQ(skel->bss->kretprobe_test7_result, 0, "kretprobe_test7_result");
+       ASSERT_EQ(skel->bss->kretprobe_test8_result, 1, "kretprobe_test8_result");

see below, even if array of ksym ptrs idea doesn't work out, at least results can be an array (which is cleaner to work with both on BPF and user space sides)

I recall in past we used to do that and we switched to specific values to be more explicit I guess.. but it might make sense in here, will try it

SNIP

Quoted 37 lines
Quoted 23 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +23 -0
+static int session_check(void *ctx, bool is_return)
+{
+       if (bpf_get_current_pid_tgid() >> 32 != pid)
+               return 1;
+
+       __u64 addr = bpf_get_func_ip(ctx);
+
+#define SET(__var, __addr) ({                  \
+       if ((const void *) addr == __addr)      \
+               __var = 1;                      \
+})
+
+       if (is_return) {
+               SET(kretprobe_test1_result, &bpf_fentry_test1);
+               SET(kretprobe_test2_result, &bpf_fentry_test2);
+               SET(kretprobe_test3_result, &bpf_fentry_test3);
+               SET(kretprobe_test4_result, &bpf_fentry_test4);
+               SET(kretprobe_test5_result, &bpf_fentry_test5);
+               SET(kretprobe_test6_result, &bpf_fentry_test6);
+               SET(kretprobe_test7_result, &bpf_fentry_test7);
+               SET(kretprobe_test8_result, &bpf_fentry_test8);
+       } else {
+               SET(kprobe_test1_result, &bpf_fentry_test1);

tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void)

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +10 -0
+               SET(kprobe_test2_result, &bpf_fentry_test2);
+               SET(kprobe_test3_result, &bpf_fentry_test3);
+               SET(kprobe_test4_result, &bpf_fentry_test4);
+               SET(kprobe_test5_result, &bpf_fentry_test5);
+               SET(kprobe_test6_result, &bpf_fentry_test6);
+               SET(kprobe_test7_result, &bpf_fentry_test7);
+               SET(kprobe_test8_result, &bpf_fentry_test8);
+       }
+
+#undef SET

curious, have you tried implementing this through a proper for loop? I wonder if something like

Quoted 3 lines

void *kfuncs[] = { &bpf_fentry_test1, ..., &bpf_fentry_test8 };

and then generic loop over this array would work. Can you please try?

yep, will try, let's see if it gets nicer

jirka

#21 lore raw

Re: [PATCH bpf-next 6/7] selftests/bpf: Add kprobe multi session test

On Wed, Apr 24, 2024 at 01:44:50PM +0200, Jiri Olsa wrote:

SNIP

Quoted 46 lines
Quoted 3 lines

see below, even if array of ksym ptrs idea doesn't work out, at least results can be an array (which is cleaner to work with both on BPF and user space sides)

I recall in past we used to do that and we switched to specific values to be more explicit I guess.. but it might make sense in here, will try it

SNIP

Quoted 37 lines
Quoted 23 lines
(quoted diff) +23 -0
+static int session_check(void *ctx, bool is_return)
+{
+       if (bpf_get_current_pid_tgid() >> 32 != pid)
+               return 1;
+
+       __u64 addr = bpf_get_func_ip(ctx);
+
+#define SET(__var, __addr) ({                  \
+       if ((const void *) addr == __addr)      \
+               __var = 1;                      \
+})
+
+       if (is_return) {
+               SET(kretprobe_test1_result, &bpf_fentry_test1);
+               SET(kretprobe_test2_result, &bpf_fentry_test2);
+               SET(kretprobe_test3_result, &bpf_fentry_test3);
+               SET(kretprobe_test4_result, &bpf_fentry_test4);
+               SET(kretprobe_test5_result, &bpf_fentry_test5);
+               SET(kretprobe_test6_result, &bpf_fentry_test6);
+               SET(kretprobe_test7_result, &bpf_fentry_test7);
+               SET(kretprobe_test8_result, &bpf_fentry_test8);
+       } else {
+               SET(kprobe_test1_result, &bpf_fentry_test1);

Review

Quoted 10 lines
(quoted diff) +10 -0
+               SET(kprobe_test2_result, &bpf_fentry_test2);
+               SET(kprobe_test3_result, &bpf_fentry_test3);
+               SET(kprobe_test4_result, &bpf_fentry_test4);
+               SET(kprobe_test5_result, &bpf_fentry_test5);
+               SET(kprobe_test6_result, &bpf_fentry_test6);
+               SET(kprobe_test7_result, &bpf_fentry_test7);
+               SET(kprobe_test8_result, &bpf_fentry_test8);
+       }
+
+#undef SET

curious, have you tried implementing this through a proper for loop? I wonder if something like

Quoted 5 lines
Quoted 3 lines

void *kfuncs[] = { &bpf_fentry_test1, ..., &bpf_fentry_test8 };

and then generic loop over this array would work. Can you please try?

yep, will try, let's see if it gets nicer

ok it looks better, I'll send new version

thanks,
jirka

---

tools/testing/selftests/bpf/bpf_kfuncs.h +2 -0
index 14ebe7d9e1a3..180030b5d828 100644
@@ -75,4 +75,6 @@ extern void bpf_key_put(struct bpf_key *key) __ksym;
 extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr,
 				      struct bpf_dynptr *sig_ptr,
 				      struct bpf_key *trusted_keyring) __ksym;
+
+extern bool bpf_session_is_return(void) __ksym;
 #endif
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +43 -0

59 lines not on this page · raw message

tools/testing/selftests/bpf/progs/kprobe_multi_session.c new file +78 -0

78 lines not on this page · raw message

author reviewer
#22 lore raw

PATCH 7/7 selftests/bpf: Add kprobe multi wrapper cookie test

In reply to Jiri Olsa

Adding kprobe multi session test that verifies the cookie value get properly propagated from entry to return program.

---
 tools/testing/selftests/bpf/bpf_kfuncs.h      |  1 +
 .../bpf/prog_tests/kprobe_multi_test.c        | 35 ++++++++++++
 .../bpf/progs/kprobe_multi_session_cookie.c   | 56 +++++++++++++++++++
 3 files changed, 92 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c
tools/testing/selftests/bpf/bpf_kfuncs.h +1 -0
index 180030b5d828..0281921cd654 100644
@@ -77,4 +77,5 @@ extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr,
 				      struct bpf_key *trusted_keyring) __ksym;
 
 extern bool bpf_session_is_return(void) __ksym;
+extern __u64 *bpf_session_cookie(void) __ksym;
 #endif
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +35 -0
index d1f116665551..2896467ca3cd 100644
@@ -5,6 +5,7 @@
 #include "kprobe_multi_empty.skel.h"
 #include "kprobe_multi_override.skel.h"
 #include "kprobe_multi_session.skel.h"
+#include "kprobe_multi_session_cookie.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "bpf/hashmap.h"
 
@@ -373,6 +374,38 @@ static void test_session_skel_api(void)
 	kprobe_multi_session__destroy(skel);
 }
 
+static void test_session_cookie_skel_api(void)
+{
+	struct kprobe_multi_session_cookie *skel = NULL;
+	LIBBPF_OPTS(bpf_kprobe_multi_opts, opts);
+	LIBBPF_OPTS(bpf_test_run_opts, topts);
+	struct bpf_link *link = NULL;
+	int err, prog_fd;
+
+	skel = kprobe_multi_session_cookie__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "fentry_raw_skel_load"))
+		goto cleanup;
+
+	skel->bss->pid = getpid();
+
+	err = kprobe_multi_session_cookie__attach(skel);
+	if (!ASSERT_OK(err, " kprobe_multi_wrapper__attach"))
+		goto cleanup;
+
+	prog_fd = bpf_program__fd(skel->progs.trigger);
+	err = bpf_prog_test_run_opts(prog_fd, &topts);
+	ASSERT_OK(err, "test_run");
+	ASSERT_EQ(topts.retval, 0, "test_run");
+
+	ASSERT_EQ(skel->bss->test_kprobe_1_result, 1, "test_kprobe_1_result");
+	ASSERT_EQ(skel->bss->test_kprobe_2_result, 2, "test_kprobe_2_result");
+	ASSERT_EQ(skel->bss->test_kprobe_3_result, 3, "test_kprobe_3_result");
+
+cleanup:
+	bpf_link__destroy(link);
+	kprobe_multi_session_cookie__destroy(skel);
+}
+
 static size_t symbol_hash(long key, void *ctx __maybe_unused)
 {
 	return str_hash((const char *) key);
@@ -739,4 +772,6 @@ void test_kprobe_multi_test(void)
 		test_attach_override();
 	if (test__start_subtest("session"))
 		test_session_skel_api();
+	if (test__start_subtest("session_cookie"))
+		test_session_cookie_skel_api();
 }
tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c new file +56 -0

56 lines not on this page · raw message

Signature · 1 line
2.44.0
Acked-by on v1 7/7
#23 lore raw

Re: [PATCH bpf-next 7/7] selftests/bpf: Add kprobe multi wrapper cookie test

Quoted 6 lines · Jiri Olsa

Adding kprobe multi session test that verifies the cookie value get properly propagated from entry to return program.

---

Forgot to update subject (still using "wrapper" naming)

overall LGTM, see nits

Quoted 11 lines
 tools/testing/selftests/bpf/bpf_kfuncs.h      |  1 +
 .../bpf/prog_tests/kprobe_multi_test.c        | 35 ++++++++++++
 .../bpf/progs/kprobe_multi_session_cookie.c   | 56 +++++++++++++++++++
 3 files changed, 92 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c
tools/testing/selftests/bpf/bpf_kfuncs.h +0 -0
index 180030b5d828..0281921cd654 100644

tools/testing/selftests/bpf/bpf_kfuncs.h extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr, · line 80

Quoted 4 lines
tools/testing/selftests/bpf/bpf_kfuncs.h +1 -0
@@ -77,4 +77,5 @@ extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr,
                                      struct bpf_key *trusted_keyring) __ksym;
 
 extern bool bpf_session_is_return(void) __ksym;
+extern __u64 *bpf_session_cookie(void) __ksym;

btw, should we use long * as return type to avoid relying on having __u64 alias be available? Long is always an 8-byte value in the BPF world, it should be fine.

Quoted 86 lines
tools/testing/selftests/bpf/bpf_kfuncs.h +0 -0
 #endif
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +35 -0
index d1f116665551..2896467ca3cd 100644
@@ -5,6 +5,7 @@
 #include "kprobe_multi_empty.skel.h"
 #include "kprobe_multi_override.skel.h"
 #include "kprobe_multi_session.skel.h"
+#include "kprobe_multi_session_cookie.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "bpf/hashmap.h"
 
@@ -373,6 +374,38 @@ static void test_session_skel_api(void)
        kprobe_multi_session__destroy(skel);
 }
 
+static void test_session_cookie_skel_api(void)
+{
+       struct kprobe_multi_session_cookie *skel = NULL;
+       LIBBPF_OPTS(bpf_kprobe_multi_opts, opts);
+       LIBBPF_OPTS(bpf_test_run_opts, topts);
+       struct bpf_link *link = NULL;
+       int err, prog_fd;
+
+       skel = kprobe_multi_session_cookie__open_and_load();
+       if (!ASSERT_OK_PTR(skel, "fentry_raw_skel_load"))
+               goto cleanup;
+
+       skel->bss->pid = getpid();
+
+       err = kprobe_multi_session_cookie__attach(skel);
+       if (!ASSERT_OK(err, " kprobe_multi_wrapper__attach"))
+               goto cleanup;
+
+       prog_fd = bpf_program__fd(skel->progs.trigger);
+       err = bpf_prog_test_run_opts(prog_fd, &topts);
+       ASSERT_OK(err, "test_run");
+       ASSERT_EQ(topts.retval, 0, "test_run");
+
+       ASSERT_EQ(skel->bss->test_kprobe_1_result, 1, "test_kprobe_1_result");
+       ASSERT_EQ(skel->bss->test_kprobe_2_result, 2, "test_kprobe_2_result");
+       ASSERT_EQ(skel->bss->test_kprobe_3_result, 3, "test_kprobe_3_result");
+
+cleanup:
+       bpf_link__destroy(link);
+       kprobe_multi_session_cookie__destroy(skel);
+}
+
 static size_t symbol_hash(long key, void *ctx __maybe_unused)
 {
        return str_hash((const char *) key);
@@ -739,4 +772,6 @@ void test_kprobe_multi_test(void)
                test_attach_override();
        if (test__start_subtest("session"))
                test_session_skel_api();
+       if (test__start_subtest("session_cookie"))
+               test_session_cookie_skel_api();
 }
tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c new file +21 -0
new file mode 100644
index 000000000000..b5c04b7b180c
@@ -0,0 +1,56 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <stdbool.h>
+#include "bpf_kfuncs.h"
+
+char _license[] SEC("license") = "GPL";
+
+int pid = 0;
+
+__u64 test_kprobe_1_result = 0;
+__u64 test_kprobe_2_result = 0;
+__u64 test_kprobe_3_result = 0;
+
+/*
+ * No tests in here, just to trigger 'bpf_fentry_test*'
+ * through tracing test_run
+ */
+SEC("fentry/bpf_modify_return_test")
+int BPF_PROG(trigger)

tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c line 31

Quoted 10 lines
tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c new file +10 -0
+{
+       return 0;
+}
+
+static int check_cookie(__u64 val, __u64 *result)
+{
+       if (bpf_get_current_pid_tgid() >> 32 != pid)
+               return 1;
+
+       __u64 *cookie = bpf_session_cookie();

we don't enforce this, but let's stick to C89 variable declaration style (or rather positioning in this case)?

Quoted 28 lines
tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c new file +25 -1
+
+       if (bpf_session_is_return())
+               *result = *cookie == val ? val : 0;
+       else
+               *cookie = val;
+       return 0;
+}
+
+SEC("kprobe.session/bpf_fentry_test1")
+int test_kprobe_1(struct pt_regs *ctx)
+{
+       return check_cookie(1, &test_kprobe_1_result);
+}
+
+SEC("kprobe.session/bpf_fentry_test1")
+int test_kprobe_2(struct pt_regs *ctx)
+{
+       return check_cookie(2, &test_kprobe_2_result);
+}
+
+SEC("kprobe.session/bpf_fentry_test1")
+int test_kprobe_3(struct pt_regs *ctx)
+{
+       return check_cookie(3, &test_kprobe_3_result);
+}
--

2.44.0

#24 lore raw

Re: [PATCH bpf-next 7/7] selftests/bpf: Add kprobe multi wrapper cookie test

Quoted 9 lines · Andrii Nakryiko
Quoted 6 lines · Jiri Olsa

Adding kprobe multi session test that verifies the cookie value get properly propagated from entry to return program.

---

Forgot to update subject (still using "wrapper" naming)

ugh, thnx

Quoted 18 lines

overall LGTM, see nits

Quoted 13 lines
 tools/testing/selftests/bpf/bpf_kfuncs.h      |  1 +
 .../bpf/prog_tests/kprobe_multi_test.c        | 35 ++++++++++++
 .../bpf/progs/kprobe_multi_session_cookie.c   | 56 +++++++++++++++++++
 3 files changed, 92 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c
tools/testing/selftests/bpf/bpf_kfuncs.h +0 -0
index 180030b5d828..0281921cd654 100644
@@ -77,4 +77,5 @@ extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr,
                                      struct bpf_key *trusted_keyring) __ksym;
 
Quoted 6 lines

tools/testing/selftests/bpf/bpf_kfuncs.h extern int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_ptr, · line 80

Quoted 2 lines
tools/testing/selftests/bpf/bpf_kfuncs.h +1 -0
 extern bool bpf_session_is_return(void) __ksym;
+extern __u64 *bpf_session_cookie(void) __ksym;

btw, should we use long * as return type to avoid relying on having __u64 alias be available? Long is always an 8-byte value in the BPF world, it should be fine.

ok, there are some __u64 in kfuncs already, but let's stay on safe side

SNIP

Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/bpf/bpf_kfuncs.h +11 -0
+/*
+ * No tests in here, just to trigger 'bpf_fentry_test*'
+ * through tracing test_run
+ */
+SEC("fentry/bpf_modify_return_test")
+int BPF_PROG(trigger)
+{
+       return 0;
+}
+
+static int check_cookie(__u64 val, __u64 *result)
Quoted 8 lines

tools/testing/selftests/bpf/bpf_kfuncs.h

Quoted 5 lines
tools/testing/selftests/bpf/bpf_kfuncs.h +5 -0
+{
+       if (bpf_get_current_pid_tgid() >> 32 != pid)
+               return 1;
+
+       __u64 *cookie = bpf_session_cookie();

we don't enforce this, but let's stick to C89 variable declaration style (or rather positioning in this case)?

ok, will change that

jirka

#25 lore raw
In reply to Jiri Olsa
Quoted 35 lines · Jiri Olsa

hi,
adding support to attach kprobe program through kprobe_multi link in a session mode, which means:
- program is attached to both function entry and return
- entry program can decided if the return program gets executed
- entry program can share u64 cookie value with return program

The initial RFC for this was posted in [0] and later discussed more and which ended up with the session idea [1]

Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.

At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.

In addition the possibility to control the return program execution and sharing data within entry and return probe allows for other use cases.

Changes from last RFC version [1]:
- changed wrapper name to session
- changed flag to adding new attach type for session:
BPF_TRACE_KPROBE_MULTI_SESSION
it's more convenient wrt filtering on kfuncs setup and seems to make more sense alltogether
- renamed bpf_kprobe_multi_is_return to bpf_session_is_return
- added bpf_session_cookie kfunc, which actually already works on current fprobe implementation (not just fprobe-on-fgraph) and it provides the shared data between entry/return probes [Andrii]

Quoted 3 lines

we could actually make the cookie size configurable.. thoughts? (it's 8 bytes atm)

Attach cookie is fixed at 8 bytes and that works pretty well. I think beyond 8 bytes there is no clearly "right" size. A common case would be to capture arguments in kprobe to handle in kretprobe, and there you might need at least 40+ bytes, which seems wasteful. So I want to say that it's probably good to hard-code it to just 8 bytes (enough to store timestamp and you can even fit in some flags if you reduce timestamp precision from nanoseconds to microseconds), or use it as an index into array or some other data structure.

let's keep it simple?

Quoted 40 lines

- better attach setup conditions changes [Andrii]
- I'm not including uprobes change atm, because it needs extra uprobe change so I'll post it separately

Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
bpf/session_data

thanks,
jirka

[0] https://lore.kernel.org/bpf/[email protected]/
[1] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (7):
bpf: Add support for kprobe multi session attach
bpf: Add support for kprobe multi session context
bpf: Add support for kprobe multi session cookie
libbpf: Add support for kprobe multi session attach
libbpf: Add kprobe session attach type name to attach_type_name selftests/bpf: Add kprobe multi session test selftests/bpf: Add kprobe multi wrapper cookie test

 include/uapi/linux/bpf.h                                        |   1 +
 kernel/bpf/btf.c                                                |   3 +++
 kernel/bpf/syscall.c                                            |   7 +++++-
 kernel/bpf/verifier.c                                           |   7 ++++++
 kernel/trace/bpf_trace.c                                        | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------
 tools/include/uapi/linux/bpf.h                                  |   1 +
 tools/lib/bpf/bpf.c                                             |   1 +
 tools/lib/bpf/libbpf.c                                          |  41 ++++++++++++++++++++++++++++++++--
 tools/lib/bpf/libbpf.h                                          |   4 +++-
 tools/testing/selftests/bpf/bpf_kfuncs.h                        |   3 +++
 tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c      |  84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/kprobe_multi_session.c        | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c |  56 ++++++++++++++++++++++++++++++++++++++++++++++
 13 files changed, 396 insertions(+), 18 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session.c
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_session_cookie.c
#26 lore raw
Quoted 49 lines · Andrii Nakryiko
Quoted 35 lines · Jiri Olsa

hi,
adding support to attach kprobe program through kprobe_multi link in a session mode, which means:
- program is attached to both function entry and return
- entry program can decided if the return program gets executed
- entry program can share u64 cookie value with return program

The initial RFC for this was posted in [0] and later discussed more and which ended up with the session idea [1]

Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.

At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.

In addition the possibility to control the return program execution and sharing data within entry and return probe allows for other use cases.

Changes from last RFC version [1]:
- changed wrapper name to session
- changed flag to adding new attach type for session:
BPF_TRACE_KPROBE_MULTI_SESSION
it's more convenient wrt filtering on kfuncs setup and seems to make more sense alltogether
- renamed bpf_kprobe_multi_is_return to bpf_session_is_return
- added bpf_session_cookie kfunc, which actually already works on current fprobe implementation (not just fprobe-on-fgraph) and it provides the shared data between entry/return probes [Andrii]

Quoted 3 lines

we could actually make the cookie size configurable.. thoughts? (it's 8 bytes atm)

Attach cookie is fixed at 8 bytes and that works pretty well. I think beyond 8 bytes there is no clearly "right" size. A common case would be to capture arguments in kprobe to handle in kretprobe, and there you might need at least 40+ bytes, which seems wasteful. So I want to say that it's probably good to hard-code it to just 8 bytes (enough to store timestamp and you can even fit in some flags if you reduce timestamp precision from nanoseconds to microseconds), or use it as an index into array or some other data structure.

Quoted 1 line

let's keep it simple?

+1 for keeping it simple. Use it as a key if you need more.

#27 lore raw
Quoted 50 lines · John Fastabend
Quoted 49 lines · Andrii Nakryiko
Quoted 35 lines · Jiri Olsa

hi,
adding support to attach kprobe program through kprobe_multi link in a session mode, which means:
- program is attached to both function entry and return
- entry program can decided if the return program gets executed
- entry program can share u64 cookie value with return program

The initial RFC for this was posted in [0] and later discussed more and which ended up with the session idea [1]

Having entry together with return probe for given function is common use case for tetragon, bpftrace and most likely for others.

At the moment if we want both entry and return probe to execute bpf program we need to create two (entry and return probe) links. The link for return probe creates extra entry probe to setup the return probe. The extra entry probe execution could be omitted if we had a way to use just single link for both entry and exit probe.

In addition the possibility to control the return program execution and sharing data within entry and return probe allows for other use cases.

Changes from last RFC version [1]:
- changed wrapper name to session
- changed flag to adding new attach type for session:
BPF_TRACE_KPROBE_MULTI_SESSION
it's more convenient wrt filtering on kfuncs setup and seems to make more sense alltogether
- renamed bpf_kprobe_multi_is_return to bpf_session_is_return
- added bpf_session_cookie kfunc, which actually already works on current fprobe implementation (not just fprobe-on-fgraph) and it provides the shared data between entry/return probes [Andrii]

Quoted 3 lines

we could actually make the cookie size configurable.. thoughts? (it's 8 bytes atm)

Attach cookie is fixed at 8 bytes and that works pretty well. I think beyond 8 bytes there is no clearly "right" size. A common case would be to capture arguments in kprobe to handle in kretprobe, and there you might need at least 40+ bytes, which seems wasteful. So I want to say that it's probably good to hard-code it to just 8 bytes (enough to store timestamp and you can even fit in some flags if you reduce timestamp precision from nanoseconds to microseconds), or use it as an index into array or some other data structure.

Quoted 3 lines
Quoted 1 line

let's keep it simple?

+1 for keeping it simple. Use it as a key if you need more.

ok, it's true then it'd be tricky wrt verifier being able to check the proper size and it does work with attach cookie so far

jirka