Skip to contentSkip to conversation

uprobe, bpf: Add session support

PATCH Superseded Jiri Olsa 4 participants 24 messages Sep 17, 2024 – Sep 23, 2024

Subsystems (11)
Kselftest BPF perf events Tracing +7 more subsystems

v4 of 9 Superseded 1 of 14 applied Latest is v9 →

lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

v4 cover uprobe, bpf: Add session support

hi,
this patchset is adding support for session uprobe attachment and using it through bpf link for bpf programs.

The session means that the uprobe consumer is executed on entry and return of probed function with additional control:
- entry callback can control execution of the return callback
- entry and return callbacks can share data/cookie

On more details please see patch #2.

It's based on Peter's perf/core:
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git perf/core

v4 changes:
- rework the uprobe consumer allocation based on Oleg's suggestions [Oleg]
- moved docs about handler return values to uprobes.h header [Oleg]
- use libbpf's attach_uprobe_multi for session attach [Andrii]
- make verifier to check session return values [Andrii]
- various small review changes
- added more tests
- added acks

patch #6 is already in bpf-next tree, but we need that as dependency

thanks,
jirka

---
Jiri Olsa (14):
uprobe: Add data pointer to consumer handlers
uprobe: Add support for session consumer
bpf: Add support for uprobe multi session attach
bpf: Add support for uprobe multi session context
bpf: Allow return values 0 and 1 for uprobe/kprobe session
libbpf: Fix uretprobe.multi.s programs auto attachment
libbpf: Add support for uprobe multi session attach selftests/bpf: Add uprobe session test
selftests/bpf: Add uprobe session cookie test selftests/bpf: Add uprobe session recursive test selftests/bpf: Add uprobe session verifier test for return value selftests/bpf: Add kprobe session verifier test for return value selftests/bpf: Add uprobe session single consumer test selftests/bpf: Add consumers stress test on single uprobe

 include/linux/uprobes.h                                            |  29 +++++++++-
 include/uapi/linux/bpf.h                                           |   1 +
 kernel/bpf/syscall.c                                               |   9 ++-
 kernel/bpf/verifier.c                                              |  10 ++++
 kernel/events/uprobes.c                                            | 150 ++++++++++++++++++++++++++++++++++++++++----------
 kernel/trace/bpf_trace.c                                           |  66 ++++++++++++++++------
 kernel/trace/trace_uprobe.c                                        |  12 ++--
 tools/include/uapi/linux/bpf.h                                     |   1 +
 tools/lib/bpf/bpf.c                                                |   1 +
 tools/lib/bpf/libbpf.c                                             |  22 +++++++-
 tools/lib/bpf/libbpf.h                                             |   4 +-
 tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c              |   2 +-
 tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c         |   2 +
 tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c         | 252 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c          |  31 +++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_consumer_stress.c   |  29 ++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_session.c           |  71 ++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c    |  48 ++++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c |  44 +++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c    |  44 +++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c          |  31 +++++++++++
 21 files changed, 798 insertions(+), 61 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_consumer_stress.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c
author reviewer
#1 lore raw

PATCH v4 1/14 uprobe: Add data pointer to consumer handlers

Adding data pointer to both entry and exit consumer handlers and all its users. The functionality itself is coming in following change.

---
 include/linux/uprobes.h                              |  4 ++--
 kernel/trace/bpf_trace.c                             |  6 ++++--
 kernel/trace/trace_uprobe.c                          | 12 ++++++++----
 .../testing/selftests/bpf/bpf_testmod/bpf_testmod.c  |  2 +-
 4 files changed, 15 insertions(+), 9 deletions(-)
include/linux/uprobes.h +2 -2
index 2b294bf1881f..bb265a632b91 100644
@@ -37,10 +37,10 @@ struct uprobe_consumer {
 	 * for the current process. If filter() is omitted or returns true,
 	 * UPROBE_HANDLER_REMOVE is effectively ignored.
 	 */
-	int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
+	int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs, __u64 *data);
 	int (*ret_handler)(struct uprobe_consumer *self,
 				unsigned long func,
-				struct pt_regs *regs);
+				struct pt_regs *regs, __u64 *data);
 	bool (*filter)(struct uprobe_consumer *self, struct mm_struct *mm);
 
 	struct list_head cons_node;
kernel/trace/bpf_trace.c +4 -2
index ac0a01cc8634..de241503c8fb 100644
@@ -3332,7 +3332,8 @@ uprobe_multi_link_filter(struct uprobe_consumer *con, struct mm_struct *mm)
 }
 
 static int
-uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs)
+uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs,
+			  __u64 *data)
 {
 	struct bpf_uprobe *uprobe;
 
@@ -3341,7 +3342,8 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs)
 }
 
 static int
-uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs)
+uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, struct pt_regs *regs,
+			      __u64 *data)
 {
 	struct bpf_uprobe *uprobe;
 
kernel/trace/trace_uprobe.c +8 -4
index f7443e996b1b..11103dde897b 100644
@@ -88,9 +88,11 @@ static struct trace_uprobe *to_trace_uprobe(struct dyn_event *ev)
 static int register_uprobe_event(struct trace_uprobe *tu);
 static int unregister_uprobe_event(struct trace_uprobe *tu);
 
-static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs);
+static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs,
+			     __u64 *data);
 static int uretprobe_dispatcher(struct uprobe_consumer *con,
-				unsigned long func, struct pt_regs *regs);
+				unsigned long func, struct pt_regs *regs,
+				__u64 *data);
 
 #ifdef CONFIG_STACK_GROWSUP
 static unsigned long adjust_stack_addr(unsigned long addr, unsigned int n)
@@ -1500,7 +1502,8 @@ trace_uprobe_register(struct trace_event_call *event, enum trace_reg type,
 	}
 }
 
-static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs)
+static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs,
+			     __u64 *data)
 {
 	struct trace_uprobe *tu;
 	struct uprobe_dispatch_data udd;
@@ -1530,7 +1533,8 @@ static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs)
 }
 
 static int uretprobe_dispatcher(struct uprobe_consumer *con,
-				unsigned long func, struct pt_regs *regs)
+				unsigned long func, struct pt_regs *regs,
+				__u64 *data)
 {
 	struct trace_uprobe *tu;
 	struct uprobe_dispatch_data udd;
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +1 -1
index 1fc16657cf42..e91ff5b285f0 100644
@@ -421,7 +421,7 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = {
 
 static int
 uprobe_ret_handler(struct uprobe_consumer *self, unsigned long func,
-		   struct pt_regs *regs)
+		   struct pt_regs *regs, __u64 *data)
 
 {
 	regs->ax  = 0x12345678deadbeef;
Signature · 1 line
2.46.0
author reviewer
#2 lore raw

PATCH v4 2/14 uprobe: Add support for session consumer

In reply to Jiri Olsa

This change allows the uprobe consumer to behave as session which means that 'handler' and 'ret_handler' callbacks are connected in a way that allows to:

- control execution of 'ret_handler' from 'handler' callback
- share data between 'handler' and 'ret_handler' callbacks

The session concept fits to our common use case where we do filtering on entry uprobe and based on the result we decide to run the return uprobe (or not).

It's also convenient to share the data between session callbacks.

To achive this we are adding new return values the uprobe consumer can return from 'handler' callback:

UPROBE_HANDLER_IGNORE
- Ignore 'ret_handler' callback for this consumer.

UPROBE_HANDLER_IWANTMYCOOKIE
- Store cookie and pass it to 'ret_handler' (if defined).

We store shared data in the return_consumer object array as part of the return_instance object. This way the handle_uretprobe_chain can find related return_consumer and its shared data.

We also store entry handler return value, for cases when there are multiple consumers on single uprobe and some of them are ignored and some of them not, in which case the return probe gets installed and we need to have a way to find out which consumer needs to be ignored.

The tricky part is when consumer is registered 'after' the uprobe entry handler is hit. In such case this consumer's 'ret_handler' gets executed as well, but it won't have the proper data pointer set, so we can filter it out.

---
 include/linux/uprobes.h |  25 ++++++-
 kernel/events/uprobes.c | 150 ++++++++++++++++++++++++++++++++--------
 2 files changed, 144 insertions(+), 31 deletions(-)
include/linux/uprobes.h +24 -1
index bb265a632b91..751b1dd4abe9 100644
@@ -23,8 +23,20 @@ struct inode;
 struct notifier_block;
 struct page;
 
+/*
+ * Allowed return values from uprobe consumer's handler callback
+ * with following meaning:
+ *
+ * UPROBE_HANDLER_REMOVE
+ * - Remove the uprobe breakpoint from current->mm.
+ * UPROBE_HANDLER_IGNORE
+ * - Ignore ret_handler callback for this consumer.
+ * UPROBE_HANDLER_IWANTMYCOOKIE
+ * - Store cookie and pass it to ret_handler (if defined).
+ */
 #define UPROBE_HANDLER_REMOVE		1
-#define UPROBE_HANDLER_MASK		1
+#define UPROBE_HANDLER_IGNORE		2
+#define UPROBE_HANDLER_IWANTMYCOOKIE	3
 
 #define MAX_URETPROBE_DEPTH		64
 
@@ -44,6 +56,8 @@ struct uprobe_consumer {
 	bool (*filter)(struct uprobe_consumer *self, struct mm_struct *mm);
 
 	struct list_head cons_node;
+
+	__u64 id;	/* set when uprobe_consumer is registered */
 };
 
 #ifdef CONFIG_UPROBES
@@ -83,14 +97,23 @@ struct uprobe_task {
 	unsigned int			depth;
 };
 
+struct return_consumer {
+	__u64	cookie;
+	__u64	id;
+	int	rc;
+};
+
 struct return_instance {
 	struct uprobe		*uprobe;
 	unsigned long		func;
 	unsigned long		stack;		/* stack pointer */
 	unsigned long		orig_ret_vaddr; /* original return address */
 	bool			chained;	/* true, if instance is nested */
+	int			consumers_cnt;
 
 	struct return_instance	*next;		/* keep as stack */
+
+	struct return_consumer	consumers[] __counted_by(consumers_cnt);
 };
 
 enum rp_check {
kernel/events/uprobes.c +120 -30

256 lines not on this page · raw message

Signature · 1 line
2.46.0
maintainer
#3 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

I don't see anything wrong after a quick glance, but I don't really understand the UPROBE_HANDLER_IGNORE logic, see below.

Quoted 3 lines · Jiri Olsa

+ * UPROBE_HANDLER_IWANTMYCOOKIE
+ * - Store cookie and pass it to ret_handler (if defined).

Cough ;) yes it was me who used this name in the previous discussion, but maybe

UPROBE_HANDLER_COOKIE

will look a bit better? Feel free to ignore.

Quoted 1 line

static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs)

...

Quoted 5 lines
kernel/events/uprobes.c +5 -0
+		if (!uc->ret_handler || rc == UPROBE_HANDLER_REMOVE)
+			continue;
+
+		/*
+		 * If alloc_return_instance and push_consumer fail, the return probe

kernel/events/uprobes.c static struct uprobe *find_active_uprobe_rcu(unsigned long bp_vaddr, int *is_swb · line 2220

Quoted 10 lines
kernel/events/uprobes.c +10 -0
+		 * won't be prepared, but we'll finish to execute all entry handlers.
+		 *
+		 * We need to store handler's return value in case the return uprobe
+		 * gets installed and contains consumers that need to be ignored.
+		 */
+		if (!ri)
+			ri = alloc_return_instance();
+
+		if (rc == UPROBE_HANDLER_IWANTMYCOOKIE || rc == UPROBE_HANDLER_IGNORE)
+			ri = push_consumer(ri, push_idx++, uc->id, cookie, rc);

So this code allocates ri (which implies prepare_uretprobe!) and calls push_consumer() even if rc == UPROBE_HANDLER_IGNORE.

Why? The comment in uprobes.h says:

UPROBE_HANDLER_IGNORE
- Ignore ret_handler callback for this consumer

but the ret_handler callback won't be ignored?

To me this code should do:

		if (!uc->ret_handler || UPROBE_HANDLER_REMOVE || UPROBE_HANDLER_IGNORE)
			continue;

		if (!ri)
			ri = alloc_return_instance();

		if (rc == UPROBE_HANDLER_IWANTMYCOOKIE)
			ri = push_consumer(...);

And,

Quoted 1 line

handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs)

...

kernel/events/uprobes.c static void · line 2262

Quoted 9 lines
kernel/events/uprobes.c +4 -1
 	list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node,
 				 srcu_read_lock_held(&uprobes_srcu)) {
+		ric = return_consumer_find(ri, &ric_idx, uc->id);
+		if (ric && ric->rc == UPROBE_HANDLER_IGNORE)
+			continue;
 		if (uc->ret_handler)
-			uc->ret_handler(uc, ri->func, regs);
+			uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL);
 	}

the UPROBE_HANDLER_IGNORE check above and the new ric->rc member should die,

		if (!uc->ret_handler)
			continue;

		ric = return_consumer_find(...);
		uc->ret_handler(..., ric ? &ric->cookie : NULL);

as we have already discussed, the session ret_handler(data) can simply do

		// my ->handler() wasn't called or it didn't return
		// UPROBE_HANDLER_IWANTMYCOOKIE
		if (!data)
			return;

at the start.

Could you explain why this can't work?

Oleg.

#4 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

Quoted 5 lines · Oleg Nesterov

I don't see anything wrong after a quick glance, but I don't really understand the UPROBE_HANDLER_IGNORE logic, see below.

Quoted 1 line · Jiri Olsa
Quoted 8 lines
Quoted 2 lines

+ * UPROBE_HANDLER_IWANTMYCOOKIE
+ * - Store cookie and pass it to ret_handler (if defined).

Cough ;) yes it was me who used this name in the previous discussion, but maybe

UPROBE_HANDLER_COOKIE

will look a bit better? Feel free to ignore.

ok, no fun it is..

Quoted 64 lines
Quoted 2 lines

static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs)

...

Quoted 5 lines
kernel/events/uprobes.c +5 -0
+		if (!uc->ret_handler || rc == UPROBE_HANDLER_REMOVE)
+			continue;
+
+		/*
+		 * If alloc_return_instance and push_consumer fail, the return probe

kernel/events/uprobes.c static struct uprobe *find_active_uprobe_rcu(unsigned long bp_vaddr, int *is_swb · line 2220

Quoted 10 lines
kernel/events/uprobes.c +10 -0
+		 * won't be prepared, but we'll finish to execute all entry handlers.
+		 *
+		 * We need to store handler's return value in case the return uprobe
+		 * gets installed and contains consumers that need to be ignored.
+		 */
+		if (!ri)
+			ri = alloc_return_instance();
+
+		if (rc == UPROBE_HANDLER_IWANTMYCOOKIE || rc == UPROBE_HANDLER_IGNORE)
+			ri = push_consumer(ri, push_idx++, uc->id, cookie, rc);

So this code allocates ri (which implies prepare_uretprobe!) and calls push_consumer() even if rc == UPROBE_HANDLER_IGNORE.

Why? The comment in uprobes.h says:

UPROBE_HANDLER_IGNORE
- Ignore ret_handler callback for this consumer

but the ret_handler callback won't be ignored?

To me this code should do:

		if (!uc->ret_handler || UPROBE_HANDLER_REMOVE || UPROBE_HANDLER_IGNORE)
			continue;

		if (!ri)
			ri = alloc_return_instance();

		if (rc == UPROBE_HANDLER_IWANTMYCOOKIE)
			ri = push_consumer(...);

And,

Quoted 1 line

handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs)

...

kernel/events/uprobes.c static void · line 2262

Quoted 9 lines
kernel/events/uprobes.c +4 -1
 	list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node,
 				 srcu_read_lock_held(&uprobes_srcu)) {
+		ric = return_consumer_find(ri, &ric_idx, uc->id);
+		if (ric && ric->rc == UPROBE_HANDLER_IGNORE)
+			continue;
 		if (uc->ret_handler)
-			uc->ret_handler(uc, ri->func, regs);
+			uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL);
 	}

the UPROBE_HANDLER_IGNORE check above and the new ric->rc member should die,

		if (!uc->ret_handler)
			continue;

		ric = return_consumer_find(...);
		uc->ret_handler(..., ric ? &ric->cookie : NULL);

as we have already discussed, the session ret_handler(data) can simply do

Quoted 8 lines
		// my ->handler() wasn't called or it didn't return
		// UPROBE_HANDLER_IWANTMYCOOKIE
		if (!data)
			return;

at the start.

Could you explain why this can't work?

I'll try ;-) it's for the case when consumer does not use UPROBE_HANDLER_IWANTMYCOOKIE

let's have 2 consumers on single uprobe, consumer-A returning UPROBE_HANDLER_IGNORE and the consumer-B returning zero, so we want the return uprobe installed, but we want just consumer-B to be executed

- so uprobe gets installed and handle_uretprobe_chain goes over all consumers calling ret_handler callback

- but we don't know consumer-A needs to be ignored, and it does not expect cookie so we have no way to find out it needs to be ignored

the change solves this by storing also return value for consumer

if all consumers ignore the ret_handler callback return uprobe is not installed

jirka

maintainer
#5 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

Damn, sorry for delay :/

And sorry, still can't understand, see below...

Quoted 53 lines · Jiri Olsa
Quoted 36 lines · Oleg Nesterov

To me this code should do:

		if (!uc->ret_handler || UPROBE_HANDLER_REMOVE || UPROBE_HANDLER_IGNORE)
			continue;

		if (!ri)
			ri = alloc_return_instance();

		if (rc == UPROBE_HANDLER_IWANTMYCOOKIE)
			ri = push_consumer(...);

And,

Quoted 1 line

handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs)

...

kernel/events/uprobes.c static void · line 2262

Quoted 9 lines
kernel/events/uprobes.c +4 -1
 	list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node,
 				 srcu_read_lock_held(&uprobes_srcu)) {
+		ric = return_consumer_find(ri, &ric_idx, uc->id);
+		if (ric && ric->rc == UPROBE_HANDLER_IGNORE)
+			continue;
 		if (uc->ret_handler)
-			uc->ret_handler(uc, ri->func, regs);
+			uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL);
 	}

the UPROBE_HANDLER_IGNORE check above and the new ric->rc member should die,

		if (!uc->ret_handler)
			continue;

		ric = return_consumer_find(...);
		uc->ret_handler(..., ric ? &ric->cookie : NULL);

as we have already discussed, the session ret_handler(data) can simply do

Quoted 8 lines
		// my ->handler() wasn't called or it didn't return
		// UPROBE_HANDLER_IWANTMYCOOKIE
		if (!data)
			return;

at the start.

Could you explain why this can't work?

I'll try ;-) it's for the case when consumer does not use UPROBE_HANDLER_IWANTMYCOOKIE

let's have 2 consumers on single uprobe, consumer-A returning UPROBE_HANDLER_IGNORE and the consumer-B returning zero, so we want the return uprobe installed, but we want just consumer-B to be executed

Quoted 5 lines

- so uprobe gets installed and handle_uretprobe_chain goes over all consumers calling ret_handler callback

- but we don't know consumer-A needs to be ignored, and it does not expect cookie so we have no way to find out it needs to be ignored

How does this differ from the case when consumer-A returns _REMOVE but another consumer returns 0?

But what I really can't understand is

and it does not
expect cookie so we have no way to find out it needs to be ignored

If we change the code as I suggested above, push_consumer() won't be called if consumer-A returns UPROBE_HANDLER_IGNORE.

This means that handle_uretprobe_chain() -> return_consumer_find() will return NULL, so handle_uretprobe_chain() won't pass the valid cookie to consumer-A's ret_handler callback, it will pass data => NULL.

So, again, why can't consumer-A's ret_handler callback do

	// my ->handler() wasn't called or it didn't return
	// UPROBE_HANDLER_IWANTMYCOOKIE
	if (!data)
		return;

at the start?

Why the UPROBE_HANDLER_IGNORE case is more problematic than the UPROBE_HANDLER_REMOVE case?

Oleg.

#6 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

Quoted 79 lines · Oleg Nesterov

Damn, sorry for delay :/

And sorry, still can't understand, see below...

Quoted 53 lines · Jiri Olsa
Quoted 36 lines · Oleg Nesterov

To me this code should do:

		if (!uc->ret_handler || UPROBE_HANDLER_REMOVE || UPROBE_HANDLER_IGNORE)
			continue;

		if (!ri)
			ri = alloc_return_instance();

		if (rc == UPROBE_HANDLER_IWANTMYCOOKIE)
			ri = push_consumer(...);

And,

Quoted 1 line

handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs)

...

Review

Quoted 9 lines
(quoted diff) +4 -1
 	list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node,
 				 srcu_read_lock_held(&uprobes_srcu)) {
+		ric = return_consumer_find(ri, &ric_idx, uc->id);
+		if (ric && ric->rc == UPROBE_HANDLER_IGNORE)
+			continue;
 		if (uc->ret_handler)
-			uc->ret_handler(uc, ri->func, regs);
+			uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL);
 	}

the UPROBE_HANDLER_IGNORE check above and the new ric->rc member should die,

		if (!uc->ret_handler)
			continue;

		ric = return_consumer_find(...);
		uc->ret_handler(..., ric ? &ric->cookie : NULL);

as we have already discussed, the session ret_handler(data) can simply do

Quoted 8 lines
		// my ->handler() wasn't called or it didn't return
		// UPROBE_HANDLER_IWANTMYCOOKIE
		if (!data)
			return;

at the start.

Could you explain why this can't work?

I'll try ;-) it's for the case when consumer does not use UPROBE_HANDLER_IWANTMYCOOKIE

let's have 2 consumers on single uprobe, consumer-A returning UPROBE_HANDLER_IGNORE and the consumer-B returning zero, so we want the return uprobe installed, but we want just consumer-B to be executed

Quoted 5 lines

- so uprobe gets installed and handle_uretprobe_chain goes over all consumers calling ret_handler callback

- but we don't know consumer-A needs to be ignored, and it does not expect cookie so we have no way to find out it needs to be ignored

How does this differ from the case when consumer-A returns _REMOVE but another consumer returns 0?

But what I really can't understand is

and it does not
expect cookie so we have no way to find out it needs to be ignored

If we change the code as I suggested above, push_consumer() won't be called if consumer-A returns UPROBE_HANDLER_IGNORE.

This means that handle_uretprobe_chain() -> return_consumer_find() will return NULL, so handle_uretprobe_chain() won't pass the valid cookie to consumer-A's ret_handler callback, it will pass data => NULL.

Quoted 6 lines

So, again, why can't consumer-A's ret_handler callback do

	// my ->handler() wasn't called or it didn't return
	// UPROBE_HANDLER_IWANTMYCOOKIE
	if (!data)
		return;

ok, I think I understand the issue now.. it all depends on 'handler' to return either UPROBE_HANDLER_IGNORE or UPROBE_HANDLER_IWANTMYCOOKIE

my idea was to make the interface more generic, so some future uprobe user won't depend on handler callback to return UPROBE_HANDLER_IWANTMYCOOKIE and can just return 0, but we can do that as follow up if it's ever needed

change below should do what you proposed originally

also on top of that.. I discussed with Andrii the possibility of dropping the UPROBE_HANDLER_IWANTMYCOOKIE completely and setup cookie for any consumer that has both 'handler' and 'ret_handler' defined, wdyt?

thanks,
jirka

---

include/linux/uprobes.h +23 -1
index bb265a632b91..5221080b1f5a 100644
@@ -23,8 +23,20 @@ struct inode;
 struct notifier_block;
 struct page;
 
+/*
+ * Allowed return values from uprobe consumer's handler callback
+ * with following meaning:
+ *
+ * UPROBE_HANDLER_REMOVE
+ * - Remove the uprobe breakpoint from current->mm.
+ * UPROBE_HANDLER_IGNORE
+ * - Ignore ret_handler callback for this consumer.
+ * UPROBE_HANDLER_IWANTMYCOOKIE
+ * - Store cookie and pass it to ret_handler (if defined).
+ */
 #define UPROBE_HANDLER_REMOVE		1
-#define UPROBE_HANDLER_MASK		1
+#define UPROBE_HANDLER_IGNORE		2
+#define UPROBE_HANDLER_IWANTMYCOOKIE	3
 
 #define MAX_URETPROBE_DEPTH		64
 
@@ -44,6 +56,8 @@ struct uprobe_consumer {
 	bool (*filter)(struct uprobe_consumer *self, struct mm_struct *mm);
 
 	struct list_head cons_node;
+
+	__u64 id;	/* set when uprobe_consumer is registered */
 };
 
 #ifdef CONFIG_UPROBES
@@ -83,14 +97,22 @@ struct uprobe_task {
 	unsigned int			depth;
 };
 
+struct return_consumer {
+	__u64	cookie;
+	__u64	id;
+};
+
 struct return_instance {
 	struct uprobe		*uprobe;
 	unsigned long		func;
 	unsigned long		stack;		/* stack pointer */
 	unsigned long		orig_ret_vaddr; /* original return address */
 	bool			chained;	/* true, if instance is nested */
+	int			consumers_cnt;
 
 	struct return_instance	*next;		/* keep as stack */
+
+	struct return_consumer	consumers[] __counted_by(consumers_cnt);
 };
 
 enum rp_check {
kernel/events/uprobes.c +114 -30

250 lines not on this page · raw message

maintainer
#7 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

Quoted 2 lines · Jiri Olsa

change below should do what you proposed originally

LGTM, just one nit below.

But I guess you need to do this on top of bpf/bpf.git, Andrii has already applied your series.

And to remind, 02/14 must be fixed in any case unless I am totally confused, handler_chain() can leak return_instance.

Quoted 3 lines

also on top of that.. I discussed with Andrii the possibility of dropping the UPROBE_HANDLER_IWANTMYCOOKIE completely and setup cookie for any consumer that has both 'handler' and 'ret_handler' defined, wdyt?

Up to you. As I said from the very beginning I won't insist on _IWANTMYCOOKIE.

kernel/events/uprobes.c static void · line 2257

Quoted 8 lines
kernel/events/uprobes.c +2 -1
 	list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node,
 				 srcu_read_lock_held(&uprobes_srcu)) {
+		ric = return_consumer_find(ri, &ric_idx, uc->id);
 		if (uc->ret_handler)
-			uc->ret_handler(uc, ri->func, regs);
+			uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL);
 	}
 	srcu_read_unlock(&uprobes_srcu, srcu_idx);

return_consumer_find() makes no sense if !uc->ret_handler, can you move

		ric = return_consumer_find(ri, &ric_idx, uc->id);

into the "if (uc->ret_handler)" block?

Oleg.

#8 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

Quoted 8 lines · Oleg Nesterov
Quoted 2 lines · Jiri Olsa

change below should do what you proposed originally

LGTM, just one nit below.

But I guess you need to do this on top of bpf/bpf.git, Andrii has already applied your series.

that seems confusing but looks like just that one fix with the commit link in [1] was applied

[1] https://lore.kernel.org/bpf/172708047825.3261420.5126267811201364070.git-patchwork-notify@kernel.org/T/#mb065649b5ab8f7ea5b03c215bdc6555a0b76c0d7

Quoted 3 lines

And to remind, 02/14 must be fixed in any case unless I am totally confused, handler_chain() can leak return_instance.

yep it was missing kfree, but it's not needed in this new version

Quoted 6 lines
Quoted 4 lines

also on top of that.. I discussed with Andrii the possibility of dropping the UPROBE_HANDLER_IWANTMYCOOKIE completely and setup cookie for any consumer that has both 'handler' and 'ret_handler' defined, wdyt?

Up to you. As I said from the very beginning I won't insist on _IWANTMYCOOKIE.

ok

Quoted 12 lines

kernel/events/uprobes.c static void · line 2257

Quoted 9 lines
kernel/events/uprobes.c +2 -1
 	list_for_each_entry_srcu(uc, &uprobe->consumers, cons_node,
 				 srcu_read_lock_held(&uprobes_srcu)) {
+		ric = return_consumer_find(ri, &ric_idx, uc->id);
 		if (uc->ret_handler)
-			uc->ret_handler(uc, ri->func, regs);
+			uc->ret_handler(uc, ri->func, regs, ric ? &ric->cookie : NULL);
 	}
 	srcu_read_unlock(&uprobes_srcu, srcu_idx);

return_consumer_find() makes no sense if !uc->ret_handler, can you move

Quoted 3 lines
		ric = return_consumer_find(ri, &ric_idx, uc->id);

into the "if (uc->ret_handler)" block?

ok, will move that

thanks,
jirka

maintainer
#9 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

Quoted 4 lines · Jiri Olsa
Quoted 2 lines · Oleg Nesterov
Quoted 1 line · Jiri Olsa
Quoted 8 lines
Quoted 6 lines
Quoted 1 line

change below should do what you proposed originally

LGTM, just one nit below.

But I guess you need to do this on top of bpf/bpf.git, Andrii has already applied your series.

that seems confusing

Yes ;)

Quoted 2 lines

but looks like just that one fix with the
commit link in [1] was applied

Ah, OK.

Quoted 4 lines
Quoted 2 lines

And to remind, 02/14 must be fixed in any case unless I am totally confused, handler_chain() can leak return_instance.

yep it was missing kfree, but it's not needed in this new version

Yes, yes, the new version looks fine to me.

Oleg.

maintainer
#10 lore raw

Re: [PATCHv4 02/14] uprobe: Add support for session consumer

In reply to Jiri Olsa
Quoted 2 lines · Jiri Olsa

static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs)

...

kernel/events/uprobes.c static struct uprobe *find_active_uprobe_rcu(unsigned long bp_vaddr, int *is_swb · line 2231

Quoted 8 lines
kernel/events/uprobes.c +8 -0
+	if (!ignore && !ZERO_OR_NULL_PTR(ri)) {
+		/*
+		 * The push_idx value has the final number of return consumers,
+		 * and ri->consumers_cnt has number of allocated consumers.
+		 */
+		ri->consumers_cnt = push_idx;
+		prepare_uretprobe(uprobe, regs, ri);
+	}

This looks wrong. ri is not kfreed if ignore == true.

But see my previous email, if we change this code as I tried to suggest the problem goes away and handler_chain() doesn't need "bool ignore".

Oleg.

author reviewer
#11 lore raw

PATCH v4 3/14 bpf: Add support for uprobe multi session attach

In reply to Jiri Olsa

Adding support to attach BPF program for entry and return probe of the same function. This is common use case which at the moment requires to create two uprobe multi links.

Adding new BPF_TRACE_UPROBE_SESSION attach type that instructs kernel to attach single link program to both entry and exit probe.

It's possible to control execution of the BPF program on return probe simply by returning zero or non zero from the entry BPF program execution to execute or not the BPF program on return probe respectively.

---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/syscall.c           |  9 ++++++--
 kernel/trace/bpf_trace.c       | 39 +++++++++++++++++++++++++++-------
 tools/include/uapi/linux/bpf.h |  1 +
 tools/lib/bpf/libbpf.c         |  1 +
 5 files changed, 41 insertions(+), 10 deletions(-)
include/uapi/linux/bpf.h +1 -0
index 35bcf52dbc65..1d93cb014884 100644
@@ -1116,6 +1116,7 @@ enum bpf_attach_type {
 	BPF_NETKIT_PRIMARY,
 	BPF_NETKIT_PEER,
 	BPF_TRACE_KPROBE_SESSION,
+	BPF_TRACE_UPROBE_SESSION,
 	__MAX_BPF_ATTACH_TYPE
 };
 
kernel/bpf/syscall.c +7 -2
index bf6c5f685ea2..1347f3000bd0 100644
@@ -4049,10 +4049,14 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
 		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI &&
 		    attach_type != BPF_TRACE_UPROBE_MULTI)
 			return -EINVAL;
+		if (prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION &&
+		    attach_type != BPF_TRACE_UPROBE_SESSION)
+			return -EINVAL;
 		if (attach_type != BPF_PERF_EVENT &&
 		    attach_type != BPF_TRACE_KPROBE_MULTI &&
 		    attach_type != BPF_TRACE_KPROBE_SESSION &&
-		    attach_type != BPF_TRACE_UPROBE_MULTI)
+		    attach_type != BPF_TRACE_UPROBE_MULTI &&
+		    attach_type != BPF_TRACE_UPROBE_SESSION)
 			return -EINVAL;
 		return 0;
 	case BPF_PROG_TYPE_SCHED_CLS:
@@ -5315,7 +5319,8 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 		else if (attr->link_create.attach_type == BPF_TRACE_KPROBE_MULTI ||
 			 attr->link_create.attach_type == BPF_TRACE_KPROBE_SESSION)
 			ret = bpf_kprobe_multi_link_attach(attr, prog);
-		else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI)
+		else if (attr->link_create.attach_type == BPF_TRACE_UPROBE_MULTI ||
+			 attr->link_create.attach_type == BPF_TRACE_UPROBE_SESSION)
 			ret = bpf_uprobe_multi_link_attach(attr, prog);
 		break;
 	default:
kernel/trace/bpf_trace.c +31 -8
index de241503c8fb..63ea457af16a 100644
@@ -1645,6 +1645,17 @@ static inline bool is_kprobe_session(const struct bpf_prog *prog)
 	return prog->expected_attach_type == BPF_TRACE_KPROBE_SESSION;
 }
 
+static inline bool is_uprobe_multi(const struct bpf_prog *prog)
+{
+	return prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI ||
+	       prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION;
+}
+
+static inline bool is_uprobe_session(const struct bpf_prog *prog)
+{
+	return prog->expected_attach_type == BPF_TRACE_UPROBE_SESSION;
+}
+
 static const struct bpf_func_proto *
 kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 {
@@ -1662,13 +1673,13 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	case BPF_FUNC_get_func_ip:
 		if (is_kprobe_multi(prog))
 			return &bpf_get_func_ip_proto_kprobe_multi;
-		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
+		if (is_uprobe_multi(prog))
 			return &bpf_get_func_ip_proto_uprobe_multi;
 		return &bpf_get_func_ip_proto_kprobe;
 	case BPF_FUNC_get_attach_cookie:
 		if (is_kprobe_multi(prog))
 			return &bpf_get_attach_cookie_proto_kmulti;
-		if (prog->expected_attach_type == BPF_TRACE_UPROBE_MULTI)
+		if (is_uprobe_multi(prog))
 			return &bpf_get_attach_cookie_proto_umulti;
 		return &bpf_get_attach_cookie_proto_trace;
 	default:
@@ -3162,6 +3173,7 @@ struct bpf_uprobe {
 	u64 cookie;
 	struct uprobe *uprobe;
 	struct uprobe_consumer consumer;
+	bool session;
 };
 
 struct bpf_uprobe_multi_link {
@@ -3336,9 +3348,13 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs,
 			  __u64 *data)
 {
 	struct bpf_uprobe *uprobe;
+	int ret;
 
 	uprobe = container_of(con, struct bpf_uprobe, consumer);
-	return uprobe_prog_run(uprobe, instruction_pointer(regs), regs);
+	ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs);
+	if (uprobe->session)
+		return ret ? UPROBE_HANDLER_IGNORE : UPROBE_HANDLER_IWANTMYCOOKIE;
+	return ret;
 }
 
 static int
@@ -3348,6 +3364,12 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s
 	struct bpf_uprobe *uprobe;
 
 	uprobe = container_of(con, struct bpf_uprobe, consumer);
+	/*
+	 * There's chance we could get called with NULL data if we registered uprobe
+	 * after it hit entry but before it hit return probe, just ignore it.
+	 */
+	if (uprobe->session && !data)
+		return 0;
 	return uprobe_prog_run(uprobe, func, regs);
 }
 
@@ -3387,7 +3409,7 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 	if (sizeof(u64) != sizeof(void *))
 		return -EOPNOTSUPP;
 
-	if (prog->expected_attach_type != BPF_TRACE_UPROBE_MULTI)
+	if (!is_uprobe_multi(prog))
 		return -EINVAL;
 
 	flags = attr->link_create.uprobe_multi.flags;
@@ -3463,11 +3485,12 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr
 
 		uprobes[i].link = link;
 
-		if (flags & BPF_F_UPROBE_MULTI_RETURN)
-			uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler;
-		else
+		if (!(flags & BPF_F_UPROBE_MULTI_RETURN))
 			uprobes[i].consumer.handler = uprobe_multi_link_handler;
-
+		if (flags & BPF_F_UPROBE_MULTI_RETURN || is_uprobe_session(prog))
+			uprobes[i].consumer.ret_handler = uprobe_multi_link_ret_handler;
+		if (is_uprobe_session(prog))
+			uprobes[i].session = true;
 		if (pid)
 			uprobes[i].consumer.filter = uprobe_multi_link_filter;
 	}
tools/include/uapi/linux/bpf.h +1 -0
index 35bcf52dbc65..1d93cb014884 100644
@@ -1116,6 +1116,7 @@ enum bpf_attach_type {
 	BPF_NETKIT_PRIMARY,
 	BPF_NETKIT_PEER,
 	BPF_TRACE_KPROBE_SESSION,
+	BPF_TRACE_UPROBE_SESSION,
 	__MAX_BPF_ATTACH_TYPE
 };
 
tools/lib/bpf/libbpf.c +1 -0
index a3be6f8fac09..274441674f92 100644
@@ -133,6 +133,7 @@ static const char * const attach_type_name[] = {
 	[BPF_NETKIT_PRIMARY]		= "netkit_primary",
 	[BPF_NETKIT_PEER]		= "netkit_peer",
 	[BPF_TRACE_KPROBE_SESSION]	= "trace_kprobe_session",
+	[BPF_TRACE_UPROBE_SESSION]	= "trace_uprobe_session",
 };
 
 static const char * const link_type_name[] = {
Signature · 1 line
2.46.0
author reviewer
#12 lore raw

PATCH v4 4/14 bpf: Add support for uprobe multi session context

In reply to Jiri Olsa

Placing bpf_session_run_ctx layer in between bpf_run_ctx and bpf_uprobe_multi_run_ctx, so the session data can be retrieved from uprobe_multi link.

Plus granting session kfuncs access to uprobe session programs.

---
 kernel/trace/bpf_trace.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)
kernel/trace/bpf_trace.c +15 -8
index 63ea457af16a..b6f377f1ce5f 100644
@@ -3186,7 +3186,7 @@ struct bpf_uprobe_multi_link {
 };
 
 struct bpf_uprobe_multi_run_ctx {
-	struct bpf_run_ctx run_ctx;
+	struct bpf_session_run_ctx session_ctx;
 	unsigned long entry_ip;
 	struct bpf_uprobe *uprobe;
 };
@@ -3299,10 +3299,15 @@ static const struct bpf_link_ops bpf_uprobe_multi_link_lops = {
 
 static int uprobe_prog_run(struct bpf_uprobe *uprobe,
 			   unsigned long entry_ip,
-			   struct pt_regs *regs)
+			   struct pt_regs *regs,
+			   bool is_return, void *data)
 {
 	struct bpf_uprobe_multi_link *link = uprobe->link;
 	struct bpf_uprobe_multi_run_ctx run_ctx = {
+		.session_ctx = {
+			.is_return = is_return,
+			.data = data,
+		},
 		.entry_ip = entry_ip,
 		.uprobe = uprobe,
 	};
@@ -3321,7 +3326,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe,
 
 	migrate_disable();
 
-	old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx);
+	old_run_ctx = bpf_set_run_ctx(&run_ctx.session_ctx.run_ctx);
 	err = bpf_prog_run(link->link.prog, regs);
 	bpf_reset_run_ctx(old_run_ctx);
 
@@ -3351,7 +3356,7 @@ uprobe_multi_link_handler(struct uprobe_consumer *con, struct pt_regs *regs,
 	int ret;
 
 	uprobe = container_of(con, struct bpf_uprobe, consumer);
-	ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs);
+	ret = uprobe_prog_run(uprobe, instruction_pointer(regs), regs, false, data);
 	if (uprobe->session)
 		return ret ? UPROBE_HANDLER_IGNORE : UPROBE_HANDLER_IWANTMYCOOKIE;
 	return ret;
@@ -3370,14 +3375,15 @@ uprobe_multi_link_ret_handler(struct uprobe_consumer *con, unsigned long func, s
 	 */
 	if (uprobe->session && !data)
 		return 0;
-	return uprobe_prog_run(uprobe, func, regs);
+	return uprobe_prog_run(uprobe, func, regs, true, data);
 }
 
 static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx)
 {
 	struct bpf_uprobe_multi_run_ctx *run_ctx;
 
-	run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx);
+	run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx,
+			       session_ctx.run_ctx);
 	return run_ctx->entry_ip;
 }
 
@@ -3385,7 +3391,8 @@ static u64 bpf_uprobe_multi_cookie(struct bpf_run_ctx *ctx)
 {
 	struct bpf_uprobe_multi_run_ctx *run_ctx;
 
-	run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx, run_ctx);
+	run_ctx = container_of(current->bpf_ctx, struct bpf_uprobe_multi_run_ctx,
+			       session_ctx.run_ctx);
 	return run_ctx->uprobe->cookie;
 }
 
@@ -3579,7 +3586,7 @@ static int bpf_kprobe_multi_filter(const struct bpf_prog *prog, u32 kfunc_id)
 	if (!btf_id_set8_contains(&kprobe_multi_kfunc_set_ids, kfunc_id))
 		return 0;
 
-	if (!is_kprobe_session(prog))
+	if (!is_kprobe_session(prog) && !is_uprobe_session(prog))
 		return -EACCES;
 
 	return 0;
Signature · 1 line
2.46.0
author reviewer
#13 lore raw

PATCH v4 5/14 bpf: Allow return values 0 and 1 for uprobe/kprobe session

In reply to Jiri Olsa

The uprobe and kprobe session program can return only 0 or 1, instruct verifier to check for that.

---
 kernel/bpf/verifier.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/bpf/verifier.c +10 -0
index d8520095ca03..988858fc37e5 100644
@@ -15668,6 +15668,16 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
 			return -ENOTSUPP;
 		}
 		break;
+	case BPF_PROG_TYPE_KPROBE:
+		switch (env->prog->expected_attach_type) {
+		case BPF_TRACE_KPROBE_SESSION:
+		case BPF_TRACE_UPROBE_SESSION:
+			range = retval_range(0, 1);
+			break;
+		default:
+			return 0;
+		}
+		break;
 	case BPF_PROG_TYPE_SK_LOOKUP:
 		range = retval_range(SK_DROP, SK_PASS);
 		break;
Signature · 1 line
2.46.0
author reviewer
#14 lore raw

PATCH v4 6/14 libbpf: Fix uretprobe.multi.s programs auto attachment

In reply to Jiri Olsa

As reported by Andrii we don't currently recognize uretprobe.multi.s programs as return probes due to using (wrong) strcmp function.

Using str_has_pfx() instead to match uretprobe.multi prefix.

Tests are passing, because the return program was executed as entry program and all counts were incremented properly.

---
 tools/lib/bpf/libbpf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/bpf/libbpf.c +1 -1
index 274441674f92..6917d4a0bd4e 100644
@@ -11684,7 +11684,7 @@ static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, stru
 		ret = 0;
 		break;
 	case 3:
-		opts.retprobe = strcmp(probe_type, "uretprobe.multi") == 0;
+		opts.retprobe = str_has_pfx(probe_type, "uretprobe.multi");
 		*link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts);
 		ret = libbpf_get_error(*link);
 		break;
Signature · 1 line
2.46.0
author reviewer
#15 lore raw

PATCH v4 7/14 libbpf: Add support for uprobe multi session attach

In reply to Jiri Olsa

Adding support to attach program in uprobe session mode with bpf_program__attach_uprobe_multi function.

Adding session bool to bpf_uprobe_multi_opts struct that allows to load and attach the bpf program via uprobe session. the attachment to create uprobe multi session.

Also adding new program loader section that allows:
SEC("uprobe.session/bpf_fentry_test*")

and loads/attaches uprobe program as uprobe session.

Adding sleepable hook (uprobe.session.s) as well.

---
 tools/lib/bpf/bpf.c    |  1 +
 tools/lib/bpf/libbpf.c | 21 ++++++++++++++++++---
 tools/lib/bpf/libbpf.h |  4 +++-
 3 files changed, 22 insertions(+), 4 deletions(-)
tools/lib/bpf/bpf.c +1 -0
index 2a4c71501a17..becdfa701c75 100644
@@ -776,6 +776,7 @@ int bpf_link_create(int prog_fd, int target_fd,
 			return libbpf_err(-EINVAL);
 		break;
 	case BPF_TRACE_UPROBE_MULTI:
+	case BPF_TRACE_UPROBE_SESSION:
 		attr.link_create.uprobe_multi.flags = OPTS_GET(opts, uprobe_multi.flags, 0);
 		attr.link_create.uprobe_multi.cnt = OPTS_GET(opts, uprobe_multi.cnt, 0);
 		attr.link_create.uprobe_multi.path = ptr_to_u64(OPTS_GET(opts, uprobe_multi.path, 0));
tools/lib/bpf/libbpf.c +18 -3
index 6917d4a0bd4e..83e56856b2b4 100644
@@ -9363,8 +9363,10 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("kprobe.session+",	KPROBE,	BPF_TRACE_KPROBE_SESSION, SEC_NONE, attach_kprobe_session),
 	SEC_DEF("uprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
 	SEC_DEF("uretprobe.multi+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_NONE, attach_uprobe_multi),
+	SEC_DEF("uprobe.session+",	KPROBE,	BPF_TRACE_UPROBE_SESSION, SEC_NONE, attach_uprobe_multi),
 	SEC_DEF("uprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
 	SEC_DEF("uretprobe.multi.s+",	KPROBE,	BPF_TRACE_UPROBE_MULTI, SEC_SLEEPABLE, attach_uprobe_multi),
+	SEC_DEF("uprobe.session.s+",	KPROBE,	BPF_TRACE_UPROBE_SESSION, SEC_SLEEPABLE, attach_uprobe_multi),
 	SEC_DEF("ksyscall+",		KPROBE,	0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("usdt+",		KPROBE,	0, SEC_USDT, attach_usdt),
@@ -11684,7 +11686,10 @@ static int attach_uprobe_multi(const struct bpf_program *prog, long cookie, stru
 		ret = 0;
 		break;
 	case 3:
-		opts.retprobe = str_has_pfx(probe_type, "uretprobe.multi");
+		if (str_has_pfx(probe_type, "uprobe.session"))
+			opts.session = true;
+		else
+			opts.retprobe = str_has_pfx(probe_type, "uretprobe.multi");
 		*link = bpf_program__attach_uprobe_multi(prog, -1, binary_path, func_name, &opts);
 		ret = libbpf_get_error(*link);
 		break;
@@ -11933,10 +11938,12 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
 	const unsigned long *ref_ctr_offsets = NULL, *offsets = NULL;
 	LIBBPF_OPTS(bpf_link_create_opts, lopts);
 	unsigned long *resolved_offsets = NULL;
+	enum bpf_attach_type attach_type;
 	int err = 0, link_fd, prog_fd;
 	struct bpf_link *link = NULL;
 	char errmsg[STRERR_BUFSIZE];
 	char full_path[PATH_MAX];
+	bool retprobe, session;
 	const __u64 *cookies;
 	const char **syms;
 	size_t cnt;
@@ -12007,12 +12014,20 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
 		offsets = resolved_offsets;
 	}
 
+	retprobe = OPTS_GET(opts, retprobe, false);
+	session  = OPTS_GET(opts, session, false);
+
+	if (retprobe && session)
+		return libbpf_err_ptr(-EINVAL);
+
+	attach_type = session ? BPF_TRACE_UPROBE_SESSION : BPF_TRACE_UPROBE_MULTI;
+
 	lopts.uprobe_multi.path = path;
 	lopts.uprobe_multi.offsets = offsets;
 	lopts.uprobe_multi.ref_ctr_offsets = ref_ctr_offsets;
 	lopts.uprobe_multi.cookies = cookies;
 	lopts.uprobe_multi.cnt = cnt;
-	lopts.uprobe_multi.flags = OPTS_GET(opts, retprobe, false) ? BPF_F_UPROBE_MULTI_RETURN : 0;
+	lopts.uprobe_multi.flags = retprobe ? BPF_F_UPROBE_MULTI_RETURN : 0;
 
 	if (pid == 0)
 		pid = getpid();
@@ -12026,7 +12041,7 @@ bpf_program__attach_uprobe_multi(const struct bpf_program *prog,
 	}
 	link->detach = &bpf_link__detach_fd;
 
-	link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &lopts);
+	link_fd = bpf_link_create(prog_fd, 0, attach_type, &lopts);
 	if (link_fd < 0) {
 		err = -errno;
 		pr_warn("prog '%s': failed to attach multi-uprobe: %s\n",
tools/lib/bpf/libbpf.h +3 -1
index 64a6a3d323e3..f6a7835dc519 100644
@@ -569,10 +569,12 @@ struct bpf_uprobe_multi_opts {
 	size_t cnt;
 	/* create return uprobes */
 	bool retprobe;
+	/* create session kprobes */
+	bool session;
 	size_t :0;
 };
 
-#define bpf_uprobe_multi_opts__last_field retprobe
+#define bpf_uprobe_multi_opts__last_field session
 
 /**
  * @brief **bpf_program__attach_uprobe_multi()** attaches a BPF program
Signature · 1 line
2.46.0
author reviewer
#16 lore raw

PATCH v4 8/14 selftests/bpf: Add uprobe session test

In reply to Jiri Olsa

Adding uprobe session test and testing that the entry program return value controls execution of the return probe program.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 47 ++++++++++++
 .../bpf/progs/uprobe_multi_session.c          | 71 +++++++++++++++++++
 2 files changed, 118 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +47 -0
index bf6ca8e3eb13..aaafd80623c5 100644
@@ -6,6 +6,7 @@
 #include "uprobe_multi.skel.h"
 #include "uprobe_multi_bench.skel.h"
 #include "uprobe_multi_usdt.skel.h"
+#include "uprobe_multi_session.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "testing_helpers.h"
 #include "../sdt.h"
@@ -615,6 +616,50 @@ static void test_link_api(void)
 	__test_link_api(child);
 }
 
+static void test_session_skel_api(void)
+{
+	struct uprobe_multi_session *skel = NULL;
+	LIBBPF_OPTS(bpf_kprobe_multi_opts, opts);
+	struct bpf_link *link = NULL;
+	int err;
+
+	skel = uprobe_multi_session__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi_session__open_and_load"))
+		goto cleanup;
+
+	skel->bss->pid = getpid();
+	skel->bss->user_ptr = test_data;
+
+	err = uprobe_multi_session__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi_session__attach"))
+		goto cleanup;
+
+	/* trigger all probes */
+	skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1;
+	skel->bss->uprobe_multi_func_2_addr = (__u64) uprobe_multi_func_2;
+	skel->bss->uprobe_multi_func_3_addr = (__u64) uprobe_multi_func_3;
+
+	uprobe_multi_func_1();
+	uprobe_multi_func_2();
+	uprobe_multi_func_3();
+
+	/*
+	 * We expect 2 for uprobe_multi_func_2 because it runs both entry/return probe,
+	 * uprobe_multi_func_[13] run just the entry probe. All expected numbers are
+	 * doubled, because we run extra test for sleepable session.
+	 */
+	ASSERT_EQ(skel->bss->uprobe_session_result[0], 2, "uprobe_multi_func_1_result");
+	ASSERT_EQ(skel->bss->uprobe_session_result[1], 4, "uprobe_multi_func_2_result");
+	ASSERT_EQ(skel->bss->uprobe_session_result[2], 2, "uprobe_multi_func_3_result");
+
+	/* We expect increase in 3 entry and 1 return session calls -> 4 */
+	ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 4, "uprobe_multi_sleep_result");
+
+cleanup:
+	bpf_link__destroy(link);
+	uprobe_multi_session__destroy(skel);
+}
+
 static void test_bench_attach_uprobe(void)
 {
 	long attach_start_ns = 0, attach_end_ns = 0;
@@ -703,4 +748,6 @@ void test_uprobe_multi_test(void)
 		test_bench_attach_usdt();
 	if (test__start_subtest("attach_api_fails"))
 		test_attach_api_fails();
+	if (test__start_subtest("session"))
+		test_session_skel_api();
 }
tools/testing/selftests/bpf/progs/uprobe_multi_session.c new file +71 -0

71 lines not on this page · raw message

Signature · 1 line
2.46.0
author reviewer
#17 lore raw

PATCH v4 9/14 selftests/bpf: Add uprobe session cookie test

In reply to Jiri Olsa

Adding uprobe session test that verifies the cookie value get properly propagated from entry to return program.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 31 ++++++++++++
 .../bpf/progs/uprobe_multi_session_cookie.c   | 48 +++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +31 -0
index aaafd80623c5..edb71b9293c9 100644
@@ -7,6 +7,7 @@
 #include "uprobe_multi_bench.skel.h"
 #include "uprobe_multi_usdt.skel.h"
 #include "uprobe_multi_session.skel.h"
+#include "uprobe_multi_session_cookie.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "testing_helpers.h"
 #include "../sdt.h"
@@ -660,6 +661,34 @@ static void test_session_skel_api(void)
 	uprobe_multi_session__destroy(skel);
 }
 
+static void test_session_cookie_skel_api(void)
+{
+	struct uprobe_multi_session_cookie *skel = NULL;
+	int err;
+
+	skel = uprobe_multi_session_cookie__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_cookie__open_and_load"))
+		goto cleanup;
+
+	skel->bss->pid = getpid();
+
+	err = uprobe_multi_session_cookie__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi_session_cookie__attach"))
+		goto cleanup;
+
+	/* trigger all probes */
+	uprobe_multi_func_1();
+	uprobe_multi_func_2();
+	uprobe_multi_func_3();
+
+	ASSERT_EQ(skel->bss->test_uprobe_1_result, 1, "test_uprobe_1_result");
+	ASSERT_EQ(skel->bss->test_uprobe_2_result, 2, "test_uprobe_2_result");
+	ASSERT_EQ(skel->bss->test_uprobe_3_result, 3, "test_uprobe_3_result");
+
+cleanup:
+	uprobe_multi_session_cookie__destroy(skel);
+}
+
 static void test_bench_attach_uprobe(void)
 {
 	long attach_start_ns = 0, attach_end_ns = 0;
@@ -750,4 +779,6 @@ void test_uprobe_multi_test(void)
 		test_attach_api_fails();
 	if (test__start_subtest("session"))
 		test_session_skel_api();
+	if (test__start_subtest("session_cookie"))
+		test_session_cookie_skel_api();
 }
tools/testing/selftests/bpf/progs/uprobe_multi_session_cookie.c new file +48 -0

48 lines not on this page · raw message

Signature · 1 line
2.46.0
author reviewer
#18 lore raw

PATCH v4 10/14 selftests/bpf: Add uprobe session recursive test

In reply to Jiri Olsa

Adding uprobe session test that verifies the cookie value is stored properly when single uprobe-ed function is executed recursively.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 57 +++++++++++++++++++
 .../progs/uprobe_multi_session_recursive.c    | 44 ++++++++++++++
 2 files changed, 101 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +57 -0
index edb71b9293c9..39d505f27784 100644
@@ -8,6 +8,7 @@
 #include "uprobe_multi_usdt.skel.h"
 #include "uprobe_multi_session.skel.h"
 #include "uprobe_multi_session_cookie.skel.h"
+#include "uprobe_multi_session_recursive.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "testing_helpers.h"
 #include "../sdt.h"
@@ -34,6 +35,12 @@ noinline void usdt_trigger(void)
 	STAP_PROBE(test, pid_filter_usdt);
 }
 
+noinline void uprobe_session_recursive(int i)
+{
+	if (i)
+		uprobe_session_recursive(i - 1);
+}
+
 struct child {
 	int go[2];
 	int c2p[2]; /* child -> parent channel */
@@ -689,6 +696,54 @@ static void test_session_cookie_skel_api(void)
 	uprobe_multi_session_cookie__destroy(skel);
 }
 
+static void test_session_recursive_skel_api(void)
+{
+	struct uprobe_multi_session_recursive *skel = NULL;
+	int i, err;
+
+	skel = uprobe_multi_session_recursive__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_recursive__open_and_load"))
+		goto cleanup;
+
+	skel->bss->pid = getpid();
+
+	err = uprobe_multi_session_recursive__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi_session_recursive__attach"))
+		goto cleanup;
+
+	for (i = 0; i < ARRAY_SIZE(skel->bss->test_uprobe_cookie_entry); i++)
+		skel->bss->test_uprobe_cookie_entry[i] = i + 1;
+
+	uprobe_session_recursive(5);
+
+	/*
+	 *                                         entry uprobe:
+	 * uprobe_session_recursive(5) {             *cookie = 1, return 0
+	 *   uprobe_session_recursive(4) {           *cookie = 2, return 1
+	 *     uprobe_session_recursive(3) {         *cookie = 3, return 0
+	 *       uprobe_session_recursive(2) {       *cookie = 4, return 1
+	 *         uprobe_session_recursive(1) {     *cookie = 5, return 0
+	 *           uprobe_session_recursive(0) {   *cookie = 6, return 1
+	 *                                          return uprobe:
+	 *           } i = 0                          not executed
+	 *         } i = 1                            test_uprobe_cookie_return[0] = 5
+	 *       } i = 2                              not executed
+	 *     } i = 3                                test_uprobe_cookie_return[1] = 3
+	 *   } i = 4                                  not executed
+	 * } i = 5                                    test_uprobe_cookie_return[2] = 1
+	 */
+
+	ASSERT_EQ(skel->bss->idx_entry, 6, "idx_entry");
+	ASSERT_EQ(skel->bss->idx_return, 3, "idx_return");
+
+	ASSERT_EQ(skel->bss->test_uprobe_cookie_return[0], 5, "test_uprobe_cookie_return[0]");
+	ASSERT_EQ(skel->bss->test_uprobe_cookie_return[1], 3, "test_uprobe_cookie_return[1]");
+	ASSERT_EQ(skel->bss->test_uprobe_cookie_return[2], 1, "test_uprobe_cookie_return[2]");
+
+cleanup:
+	uprobe_multi_session_recursive__destroy(skel);
+}
+
 static void test_bench_attach_uprobe(void)
 {
 	long attach_start_ns = 0, attach_end_ns = 0;
@@ -781,4 +836,6 @@ void test_uprobe_multi_test(void)
 		test_session_skel_api();
 	if (test__start_subtest("session_cookie"))
 		test_session_cookie_skel_api();
+	if (test__start_subtest("session_cookie_recursive"))
+		test_session_recursive_skel_api();
 }
tools/testing/selftests/bpf/progs/uprobe_multi_session_recursive.c new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.46.0
author reviewer
#19 lore raw

PATCH v4 11/14 selftests/bpf: Add uprobe session verifier test for return value

In reply to Jiri Olsa

Making sure uprobe.session program can return only [0,1] values.

---
 .../bpf/prog_tests/uprobe_multi_test.c        |  2 ++
 .../bpf/progs/uprobe_multi_verifier.c         | 31 +++++++++++++++++++
 2 files changed, 33 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +2 -0
index 39d505f27784..210cdb620ee0 100644
@@ -9,6 +9,7 @@
 #include "uprobe_multi_session.skel.h"
 #include "uprobe_multi_session_cookie.skel.h"
 #include "uprobe_multi_session_recursive.skel.h"
+#include "uprobe_multi_verifier.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "testing_helpers.h"
 #include "../sdt.h"
@@ -838,4 +839,5 @@ void test_uprobe_multi_test(void)
 		test_session_cookie_skel_api();
 	if (test__start_subtest("session_cookie_recursive"))
 		test_session_recursive_skel_api();
+	RUN_TESTS(uprobe_multi_verifier);
 }
tools/testing/selftests/bpf/progs/uprobe_multi_verifier.c new file +31 -0
new file mode 100644
index 000000000000..fe49f2cb5360
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/usdt.bpf.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+
+SEC("uprobe.session")
+__success
+int uprobe_sesison_return_0(struct pt_regs *ctx)
+{
+	return 0;
+}
+
+SEC("uprobe.session")
+__success
+int uprobe_sesison_return_1(struct pt_regs *ctx)
+{
+	return 1;
+}
+
+SEC("uprobe.session")
+__failure
+__msg("At program exit the register R0 has smin=2 smax=2 should have been in [0, 1]")
+int uprobe_sesison_return_2(struct pt_regs *ctx)
+{
+	return 2;
+}
Signature · 1 line
2.46.0
author reviewer
#20 lore raw

PATCH v4 12/14 selftests/bpf: Add kprobe session verifier test for return value

In reply to Jiri Olsa

Making sure kprobe.session program can return only [0,1] values.

---
 .../bpf/prog_tests/kprobe_multi_test.c        |  2 ++
 .../bpf/progs/kprobe_multi_verifier.c         | 31 +++++++++++++++++++
 2 files changed, 33 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +2 -0
index 960c9323d1e0..66ab1cae923e 100644
@@ -6,6 +6,7 @@
 #include "kprobe_multi_override.skel.h"
 #include "kprobe_multi_session.skel.h"
 #include "kprobe_multi_session_cookie.skel.h"
+#include "kprobe_multi_verifier.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "bpf/hashmap.h"
 
@@ -764,4 +765,5 @@ void test_kprobe_multi_test(void)
 		test_session_skel_api();
 	if (test__start_subtest("session_cookie"))
 		test_session_cookie_skel_api();
+	RUN_TESTS(kprobe_multi_verifier);
 }
tools/testing/selftests/bpf/progs/kprobe_multi_verifier.c new file +31 -0
new file mode 100644
index 000000000000..288577e81deb
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/usdt.bpf.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+
+SEC("kprobe.session")
+__success
+int kprobe_session_return_0(struct pt_regs *ctx)
+{
+	return 0;
+}
+
+SEC("kprobe.session")
+__success
+int kprobe_session_return_1(struct pt_regs *ctx)
+{
+	return 1;
+}
+
+SEC("kprobe.session")
+__failure
+__msg("At program exit the register R0 has smin=2 smax=2 should have been in [0, 1]")
+int kprobe_session_return_2(struct pt_regs *ctx)
+{
+	return 2;
+}
Signature · 1 line
2.46.0
author reviewer
#21 lore raw

PATCH v4 13/14 selftests/bpf: Add uprobe session single consumer test

In reply to Jiri Olsa

Testing that the session ret_handler bypass works on single uprobe with multiple consumers, each with different session ignore return value.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 33 ++++++++++++++
 .../bpf/progs/uprobe_multi_session_single.c   | 44 +++++++++++++++++++
 2 files changed, 77 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +33 -0
index 210cdb620ee0..594aa8c06f58 100644
@@ -7,6 +7,7 @@
 #include "uprobe_multi_bench.skel.h"
 #include "uprobe_multi_usdt.skel.h"
 #include "uprobe_multi_session.skel.h"
+#include "uprobe_multi_session_single.skel.h"
 #include "uprobe_multi_session_cookie.skel.h"
 #include "uprobe_multi_session_recursive.skel.h"
 #include "uprobe_multi_verifier.skel.h"
@@ -669,6 +670,36 @@ static void test_session_skel_api(void)
 	uprobe_multi_session__destroy(skel);
 }
 
+static void test_session_single_skel_api(void)
+{
+	struct uprobe_multi_session_single *skel = NULL;
+	LIBBPF_OPTS(bpf_kprobe_multi_opts, opts);
+	int err;
+
+	skel = uprobe_multi_session_single__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi_session_single__open_and_load"))
+		goto cleanup;
+
+	skel->bss->pid = getpid();
+
+	err = uprobe_multi_session_single__attach(skel);
+	if (!ASSERT_OK(err, "uprobe_multi_session_single__attach"))
+		goto cleanup;
+
+	uprobe_multi_func_1();
+
+	/*
+	 * We expect consumer 0 and 2 to trigger just entry handler (value 1)
+	 * and consumer 1 to hit both (value 2).
+	 */
+	ASSERT_EQ(skel->bss->uprobe_session_result[0], 1, "uprobe_session_result_0");
+	ASSERT_EQ(skel->bss->uprobe_session_result[1], 2, "uprobe_session_result_1");
+	ASSERT_EQ(skel->bss->uprobe_session_result[2], 1, "uprobe_session_result_2");
+
+cleanup:
+	uprobe_multi_session_single__destroy(skel);
+}
+
 static void test_session_cookie_skel_api(void)
 {
 	struct uprobe_multi_session_cookie *skel = NULL;
@@ -835,6 +866,8 @@ void test_uprobe_multi_test(void)
 		test_attach_api_fails();
 	if (test__start_subtest("session"))
 		test_session_skel_api();
+	if (test__start_subtest("session_single"))
+		test_session_single_skel_api();
 	if (test__start_subtest("session_cookie"))
 		test_session_cookie_skel_api();
 	if (test__start_subtest("session_cookie_recursive"))
tools/testing/selftests/bpf/progs/uprobe_multi_session_single.c new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.46.0
author reviewer
#22 lore raw

PATCH v4 14/14 selftests/bpf: Add consumers stress test on single uprobe

In reply to Jiri Olsa

We create multiple threads each trying to attach and detach uprobe consumers on the same uprobe, while main thread is hitting on that uprobe. All that for 5 seconds.

---
 .../bpf/prog_tests/uprobe_multi_test.c        | 82 +++++++++++++++++++
 .../bpf/progs/uprobe_multi_consumer_stress.c  | 29 +++++++
 2 files changed, 111 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_multi_consumer_stress.c
tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +82 -0
index 594aa8c06f58..dcc128507212 100644
@@ -3,6 +3,7 @@
 #include <unistd.h>
 #include <pthread.h>
 #include <test_progs.h>
+#include <time.h>
 #include "uprobe_multi.skel.h"
 #include "uprobe_multi_bench.skel.h"
 #include "uprobe_multi_usdt.skel.h"
@@ -10,6 +11,7 @@
 #include "uprobe_multi_session_single.skel.h"
 #include "uprobe_multi_session_cookie.skel.h"
 #include "uprobe_multi_session_recursive.skel.h"
+#include "uprobe_multi_consumer_stress.skel.h"
 #include "uprobe_multi_verifier.skel.h"
 #include "bpf/libbpf_internal.h"
 #include "testing_helpers.h"
@@ -848,6 +850,84 @@ static void test_bench_attach_usdt(void)
 	printf("%s: detached in %7.3lfs\n", __func__, detach_delta);
 }
 
+static int done;
+
+static void *worker(void *p)
+{
+	struct uprobe_multi_consumer_stress *skel = p;
+	struct bpf_link *link = NULL;
+
+	srand(time(NULL));
+
+	while (!done) {
+		LIBBPF_OPTS(bpf_uprobe_multi_opts, opts);
+		struct bpf_program *prog;
+
+		switch (rand() % 4) {
+		case 0:
+			prog = skel->progs.uprobe_session_0;
+			opts.session = true;
+			break;
+		case 1:
+			prog = skel->progs.uprobe_session_1;
+			opts.session = true;
+			break;
+		case 2:
+			prog = skel->progs.uprobe;
+			break;
+		case 3:
+			prog = skel->progs.uretprobe;
+			opts.retprobe = true;
+			break;
+		}
+
+		link = bpf_program__attach_uprobe_multi(prog, -1, "/proc/self/exe",
+							"uprobe_multi_func_1", &opts);
+		bpf_link__destroy(link);
+	}
+
+	return NULL;
+}
+
+#define THREAD_COUNT 4
+
+static void test_session_consumer_stress(void)
+{
+	struct uprobe_multi_consumer_stress *skel;
+	pthread_t threads[THREAD_COUNT];
+	time_t start;
+	int i, err;
+
+	/*
+	 * We create multiple threads each trying to attach and detach uprobe
+	 * consumer on the same uprobe, while main thread is hitting on that
+	 * uprobe. All that for 5 seconds.
+	 */
+	skel = uprobe_multi_consumer_stress__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "uprobe_multi_consumer_stress"))
+		goto cleanup;
+
+	for (i = 0; i < THREAD_COUNT; i++) {
+		err = pthread_create(threads + i, NULL, worker, skel);
+		if (!ASSERT_OK(err, "pthread_create"))
+			goto join;
+	}
+
+	start = time(NULL);
+
+	while (start + 5 > time(NULL))
+		uprobe_multi_func_1();
+
+	done = 1;
+
+join:
+	for (i = 0; i < THREAD_COUNT; i++)
+		pthread_join(threads[i], NULL);
+
+cleanup:
+	uprobe_multi_consumer_stress__destroy(skel);
+}
+
 void test_uprobe_multi_test(void)
 {
 	if (test__start_subtest("skel_api"))
@@ -872,5 +952,7 @@ void test_uprobe_multi_test(void)
 		test_session_cookie_skel_api();
 	if (test__start_subtest("session_cookie_recursive"))
 		test_session_recursive_skel_api();
+	if (test__start_subtest("consumer_stress"))
+		test_session_consumer_stress();
 	RUN_TESTS(uprobe_multi_verifier);
 }
tools/testing/selftests/bpf/progs/uprobe_multi_consumer_stress.c new file +29 -0
new file mode 100644
index 000000000000..5390108a21ff
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+SEC("uprobe.session")
+int uprobe_session_0(struct pt_regs *ctx)
+{
+	return 0;
+}
+
+SEC("uprobe.session")
+int uprobe_session_1(struct pt_regs *ctx)
+{
+	return 1;
+}
+
+SEC("uprobe.multi")
+int uprobe(struct pt_regs *ctx)
+{
+	return 0;
+}
+
+SEC("uprobe.multi")
+int uretprobe(struct pt_regs *ctx)
+{
+	return 0;
+}
Signature · 1 line
2.46.0
#23 lore raw
In reply to Jiri Olsa
This series was applied to bpf/bpf.git (master) by Andrii Nakryiko <[email protected]>:

Hello:

This series was applied to bpf/bpf.git (master)
by Andrii Nakryiko <[email protected]>:

Quoted 10 lines · On Tue, 17 Sep 2024 10:50:10 +0200 you wrote:

hi,
this patchset is adding support for session uprobe attachment and using it through bpf link for bpf programs.

The session means that the uprobe consumer is executed on entry and return of probed function with additional control:
- entry callback can control execution of the return callback
- entry and return callbacks can share data/cookie

[...]

Here is the summary with links:
- [PATCHv4,01/14] uprobe: Add data pointer to consumer handlers (no matching commit)
- [PATCHv4,02/14] uprobe: Add support for session consumer
(no matching commit)
- [PATCHv4,03/14] bpf: Add support for uprobe multi session attach (no matching commit)
- [PATCHv4,04/14] bpf: Add support for uprobe multi session context (no matching commit)
- [PATCHv4,05/14] bpf: Allow return values 0 and 1 for uprobe/kprobe session (no matching commit)
- [PATCHv4,06/14] libbpf: Fix uretprobe.multi.s programs auto attachment

    https://git.kernel.org/bpf/bpf/c/8c8b47597403

- [PATCHv4,07/14] libbpf: Add support for uprobe multi session attach (no matching commit)
- [PATCHv4,08/14] selftests/bpf: Add uprobe session test
(no matching commit)
- [PATCHv4,09/14] selftests/bpf: Add uprobe session cookie test
(no matching commit)
- [PATCHv4,10/14] selftests/bpf: Add uprobe session recursive test (no matching commit)
- [PATCHv4,11/14] selftests/bpf: Add uprobe session verifier test for return value (no matching commit)
- [PATCHv4,12/14] selftests/bpf: Add kprobe session verifier test for return value (no matching commit)
- [PATCHv4,13/14] selftests/bpf: Add uprobe session single consumer test (no matching commit)
- [PATCHv4,14/14] selftests/bpf: Add consumers stress test on single uprobe (no matching commit)

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html