Skip to contentSkip to conversation

kprobe/bpf: Add support to attach multiple kprobes

RFC In review Jiri Olsa 6 participants 52 messages Jan 4, 2022 – Jan 24, 2022

Subsystems (16)
lore
Share & tools
lore

Jiri Olsa

author
#0 lore raw

cover kprobe/bpf: Add support to attach multiple kprobes

hi,
adding support to attach multiple kprobes within single syscall and speed up attachment of many kprobes.

The previous attempt [1] wasn't fast enough, so coming with new approach that adds new kprobe interface.

The attachment speed of of this approach (tested in bpftrace) is now comparable to ftrace tracer attachment speed.. fast ;-)

The limit of this approach is forced by using ftrace as attach layer, so it allows only kprobes on function's entry (plus return probes).

This patchset contains:
- kprobes support to register multiple kprobes with current kprobe API (patches 1 - 8)
- bpf support ot create new kprobe link allowing to attach multiple addresses (patches 9 - 14)

We don't need to care about multiple probes on same functions because it's taken care on the ftrace_ops layer.

Also available at:
https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
kprobe/multi

thanks,
jirka

[1] https://lore.kernel.org/bpf/[email protected]/

---
Jiri Olsa (13):
ftrace: Add ftrace_set_filter_ips function
kprobe: Keep traced function address
kprobe: Add support to register multiple ftrace kprobes
kprobe: Add support to register multiple ftrace kretprobes
kprobe: Allow to get traced function address for multi ftrace kprobes samples/kprobes: Add support for multi kprobe interface samples/kprobes: Add support for multi kretprobe interface
bpf: Add kprobe link for attaching raw kprobes
libbpf: Add libbpf__kallsyms_parse function
libbpf: Add bpf_link_create support for multi kprobes
libbpf: Add bpf_program__attach_kprobe_opts for multi kprobes selftest/bpf: Add raw kprobe attach test
selftest/bpf: Add bpf_cookie test for raw_k[ret]probe

 arch/Kconfig                                             |   3 ++
 arch/x86/Kconfig                                         |   1 +
 arch/x86/kernel/kprobes/ftrace.c                         |  51 +++++++++++++-----
 include/linux/bpf_types.h                                |   1 +
 include/linux/ftrace.h                                   |   3 ++
 include/linux/kprobes.h                                  |  55 ++++++++++++++++++++
 include/uapi/linux/bpf.h                                 |  12 +++++
 kernel/bpf/syscall.c                                     | 191 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 kernel/kprobes.c                                         | 264 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------
 kernel/trace/bpf_trace.c                                 |   7 ++-
 kernel/trace/ftrace.c                                    |  53 +++++++++++++++----
 samples/kprobes/kprobe_example.c                         |  47 +++++++++++++++--
 samples/kprobes/kretprobe_example.c                      |  43 +++++++++++++++-
 tools/include/uapi/linux/bpf.h                           |  12 +++++
 tools/lib/bpf/bpf.c                                      |   5 ++
 tools/lib/bpf/bpf.h                                      |   7 ++-
 tools/lib/bpf/libbpf.c                                   | 186 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------
 tools/lib/bpf/libbpf_internal.h                          |   5 ++
 tools/testing/selftests/bpf/prog_tests/bpf_cookie.c      |  42 +++++++++++++++
 tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c |  92 +++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/get_func_ip_test.c     |   4 +-
 tools/testing/selftests/bpf/progs/raw_kprobe.c           |  58 +++++++++++++++++++++
 tools/testing/selftests/bpf/progs/test_bpf_cookie.c      |  24 ++++++++-
 23 files changed, 1062 insertions(+), 104 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c
 create mode 100644 tools/testing/selftests/bpf/progs/raw_kprobe.c
author
#1 lore raw

PATCH 1/13 ftrace: Add ftrace_set_filter_ips function

Adding ftrace_set_filter_ips function to be able to set filter on multiple ip addresses at once.

With the kprobe multi attach interface we have cases where we need to initialize ftrace_ops object with thousands of functions, so having single function diving into ftrace_hash_move_and_update_ops with ftrace_lock is faster.

The functions ips are passed as unsigned long array with count.

---
 include/linux/ftrace.h |  3 +++
 kernel/trace/ftrace.c  | 53 +++++++++++++++++++++++++++++++++++-------
 2 files changed, 47 insertions(+), 9 deletions(-)
include/linux/ftrace.h +3 -0
index 9999e29187de..60847cbce0da 100644
@@ -512,6 +512,8 @@ struct dyn_ftrace {
 
 int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip,
 			 int remove, int reset);
+int ftrace_set_filter_ips(struct ftrace_ops *ops, unsigned long *ips,
+			  unsigned int cnt, int remove, int reset);
 int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
 int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
@@ -802,6 +804,7 @@ static inline unsigned long ftrace_location(unsigned long ip)
 #define ftrace_regex_open(ops, flag, inod, file) ({ -ENODEV; })
 #define ftrace_set_early_filter(ops, buf, enable) do { } while (0)
 #define ftrace_set_filter_ip(ops, ip, remove, reset) ({ -ENODEV; })
+#define ftrace_set_filter_ips(ops, ips, cnt, remove, reset) ({ -ENODEV; })
 #define ftrace_set_filter(ops, buf, len, reset) ({ -ENODEV; })
 #define ftrace_set_notrace(ops, buf, len, reset) ({ -ENODEV; })
 #define ftrace_free_filter(ops) do { } while (0)
kernel/trace/ftrace.c +44 -9

95 lines not on this page · raw message

Signature · 1 line
2.33.1
author
#2 lore raw

PATCH 2/13 kprobe: Keep traced function address

In reply to Jiri Olsa

The bpf_get_func_ip_kprobe helper should return traced function address, but it's doing so only for kprobes that are placed on the function entry.

If kprobe is placed within the function, bpf_get_func_ip_kprobe returns that address instead of function entry.

Storing the function entry directly in kprobe object, so it could be used in bpf_get_func_ip_kprobe helper.

---
 include/linux/kprobes.h                              |  3 +++
 kernel/kprobes.c                                     | 12 ++++++++++++
 kernel/trace/bpf_trace.c                             |  2 +-
 tools/testing/selftests/bpf/progs/get_func_ip_test.c |  4 ++--
 4 files changed, 18 insertions(+), 3 deletions(-)
include/linux/kprobes.h +3 -0
index 8c8f7a4d93af..a204df4fef96 100644
@@ -74,6 +74,9 @@ struct kprobe {
 	/* Offset into the symbol */
 	unsigned int offset;
 
+	/* traced function address */
+	unsigned long func_addr;
+
 	/* Called before addr is executed. */
 	kprobe_pre_handler_t pre_handler;
 
kernel/kprobes.c +12 -0
index d20ae8232835..c4060a8da050 100644
@@ -1310,6 +1310,7 @@ static void init_aggr_kprobe(struct kprobe *ap, struct kprobe *p)
 	copy_kprobe(p, ap);
 	flush_insn_slot(ap);
 	ap->addr = p->addr;
+	ap->func_addr = p->func_addr;
 	ap->flags = p->flags & ~KPROBE_FLAG_OPTIMIZED;
 	ap->pre_handler = aggr_pre_handler;
 	/* We don't care the kprobe which has gone. */
@@ -1588,6 +1589,16 @@ static int check_kprobe_address_safe(struct kprobe *p,
 	return ret;
 }
 
+static unsigned long resolve_func_addr(kprobe_opcode_t *addr)
+{
+	char str[KSYM_SYMBOL_LEN];
+	unsigned long offset;
+
+	if (kallsyms_lookup((unsigned long) addr, NULL, &offset, NULL, str))
+		return (unsigned long) addr - offset;
+	return 0;
+}
+
 int register_kprobe(struct kprobe *p)
 {
 	int ret;
@@ -1600,6 +1611,7 @@ int register_kprobe(struct kprobe *p)
 	if (IS_ERR(addr))
 		return PTR_ERR(addr);
 	p->addr = addr;
+	p->func_addr = resolve_func_addr(addr);
 
 	ret = warn_kprobe_rereg(p);
 	if (ret)
kernel/trace/bpf_trace.c +1 -1
index 21aa30644219..25631253084a 100644
@@ -1026,7 +1026,7 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
 {
 	struct kprobe *kp = kprobe_running();
 
-	return kp ? (uintptr_t)kp->addr : 0;
+	return kp ? (uintptr_t)kp->func_addr : 0;
 }
 
 static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
tools/testing/selftests/bpf/progs/get_func_ip_test.c +2 -2
index a587aeca5ae0..e988aefa567e 100644
@@ -69,7 +69,7 @@ int test6(struct pt_regs *ctx)
 {
 	__u64 addr = bpf_get_func_ip(ctx);
 
-	test6_result = (const void *) addr == &bpf_fentry_test6 + 5;
+	test6_result = (const void *) addr == &bpf_fentry_test6;
 	return 0;
 }
 
@@ -79,6 +79,6 @@ int test7(struct pt_regs *ctx)
 {
 	__u64 addr = bpf_get_func_ip(ctx);
 
-	test7_result = (const void *) addr == &bpf_fentry_test7 + 5;
+	test7_result = (const void *) addr == &bpf_fentry_test7;
 	return 0;
 }
Signature · 1 line
2.33.1
author
#7 lore raw

PATCH 3/13 kprobe: Add support to register multiple ftrace kprobes

In reply to Jiri Olsa

Adding support to register kprobe on multiple addresses within single kprobe object instance.

It uses the CONFIG_KPROBES_ON_FTRACE feature (so it's only available for function entry addresses) and separated ftrace_ops object placed directly in the kprobe object.

There's new CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE config option, enabled for archs that support multi kprobes.

It registers all the provided addresses in the ftrace_ops object filter and adds separate ftrace_ops callback.

To register multi kprobe user provides array of addresses or symbols with their count, like:

  struct kprobe kp = {};

  kp.multi.symbols = (const char **) symbols;
  kp.multi.cnt = cnt;
  ...

  err = register_kprobe(&kp);
---
 arch/Kconfig                     |   3 +
 arch/x86/Kconfig                 |   1 +
 arch/x86/kernel/kprobes/ftrace.c |  48 ++++++--
 include/linux/kprobes.h          |  25 ++++
 kernel/kprobes.c                 | 204 +++++++++++++++++++++++++------
 5 files changed, 235 insertions(+), 46 deletions(-)
arch/Kconfig +3 -0
index d3c4ab249e9c..0131636e1ef8 100644
@@ -191,6 +191,9 @@ config HAVE_OPTPROBES
 config HAVE_KPROBES_ON_FTRACE
 	bool
 
+config HAVE_KPROBES_MULTI_ON_FTRACE
+	bool
+
 config ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
 	bool
 	help
arch/x86/Kconfig +1 -0
index 5c2ccb85f2ef..0c870238016a 100644
@@ -217,6 +217,7 @@ config X86
 	select HAVE_KERNEL_ZSTD
 	select HAVE_KPROBES
 	select HAVE_KPROBES_ON_FTRACE
+	select HAVE_KPROBES_MULTI_ON_FTRACE
 	select HAVE_FUNCTION_ERROR_INJECTION
 	select HAVE_KRETPROBES
 	select HAVE_KVM
arch/x86/kernel/kprobes/ftrace.c +36 -12

69 lines not on this page · raw message

include/linux/kprobes.h +25 -0
index a204df4fef96..03fd86ef69cb 100644
@@ -68,6 +68,16 @@ struct kprobe {
 	/* location of the probe point */
 	kprobe_opcode_t *addr;
 
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	/* location of the multi probe points */
+	struct {
+		const char **symbols;
+		kprobe_opcode_t **addrs;
+		unsigned int cnt;
+		struct ftrace_ops ops;
+	} multi;
+#endif
+
 	/* Allow user to indicate symbol name of the probe point */
 	const char *symbol_name;
 
@@ -105,6 +115,7 @@ struct kprobe {
 				   * this flag is only for optimized_kprobe.
 				   */
 #define KPROBE_FLAG_FTRACE	8 /* probe is using ftrace */
+#define KPROBE_FLAG_MULTI      16 /* probe multi addresses */
 
 /* Has this kprobe gone ? */
 static inline bool kprobe_gone(struct kprobe *p)
@@ -130,6 +141,18 @@ static inline bool kprobe_ftrace(struct kprobe *p)
 	return p->flags & KPROBE_FLAG_FTRACE;
 }
 
+/* Is this ftrace multi kprobe ? */
+static inline bool kprobe_ftrace_multi(struct kprobe *p)
+{
+	return kprobe_ftrace(p) && (p->flags & KPROBE_FLAG_MULTI);
+}
+
+/* Is this single kprobe ? */
+static inline bool kprobe_single(struct kprobe *p)
+{
+	return !(p->flags & KPROBE_FLAG_MULTI);
+}
+
 /*
  * Function-return probe -
  * Note:
@@ -365,6 +388,8 @@ static inline void wait_for_kprobe_optimizer(void) { }
 #ifdef CONFIG_KPROBES_ON_FTRACE
 extern void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
 				  struct ftrace_ops *ops, struct ftrace_regs *fregs);
+extern void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip,
+					struct ftrace_ops *ops, struct ftrace_regs *fregs);
 extern int arch_prepare_kprobe_ftrace(struct kprobe *p);
 #else
 static inline int arch_prepare_kprobe_ftrace(struct kprobe *p)
kernel/kprobes.c +170 -34

294 lines not on this page · raw message

Signature · 1 line
2.33.1
author
#9 lore raw

PATCH 4/13 kprobe: Add support to register multiple ftrace kretprobes

In reply to Jiri Olsa

Adding support to register kretprobe on multiple addresses within single kprobe object instance.

The interface stays same as for kprobes, the patch just adds blacklist check for each address, which is special for kretprobes.

---
 include/linux/kprobes.h |  1 +
 kernel/kprobes.c        | 46 ++++++++++++++++++++++++++++++++---------
 2 files changed, 37 insertions(+), 10 deletions(-)
include/linux/kprobes.h +1 -0
index 03fd86ef69cb..a31da6202b5c 100644
@@ -75,6 +75,7 @@ struct kprobe {
 		kprobe_opcode_t **addrs;
 		unsigned int cnt;
 		struct ftrace_ops ops;
+		bool check_kretprobe;
 	} multi;
 #endif
 
kernel/kprobes.c +36 -10

87 lines not on this page · raw message

Signature · 1 line
2.33.1
author
#10 lore raw

PATCH 5/13 kprobe: Allow to get traced function address for multi ftrace kprobes

In reply to Jiri Olsa

The current bpf_get_func_ip_kprobe helper does not work properly, when used in ebpf program triggered by the new multi kprobes.

We can't use kprobe's func_addr in bpf_get_func_ip_kprobe helper, because there are multiple functions registered for single kprobe object.

Adding new per cpu variable current_ftrace_multi_addr and extra address in kretprobe_instance object to keep current traced function address for each cpu for both kprobe handler and kretprobe trampoline.

The address value is set/passed as follows, for kprobe:

  kprobe_ftrace_multi_handler
  {
    old = kprobe_ftrace_multi_addr_set(ip);
    handler..
    kprobe_ftrace_multi_addr_set(old);
  }

For kretprobe:

  kprobe_ftrace_multi_handler
  {
    old = kprobe_ftrace_multi_addr_set(ip);
    ...
      pre_handler_kretprobe
      {
        ri->ftrace_multi_addr = kprobe_ftrace_multi_addr
      }
    ...
    kprobe_ftrace_multi_addr_set(old);
  }

  __kretprobe_trampoline_handler
  {
    prev_func_addr = kprobe_ftrace_multi_addr_set(ri->ftrace_multi_addr);
    handler..
    kprobe_ftrace_multi_addr_set(prev_func_addr);
  }
---
 arch/x86/kernel/kprobes/ftrace.c |  3 +++
 include/linux/kprobes.h          | 26 ++++++++++++++++++++++++++
 kernel/kprobes.c                 |  6 ++++++
 kernel/trace/bpf_trace.c         |  7 ++++++-
 4 files changed, 41 insertions(+), 1 deletion(-)
arch/x86/kernel/kprobes/ftrace.c +3 -0
index ac4d256b89c6..8caaa58c3a64 100644
@@ -72,6 +72,7 @@ NOKPROBE_SYMBOL(kprobe_ftrace_handler);
 void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip,
 				 struct ftrace_ops *ops, struct ftrace_regs *fregs)
 {
+	unsigned long old;
 	struct kprobe *p;
 	int bit;
 
@@ -79,8 +80,10 @@ void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip,
 	if (bit < 0)
 		return;
 
+	old = kprobe_ftrace_multi_addr_set(ip);
 	p = container_of(ops, struct kprobe, multi.ops);
 	ftrace_handler(p, ip, fregs);
+	kprobe_ftrace_multi_addr_set(old);
 
 	ftrace_test_recursion_unlock(bit);
 }
include/linux/kprobes.h +26 -0
index a31da6202b5c..3f0522b9538b 100644
@@ -191,6 +191,7 @@ struct kretprobe_instance {
 	struct kretprobe_holder *rph;
 	kprobe_opcode_t *ret_addr;
 	void *fp;
+	unsigned long ftrace_multi_addr;
 	char data[];
 };
 
@@ -387,16 +388,37 @@ static inline void wait_for_kprobe_optimizer(void) { }
 #endif /* CONFIG_OPTPROBES */
 
 #ifdef CONFIG_KPROBES_ON_FTRACE
+DECLARE_PER_CPU(unsigned long, current_ftrace_multi_addr);
 extern void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
 				  struct ftrace_ops *ops, struct ftrace_regs *fregs);
 extern void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip,
 					struct ftrace_ops *ops, struct ftrace_regs *fregs);
 extern int arch_prepare_kprobe_ftrace(struct kprobe *p);
+
+static inline unsigned long kprobe_ftrace_multi_addr(void)
+{
+	return __this_cpu_read(current_ftrace_multi_addr);
+}
+static inline unsigned long kprobe_ftrace_multi_addr_set(unsigned long addr)
+{
+	unsigned long old = __this_cpu_read(current_ftrace_multi_addr);
+
+	__this_cpu_write(current_ftrace_multi_addr, addr);
+	return old;
+}
 #else
 static inline int arch_prepare_kprobe_ftrace(struct kprobe *p)
 {
 	return -EINVAL;
 }
+static inline unsigned long kprobe_ftrace_multi_addr_set(unsigned long addr)
+{
+	return 0;
+}
+static inline unsigned long kprobe_ftrace_multi_addr(void)
+{
+	return 0;
+}
 #endif /* CONFIG_KPROBES_ON_FTRACE */
 
 /* Get the kprobe at this addr (if any) - called with preemption disabled */
@@ -514,6 +536,10 @@ static inline int kprobe_get_kallsym(unsigned int symnum, unsigned long *value,
 {
 	return -ERANGE;
 }
+static inline unsigned long kprobe_ftrace_multi_addr(void)
+{
+	return 0;
+}
 #endif /* CONFIG_KPROBES */
 
 static inline int disable_kretprobe(struct kretprobe *rp)
kernel/kprobes.c +6 -0
index 04fc411ca30c..6ba249f3a0cb 100644
@@ -1064,6 +1064,8 @@ static bool in_kretprobe_blacklist(void *addr)
 }
 
 #ifdef CONFIG_KPROBES_ON_FTRACE
+DEFINE_PER_CPU(unsigned long, current_ftrace_multi_addr);
+
 static struct ftrace_ops kprobe_ftrace_ops __read_mostly = {
 	.func = kprobe_ftrace_handler,
 	.flags = FTRACE_OPS_FL_SAVE_REGS,
@@ -2106,11 +2108,14 @@ unsigned long __kretprobe_trampoline_handler(struct pt_regs *regs,
 		rp = get_kretprobe(ri);
 		if (rp && rp->handler) {
 			struct kprobe *prev = kprobe_running();
+			unsigned long prev_func_addr;
 
+			prev_func_addr = kprobe_ftrace_multi_addr_set(ri->ftrace_multi_addr);
 			__this_cpu_write(current_kprobe, &rp->kp);
 			ri->ret_addr = correct_ret_addr;
 			rp->handler(ri, regs);
 			__this_cpu_write(current_kprobe, prev);
+			kprobe_ftrace_multi_addr_set(prev_func_addr);
 		}
 		if (first == node)
 			break;
@@ -2161,6 +2166,7 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs)
 	}
 
 	arch_prepare_kretprobe(ri, regs);
+	ri->ftrace_multi_addr = kprobe_ftrace_multi_addr();
 
 	__llist_add(&ri->llist, &current->kretprobe_instances);
 
kernel/trace/bpf_trace.c +6 -1
index 25631253084a..39f4d476cfca 100644
@@ -1026,7 +1026,12 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
 {
 	struct kprobe *kp = kprobe_running();
 
-	return kp ? (uintptr_t)kp->func_addr : 0;
+	if (!kp)
+		return 0;
+	if (kprobe_ftrace_multi(kp))
+		return (uintptr_t) kprobe_ftrace_multi_addr();
+	else
+		return (uintptr_t) kp->func_addr;
 }
 
 static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
Signature · 1 line
2.33.1
author
#12 lore raw

PATCH 6/13 samples/kprobes: Add support for multi kprobe interface

In reply to Jiri Olsa

Adding support to test multi kprobe interface. It's now possible to register multiple functions for the module handler, like:

  # modprobe kprobe_example symbol='sched_fork,kernel_clone'
---
 samples/kprobes/kprobe_example.c | 47 +++++++++++++++++++++++++++++---
 1 file changed, 43 insertions(+), 4 deletions(-)
samples/kprobes/kprobe_example.c +43 -4
index f991a66b5b02..4ae31184f098 100644
@@ -15,8 +15,10 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/kprobes.h>
+#include <linux/slab.h>
+#include <linux/string.h>
 
-#define MAX_SYMBOL_LEN	64
+#define MAX_SYMBOL_LEN	4096
 static char symbol[MAX_SYMBOL_LEN] = "kernel_clone";
 module_param_string(symbol, symbol, sizeof(symbol), 0644);
 
@@ -97,17 +99,54 @@ static void __kprobes handler_post(struct kprobe *p, struct pt_regs *regs,
 
 static int __init kprobe_init(void)
 {
+	char **symbols = NULL;
 	int ret;
+
 	kp.pre_handler = handler_pre;
 	kp.post_handler = handler_post;
 
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	if (strchr(symbol, ',')) {
+		char *p, *tmp;
+		int cnt;
+
+		tmp = kstrdup(symbol, GFP_KERNEL);
+		if (!tmp)
+			return -ENOMEM;
+
+		p = strchr(tmp, ',');
+		while (p) {
+			*p = ' ';
+			p = strchr(p + 1, ',');
+		}
+
+		symbols = argv_split(GFP_KERNEL, tmp, &cnt);
+		kfree(tmp);
+		if (!symbols) {
+			ret = -ENOMEM;
+			goto out;
+		}
+
+		kp.multi.symbols = (const char **) symbols;
+		kp.multi.cnt = cnt;
+	}
+#endif
+
 	ret = register_kprobe(&kp);
 	if (ret < 0) {
 		pr_err("register_kprobe failed, returned %d\n", ret);
-		return ret;
+		goto out;
 	}
-	pr_info("Planted kprobe at %p\n", kp.addr);
-	return 0;
+
+	if (symbols)
+		pr_info("Planted multi kprobe to %s\n", symbol);
+	else
+		pr_info("Planted kprobe at %p\n", kp.addr);
+
+out:
+	if (symbols)
+		argv_free(symbols);
+	return ret;
 }
 
 static void __exit kprobe_exit(void)
Signature · 1 line
2.33.1
author
#13 lore raw

PATCH 7/13 samples/kprobes: Add support for multi kretprobe interface

In reply to Jiri Olsa

Adding support to test multi kprobe interface. It's now possible to register multiple functions for the module handler, like:

  # modprobe kretprobe_example.ko func='sched_fork,kernel_clone'
---
 samples/kprobes/kretprobe_example.c | 43 +++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)
samples/kprobes/kretprobe_example.c +41 -2
index 228321ecb161..2181cf0d6e4a 100644
@@ -25,6 +25,8 @@
 #include <linux/ktime.h>
 #include <linux/limits.h>
 #include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/string.h>
 
 static char func_name[NAME_MAX] = "kernel_clone";
 module_param_string(func, func_name, NAME_MAX, S_IRUGO);
@@ -80,17 +82,54 @@ static struct kretprobe my_kretprobe = {
 
 static int __init kretprobe_init(void)
 {
+	char **symbols = NULL;
 	int ret;
 
 	my_kretprobe.kp.symbol_name = func_name;
+
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	if (strchr(func_name, ',')) {
+		char *p, *tmp;
+		int cnt;
+
+		tmp = kstrdup(func_name, GFP_KERNEL);
+		if (!tmp)
+			return -ENOMEM;
+
+		p = strchr(tmp, ',');
+		while (p) {
+			*p = ' ';
+			p = strchr(p + 1, ',');
+		}
+
+		symbols = argv_split(GFP_KERNEL, tmp, &cnt);
+		kfree(tmp);
+		if (!symbols) {
+			ret = -ENOMEM;
+			goto out;
+		}
+
+		my_kretprobe.kp.multi.symbols = (const char **) symbols;
+		my_kretprobe.kp.multi.cnt = cnt;
+	}
+#endif
 	ret = register_kretprobe(&my_kretprobe);
 	if (ret < 0) {
 		pr_err("register_kretprobe failed, returned %d\n", ret);
 		return ret;
 	}
-	pr_info("Planted return probe at %s: %p\n",
+
+	if (symbols) {
+		pr_info("Planted multi return kprobe to %s\n", func_name);
+	} else {
+		pr_info("Planted return probe at %s: %p\n",
 			my_kretprobe.kp.symbol_name, my_kretprobe.kp.addr);
-	return 0;
+	}
+
+out:
+	if (symbols)
+		argv_free(symbols);
+	return ret;
 }
 
 static void __exit kretprobe_exit(void)
Signature · 1 line
2.33.1
author
#14 lore raw

PATCH 8/13 bpf: Add kprobe link for attaching raw kprobes

In reply to Jiri Olsa

Adding new link type BPF_LINK_TYPE_KPROBE to attach kprobes directly through register_kprobe/kretprobe API.

Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program.

The new link allows to create multiple kprobes link by using new link_create interface:

  struct {
    __aligned_u64   addrs;
    __u32           cnt;
    __u64           bpf_cookie;
  } kprobe;

Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe.

---
 include/linux/bpf_types.h      |   1 +
 include/uapi/linux/bpf.h       |  12 +++
 kernel/bpf/syscall.c           | 191 ++++++++++++++++++++++++++++++++-
 tools/include/uapi/linux/bpf.h |  12 +++
 4 files changed, 211 insertions(+), 5 deletions(-)
include/linux/bpf_types.h +1 -0
index 48a91c51c015..a9000feab34e 100644
@@ -140,3 +140,4 @@ BPF_LINK_TYPE(BPF_LINK_TYPE_XDP, xdp)
 #ifdef CONFIG_PERF_EVENTS
 BPF_LINK_TYPE(BPF_LINK_TYPE_PERF_EVENT, perf)
 #endif
+BPF_LINK_TYPE(BPF_LINK_TYPE_KPROBE, kprobe)
include/uapi/linux/bpf.h +12 -0
index b0383d371b9a..5216b333c688 100644
@@ -995,6 +995,7 @@ enum bpf_attach_type {
 	BPF_SK_REUSEPORT_SELECT,
 	BPF_SK_REUSEPORT_SELECT_OR_MIGRATE,
 	BPF_PERF_EVENT,
+	BPF_TRACE_RAW_KPROBE,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1009,6 +1010,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_NETNS = 5,
 	BPF_LINK_TYPE_XDP = 6,
 	BPF_LINK_TYPE_PERF_EVENT = 7,
+	BPF_LINK_TYPE_KPROBE = 8,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE		(1U << 4)
 
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1465,6 +1472,11 @@ union bpf_attr {
 				 */
 				__u64		bpf_cookie;
 			} perf_event;
+			struct {
+				__aligned_u64	addrs;
+				__u32		cnt;
+				__u64		bpf_cookie;
+			} kprobe;
 		};
 	} link_create;
 
kernel/bpf/syscall.c +186 -5

233 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +12 -0
index b0383d371b9a..5216b333c688 100644
@@ -995,6 +995,7 @@ enum bpf_attach_type {
 	BPF_SK_REUSEPORT_SELECT,
 	BPF_SK_REUSEPORT_SELECT_OR_MIGRATE,
 	BPF_PERF_EVENT,
+	BPF_TRACE_RAW_KPROBE,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1009,6 +1010,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_NETNS = 5,
 	BPF_LINK_TYPE_XDP = 6,
 	BPF_LINK_TYPE_PERF_EVENT = 7,
+	BPF_LINK_TYPE_KPROBE = 8,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE		(1U << 4)
 
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1465,6 +1472,11 @@ union bpf_attr {
 				 */
 				__u64		bpf_cookie;
 			} perf_event;
+			struct {
+				__aligned_u64	addrs;
+				__u32		cnt;
+				__u64		bpf_cookie;
+			} kprobe;
 		};
 	} link_create;
 
Signature · 1 line
2.33.1
author
#19 lore raw

PATCH 9/13 libbpf: Add libbpf__kallsyms_parse function

In reply to Jiri Olsa

Move the kallsyms parsing in internal libbpf__kallsyms_parse function, so it can be used from other places.

It will be used in following changes.

---
 tools/lib/bpf/libbpf.c          | 62 ++++++++++++++++++++-------------
 tools/lib/bpf/libbpf_internal.h |  5 +++
 2 files changed, 43 insertions(+), 24 deletions(-)
tools/lib/bpf/libbpf.c +38 -24
index 9cb99d1e2385..25512b4dbc8c 100644
@@ -7146,12 +7146,10 @@ static int bpf_object__sanitize_maps(struct bpf_object *obj)
 	return 0;
 }
 
-static int bpf_object__read_kallsyms_file(struct bpf_object *obj)
+int libbpf__kallsyms_parse(void *arg, kallsyms_cb_t cb)
 {
 	char sym_type, sym_name[500];
 	unsigned long long sym_addr;
-	const struct btf_type *t;
-	struct extern_desc *ext;
 	int ret, err = 0;
 	FILE *f;
 
@@ -7170,35 +7168,51 @@ static int bpf_object__read_kallsyms_file(struct bpf_object *obj)
 		if (ret != 3) {
 			pr_warn("failed to read kallsyms entry: %d\n", ret);
 			err = -EINVAL;
-			goto out;
+			break;
 		}
 
-		ext = find_extern_by_name(obj, sym_name);
-		if (!ext || ext->type != EXT_KSYM)
-			continue;
-
-		t = btf__type_by_id(obj->btf, ext->btf_id);
-		if (!btf_is_var(t))
-			continue;
-
-		if (ext->is_set && ext->ksym.addr != sym_addr) {
-			pr_warn("extern (ksym) '%s' resolution is ambiguous: 0x%llx or 0x%llx\n",
-				sym_name, ext->ksym.addr, sym_addr);
-			err = -EINVAL;
-			goto out;
-		}
-		if (!ext->is_set) {
-			ext->is_set = true;
-			ext->ksym.addr = sym_addr;
-			pr_debug("extern (ksym) %s=0x%llx\n", sym_name, sym_addr);
-		}
+		err = cb(arg, sym_addr, sym_type, sym_name);
+		if (err)
+			break;
 	}
 
-out:
 	fclose(f);
 	return err;
 }
 
+static int kallsyms_cb(void *arg, unsigned long long sym_addr,
+		       char sym_type, const char *sym_name)
+{
+	struct bpf_object *obj = arg;
+	const struct btf_type *t;
+	struct extern_desc *ext;
+
+	ext = find_extern_by_name(obj, sym_name);
+	if (!ext || ext->type != EXT_KSYM)
+		return 0;
+
+	t = btf__type_by_id(obj->btf, ext->btf_id);
+	if (!btf_is_var(t))
+		return 0;
+
+	if (ext->is_set && ext->ksym.addr != sym_addr) {
+		pr_warn("extern (ksym) '%s' resolution is ambiguous: 0x%llx or 0x%llx\n",
+			sym_name, ext->ksym.addr, sym_addr);
+		return -EINVAL;
+	}
+	if (!ext->is_set) {
+		ext->is_set = true;
+		ext->ksym.addr = sym_addr;
+		pr_debug("extern (ksym) %s=0x%llx\n", sym_name, sym_addr);
+	}
+	return 0;
+}
+
+static int bpf_object__read_kallsyms_file(struct bpf_object *obj)
+{
+	return libbpf__kallsyms_parse(obj, kallsyms_cb);
+}
+
 static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name,
 			    __u16 kind, struct btf **res_btf,
 			    struct module_btf **res_mod_btf)
tools/lib/bpf/libbpf_internal.h +5 -0
index 1565679eb432..dc544d239509 100644
@@ -446,6 +446,11 @@ __s32 btf__find_by_name_kind_own(const struct btf *btf, const char *type_name,
 
 extern enum libbpf_strict_mode libbpf_mode;
 
+typedef int (*kallsyms_cb_t)(void *arg, unsigned long long sym_addr,
+			     char sym_type, const char *sym_name);
+
+int libbpf__kallsyms_parse(void *arg, kallsyms_cb_t cb);
+
 /* handle direct returned errors */
 static inline int libbpf_err(int ret)
 {
Signature · 1 line
2.33.1
author
#20 lore raw

PATCH 10/13 libbpf: Add bpf_link_create support for multi kprobes

In reply to Jiri Olsa

Adding new kprobe struct in bpf_link_create_opts object to pass multi kprobe data to link_create attr API.

---
 tools/lib/bpf/bpf.c | 5 +++++
 tools/lib/bpf/bpf.h | 7 ++++++-
 2 files changed, 11 insertions(+), 1 deletion(-)
tools/lib/bpf/bpf.c +5 -0
index 9b64eed2b003..40cad575ad62 100644
@@ -848,6 +848,11 @@ int bpf_link_create(int prog_fd, int target_fd,
 		if (!OPTS_ZEROED(opts, perf_event))
 			return libbpf_err(-EINVAL);
 		break;
+	case BPF_TRACE_RAW_KPROBE:
+		attr.link_create.kprobe.addrs = OPTS_GET(opts, kprobe.addrs, 0);
+		attr.link_create.kprobe.cnt = OPTS_GET(opts, kprobe.cnt, 0);
+		attr.link_create.kprobe.bpf_cookie = OPTS_GET(opts, kprobe.bpf_cookie, 0);
+		break;
 	default:
 		if (!OPTS_ZEROED(opts, flags))
 			return libbpf_err(-EINVAL);
tools/lib/bpf/bpf.h +6 -1
index 00619f64a040..9611023138b1 100644
@@ -298,10 +298,15 @@ struct bpf_link_create_opts {
 		struct {
 			__u64 bpf_cookie;
 		} perf_event;
+		struct {
+			__u64 addrs;
+			__u32 cnt;
+			__u64 bpf_cookie;
+		} kprobe;
 	};
 	size_t :0;
 };
-#define bpf_link_create_opts__last_field perf_event
+#define bpf_link_create_opts__last_field kprobe.bpf_cookie
 
 LIBBPF_API int bpf_link_create(int prog_fd, int target_fd,
 			       enum bpf_attach_type attach_type,
Signature · 1 line
2.33.1
author
#21 lore raw

PATCH 11/13 libbpf: Add bpf_program__attach_kprobe_opts for multi kprobes

In reply to Jiri Olsa

SEC("raw_kprobe/bpf_fentry_test*")

to attach program to all bpf_fentry_test* functions.

---
 tools/lib/bpf/libbpf.c | 124 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 123 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf.c +123 -1
index 25512b4dbc8c..0061ab02fc5a 100644
@@ -10000,6 +10000,125 @@ static int perf_event_kprobe_open_legacy(const char *probe_name, bool retprobe,
 	return pfd;
 }
 
+struct kprobe_resolve_raw_kprobe {
+	const char *name;
+	__u64 *addrs;
+	__u32 alloc;
+	__u32 cnt;
+};
+
+static bool glob_matches(const char *glob, const char *s)
+{
+	int n = strlen(glob);
+
+	if (n == 1 && glob[0] == '*')
+		return true;
+
+	if (glob[0] == '*' && glob[n - 1] == '*') {
+		const char *subs;
+		/* substring match */
+
+		/* this is hacky, but we don't want to allocate
+		 * for no good reason
+		 */
+		((char *)glob)[n - 1] = '\0';
+		subs = strstr(s, glob + 1);
+		((char *)glob)[n - 1] = '*';
+
+		return subs != NULL;
+	} else if (glob[0] == '*') {
+		size_t nn = strlen(s);
+		/* suffix match */
+
+		/* too short for a given suffix */
+		if (nn < n - 1)
+			return false;
+		return strcmp(s + nn - (n - 1), glob + 1) == 0;
+	} else if (glob[n - 1] == '*') {
+		/* prefix match */
+		return strncmp(s, glob, n - 1) == 0;
+	} else {
+		/* exact match */
+		return strcmp(glob, s) == 0;
+	}
+}
+
+static int
+kprobe_resolve_raw_kprobe_cb(void *arg, unsigned long long sym_addr,
+			     char sym_type, const char *sym_name)
+{
+	struct kprobe_resolve_raw_kprobe *res = arg;
+	__u64 *p;
+
+	if (!glob_matches(res->name, sym_name))
+		return 0;
+
+	if (res->cnt == res->alloc) {
+		res->alloc = max((__u32) 16, res->alloc * 3 / 2);
+		p = libbpf_reallocarray(res->addrs, res->alloc, sizeof(__u32));
+		if (!p)
+			return -ENOMEM;
+		res->addrs = p;
+	}
+	res->addrs[res->cnt++] = sym_addr;
+	return 0;
+}
+
+static struct bpf_link *
+attach_raw_kprobe_opts(const struct bpf_program *prog,
+		       const char *func_name,
+		       const struct bpf_kprobe_opts *kopts)
+{
+	DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts);
+	struct kprobe_resolve_raw_kprobe res = {
+		.name = func_name,
+	};
+	struct bpf_link *link = NULL;
+	char errmsg[STRERR_BUFSIZE];
+	int err, link_fd, prog_fd;
+	__u64 bpf_cookie;
+	bool retprobe;
+
+	err = libbpf__kallsyms_parse(&res, kprobe_resolve_raw_kprobe_cb);
+	if (err)
+		goto error;
+	if (!res.cnt) {
+		err = -ENOENT;
+		goto error;
+	}
+
+	retprobe = OPTS_GET(kopts, retprobe, false);
+	bpf_cookie = OPTS_GET(kopts, bpf_cookie, 0);
+
+	opts.kprobe.addrs = (__u64) res.addrs;
+	opts.kprobe.cnt = res.cnt;
+	opts.kprobe.bpf_cookie = bpf_cookie;
+	opts.flags = retprobe ? BPF_F_KPROBE_RETURN : 0;
+
+	link = calloc(1, sizeof(*link));
+	if (!link)
+		return libbpf_err_ptr(-ENOMEM);
+	link->detach = &bpf_link__detach_fd;
+
+	prog_fd = bpf_program__fd(prog);
+	link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts);
+	if (link_fd < 0) {
+		err = -errno;
+		pr_warn("prog '%s': failed to attach to %s: %s\n",
+			prog->name, res.name,
+			libbpf_strerror_r(err, errmsg, sizeof(errmsg)));
+		goto error;
+	}
+	link->fd = link_fd;
+	free(res.addrs);
+	return link;
+
+error:
+	free(link);
+	free(res.addrs);
+	return libbpf_err_ptr(err);
+}
+
 struct bpf_link *
 bpf_program__attach_kprobe_opts(const struct bpf_program *prog,
 				const char *func_name,
@@ -10016,6 +10135,9 @@ bpf_program__attach_kprobe_opts(const struct bpf_program *prog,
 	if (!OPTS_VALID(opts, bpf_kprobe_opts))
 		return libbpf_err_ptr(-EINVAL);
 
+	if (strchr(func_name, '*'))
+		return attach_raw_kprobe_opts(prog, func_name, opts);
+
 	retprobe = OPTS_GET(opts, retprobe, false);
 	offset = OPTS_GET(opts, offset, 0);
 	pe_opts.bpf_cookie = OPTS_GET(opts, bpf_cookie, 0);
@@ -10096,7 +10218,7 @@ static struct bpf_link *attach_kprobe(const struct bpf_program *prog, long cooki
 	else
 		func_name = prog->sec_name + sizeof("kprobe/") - 1;
 
-	n = sscanf(func_name, "%m[a-zA-Z0-9_.]+%li", &func, &offset);
+	n = sscanf(func_name, "%m[a-zA-Z0-9_.*]+%li", &func, &offset);
 	if (n < 1) {
 		err = -EINVAL;
 		pr_warn("kprobe name is invalid: %s\n", func_name);
Signature · 1 line
2.33.1
author
#22 lore raw

PATCH 12/13 selftest/bpf: Add raw kprobe attach test

In reply to Jiri Olsa

Adding raw kprobe attach test that uses new interface to attach multiple kprobes with new kprobe link.

The test is attaching to bpf_fentry_test* functions and single trampoline program to use bpf_prog_test_run to trigger bpf_fentry_test* functions.

---
 .../bpf/prog_tests/raw_kprobe_test.c          | 92 +++++++++++++++++++
 .../testing/selftests/bpf/progs/raw_kprobe.c  | 58 ++++++++++++
 2 files changed, 150 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c
 create mode 100644 tools/testing/selftests/bpf/progs/raw_kprobe.c
tools/testing/selftests/bpf/prog_tests/raw_kprobe_test.c new file +92 -0
new file mode 100644
index 000000000000..5ade44c57c9e
@@ -0,0 +1,92 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include "raw_kprobe.skel.h"
+#include "trace_helpers.h"
+
+static void test_skel_api(void)
+{
+	struct raw_kprobe *skel = NULL;
+	__u32 duration = 0, retval;
+	int err, prog_fd;
+
+	skel = raw_kprobe__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "raw_kprobe__open_and_load"))
+		goto cleanup;
+
+	err = raw_kprobe__attach(skel);
+	if (!ASSERT_OK(err, "raw_kprobe__attach"))
+		goto cleanup;
+
+	prog_fd = bpf_program__fd(skel->progs.test1);
+	err = bpf_prog_test_run(prog_fd, 1, NULL, 0,
+				NULL, NULL, &retval, &duration);
+	ASSERT_OK(err, "test_run");
+	ASSERT_EQ(retval, 0, "test_run");
+
+	ASSERT_EQ(skel->bss->test2_result, 8, "test2_result");
+	ASSERT_EQ(skel->bss->test3_result, 8, "test3_result");
+
+cleanup:
+	raw_kprobe__destroy(skel);
+}
+
+static void test_link_api(void)
+{
+	DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts);
+	int err, prog_fd, link1_fd = -1, link2_fd = -1;
+	struct raw_kprobe *skel = NULL;
+	__u32 duration = 0, retval;
+	__u64 addrs[8];
+
+	skel = raw_kprobe__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "fentry_raw_skel_load"))
+		goto cleanup;
+
+	kallsyms_find("bpf_fentry_test1", &addrs[0]);
+	kallsyms_find("bpf_fentry_test2", &addrs[1]);
+	kallsyms_find("bpf_fentry_test3", &addrs[2]);
+	kallsyms_find("bpf_fentry_test4", &addrs[3]);
+	kallsyms_find("bpf_fentry_test5", &addrs[4]);
+	kallsyms_find("bpf_fentry_test6", &addrs[5]);
+	kallsyms_find("bpf_fentry_test7", &addrs[6]);
+	kallsyms_find("bpf_fentry_test8", &addrs[7]);
+
+	opts.kprobe.addrs = (__u64) addrs;
+	opts.kprobe.cnt = 8;
+
+	prog_fd = bpf_program__fd(skel->progs.test2);
+	link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts);
+	if (!ASSERT_GE(link1_fd, 0, "link_fd"))
+		goto cleanup;
+
+	opts.flags = BPF_F_KPROBE_RETURN;
+	prog_fd = bpf_program__fd(skel->progs.test3);
+	link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts);
+	if (!ASSERT_GE(link2_fd, 0, "link_fd"))
+		goto cleanup;
+
+	skel->bss->test2_result = 0;
+	skel->bss->test3_result = 0;
+
+	prog_fd = bpf_program__fd(skel->progs.test1);
+	err = bpf_prog_test_run(prog_fd, 1, NULL, 0,
+				NULL, NULL, &retval, &duration);
+	ASSERT_OK(err, "test_run");
+	ASSERT_EQ(retval, 0, "test_run");
+
+	ASSERT_EQ(skel->bss->test2_result, 8, "test2_result");
+	ASSERT_EQ(skel->bss->test3_result, 8, "test3_result");
+
+cleanup:
+	if (link1_fd != -1)
+		close(link1_fd);
+	if (link2_fd != -1)
+		close(link2_fd);
+	raw_kprobe__destroy(skel);
+}
+
+void test_raw_kprobe_test(void)
+{
+	test_skel_api();
+	test_link_api();
+}
tools/testing/selftests/bpf/progs/raw_kprobe.c new file +58 -0

58 lines not on this page · raw message

Signature · 1 line
2.33.1
author
#23 lore raw

PATCH 13/13 selftest/bpf: Add bpf_cookie test for raw_k[ret]probe

In reply to Jiri Olsa

Adding bpf_cookie test for raw_k[ret]probes.

Using the bpf_link_create interface directly and single trampoline program to trigger the bpf_fentry_test1 execution.

---
 .../selftests/bpf/prog_tests/bpf_cookie.c     | 42 +++++++++++++++++++
 .../selftests/bpf/progs/test_bpf_cookie.c     | 24 ++++++++++-
 2 files changed, 65 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +42 -0
index 5eea3c3a40fe..aee01dd3a823 100644
@@ -57,6 +57,46 @@ static void kprobe_subtest(struct test_bpf_cookie *skel)
 	bpf_link__destroy(retlink2);
 }
 
+static void rawkprobe_subtest(struct test_bpf_cookie *skel)
+{
+	DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts);
+	int err, prog_fd, link1_fd = -1, link2_fd = -1;
+	__u32 duration = 0, retval;
+	__u64 addr;
+
+	kallsyms_find("bpf_fentry_test1", &addr);
+
+	opts.kprobe.addrs = (__u64) &addr;
+	opts.kprobe.cnt = 1;
+	opts.kprobe.bpf_cookie = 0x1;
+	prog_fd = bpf_program__fd(skel->progs.handle_raw_kprobe);
+
+	link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts);
+	if (!ASSERT_GE(link1_fd, 0, "link1_fd"))
+		return;
+
+	opts.flags = BPF_F_KPROBE_RETURN;
+	opts.kprobe.bpf_cookie = 0x2;
+	prog_fd = bpf_program__fd(skel->progs.handle_raw_kretprobe);
+
+	link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_RAW_KPROBE, &opts);
+	if (!ASSERT_GE(link2_fd, 0, "link2_fd"))
+		goto cleanup;
+
+	prog_fd = bpf_program__fd(skel->progs.raw_trigger);
+	err = bpf_prog_test_run(prog_fd, 1, NULL, 0,
+				NULL, NULL, &retval, &duration);
+	ASSERT_OK(err, "test_run");
+	ASSERT_EQ(retval, 0, "test_run");
+
+	ASSERT_EQ(skel->bss->raw_kprobe_res, 0x1, "raw_kprobe_res");
+	ASSERT_EQ(skel->bss->raw_kretprobe_res, 0x2, "raw_kretprobe_res");
+
+cleanup:
+	close(link1_fd);
+	close(link2_fd);
+}
+
 static void uprobe_subtest(struct test_bpf_cookie *skel)
 {
 	DECLARE_LIBBPF_OPTS(bpf_uprobe_opts, opts);
@@ -243,6 +283,8 @@ void test_bpf_cookie(void)
 
 	if (test__start_subtest("kprobe"))
 		kprobe_subtest(skel);
+	if (test__start_subtest("rawkprobe"))
+		rawkprobe_subtest(skel);
 	if (test__start_subtest("uprobe"))
 		uprobe_subtest(skel);
 	if (test__start_subtest("tracepoint"))
tools/testing/selftests/bpf/progs/test_bpf_cookie.c +23 -1
index 2d3a7710e2ce..409f87464b1f 100644
@@ -8,8 +8,9 @@
 int my_tid;
 
 int kprobe_res;
-int kprobe_multi_res;
 int kretprobe_res;
+int raw_kprobe_res;
+int raw_kretprobe_res;
 int uprobe_res;
 int uretprobe_res;
 int tp_res;
@@ -37,6 +38,27 @@ int handle_kretprobe(struct pt_regs *ctx)
 	return 0;
 }
 
+SEC("kprobe/bpf_fentry_test1")
+int handle_raw_kprobe(struct pt_regs *ctx)
+{
+	update(ctx, &raw_kprobe_res);
+	return 0;
+}
+
+SEC("kretprobe/bpf_fentry_test1")
+int handle_raw_kretprobe(struct pt_regs *ctx)
+{
+	update(ctx, &raw_kretprobe_res);
+	return 0;
+}
+
+/* just to trigger bpf_fentry_test1 through tracing test_run */
+SEC("fentry/bpf_modify_return_test")
+int BPF_PROG(raw_trigger)
+{
+	return 0;
+}
+
 SEC("uprobe/trigger_func")
 int handle_uprobe(struct pt_regs *ctx)
 {
Signature · 1 line
2.33.1
#24 lore raw
In reply to Jiri Olsa
Quoted 10 lines · Jiri Olsa

hi,
adding support to attach multiple kprobes within single syscall and speed up attachment of many kprobes.

The previous attempt [1] wasn't fast enough, so coming with new approach that adds new kprobe interface.

The attachment speed of of this approach (tested in bpftrace) is now comparable to ftrace tracer attachment speed.. fast ;-)

What are the absolute numbers?
How quickly a single bpf prog can attach to 1k kprobes?

author
#25 lore raw
Quoted 6 lines · Alexei Starovoitov
Quoted 5 lines · Jiri Olsa

hi,
adding support to attach multiple kprobes within single syscall and speed up attachment of many kprobes.

Quoted 9 lines
Quoted 5 lines

The previous attempt [1] wasn't fast enough, so coming with new approach that adds new kprobe interface.

The attachment speed of of this approach (tested in bpftrace) is now comparable to ftrace tracer attachment speed.. fast ;-)

What are the absolute numbers?
How quickly a single bpf prog can attach to 1k kprobes?

I'd need to write special tool for 1k kprobes exactly, we could do some benchmark selftest for that

I tested following counts with current bpftrace interface for now (note it includes both attach and detach)

2 seconds for 673 kprobes:

	# perf stat -e cycles:u,cycles:k ./src/bpftrace  -e 'kprobe:kvm* {  } i:ms:10 { printf("KRAVA\n"); exit() }' 
	Attaching 2 probes...
	Attaching 673 functions
	KRAVA
	 Performance counter stats for './src/bpftrace -e kprobe:kvm* {  } i:ms:10 { printf("KRAVA\n"); exit() }':

	     1,695,142,901      cycles:u                                                    
	     1,909,616,944      cycles:k                                                    

	       1.990434019 seconds time elapsed

	       0.767746000 seconds user
	       0.921166000 seconds sys

5 seconds for 3337 kprobes:

	# perf stat -e cycles:u,cycles:k ./src/bpftrace  -e 'kprobe:x* {  } i:ms:10 { printf("KRAVA\n"); exit() }' 
	Attaching 2 probes...
	Attaching 3337 functions
	KRAVA
	 Performance counter stats for './src/bpftrace -e kprobe:x* {  } i:ms:10 { printf("KRAVA\n"); exit() }':

	     1,731,646,061      cycles:u                                                    
	     9,815,306,940      cycles:k                                                    

	       5.196176904 seconds time elapsed

	       0.780508000 seconds user
	       4.078170000 seconds sys

lot of the time above is spent in kallsyms:

	    42.70%  bpftrace  [kernel.kallsyms]     [k] kallsyms_expand_symbol.constprop.0
	     5.11%  bpftrace  [kernel.kallsyms]     [k] insn_get_prefixes.part.0
	     3.91%  bpftrace  [kernel.kallsyms]     [k] insn_decode
	     3.09%  bpftrace  [kernel.kallsyms]     [k] arch_jump_entry_size
	     1.98%  bpftrace  [kernel.kallsyms]     [k] __lock_acquire
	     1.51%  bpftrace  [kernel.kallsyms]     [k] static_call_text_reserved

by checking if the address is on the kprobe blacklist:

	    42.70%  bpftrace  [kernel.kallsyms]     [k] kallsyms_expand_symbol.constprop.0
		    |
		    ---kallsyms_expand_symbol.constprop.0
		       |          
			--42.22%--kallsyms_lookup_name
				  within_kprobe_blacklist.part.0
				  check_kprobe_address
				  register_kprobe
				  bpf_kprobe_link_attach
				  __sys_bpf
				  __x64_sys_bpf
				  do_syscall_64
				  entry_SYSCALL_64_after_hwframe
				  syscall
				  bpftrace::AttachedProbe::attach_kprobe

I could revive that patch that did bsearch on kallsyms or we could add 'do-not-check-kprobe-blacklist' unsafe mode to get more speed

jirka

maintainer
#3 lore raw

Re: [PATCH 02/13] kprobe: Keep traced function address

In reply to Jiri Olsa
Quoted 9 lines · On Tue, 4 Jan 2022 09:09:32 +0100 Jiri Olsa <[email protected]> wrote:

The bpf_get_func_ip_kprobe helper should return traced function address, but it's doing so only for kprobes that are placed on the function entry.

If kprobe is placed within the function, bpf_get_func_ip_kprobe returns that address instead of function entry.

Storing the function entry directly in kprobe object, so it could be used in bpf_get_func_ip_kprobe helper.

Hmm, please do this in bpf side, which should have some data structure around the kprobe itself. Do not add this "specialized" field to the kprobe data structure.

Thank you,

Quoted 97 lines
---
 include/linux/kprobes.h                              |  3 +++
 kernel/kprobes.c                                     | 12 ++++++++++++
 kernel/trace/bpf_trace.c                             |  2 +-
 tools/testing/selftests/bpf/progs/get_func_ip_test.c |  4 ++--
 4 files changed, 18 insertions(+), 3 deletions(-)
include/linux/kprobes.h +3 -0
index 8c8f7a4d93af..a204df4fef96 100644
@@ -74,6 +74,9 @@ struct kprobe {
 	/* Offset into the symbol */
 	unsigned int offset;
 
+	/* traced function address */
+	unsigned long func_addr;
+
 	/* Called before addr is executed. */
 	kprobe_pre_handler_t pre_handler;
 
kernel/kprobes.c +12 -0
index d20ae8232835..c4060a8da050 100644
@@ -1310,6 +1310,7 @@ static void init_aggr_kprobe(struct kprobe *ap, struct kprobe *p)
 	copy_kprobe(p, ap);
 	flush_insn_slot(ap);
 	ap->addr = p->addr;
+	ap->func_addr = p->func_addr;
 	ap->flags = p->flags & ~KPROBE_FLAG_OPTIMIZED;
 	ap->pre_handler = aggr_pre_handler;
 	/* We don't care the kprobe which has gone. */
@@ -1588,6 +1589,16 @@ static int check_kprobe_address_safe(struct kprobe *p,
 	return ret;
 }
 
+static unsigned long resolve_func_addr(kprobe_opcode_t *addr)
+{
+	char str[KSYM_SYMBOL_LEN];
+	unsigned long offset;
+
+	if (kallsyms_lookup((unsigned long) addr, NULL, &offset, NULL, str))
+		return (unsigned long) addr - offset;
+	return 0;
+}
+
 int register_kprobe(struct kprobe *p)
 {
 	int ret;
@@ -1600,6 +1611,7 @@ int register_kprobe(struct kprobe *p)
 	if (IS_ERR(addr))
 		return PTR_ERR(addr);
 	p->addr = addr;
+	p->func_addr = resolve_func_addr(addr);
 
 	ret = warn_kprobe_rereg(p);
 	if (ret)
kernel/trace/bpf_trace.c +1 -1
index 21aa30644219..25631253084a 100644
@@ -1026,7 +1026,7 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
 {
 	struct kprobe *kp = kprobe_running();
 
-	return kp ? (uintptr_t)kp->addr : 0;
+	return kp ? (uintptr_t)kp->func_addr : 0;
 }
 
 static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
tools/testing/selftests/bpf/progs/get_func_ip_test.c +2 -2
index a587aeca5ae0..e988aefa567e 100644
@@ -69,7 +69,7 @@ int test6(struct pt_regs *ctx)
 {
 	__u64 addr = bpf_get_func_ip(ctx);
 
-	test6_result = (const void *) addr == &bpf_fentry_test6 + 5;
+	test6_result = (const void *) addr == &bpf_fentry_test6;
 	return 0;
 }
 
@@ -79,6 +79,6 @@ int test7(struct pt_regs *ctx)
 {
 	__u64 addr = bpf_get_func_ip(ctx);
 
-	test7_result = (const void *) addr == &bpf_fentry_test7 + 5;
+	test7_result = (const void *) addr == &bpf_fentry_test7;
 	return 0;
 }
Signature · 1 line
2.33.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
maintainer
#8 lore raw

Re: [PATCH 03/13] kprobe: Add support to register multiple ftrace kprobes

In reply to Jiri Olsa
Quoted 16 lines · On Tue, 4 Jan 2022 09:09:33 +0100 Jiri Olsa <[email protected]> wrote:

Adding support to register kprobe on multiple addresses within single kprobe object instance.

It uses the CONFIG_KPROBES_ON_FTRACE feature (so it's only available for function entry addresses) and separated ftrace_ops object placed directly in the kprobe object.

There's new CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE config option, enabled for archs that support multi kprobes.

It registers all the provided addresses in the ftrace_ops object filter and adds separate ftrace_ops callback.

To register multi kprobe user provides array of addresses or symbols with their count, like:

Quoted 7 lines
  struct kprobe kp = {};

  kp.multi.symbols = (const char **) symbols;
  kp.multi.cnt = cnt;
  ...

  err = register_kprobe(&kp);

I would like to keep the kprobe itself as simple as possible, which also should provide a consistent probe handling model.

I understand that you consider the overhead of having multiple probes, but as far as I can see, this implementation design smells no good, sorry.
The worst point is that the multi kprobe only supports function entry (and only available if FTRACE is enabled). Then, that is FTRACE, not kprobes.

Yes, kprobe supports probing on FTRACE by using FTRACE, but that does not mean kprobes wrapps FTRACE. That is just for "avoidance" of the limitation (because we can not put a breakpoint on self-modified code.)

If you need a probe which support multiple address but only on function entry, that should be something like 'fprobe', not kprobes.
IMHO, that should be implemented as similar but different APIs because those are simply different.

So, can't we use ftrace directly from bpf? I don't think there is no reason that the bpf sticks on kprobes APIs.

Thank you,

Quoted 480 lines
---
 arch/Kconfig                     |   3 +
 arch/x86/Kconfig                 |   1 +
 arch/x86/kernel/kprobes/ftrace.c |  48 ++++++--
 include/linux/kprobes.h          |  25 ++++
 kernel/kprobes.c                 | 204 +++++++++++++++++++++++++------
 5 files changed, 235 insertions(+), 46 deletions(-)
arch/Kconfig +3 -0
index d3c4ab249e9c..0131636e1ef8 100644
@@ -191,6 +191,9 @@ config HAVE_OPTPROBES
 config HAVE_KPROBES_ON_FTRACE
 	bool
 
+config HAVE_KPROBES_MULTI_ON_FTRACE
+	bool
+
 config ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
 	bool
 	help
arch/x86/Kconfig +1 -0
index 5c2ccb85f2ef..0c870238016a 100644
@@ -217,6 +217,7 @@ config X86
 	select HAVE_KERNEL_ZSTD
 	select HAVE_KPROBES
 	select HAVE_KPROBES_ON_FTRACE
+	select HAVE_KPROBES_MULTI_ON_FTRACE
 	select HAVE_FUNCTION_ERROR_INJECTION
 	select HAVE_KRETPROBES
 	select HAVE_KVM
arch/x86/kernel/kprobes/ftrace.c +36 -12
index dd2ec14adb77..ac4d256b89c6 100644
@@ -12,22 +12,14 @@
 
 #include "common.h"
 
-/* Ftrace callback handler for kprobes -- called under preempt disabled */
-void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
-			   struct ftrace_ops *ops, struct ftrace_regs *fregs)
+static void ftrace_handler(struct kprobe *p, unsigned long ip,
+			   struct ftrace_regs *fregs)
 {
 	struct pt_regs *regs = ftrace_get_regs(fregs);
-	struct kprobe *p;
 	struct kprobe_ctlblk *kcb;
-	int bit;
 
-	bit = ftrace_test_recursion_trylock(ip, parent_ip);
-	if (bit < 0)
-		return;
-
-	p = get_kprobe((kprobe_opcode_t *)ip);
 	if (unlikely(!p) || kprobe_disabled(p))
-		goto out;
+		return;
 
 	kcb = get_kprobe_ctlblk();
 	if (kprobe_running()) {
@@ -57,11 +49,43 @@ void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
 		 */
 		__this_cpu_write(current_kprobe, NULL);
 	}
-out:
+}
+
+/* Ftrace callback handler for kprobes -- called under preempt disabled */
+void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
+			   struct ftrace_ops *ops, struct ftrace_regs *fregs)
+{
+	struct kprobe *p;
+	int bit;
+
+	bit = ftrace_test_recursion_trylock(ip, parent_ip);
+	if (bit < 0)
+		return;
+
+	p = get_kprobe((kprobe_opcode_t *)ip);
+	ftrace_handler(p, ip, fregs);
+
 	ftrace_test_recursion_unlock(bit);
 }
 NOKPROBE_SYMBOL(kprobe_ftrace_handler);
 
+void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip,
+				 struct ftrace_ops *ops, struct ftrace_regs *fregs)
+{
+	struct kprobe *p;
+	int bit;
+
+	bit = ftrace_test_recursion_trylock(ip, parent_ip);
+	if (bit < 0)
+		return;
+
+	p = container_of(ops, struct kprobe, multi.ops);
+	ftrace_handler(p, ip, fregs);
+
+	ftrace_test_recursion_unlock(bit);
+}
+NOKPROBE_SYMBOL(kprobe_ftrace_multi_handler);
+
 int arch_prepare_kprobe_ftrace(struct kprobe *p)
 {
 	p->ainsn.insn = NULL;
include/linux/kprobes.h +25 -0
index a204df4fef96..03fd86ef69cb 100644
@@ -68,6 +68,16 @@ struct kprobe {
 	/* location of the probe point */
 	kprobe_opcode_t *addr;
 
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	/* location of the multi probe points */
+	struct {
+		const char **symbols;
+		kprobe_opcode_t **addrs;
+		unsigned int cnt;
+		struct ftrace_ops ops;
+	} multi;
+#endif
+
 	/* Allow user to indicate symbol name of the probe point */
 	const char *symbol_name;
 
@@ -105,6 +115,7 @@ struct kprobe {
 				   * this flag is only for optimized_kprobe.
 				   */
 #define KPROBE_FLAG_FTRACE	8 /* probe is using ftrace */
+#define KPROBE_FLAG_MULTI      16 /* probe multi addresses */
 
 /* Has this kprobe gone ? */
 static inline bool kprobe_gone(struct kprobe *p)
@@ -130,6 +141,18 @@ static inline bool kprobe_ftrace(struct kprobe *p)
 	return p->flags & KPROBE_FLAG_FTRACE;
 }
 
+/* Is this ftrace multi kprobe ? */
+static inline bool kprobe_ftrace_multi(struct kprobe *p)
+{
+	return kprobe_ftrace(p) && (p->flags & KPROBE_FLAG_MULTI);
+}
+
+/* Is this single kprobe ? */
+static inline bool kprobe_single(struct kprobe *p)
+{
+	return !(p->flags & KPROBE_FLAG_MULTI);
+}
+
 /*
  * Function-return probe -
  * Note:
@@ -365,6 +388,8 @@ static inline void wait_for_kprobe_optimizer(void) { }
 #ifdef CONFIG_KPROBES_ON_FTRACE
 extern void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
 				  struct ftrace_ops *ops, struct ftrace_regs *fregs);
+extern void kprobe_ftrace_multi_handler(unsigned long ip, unsigned long parent_ip,
+					struct ftrace_ops *ops, struct ftrace_regs *fregs);
 extern int arch_prepare_kprobe_ftrace(struct kprobe *p);
 #else
 static inline int arch_prepare_kprobe_ftrace(struct kprobe *p)
kernel/kprobes.c +170 -34
index c4060a8da050..e7729e20d85c 100644
@@ -44,6 +44,7 @@
 #include <asm/cacheflush.h>
 #include <asm/errno.h>
 #include <linux/uaccess.h>
+#include <linux/ftrace.h>
 
 #define KPROBE_HASH_BITS 6
 #define KPROBE_TABLE_SIZE (1 << KPROBE_HASH_BITS)
@@ -1022,6 +1023,35 @@ static struct kprobe *alloc_aggr_kprobe(struct kprobe *p)
 }
 #endif /* CONFIG_OPTPROBES */
 
+static int check_kprobe_address(unsigned long addr)
+{
+	/* Ensure it is not in reserved area nor out of text */
+	return !kernel_text_address(addr) ||
+		within_kprobe_blacklist(addr) ||
+		jump_label_text_reserved((void *) addr, (void *) addr) ||
+		static_call_text_reserved((void *) addr, (void *) addr) ||
+		find_bug(addr);
+}
+
+static int check_ftrace_location(unsigned long addr, struct kprobe *p)
+{
+	unsigned long ftrace_addr;
+
+	ftrace_addr = ftrace_location(addr);
+	if (ftrace_addr) {
+#ifdef CONFIG_KPROBES_ON_FTRACE
+		/* Given address is not on the instruction boundary */
+		if (addr != ftrace_addr)
+			return -EILSEQ;
+		if (p)
+			p->flags |= KPROBE_FLAG_FTRACE;
+#else	/* !CONFIG_KPROBES_ON_FTRACE */
+		return -EINVAL;
+#endif
+	}
+	return 0;
+}
+
 #ifdef CONFIG_KPROBES_ON_FTRACE
 static struct ftrace_ops kprobe_ftrace_ops __read_mostly = {
 	.func = kprobe_ftrace_handler,
@@ -1043,6 +1073,13 @@ static int __arm_kprobe_ftrace(struct kprobe *p, struct ftrace_ops *ops,
 
 	lockdep_assert_held(&kprobe_mutex);
 
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	if (kprobe_ftrace_multi(p)) {
+		ret = register_ftrace_function(&p->multi.ops);
+		WARN(ret < 0, "Failed to register kprobe-multi-ftrace (error %d)\n", ret);
+		return ret;
+	}
+#endif
 	ret = ftrace_set_filter_ip(ops, (unsigned long)p->addr, 0, 0);
 	if (WARN_ONCE(ret < 0, "Failed to arm kprobe-ftrace at %pS (error %d)\n", p->addr, ret))
 		return ret;
@@ -1081,6 +1118,13 @@ static int __disarm_kprobe_ftrace(struct kprobe *p, struct ftrace_ops *ops,
 
 	lockdep_assert_held(&kprobe_mutex);
 
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	if (kprobe_ftrace_multi(p)) {
+		ret = unregister_ftrace_function(&p->multi.ops);
+		WARN(ret < 0, "Failed to unregister kprobe-ftrace (error %d)\n", ret);
+		return ret;
+	}
+#endif
 	if (*cnt == 1) {
 		ret = unregister_ftrace_function(ops);
 		if (WARN(ret < 0, "Failed to unregister kprobe-ftrace (error %d)\n", ret))
@@ -1103,6 +1147,94 @@ static int disarm_kprobe_ftrace(struct kprobe *p)
 		ipmodify ? &kprobe_ipmodify_ops : &kprobe_ftrace_ops,
 		ipmodify ? &kprobe_ipmodify_enabled : &kprobe_ftrace_enabled);
 }
+
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+/*
+ * In addition to standard kprobe address check for multi
+ * ftrace kprobes we also allow only:
+ * - ftrace managed function entry address
+ * - kernel core only address
+ */
+static unsigned long check_ftrace_addr(unsigned long addr)
+{
+	int err;
+
+	if (!addr)
+		return -EINVAL;
+	err = check_ftrace_location(addr, NULL);
+	if (err)
+		return err;
+	if (check_kprobe_address(addr))
+		return -EINVAL;
+	if (__module_text_address(addr))
+		return -EINVAL;
+	return 0;
+}
+
+static int check_ftrace_multi(struct kprobe *p)
+{
+	kprobe_opcode_t **addrs = p->multi.addrs;
+	const char **symbols = p->multi.symbols;
+	unsigned int i, cnt = p->multi.cnt;
+	unsigned long addr, *ips;
+	int err;
+
+	if ((symbols && addrs) || (!symbols && !addrs))
+		return -EINVAL;
+
+	/* do we want sysctl for this? */
+	if (cnt >= 20000)
+		return -E2BIG;
+
+	ips = kmalloc(sizeof(*ips) * cnt, GFP_KERNEL);
+	if (!ips)
+		return -ENOMEM;
+
+	for (i = 0; i < cnt; i++) {
+		if (symbols)
+			addr = (unsigned long) kprobe_lookup_name(symbols[i], 0);
+		else
+			addr = (unsigned long) addrs[i];
+		ips[i] = addr;
+	}
+
+	jump_label_lock();
+	preempt_disable();
+
+	for (i = 0; i < cnt; i++) {
+		err = check_ftrace_addr(ips[i]);
+		if (err)
+			break;
+	}
+
+	preempt_enable();
+	jump_label_unlock();
+
+	if (err)
+		goto out;
+
+	err = ftrace_set_filter_ips(&p->multi.ops, ips, cnt, 0, 0);
+	if (err)
+		goto out;
+
+	p->multi.ops.func = kprobe_ftrace_multi_handler;
+	p->multi.ops.flags = FTRACE_OPS_FL_SAVE_REGS|FTRACE_OPS_FL_DYNAMIC;
+
+	p->flags |= KPROBE_FLAG_MULTI|KPROBE_FLAG_FTRACE;
+	if (p->post_handler)
+		p->multi.ops.flags |= FTRACE_OPS_FL_IPMODIFY;
+
+out:
+	kfree(ips);
+	return err;
+}
+
+static void free_ftrace_multi(struct kprobe *p)
+{
+	ftrace_free_filter(&p->multi.ops);
+}
+#endif
+
 #else	/* !CONFIG_KPROBES_ON_FTRACE */
 static inline int arm_kprobe_ftrace(struct kprobe *p)
 {
@@ -1489,6 +1621,9 @@ static struct kprobe *__get_valid_kprobe(struct kprobe *p)
 
 	lockdep_assert_held(&kprobe_mutex);
 
+	if (kprobe_ftrace_multi(p))
+		return p;
+
 	ap = get_kprobe(p->addr);
 	if (unlikely(!ap))
 		return NULL;
@@ -1520,41 +1655,18 @@ static inline int warn_kprobe_rereg(struct kprobe *p)
 	return ret;
 }
 
-static int check_ftrace_location(struct kprobe *p)
-{
-	unsigned long ftrace_addr;
-
-	ftrace_addr = ftrace_location((unsigned long)p->addr);
-	if (ftrace_addr) {
-#ifdef CONFIG_KPROBES_ON_FTRACE
-		/* Given address is not on the instruction boundary */
-		if ((unsigned long)p->addr != ftrace_addr)
-			return -EILSEQ;
-		p->flags |= KPROBE_FLAG_FTRACE;
-#else	/* !CONFIG_KPROBES_ON_FTRACE */
-		return -EINVAL;
-#endif
-	}
-	return 0;
-}
-
 static int check_kprobe_address_safe(struct kprobe *p,
 				     struct module **probed_mod)
 {
 	int ret;
 
-	ret = check_ftrace_location(p);
+	ret = check_ftrace_location((unsigned long) p->addr, p);
 	if (ret)
 		return ret;
 	jump_label_lock();
 	preempt_disable();
 
-	/* Ensure it is not in reserved area nor out of text */
-	if (!kernel_text_address((unsigned long) p->addr) ||
-	    within_kprobe_blacklist((unsigned long) p->addr) ||
-	    jump_label_text_reserved(p->addr, p->addr) ||
-	    static_call_text_reserved(p->addr, p->addr) ||
-	    find_bug((unsigned long)p->addr)) {
+	if (check_kprobe_address((unsigned long) p->addr)) {
 		ret = -EINVAL;
 		goto out;
 	}
@@ -1599,13 +1711,16 @@ static unsigned long resolve_func_addr(kprobe_opcode_t *addr)
 	return 0;
 }
 
-int register_kprobe(struct kprobe *p)
+static int check_addr(struct kprobe *p, struct module **probed_mod)
 {
 	int ret;
-	struct kprobe *old_p;
-	struct module *probed_mod;
 	kprobe_opcode_t *addr;
 
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	if (p->multi.cnt)
+		return check_ftrace_multi(p);
+#endif
+
 	/* Adjust probe address from symbol */
 	addr = kprobe_addr(p);
 	if (IS_ERR(addr))
@@ -1616,13 +1731,21 @@ int register_kprobe(struct kprobe *p)
 	ret = warn_kprobe_rereg(p);
 	if (ret)
 		return ret;
+	return check_kprobe_address_safe(p, probed_mod);
+}
+
+int register_kprobe(struct kprobe *p)
+{
+	struct module *probed_mod = NULL;
+	struct kprobe *old_p;
+	int ret;
 
 	/* User can pass only KPROBE_FLAG_DISABLED to register_kprobe */
 	p->flags &= KPROBE_FLAG_DISABLED;
 	p->nmissed = 0;
 	INIT_LIST_HEAD(&p->list);
 
-	ret = check_kprobe_address_safe(p, &probed_mod);
+	ret = check_addr(p, &probed_mod);
 	if (ret)
 		return ret;
 
@@ -1644,14 +1767,21 @@ int register_kprobe(struct kprobe *p)
 	if (ret)
 		goto out;
 
-	INIT_HLIST_NODE(&p->hlist);
-	hlist_add_head_rcu(&p->hlist,
-		       &kprobe_table[hash_ptr(p->addr, KPROBE_HASH_BITS)]);
+	/*
+	 * Multi ftrace kprobes do not have single address,
+	 * so they are not stored in the kprobe_table hash.
+	 */
+	if (kprobe_single(p)) {
+		INIT_HLIST_NODE(&p->hlist);
+		hlist_add_head_rcu(&p->hlist,
+			       &kprobe_table[hash_ptr(p->addr, KPROBE_HASH_BITS)]);
+	}
 
 	if (!kprobes_all_disarmed && !kprobe_disabled(p)) {
 		ret = arm_kprobe(p);
 		if (ret) {
-			hlist_del_rcu(&p->hlist);
+			if (kprobe_single(p))
+				hlist_del_rcu(&p->hlist);
 			synchronize_rcu();
 			goto out;
 		}
@@ -1778,7 +1908,13 @@ static int __unregister_kprobe_top(struct kprobe *p)
 	return 0;
 
 disarmed:
-	hlist_del_rcu(&ap->hlist);
+	if (kprobe_single(ap))
+		hlist_del_rcu(&ap->hlist);
+
+#ifdef CONFIG_HAVE_KPROBES_MULTI_ON_FTRACE
+	if (kprobe_ftrace_multi(ap))
+		free_ftrace_multi(ap);
+#endif
 	return 0;
 }
 
Signature · 1 line
2.33.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
maintainer
#26 lore raw
In reply to Jiri Olsa
Quoted 6 lines · On Tue, 4 Jan 2022 09:09:30 +0100 Jiri Olsa <[email protected]> wrote:

hi,
adding support to attach multiple kprobes within single syscall and speed up attachment of many kprobes.

The previous attempt [1] wasn't fast enough, so coming with new approach that adds new kprobe interface.

Yes, since register_kprobes() just registers multiple kprobes on array. This is designed for dozens of kprobes.

Quoted 2 lines

The attachment speed of of this approach (tested in bpftrace) is now comparable to ftrace tracer attachment speed.. fast ;-)

Yes, because that if ftrace, not kprobes.

Quoted 3 lines

The limit of this approach is forced by using ftrace as attach layer, so it allows only kprobes on function's entry (plus return probes).

Note that you also need to multiply the number of instances.

Quoted 9 lines

This patchset contains:
- kprobes support to register multiple kprobes with current kprobe API (patches 1 - 8)
- bpf support ot create new kprobe link allowing to attach multiple addresses (patches 9 - 14)

We don't need to care about multiple probes on same functions because it's taken care on the ftrace_ops layer.

Hmm, I think there may be a time to split the "kprobe as an interface for the software breakpoint" and "kprobe as a wrapper interface for the callbacks of various instrumentations", like 'raw_kprobe'(or kswbp) and 'kprobes'.
And this may be called as 'fprobe' as ftrace_ops wrapper. (But if the bpf is enough flexible, this kind of intermediate layer
may not be needed, it can use ftrace_ops directly, eventually)

Jiri, have you already considered to use ftrace_ops from the bpf directly? Are there any issues?
(bpf depends on 'kprobe' widely?)

Thank you,

Signature · 1 line
Masami Hiramatsu <[email protected]>
#5 lore raw

Re: [PATCH 02/13] kprobe: Keep traced function address

In reply to Jiri Olsa
Quoted 24 lines · Jiri Olsa

The bpf_get_func_ip_kprobe helper should return traced function address, but it's doing so only for kprobes that are placed on the function entry.

If kprobe is placed within the function, bpf_get_func_ip_kprobe returns that address instead of function entry.

Storing the function entry directly in kprobe object, so it could be used in bpf_get_func_ip_kprobe helper.

---
 include/linux/kprobes.h                              |  3 +++
 kernel/kprobes.c                                     | 12 ++++++++++++
 kernel/trace/bpf_trace.c                             |  2 +-
 tools/testing/selftests/bpf/progs/get_func_ip_test.c |  4 ++--
 4 files changed, 18 insertions(+), 3 deletions(-)
include/linux/kprobes.h +0 -0
index 8c8f7a4d93af..a204df4fef96 100644

include/linux/kprobes.h struct kprobe { · line 79

Quoted 6 lines
include/linux/kprobes.h +3 -0
@@ -74,6 +74,9 @@ struct kprobe {
        /* Offset into the symbol */
        unsigned int offset;
 
+       /* traced function address */
+       unsigned long func_addr;
+

keep in mind that we'll also need (maybe in a follow up series) to store bpf_cookie somewhere close to this func_addr as well. Just mentioning to keep in mind as you decide with Masami where to put it.

Quoted 67 lines
include/linux/kprobes.h +0 -0
        /* Called before addr is executed. */
        kprobe_pre_handler_t pre_handler;
 
kernel/kprobes.c +12 -0
index d20ae8232835..c4060a8da050 100644
@@ -1310,6 +1310,7 @@ static void init_aggr_kprobe(struct kprobe *ap, struct kprobe *p)
        copy_kprobe(p, ap);
        flush_insn_slot(ap);
        ap->addr = p->addr;
+       ap->func_addr = p->func_addr;
        ap->flags = p->flags & ~KPROBE_FLAG_OPTIMIZED;
        ap->pre_handler = aggr_pre_handler;
        /* We don't care the kprobe which has gone. */
@@ -1588,6 +1589,16 @@ static int check_kprobe_address_safe(struct kprobe *p,
        return ret;
 }
 
+static unsigned long resolve_func_addr(kprobe_opcode_t *addr)
+{
+       char str[KSYM_SYMBOL_LEN];
+       unsigned long offset;
+
+       if (kallsyms_lookup((unsigned long) addr, NULL, &offset, NULL, str))
+               return (unsigned long) addr - offset;
+       return 0;
+}
+
 int register_kprobe(struct kprobe *p)
 {
        int ret;
@@ -1600,6 +1611,7 @@ int register_kprobe(struct kprobe *p)
        if (IS_ERR(addr))
                return PTR_ERR(addr);
        p->addr = addr;
+       p->func_addr = resolve_func_addr(addr);
 
        ret = warn_kprobe_rereg(p);
        if (ret)
kernel/trace/bpf_trace.c +1 -1
index 21aa30644219..25631253084a 100644
@@ -1026,7 +1026,7 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
 {
        struct kprobe *kp = kprobe_running();
 
-       return kp ? (uintptr_t)kp->addr : 0;
+       return kp ? (uintptr_t)kp->func_addr : 0;
 }
 
 static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
tools/testing/selftests/bpf/progs/get_func_ip_test.c +1 -1
index a587aeca5ae0..e988aefa567e 100644
@@ -69,7 +69,7 @@ int test6(struct pt_regs *ctx)
 {
        __u64 addr = bpf_get_func_ip(ctx);
 
-       test6_result = (const void *) addr == &bpf_fentry_test6 + 5;
+       test6_result = (const void *) addr == &bpf_fentry_test6;
        return 0;
 }
 

tools/testing/selftests/bpf/progs/get_func_ip_test.c int test7(struct pt_regs *ctx) · line 82

Quoted 5 lines
tools/testing/selftests/bpf/progs/get_func_ip_test.c +1 -1
@@ -79,6 +79,6 @@ int test7(struct pt_regs *ctx)
 {
        __u64 addr = bpf_get_func_ip(ctx);
 
-       test7_result = (const void *) addr == &bpf_fentry_test7 + 5;
+       test7_result = (const void *) addr == &bpf_fentry_test7;

we can treat this as a bug fix for bpf_get_func_ip() for kprobes, right? I think "Fixes: " tag is in order then.

Quoted 6 lines
tools/testing/selftests/bpf/progs/get_func_ip_test.c +0 -1
        return 0;
 }
 
--

2.33.1

#11 lore raw

Re: [PATCH 05/13] kprobe: Allow to get traced function address for multi ftrace kprobes

In reply to Jiri Olsa
Quoted 33 lines · Jiri Olsa

The current bpf_get_func_ip_kprobe helper does not work properly, when used in ebpf program triggered by the new multi kprobes.

We can't use kprobe's func_addr in bpf_get_func_ip_kprobe helper, because there are multiple functions registered for single kprobe object.

Adding new per cpu variable current_ftrace_multi_addr and extra address in kretprobe_instance object to keep current traced function address for each cpu for both kprobe handler and kretprobe trampoline.

The address value is set/passed as follows, for kprobe:

  kprobe_ftrace_multi_handler
  {
    old = kprobe_ftrace_multi_addr_set(ip);
    handler..
    kprobe_ftrace_multi_addr_set(old);
  }

For kretprobe:

  kprobe_ftrace_multi_handler
  {
    old = kprobe_ftrace_multi_addr_set(ip);
    ...
      pre_handler_kretprobe
      {
        ri->ftrace_multi_addr = kprobe_ftrace_multi_addr
      }
    ...
    kprobe_ftrace_multi_addr_set(old);
Quoted 9 lines
  }

  __kretprobe_trampoline_handler
  {
    prev_func_addr = kprobe_ftrace_multi_addr_set(ri->ftrace_multi_addr);
    handler..
    kprobe_ftrace_multi_addr_set(prev_func_addr);
  }

Is it possible to record or calculate the multi-kprobe "instance index" (i.e., which position in addrs array did we get triggered for)? If yes, then storing that index would allow to fetch both IP and cookie value with just one per-cpu variable.

Quoted 8 lines
---
 arch/x86/kernel/kprobes/ftrace.c |  3 +++
 include/linux/kprobes.h          | 26 ++++++++++++++++++++++++++
 kernel/kprobes.c                 |  6 ++++++
 kernel/trace/bpf_trace.c         |  7 ++++++-
 4 files changed, 41 insertions(+), 1 deletion(-)

[...]

#15 lore raw

Re: [PATCH 08/13] bpf: Add kprobe link for attaching raw kprobes

In reply to Jiri Olsa
Quoted 7 lines · Jiri Olsa

Adding new link type BPF_LINK_TYPE_KPROBE to attach kprobes directly through register_kprobe/kretprobe API.

Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program.

Quoted 7 lines

The new link allows to create multiple kprobes link by using new link_create interface:

  struct {
    __aligned_u64   addrs;
    __u32           cnt;
    __u64           bpf_cookie;

I'm afraid bpf_cookie has to be different for each addr, otherwise it's severely limiting. So it would be an array of cookies alongside an array of addresses

Quoted 13 lines
  } kprobe;

Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe.

---
 include/linux/bpf_types.h      |   1 +
 include/uapi/linux/bpf.h       |  12 +++
 kernel/bpf/syscall.c           | 191 ++++++++++++++++++++++++++++++++-
 tools/include/uapi/linux/bpf.h |  12 +++
 4 files changed, 211 insertions(+), 5 deletions(-)

[...]

include/uapi/linux/bpf.h enum bpf_link_type { · line 1120

Quoted 9 lines
include/uapi/linux/bpf.h +5 -0
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE                (1U << 4)
 
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN    (1U << 0)
+

we have plenty of flexibility to have per-link type fields, so why not add bool is_retprobe next to addrs and cnt?

Quoted 15 lines
include/uapi/linux/bpf.h +5 -0
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1465,6 +1472,11 @@ union bpf_attr {
                                 */
                                __u64           bpf_cookie;
                        } perf_event;
+                       struct {
+                               __aligned_u64   addrs;
+                               __u32           cnt;
+                               __u64           bpf_cookie;
+                       } kprobe;
                };
        } link_create;
 

[...]

author
#27 lore raw
In reply to Masami Hiramatsu
Quoted 42 lines · Masami Hiramatsu
Quoted 6 lines · On Tue, 4 Jan 2022 09:09:30 +0100 Jiri Olsa <[email protected]> wrote:

hi,
adding support to attach multiple kprobes within single syscall and speed up attachment of many kprobes.

The previous attempt [1] wasn't fast enough, so coming with new approach that adds new kprobe interface.

Yes, since register_kprobes() just registers multiple kprobes on array. This is designed for dozens of kprobes.

Quoted 2 lines

The attachment speed of of this approach (tested in bpftrace) is now comparable to ftrace tracer attachment speed.. fast ;-)

Yes, because that if ftrace, not kprobes.

Quoted 3 lines

The limit of this approach is forced by using ftrace as attach layer, so it allows only kprobes on function's entry (plus return probes).

Note that you also need to multiply the number of instances.

Quoted 9 lines

This patchset contains:
- kprobes support to register multiple kprobes with current kprobe API (patches 1 - 8)
- bpf support ot create new kprobe link allowing to attach multiple addresses (patches 9 - 14)

We don't need to care about multiple probes on same functions because it's taken care on the ftrace_ops layer.

Hmm, I think there may be a time to split the "kprobe as an interface for the software breakpoint" and "kprobe as a wrapper interface for the callbacks of various instrumentations", like 'raw_kprobe'(or kswbp) and 'kprobes'.
And this may be called as 'fprobe' as ftrace_ops wrapper. (But if the bpf is enough flexible, this kind of intermediate layer
may not be needed, it can use ftrace_ops directly, eventually)

Quoted 3 lines

Jiri, have you already considered to use ftrace_ops from the bpf directly? Are there any issues?
(bpf depends on 'kprobe' widely?)

at the moment there's not ftrace public interface for the return probe merged in, so to get the kretprobe working I had to use kprobe interface

but.. there are patches Steven shared some time ago, that do that and make graph_ops available as kernel interface

I recall we considered graph_ops interface before as common attach layer for trampolines, which was bad, but it might actually make sense for kprobes

I'll need to check it in more details but I think both graph_ops and kprobe do about similar thing wrt hooking return probe, so it should be comparable.. and they are already doing the same for the entry hook, because kprobe is mostly using ftrace for that

we would not need to introduce new program type - kprobe programs should be able to run from ftrace callbacks just fine

so we would have:
- kprobe type programs attaching to:
- new BPF_LINK_TYPE_FPROBE link using the graph_ops as attachment layer

jirka

author
#4 lore raw

Re: [PATCH 02/13] kprobe: Keep traced function address

In reply to Masami Hiramatsu
Quoted 10 lines · Masami Hiramatsu
Quoted 7 lines · On Tue, 4 Jan 2022 09:09:32 +0100 Jiri Olsa <[email protected]> wrote:

The bpf_get_func_ip_kprobe helper should return traced function address, but it's doing so only for kprobes that are placed on the function entry.

If kprobe is placed within the function, bpf_get_func_ip_kprobe returns that address instead of function entry.

Quoted 6 lines
Quoted 2 lines

Storing the function entry directly in kprobe object, so it could be used in bpf_get_func_ip_kprobe helper.

Hmm, please do this in bpf side, which should have some data structure around the kprobe itself. Do not add this "specialized" field to the kprobe data structure.

ok, will check

thanks,
jirka

Quoted 105 lines

Thank you,

Quoted 97 lines
---
 include/linux/kprobes.h                              |  3 +++
 kernel/kprobes.c                                     | 12 ++++++++++++
 kernel/trace/bpf_trace.c                             |  2 +-
 tools/testing/selftests/bpf/progs/get_func_ip_test.c |  4 ++--
 4 files changed, 18 insertions(+), 3 deletions(-)
include/linux/kprobes.h +3 -0
index 8c8f7a4d93af..a204df4fef96 100644
@@ -74,6 +74,9 @@ struct kprobe {
 	/* Offset into the symbol */
 	unsigned int offset;
 
+	/* traced function address */
+	unsigned long func_addr;
+
 	/* Called before addr is executed. */
 	kprobe_pre_handler_t pre_handler;
 
kernel/kprobes.c +12 -0
index d20ae8232835..c4060a8da050 100644
@@ -1310,6 +1310,7 @@ static void init_aggr_kprobe(struct kprobe *ap, struct kprobe *p)
 	copy_kprobe(p, ap);
 	flush_insn_slot(ap);
 	ap->addr = p->addr;
+	ap->func_addr = p->func_addr;
 	ap->flags = p->flags & ~KPROBE_FLAG_OPTIMIZED;
 	ap->pre_handler = aggr_pre_handler;
 	/* We don't care the kprobe which has gone. */
@@ -1588,6 +1589,16 @@ static int check_kprobe_address_safe(struct kprobe *p,
 	return ret;
 }
 
+static unsigned long resolve_func_addr(kprobe_opcode_t *addr)
+{
+	char str[KSYM_SYMBOL_LEN];
+	unsigned long offset;
+
+	if (kallsyms_lookup((unsigned long) addr, NULL, &offset, NULL, str))
+		return (unsigned long) addr - offset;
+	return 0;
+}
+
 int register_kprobe(struct kprobe *p)
 {
 	int ret;
@@ -1600,6 +1611,7 @@ int register_kprobe(struct kprobe *p)
 	if (IS_ERR(addr))
 		return PTR_ERR(addr);
 	p->addr = addr;
+	p->func_addr = resolve_func_addr(addr);
 
 	ret = warn_kprobe_rereg(p);
 	if (ret)
kernel/trace/bpf_trace.c +1 -1
index 21aa30644219..25631253084a 100644
@@ -1026,7 +1026,7 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
 {
 	struct kprobe *kp = kprobe_running();
 
-	return kp ? (uintptr_t)kp->addr : 0;
+	return kp ? (uintptr_t)kp->func_addr : 0;
 }
 
 static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
tools/testing/selftests/bpf/progs/get_func_ip_test.c +2 -2
index a587aeca5ae0..e988aefa567e 100644
@@ -69,7 +69,7 @@ int test6(struct pt_regs *ctx)
 {
 	__u64 addr = bpf_get_func_ip(ctx);
 
-	test6_result = (const void *) addr == &bpf_fentry_test6 + 5;
+	test6_result = (const void *) addr == &bpf_fentry_test6;
 	return 0;
 }
 
@@ -79,6 +79,6 @@ int test7(struct pt_regs *ctx)
 {
 	__u64 addr = bpf_get_func_ip(ctx);
 
-	test7_result = (const void *) addr == &bpf_fentry_test7 + 5;
+	test7_result = (const void *) addr == &bpf_fentry_test7;
 	return 0;
 }
Signature · 1 line
2.33.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
author
#6 lore raw

Re: [PATCH 02/13] kprobe: Keep traced function address

In reply to Andrii Nakryiko
Quoted 28 lines · Andrii Nakryiko
Quoted 27 lines · Jiri Olsa

The bpf_get_func_ip_kprobe helper should return traced function address, but it's doing so only for kprobes that are placed on the function entry.

If kprobe is placed within the function, bpf_get_func_ip_kprobe returns that address instead of function entry.

Storing the function entry directly in kprobe object, so it could be used in bpf_get_func_ip_kprobe helper.

---
 include/linux/kprobes.h                              |  3 +++
 kernel/kprobes.c                                     | 12 ++++++++++++
 kernel/trace/bpf_trace.c                             |  2 +-
 tools/testing/selftests/bpf/progs/get_func_ip_test.c |  4 ++--
 4 files changed, 18 insertions(+), 3 deletions(-)
include/linux/kprobes.h +0 -0
index 8c8f7a4d93af..a204df4fef96 100644
@@ -74,6 +74,9 @@ struct kprobe {
        /* Offset into the symbol */
        unsigned int offset;
 
Quoted 7 lines

include/linux/kprobes.h struct kprobe { · line 79

Quoted 3 lines
include/linux/kprobes.h +3 -0
+       /* traced function address */
+       unsigned long func_addr;
+

keep in mind that we'll also need (maybe in a follow up series) to store bpf_cookie somewhere close to this func_addr as well. Just mentioning to keep in mind as you decide with Masami where to put it.

ok

SNIP

Quoted 30 lines
Quoted 30 lines
kernel/trace/bpf_trace.c +1 -1
index 21aa30644219..25631253084a 100644
@@ -1026,7 +1026,7 @@ BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs)
 {
        struct kprobe *kp = kprobe_running();
 
-       return kp ? (uintptr_t)kp->addr : 0;
+       return kp ? (uintptr_t)kp->func_addr : 0;
 }
 
 static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = {
tools/testing/selftests/bpf/progs/get_func_ip_test.c +1 -1
index a587aeca5ae0..e988aefa567e 100644
@@ -69,7 +69,7 @@ int test6(struct pt_regs *ctx)
 {
        __u64 addr = bpf_get_func_ip(ctx);
 
-       test6_result = (const void *) addr == &bpf_fentry_test6 + 5;
+       test6_result = (const void *) addr == &bpf_fentry_test6;
        return 0;
 }
 
@@ -79,6 +79,6 @@ int test7(struct pt_regs *ctx)
 {
        __u64 addr = bpf_get_func_ip(ctx);
 
Quoted 5 lines

tools/testing/selftests/bpf/progs/get_func_ip_test.c int test7(struct pt_regs *ctx) · line 82

Quoted 2 lines
tools/testing/selftests/bpf/progs/get_func_ip_test.c +1 -1
-       test7_result = (const void *) addr == &bpf_fentry_test7 + 5;
+       test7_result = (const void *) addr == &bpf_fentry_test7;

we can treat this as a bug fix for bpf_get_func_ip() for kprobes, right? I think "Fixes: " tag is in order then.

true, will add that in next version

thanks,
jirka

author
#16 lore raw

Re: [PATCH 08/13] bpf: Add kprobe link for attaching raw kprobes

In reply to Andrii Nakryiko
Quoted 11 lines · Andrii Nakryiko
Quoted 10 lines · Jiri Olsa

Adding new link type BPF_LINK_TYPE_KPROBE to attach kprobes directly through register_kprobe/kretprobe API.

Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program.

The new link allows to create multiple kprobes link by using new link_create interface:

Quoted 8 lines
Quoted 4 lines
  struct {
    __aligned_u64   addrs;
    __u32           cnt;
    __u64           bpf_cookie;

I'm afraid bpf_cookie has to be different for each addr, otherwise it's severely limiting. So it would be an array of cookies alongside an array of addresses

ok

Quoted 21 lines
Quoted 14 lines
  } kprobe;

Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe.

---
 include/linux/bpf_types.h      |   1 +
 include/uapi/linux/bpf.h       |  12 +++
 kernel/bpf/syscall.c           | 191 ++++++++++++++++++++++++++++++++-
 tools/include/uapi/linux/bpf.h |  12 +++
 4 files changed, 211 insertions(+), 5 deletions(-)

[...]

Quoted 4 lines
include/uapi/linux/bpf.h +0 -0
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE                (1U << 4)
 
Quoted 8 lines

include/uapi/linux/bpf.h enum bpf_link_type { · line 1120

Quoted 5 lines
include/uapi/linux/bpf.h +5 -0
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN    (1U << 0)
+

we have plenty of flexibility to have per-link type fields, so why not add bool is_retprobe next to addrs and cnt?

well I thought if I do that, people would suggest to use the empty flags field instead ;-)

we can move it there as you suggest, but I wonder it's good idea to use bool in uapi headers, because the bool size definition is vague

jirka

Quoted 19 lines
Quoted 16 lines
include/uapi/linux/bpf.h +5 -0
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1465,6 +1472,11 @@ union bpf_attr {
                                 */
                                __u64           bpf_cookie;
                        } perf_event;
+                       struct {
+                               __aligned_u64   addrs;
+                               __u32           cnt;
+                               __u64           bpf_cookie;
+                       } kprobe;
                };
        } link_create;
 

[...]

maintainer
#28 lore raw
In reply to Jiri Olsa
Quoted 43 lines · On Thu, 6 Jan 2022 09:29:02 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 42 lines · Masami Hiramatsu
Quoted 6 lines · On Tue, 4 Jan 2022 09:09:30 +0100 Jiri Olsa <[email protected]> wrote:

hi,
adding support to attach multiple kprobes within single syscall and speed up attachment of many kprobes.

The previous attempt [1] wasn't fast enough, so coming with new approach that adds new kprobe interface.

Yes, since register_kprobes() just registers multiple kprobes on array. This is designed for dozens of kprobes.

Quoted 2 lines

The attachment speed of of this approach (tested in bpftrace) is now comparable to ftrace tracer attachment speed.. fast ;-)

Yes, because that if ftrace, not kprobes.

Quoted 3 lines

The limit of this approach is forced by using ftrace as attach layer, so it allows only kprobes on function's entry (plus return probes).

Note that you also need to multiply the number of instances.

Quoted 9 lines

This patchset contains:
- kprobes support to register multiple kprobes with current kprobe API (patches 1 - 8)
- bpf support ot create new kprobe link allowing to attach multiple addresses (patches 9 - 14)

We don't need to care about multiple probes on same functions because it's taken care on the ftrace_ops layer.

Hmm, I think there may be a time to split the "kprobe as an interface for the software breakpoint" and "kprobe as a wrapper interface for the callbacks of various instrumentations", like 'raw_kprobe'(or kswbp) and 'kprobes'.
And this may be called as 'fprobe' as ftrace_ops wrapper. (But if the bpf is enough flexible, this kind of intermediate layer
may not be needed, it can use ftrace_ops directly, eventually)

Quoted 7 lines
Quoted 3 lines

Jiri, have you already considered to use ftrace_ops from the bpf directly? Are there any issues?
(bpf depends on 'kprobe' widely?)

at the moment there's not ftrace public interface for the return probe merged in, so to get the kretprobe working I had to use kprobe interface

Yeah, I found that too. We have to ask Steve to salvage it ;)

Quoted 6 lines

but.. there are patches Steven shared some time ago, that do that and make graph_ops available as kernel interface

I recall we considered graph_ops interface before as common attach layer for trampolines, which was bad, but it might actually make sense for kprobes

I started working on making 'fprobe' which will provide multiple function probe with similar interface of kprobes. See attached patch. Then you can use it in bpf, maybe with an union like

union {
struct kprobe kp; // for function body
struct fprobe fp; // for function entry and return
};

At this moment, fprobe only support entry_handler, but when we re-start the generic graph_ops interface, it is easy to expand to support exit_handler.
If this works, I think kretprobe can be phased out, since at that moment, kprobe_event can replace it with the fprobe exit_handler. (This is a benefit of decoupling the instrumentation layer from the event layer. It can choose the best way without changing user interface.)

Quoted 7 lines

I'll need to check it in more details but I think both graph_ops and kprobe do about similar thing wrt hooking return probe, so it should be comparable.. and they are already doing the same for the entry hook, because kprobe is mostly using ftrace for that

we would not need to introduce new program type - kprobe programs should be able to run from ftrace callbacks just fine

That seems to bind your mind. The program type is just a programing 'model' of the bpf. You can choose the best implementation to provide equal functionality. 'kprobe' in bpf is just a name that you call some instrumentations which can probe kernel code.

Thank you,

Quoted 7 lines

so we would have:
- kprobe type programs attaching to:
- new BPF_LINK_TYPE_FPROBE link using the graph_ops as attachment layer

jirka

Signature · 1 line
Masami Hiramatsu <[email protected]>

From 269b86597c166d6d4c5dd564168237603533165a Mon Sep 17 00:00:00 2001
From: Masami Hiramatsu <[email protected]>
Date: Thu, 6 Jan 2022 15:40:36 +0900
Subject: [PATCH] fprobe: Add ftrace based probe APIs

The fprobe is a wrapper API for ftrace function tracer. Unlike kprobes, this probes only supports the function entry, but it can probe multiple functions by one fprobe. The usage is almost same as the kprobe, user will specify the function names by fprobe::syms, the number of syms by fprobe::nsyms, and the user handler by fprobe::handler.

struct fprobe = { 0 };
const char *targets[] = {"func1", "func2", "func3"};

fprobe.handler = user_handler;
fprobe.nsyms = ARRAY_SIZE(targets);
fprobe.syms = targets;

ret = register_fprobe(&fprobe);
...

---
 include/linux/fprobes.h |  52 ++++++++++++++++
 kernel/trace/Kconfig    |  10 ++++
 kernel/trace/Makefile   |   1 +
 kernel/trace/fprobes.c  | 128 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 191 insertions(+)
 create mode 100644 include/linux/fprobes.h
 create mode 100644 kernel/trace/fprobes.c
include/linux/fprobes.h new file +52 -0
new file mode 100644
index 000000000000..22db748bf491
@@ -0,0 +1,52 @@
+#ifndef _LINUX_FPROBES_H
+#define _LINUX_FPROBES_H
+/* Simple ftrace probe wrapper */
+
+#include <linux/compiler.h>
+#include <linux/ftrace.h>
+
+struct fprobe {
+	const char		**syms;
+	unsigned long		*addrs;
+	unsigned int		nsyms;
+
+	struct ftrace_ops	ftrace;
+	unsigned long		nmissed;
+	unsigned int		flags;
+	void (*handler) (struct fprobe *, struct pt_regs *);
+};
+
+#define FPROBE_FL_DISABLED	1
+
+static inline bool fprobe_disabled(struct fprobe *fp)
+{
+	return (fp) ? fp->flags & FPROBE_FL_DISABLED : false;
+}
+
+#ifdef CONFIG_FPROBES
+int register_fprobe(struct fprobe *fp);
+int unregister_fprobe(struct fprobe *fp);
+#else
+static inline int register_fprobe(struct fprobe *fp)
+{
+	return -ENOTSUPP;
+}
+static inline int unregister_fprobe(struct fprobe *fp)
+{
+	return -ENOTSUPP;
+}
+#endif
+
+static inline void disable_fprobe(struct fprobe *fp)
+{
+	if (fp)
+		fp->flags |= FPROBE_FL_DISABLED;
+}
+
+static inline void enable_fprobe(struct fprobe *fp)
+{
+	if (fp)
+		fp->flags &= ~FPROBE_FL_DISABLED;
+}
+
+#endif
kernel/trace/Kconfig +10 -0
index 420ff4bc67fd..45a3618a20a7 100644
@@ -223,6 +223,16 @@ config DYNAMIC_FTRACE_WITH_ARGS
 	depends on DYNAMIC_FTRACE
 	depends on HAVE_DYNAMIC_FTRACE_WITH_ARGS
 
+config FPROBES
+	bool "Kernel Function Probe (fprobe)"
+	depends on FUNCTION_TRACER
+	depends on DYNAMIC_FTRACE_WITH_REGS
+	default n
+	help
+	  This option enables kernel function probe feature, which is
+	  similar to kprobes, but probes only for kernel function entries
+	  and it can probe multiple functions by one fprobe.
+
 config FUNCTION_PROFILER
 	bool "Kernel function profiler"
 	depends on FUNCTION_TRACER
kernel/trace/Makefile +1 -0
index bedc5caceec7..47a37a3bb974 100644
@@ -97,6 +97,7 @@ obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o
 obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o
 obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o
 obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
+obj-$(CONFIG_FPROBES) += fprobes.o
 
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
kernel/trace/fprobes.c new file +128 -0

128 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#29 lore raw

On Thu, Jan 06, 2022 at 10:59:43PM +0900, Masami Hiramatsu wrote:

SNIP

Quoted 13 lines
Quoted 13 lines
Quoted 12 lines

Hmm, I think there may be a time to split the "kprobe as an interface for the software breakpoint" and "kprobe as a wrapper interface for the callbacks of various instrumentations", like 'raw_kprobe'(or kswbp) and 'kprobes'.
And this may be called as 'fprobe' as ftrace_ops wrapper. (But if the bpf is enough flexible, this kind of intermediate layer
may not be needed, it can use ftrace_ops directly, eventually)

Jiri, have you already considered to use ftrace_ops from the bpf directly? Are there any issues?
(bpf depends on 'kprobe' widely?)

Quoted 5 lines
Quoted 3 lines

at the moment there's not ftrace public interface for the return probe merged in, so to get the kretprobe working I had to use kprobe interface

Yeah, I found that too. We have to ask Steve to salvage it ;)

I got those patches rebased like half a year ago upstream code, so should be easy to revive them

Quoted 17 lines
Quoted 7 lines

but.. there are patches Steven shared some time ago, that do that and make graph_ops available as kernel interface

I recall we considered graph_ops interface before as common attach layer for trampolines, which was bad, but it might actually make sense for kprobes

I started working on making 'fprobe' which will provide multiple function probe with similar interface of kprobes. See attached patch. Then you can use it in bpf, maybe with an union like

union {
struct kprobe kp; // for function body
struct fprobe fp; // for function entry and return
};

Quoted 9 lines

At this moment, fprobe only support entry_handler, but when we re-start the generic graph_ops interface, it is easy to expand to support exit_handler.
If this works, I think kretprobe can be phased out, since at that moment, kprobe_event can replace it with the fprobe exit_handler. (This is a benefit of decoupling the instrumentation layer from the event layer. It can choose the best way without changing user interface.)

I can resend out graph_ops patches if you want to base it directly on that

Quoted 5 lines
Quoted 5 lines

I'll need to check it in more details but I think both graph_ops and kprobe do about similar thing wrt hooking return probe, so it should be comparable.. and they are already doing the same for the entry hook, because kprobe is mostly using ftrace for that

Quoted 7 lines
Quoted 2 lines

we would not need to introduce new program type - kprobe programs should be able to run from ftrace callbacks just fine

That seems to bind your mind. The program type is just a programing 'model' of the bpf. You can choose the best implementation to provide equal functionality. 'kprobe' in bpf is just a name that you call some instrumentations which can probe kernel code.

I don't want to introduce new type, there's some dependencies in bpf verifier and helpers code we'd need to handle for that

I'm looking for solution for current kprobe bpf program type to be registered for multiple addresses quickly

Quoted 54 lines

Thank you,

Quoted 7 lines

so we would have:
- kprobe type programs attaching to:
- new BPF_LINK_TYPE_FPROBE link using the graph_ops as attachment layer

jirka

Signature · 34 lines
Masami Hiramatsu <[email protected]>

From 269b86597c166d6d4c5dd564168237603533165a Mon Sep 17 00:00:00 2001
From: Masami Hiramatsu <[email protected]>
Date: Thu, 6 Jan 2022 15:40:36 +0900
Subject: [PATCH] fprobe: Add ftrace based probe APIs

The fprobe is a wrapper API for ftrace function tracer.
Unlike kprobes, this probes only supports the function entry, but
it can probe multiple functions by one fprobe. The usage is almost
same as the kprobe, user will specify the function names by
fprobe::syms, the number of syms by fprobe::nsyms, and the user
handler by fprobe::handler.

struct fprobe = { 0 };
const char *targets[] = {"func1", "func2", "func3"};

fprobe.handler = user_handler;
fprobe.nsyms = ARRAY_SIZE(targets);
fprobe.syms = targets;

ret = register_fprobe(&fprobe);
...


Signed-off-by: Masami Hiramatsu <[email protected]>
---
 include/linux/fprobes.h |  52 ++++++++++++++++
 kernel/trace/Kconfig    |  10 ++++
 kernel/trace/Makefile   |   1 +
 kernel/trace/fprobes.c  | 128 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 191 insertions(+)
 create mode 100644 include/linux/fprobes.h
 create mode 100644 kernel/trace/fprobes.c
include/linux/fprobes.h new file +0 -0
new file mode 100644
index 000000000000..22db748bf491

include/linux/fprobes.h line 10

Quoted 10 lines
include/linux/fprobes.h new file +10 -0
@@ -0,0 +1,52 @@
+#ifndef _LINUX_FPROBES_H
+#define _LINUX_FPROBES_H
+/* Simple ftrace probe wrapper */
+
+#include <linux/compiler.h>
+#include <linux/ftrace.h>
+
+struct fprobe {
+	const char		**syms;
+	unsigned long		*addrs;

could you add array of user data for each addr/sym?

SNIP

Quoted 43 lines
include/linux/fprobes.h new file +43 -0
+static int populate_func_addresses(struct fprobe *fp)
+{
+	unsigned int i;
+
+	fp->addrs = kmalloc(sizeof(void *) * fp->nsyms, GFP_KERNEL);
+	if (!fp->addrs)
+		return -ENOMEM;
+
+	for (i = 0; i < fp->nsyms; i++) {
+		fp->addrs[i] = kallsyms_lookup_name(fp->syms[i]);
+		if (!fp->addrs[i]) {
+			kfree(fp->addrs);
+			fp->addrs = NULL;
+			return -ENOENT;
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * register_fprobe - Register fprobe to ftrace
+ * @fp: A fprobe data structure to be registered.
+ *
+ * This expects the user set @fp::syms or @fp::addrs (not both),
+ * @fp::nsyms (number of entries of @fp::syms or @fp::addrs) and
+ * @fp::handler. Other fields are initialized by this function.
+ */
+int register_fprobe(struct fprobe *fp)
+{
+	unsigned int i;
+	int ret;
+
+	if (!fp)
+		return -EINVAL;
+
+	if (!fp->nsyms || (!fp->syms && !fp->addrs) || (fp->syms && fp->addrs))
+		return -EINVAL;
+
+	if (fp->syms) {
+		ret = populate_func_addresses(fp);
+		if (ret < 0)
+			return ret;

include/linux/fprobes.h

Quoted 10 lines
include/linux/fprobes.h new file +10 -0
+	}
+
+	fp->ftrace.func = fprobe_handler;
+	fp->ftrace.flags = FTRACE_OPS_FL_SAVE_REGS;
+
+	for (i = 0; i < fp->nsyms; i++) {
+		ret = ftrace_set_filter_ip(&fp->ftrace, fp->addrs[i], 0, 0);
+		if (ret < 0)
+			goto error;
+	}

I introduced ftrace_set_filter_ips, because loop like above was slow:
https://lore.kernel.org/bpf/[email protected]/

thanks,
jirka

Quoted 42 lines
include/linux/fprobes.h new file +39 -0
+
+	fp->nmissed = 0;
+	ret = register_ftrace_function(&fp->ftrace);
+	if (!ret)
+		return ret;
+
+error:
+	if (fp->syms) {
+		kfree(fp->addrs);
+		fp->addrs = NULL;
+	}
+
+	return ret;
+}
+
+/**
+ * unregister_fprobe - Unregister fprobe from ftrace
+ * @fp: A fprobe data structure to be unregistered.
+ */
+int unregister_fprobe(struct fprobe *fp)
+{
+	int ret;
+
+	if (!fp)
+		return -EINVAL;
+
+	if (!fp->nsyms || !fp->addrs)
+		return -EINVAL;
+
+	ret = unregister_ftrace_function(&fp->ftrace);
+
+	if (fp->syms) {
+		/* fp->addrs is allocated by register_fprobe() */
+		kfree(fp->addrs);
+		fp->addrs = NULL;
+	}
+
+	return ret;
+}
Signature · 1 line
2.25.1
maintainer
#31 lore raw
In reply to Masami Hiramatsu
Quoted 5 lines · On Thu, 6 Jan 2022 22:59:43 +0900 Masami Hiramatsu <[email protected]> wrote:
Quoted 3 lines

at the moment there's not ftrace public interface for the return probe merged in, so to get the kretprobe working I had to use kprobe interface

Yeah, I found that too. We have to ask Steve to salvage it ;)

I have one more week of being unemployed (and I'm done with my office renovation), so perhaps I'll start looking into this.

This was the work to merge function graph tracer with kretprobes, right?

-- Steve

#17 lore raw

Re: [PATCH 08/13] bpf: Add kprobe link for attaching raw kprobes

In reply to Jiri Olsa
Quoted 54 lines · Jiri Olsa
Quoted 11 lines · Andrii Nakryiko
Quoted 10 lines · Jiri Olsa

Adding new link type BPF_LINK_TYPE_KPROBE to attach kprobes directly through register_kprobe/kretprobe API.

Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program.

The new link allows to create multiple kprobes link by using new link_create interface:

Quoted 8 lines
Quoted 4 lines
  struct {
    __aligned_u64   addrs;
    __u32           cnt;
    __u64           bpf_cookie;

I'm afraid bpf_cookie has to be different for each addr, otherwise it's severely limiting. So it would be an array of cookies alongside an array of addresses

ok

Quoted 21 lines
Quoted 14 lines
  } kprobe;

Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe.

---
 include/linux/bpf_types.h      |   1 +
 include/uapi/linux/bpf.h       |  12 +++
 kernel/bpf/syscall.c           | 191 ++++++++++++++++++++++++++++++++-
 tools/include/uapi/linux/bpf.h |  12 +++
 4 files changed, 211 insertions(+), 5 deletions(-)

[...]

Quoted 4 lines
include/uapi/linux/bpf.h +0 -0
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE                (1U << 4)
 
Quoted 8 lines

include/uapi/linux/bpf.h enum bpf_link_type { · line 1120

Quoted 5 lines
include/uapi/linux/bpf.h +5 -0
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN    (1U << 0)
+

we have plenty of flexibility to have per-link type fields, so why not add bool is_retprobe next to addrs and cnt?

Quoted 5 lines

well I thought if I do that, people would suggest to use the empty flags field instead ;-)

we can move it there as you suggest, but I wonder it's good idea to use bool in uapi headers, because the bool size definition is vague

Good point. No 'bool' please.
grep bool include/uapi/linux/
Only gives openvswitch.h and it's guarded by ifdef KERNEL So not a single uapi header has bool in it.

#32 lore raw
In reply to Masami Hiramatsu
Quoted 5 lines · Masami Hiramatsu

That seems to bind your mind. The program type is just a programing 'model' of the bpf. You can choose the best implementation to provide equal functionality. 'kprobe' in bpf is just a name that you call some instrumentations which can probe kernel code.

No. We're not going to call it "fprobe" or any other name. From bpf user's pov it's going to be "multi attach kprobe", because this is how everyone got to know kprobes.
The 99% usage is at the beginning of the funcs.
When users say "kprobe" they don't care how kernel attaches it. The func entry limitation for "multi attach kprobe" is a no-brainer.

And we need both "multi attach kprobe" and "multi attach kretprobe" at the same time. It's no go to implement one first and the other some time later.

#18 lore raw

Re: [PATCH 08/13] bpf: Add kprobe link for attaching raw kprobes

In reply to Alexei Starovoitov
Quoted 59 lines · Alexei Starovoitov
Quoted 57 lines · Jiri Olsa
Quoted 11 lines · Andrii Nakryiko
Quoted 10 lines · Jiri Olsa

Adding new link type BPF_LINK_TYPE_KPROBE to attach kprobes directly through register_kprobe/kretprobe API.

Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program.

The new link allows to create multiple kprobes link by using new link_create interface:

Quoted 8 lines
Quoted 4 lines
  struct {
    __aligned_u64   addrs;
    __u32           cnt;
    __u64           bpf_cookie;

I'm afraid bpf_cookie has to be different for each addr, otherwise it's severely limiting. So it would be an array of cookies alongside an array of addresses

ok

Quoted 21 lines
Quoted 14 lines
  } kprobe;

Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe.

---
 include/linux/bpf_types.h      |   1 +
 include/uapi/linux/bpf.h       |  12 +++
 kernel/bpf/syscall.c           | 191 ++++++++++++++++++++++++++++++++-
 tools/include/uapi/linux/bpf.h |  12 +++
 4 files changed, 211 insertions(+), 5 deletions(-)

[...]

Quoted 4 lines
include/uapi/linux/bpf.h +0 -0
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE                (1U << 4)
 
Quoted 8 lines

include/uapi/linux/bpf.h enum bpf_link_type { · line 1120

Quoted 5 lines
include/uapi/linux/bpf.h +5 -0
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN    (1U << 0)
+

we have plenty of flexibility to have per-link type fields, so why not add bool is_retprobe next to addrs and cnt?

well I thought if I do that, people would suggest to use the empty flags field instead ;-)

Quoted 7 lines
Quoted 2 lines

we can move it there as you suggest, but I wonder it's good idea to use bool in uapi headers, because the bool size definition is vague

Good point. No 'bool' please.
grep bool include/uapi/linux/
Only gives openvswitch.h and it's guarded by ifdef KERNEL So not a single uapi header has bool in it.

Yeah, I don't insist on bool specifically.

But I was trying to avoid link_create.flags to become map_flags where we have various flags, each taking a separate bit, but most flags don't apply to most map types. Ideally link_create.flags would have few flags that apply to all or most link types (i.e., something orthogonal to a specific link type), and for this case we could have kprobe-specific flags (link_create.kprobe.flags) to adjust kprobe link creation behavior.

But I don't know, maybe I'm overthinking this.

maintainer
#33 lore raw
In reply to Alexei Starovoitov
Quoted 7 lines · On Thu, 6 Jan 2022 09:40:17 -0800 Alexei Starovoitov…
Quoted 5 lines · Masami Hiramatsu

That seems to bind your mind. The program type is just a programing 'model' of the bpf. You can choose the best implementation to provide equal functionality. 'kprobe' in bpf is just a name that you call some instrumentations which can probe kernel code.

Quoted 6 lines

No. We're not going to call it "fprobe" or any other name. From bpf user's pov it's going to be "multi attach kprobe", because this is how everyone got to know kprobes.
The 99% usage is at the beginning of the funcs.
When users say "kprobe" they don't care how kernel attaches it. The func entry limitation for "multi attach kprobe" is a no-brainer.

Agreed. I think I might mislead you. From the bpf user pov, it always be shown as 'multi attached kprobes (but only for the function entry)' the 'fprobe' is kernel internal API name.

Quoted 3 lines

And we need both "multi attach kprobe" and "multi attach kretprobe" at the same time. It's no go to implement one first and the other some time later.

You can provide the interface to user space, but the kernel implementation is optimized step by step. We can start it with using real multiple kretprobes, and then, switch to 'fprobe' after integrating fgraph callback. :)

Thank you,

Signature · 1 line
Masami Hiramatsu <[email protected]>
#34 lore raw
Quoted 22 lines · Masami Hiramatsu
Quoted 7 lines · On Thu, 6 Jan 2022 09:40:17 -0800 Alexei Starovoitov…
Quoted 5 lines · Masami Hiramatsu

That seems to bind your mind. The program type is just a programing 'model' of the bpf. You can choose the best implementation to provide equal functionality. 'kprobe' in bpf is just a name that you call some instrumentations which can probe kernel code.

Quoted 6 lines

No. We're not going to call it "fprobe" or any other name. From bpf user's pov it's going to be "multi attach kprobe", because this is how everyone got to know kprobes.
The 99% usage is at the beginning of the funcs.
When users say "kprobe" they don't care how kernel attaches it. The func entry limitation for "multi attach kprobe" is a no-brainer.

Agreed. I think I might mislead you. From the bpf user pov, it always be shown as 'multi attached kprobes (but only for the function entry)' the 'fprobe' is kernel internal API name.

Quoted 8 lines
Quoted 3 lines

And we need both "multi attach kprobe" and "multi attach kretprobe" at the same time. It's no go to implement one first and the other some time later.

You can provide the interface to user space, but the kernel implementation is optimized step by step. We can start it with using real multiple kretprobes, and then, switch to 'fprobe' after integrating fgraph callback. :)

Sounds good to me.
My point was that users often want to say:
"profile speed of all foo* functions".
To perform such a command a tracer would need to attach kprobes and kretprobes to all such functions. The speed of attach/detach has to be fast. Currently tracers artificially limit the regex just because attach/detach is so slow that the user will likely Ctrl-C instead of waiting for many seconds.

maintainer
#30 lore raw
In reply to Jiri Olsa
Quoted 17 lines · On Thu, 6 Jan 2022 15:57:10 +0100 Jiri Olsa <[email protected]> wrote:

On Thu, Jan 06, 2022 at 10:59:43PM +0900, Masami Hiramatsu wrote:

SNIP

Quoted 13 lines
Quoted 13 lines
Quoted 12 lines

Hmm, I think there may be a time to split the "kprobe as an interface for the software breakpoint" and "kprobe as a wrapper interface for the callbacks of various instrumentations", like 'raw_kprobe'(or kswbp) and 'kprobes'.
And this may be called as 'fprobe' as ftrace_ops wrapper. (But if the bpf is enough flexible, this kind of intermediate layer
may not be needed, it can use ftrace_ops directly, eventually)

Jiri, have you already considered to use ftrace_ops from the bpf directly? Are there any issues?
(bpf depends on 'kprobe' widely?)

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

at the moment there's not ftrace public interface for the return probe merged in, so to get the kretprobe working I had to use kprobe interface

Yeah, I found that too. We have to ask Steve to salvage it ;)

I got those patches rebased like half a year ago upstream code, so should be easy to revive them

Nice! :)

Quoted 28 lines
Quoted 18 lines
Quoted 8 lines

but.. there are patches Steven shared some time ago, that do that and make graph_ops available as kernel interface

I recall we considered graph_ops interface before as common attach layer for trampolines, which was bad, but it might actually make sense for kprobes

I started working on making 'fprobe' which will provide multiple function probe with similar interface of kprobes. See attached patch. Then you can use it in bpf, maybe with an union like

union {
struct kprobe kp; // for function body
struct fprobe fp; // for function entry and return
};

Quoted 9 lines

At this moment, fprobe only support entry_handler, but when we re-start the generic graph_ops interface, it is easy to expand to support exit_handler.
If this works, I think kretprobe can be phased out, since at that moment, kprobe_event can replace it with the fprobe exit_handler. (This is a benefit of decoupling the instrumentation layer from the event layer. It can choose the best way without changing user interface.)

Quoted 2 lines

I can resend out graph_ops patches if you want to base it directly on that

Yes, that's very helpful. Now I'm considering to use it (or via fprobe) from kretprobes like ftrace-based kprobe.

Quoted 13 lines
Quoted 5 lines
Quoted 5 lines

I'll need to check it in more details but I think both graph_ops and kprobe do about similar thing wrt hooking return probe, so it should be comparable.. and they are already doing the same for the entry hook, because kprobe is mostly using ftrace for that

Quoted 7 lines
Quoted 2 lines

we would not need to introduce new program type - kprobe programs should be able to run from ftrace callbacks just fine

That seems to bind your mind. The program type is just a programing 'model' of the bpf. You can choose the best implementation to provide equal functionality. 'kprobe' in bpf is just a name that you call some instrumentations which can probe kernel code.

Quoted 5 lines

I don't want to introduce new type, there's some dependencies in bpf verifier and helpers code we'd need to handle for that

I'm looking for solution for current kprobe bpf program type to be registered for multiple addresses quickly

Yes, as I replied to Alex, the bpf program type itself keeps 'kprobe'. For example, you've introduced bpf_kprobe_link at [8/13],

struct bpf_kprobe_link {
	struct bpf_link link;
	union {
		struct kretprobe rp;
		struct fprobe fp;
	};
	bool is_return;
	bool is_fentry;
	kprobe_opcode_t **addrs;
	u32 cnt;
	u64 bpf_cookie;
};

If all "addrs" are function entry, ::fp will be used. If cnt == 1 then use ::rp.

Quoted 54 lines
Quoted 54 lines
Quoted 6 lines

so we would have:
- kprobe type programs attaching to:
- new BPF_LINK_TYPE_FPROBE link using the graph_ops as attachment layer

jirka

Signature · 34 lines
Masami Hiramatsu <[email protected]>

From 269b86597c166d6d4c5dd564168237603533165a Mon Sep 17 00:00:00 2001
From: Masami Hiramatsu <[email protected]>
Date: Thu, 6 Jan 2022 15:40:36 +0900
Subject: [PATCH] fprobe: Add ftrace based probe APIs

The fprobe is a wrapper API for ftrace function tracer.
Unlike kprobes, this probes only supports the function entry, but
it can probe multiple functions by one fprobe. The usage is almost
same as the kprobe, user will specify the function names by
fprobe::syms, the number of syms by fprobe::nsyms, and the user
handler by fprobe::handler.

struct fprobe = { 0 };
const char *targets[] = {"func1", "func2", "func3"};

fprobe.handler = user_handler;
fprobe.nsyms = ARRAY_SIZE(targets);
fprobe.syms = targets;

ret = register_fprobe(&fprobe);
...


Signed-off-by: Masami Hiramatsu <[email protected]>
---
 include/linux/fprobes.h |  52 ++++++++++++++++
 kernel/trace/Kconfig    |  10 ++++
 kernel/trace/Makefile   |   1 +
 kernel/trace/fprobes.c  | 128 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 191 insertions(+)
 create mode 100644 include/linux/fprobes.h
 create mode 100644 kernel/trace/fprobes.c
include/linux/fprobes.h new file +4 -0
new file mode 100644
index 000000000000..22db748bf491
@@ -0,0 +1,52 @@
+#ifndef _LINUX_FPROBES_H
+#define _LINUX_FPROBES_H
+/* Simple ftrace probe wrapper */
+
Quoted 8 lines

include/linux/fprobes.h line 10

Quoted 6 lines
include/linux/fprobes.h new file +6 -0
+#include <linux/compiler.h>
+#include <linux/ftrace.h>
+
+struct fprobe {
+	const char		**syms;
+	unsigned long		*addrs;

could you add array of user data for each addr/sym?

OK, something like this?

	void	**user_data;

But note that you need O(N) to search the entry corresponding to a specific address. To reduce the overhead, we may need to sort the array in advance (e.g. when registering it).

Quoted 51 lines

SNIP

Quoted 48 lines
include/linux/fprobes.h new file +48 -0
+static int populate_func_addresses(struct fprobe *fp)
+{
+	unsigned int i;
+
+	fp->addrs = kmalloc(sizeof(void *) * fp->nsyms, GFP_KERNEL);
+	if (!fp->addrs)
+		return -ENOMEM;
+
+	for (i = 0; i < fp->nsyms; i++) {
+		fp->addrs[i] = kallsyms_lookup_name(fp->syms[i]);
+		if (!fp->addrs[i]) {
+			kfree(fp->addrs);
+			fp->addrs = NULL;
+			return -ENOENT;
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * register_fprobe - Register fprobe to ftrace
+ * @fp: A fprobe data structure to be registered.
+ *
+ * This expects the user set @fp::syms or @fp::addrs (not both),
+ * @fp::nsyms (number of entries of @fp::syms or @fp::addrs) and
+ * @fp::handler. Other fields are initialized by this function.
+ */
+int register_fprobe(struct fprobe *fp)
+{
+	unsigned int i;
+	int ret;
+
+	if (!fp)
+		return -EINVAL;
+
+	if (!fp->nsyms || (!fp->syms && !fp->addrs) || (fp->syms && fp->addrs))
+		return -EINVAL;
+
+	if (fp->syms) {
+		ret = populate_func_addresses(fp);
+		if (ret < 0)
+			return ret;
+	}
+
+	fp->ftrace.func = fprobe_handler;
+	fp->ftrace.flags = FTRACE_OPS_FL_SAVE_REGS;
+
Quoted 8 lines

include/linux/fprobes.h

Quoted 5 lines
include/linux/fprobes.h new file +5 -0
+	for (i = 0; i < fp->nsyms; i++) {
+		ret = ftrace_set_filter_ip(&fp->ftrace, fp->addrs[i], 0, 0);
+		if (ret < 0)
+			goto error;
+	}

I introduced ftrace_set_filter_ips, because loop like above was slow:
https://lore.kernel.org/bpf/[email protected]/

Ah, thanks for noticing!

Thank you,

Quoted 47 lines

thanks,
jirka

Quoted 42 lines
include/linux/fprobes.h new file +39 -0
+
+	fp->nmissed = 0;
+	ret = register_ftrace_function(&fp->ftrace);
+	if (!ret)
+		return ret;
+
+error:
+	if (fp->syms) {
+		kfree(fp->addrs);
+		fp->addrs = NULL;
+	}
+
+	return ret;
+}
+
+/**
+ * unregister_fprobe - Unregister fprobe from ftrace
+ * @fp: A fprobe data structure to be unregistered.
+ */
+int unregister_fprobe(struct fprobe *fp)
+{
+	int ret;
+
+	if (!fp)
+		return -EINVAL;
+
+	if (!fp->nsyms || !fp->addrs)
+		return -EINVAL;
+
+	ret = unregister_ftrace_function(&fp->ftrace);
+
+	if (fp->syms) {
+		/* fp->addrs is allocated by register_fprobe() */
+		kfree(fp->addrs);
+		fp->addrs = NULL;
+	}
+
+	return ret;
+}
Signature · 1 line
2.25.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
maintainer
#35 lore raw
In reply to Alexei Starovoitov
Quoted 32 lines · On Thu, 6 Jan 2022 16:20:05 -0800 Alexei Starovoitov…
Quoted 22 lines · Masami Hiramatsu
Quoted 7 lines · On Thu, 6 Jan 2022 09:40:17 -0800 Alexei Starovoitov…
Quoted 5 lines · Masami Hiramatsu

That seems to bind your mind. The program type is just a programing 'model' of the bpf. You can choose the best implementation to provide equal functionality. 'kprobe' in bpf is just a name that you call some instrumentations which can probe kernel code.

Quoted 6 lines

No. We're not going to call it "fprobe" or any other name. From bpf user's pov it's going to be "multi attach kprobe", because this is how everyone got to know kprobes.
The 99% usage is at the beginning of the funcs.
When users say "kprobe" they don't care how kernel attaches it. The func entry limitation for "multi attach kprobe" is a no-brainer.

Agreed. I think I might mislead you. From the bpf user pov, it always be shown as 'multi attached kprobes (but only for the function entry)' the 'fprobe' is kernel internal API name.

Quoted 8 lines
Quoted 3 lines

And we need both "multi attach kprobe" and "multi attach kretprobe" at the same time. It's no go to implement one first and the other some time later.

You can provide the interface to user space, but the kernel implementation is optimized step by step. We can start it with using real multiple kretprobes, and then, switch to 'fprobe' after integrating fgraph callback. :)

Quoted 5 lines

Sounds good to me.
My point was that users often want to say:
"profile speed of all foo* functions".
To perform such a command a tracer would need to attach kprobes and kretprobes to all such functions.

Yeah, I know. That is more than 10 years issue since systemtap. :)

Quoted 1 line

The speed of attach/detach has to be fast.

Yes, that's why I provided register/unregister_kprobes() but it sounds not enough (and maybe not optimized enough because all handlers are same)

Quoted 3 lines

Currently tracers artificially limit the regex just because attach/detach is so slow that the user will likely Ctrl-C instead of waiting for many seconds.

Ah, OK.
Anyway I also would like to fix that issue. If user wants only function entry/exit, it should be done by ftrace. But since the syntax (and user's mind model) it should be done via 'kprobe', so transparently converting such request to ftrace is needed.

Thank you,

Signature · 1 line
Masami Hiramatsu <[email protected]>
maintainer
#36 lore raw

cover fprobe: Introduce fprobe function entry/exit probe

In reply to Jiri Olsa

Hi Jiri,

Here is a short series of patches, which shows what I replied to your series.

This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return, which I cloned from kretprobe.

I also rewrite your [08/13] bpf patch to use this fprobe instead of kprobes. I didn't tested that one, but the sample module seems to work. Please test bpf part with your libbpf updates.

BTW, while implementing the fprobe, I introduced the per-probe point private data, but I'm not sure why you need it. It seems that data is not used from bpf...

If this is good for you, I would like to proceed this with the rethook and rewrite the kretprobe to use the rethook to hook the functions. That should be much cleaner (and easy to prepare for the fgraph tracer integration)

Thank you,

---

Jiri Olsa (1):
bpf: Add kprobe link for attaching raw kprobes

Masami Hiramatsu (5):
fprobe: Add ftrace based probe APIs
rethook: Add a generic return hook
rethook: x86: Add rethook x86 implementation
fprobe: Add exit_handler support
fprobe: Add sample program for fprobe

 arch/x86/Kconfig                |    1 
 arch/x86/kernel/Makefile        |    1 
 arch/x86/kernel/rethook.c       |  115 ++++++++++++++++++++
 include/linux/bpf_types.h       |    1 
 include/linux/fprobes.h         |   75 +++++++++++++
 include/linux/rethook.h         |   74 +++++++++++++
 include/linux/sched.h           |    3 +
 include/uapi/linux/bpf.h        |   12 ++
 kernel/bpf/syscall.c            |  199 +++++++++++++++++++++++++++++++++-
 kernel/exit.c                   |    2 
 kernel/fork.c                   |    3 +
 kernel/trace/Kconfig            |   22 ++++
 kernel/trace/Makefile           |    2 
 kernel/trace/fprobes.c          |  187 ++++++++++++++++++++++++++++++++
 kernel/trace/rethook.c          |  226 +++++++++++++++++++++++++++++++++++++++
 samples/Kconfig                 |    6 +
 samples/Makefile                |    1 
 samples/fprobe/Makefile         |    3 +
 samples/fprobe/fprobe_example.c |  103 ++++++++++++++++++
 tools/include/uapi/linux/bpf.h  |   12 ++
 20 files changed, 1043 insertions(+), 5 deletions(-)
 create mode 100644 arch/x86/kernel/rethook.c
 create mode 100644 include/linux/fprobes.h
 create mode 100644 include/linux/rethook.h
 create mode 100644 kernel/trace/fprobes.c
 create mode 100644 kernel/trace/rethook.c
 create mode 100644 samples/fprobe/Makefile
 create mode 100644 samples/fprobe/fprobe_example.c

--
Masami Hiramatsu (Linaro) <[email protected]>

maintainer
#37 lore raw

RFC PATCH 1/6 fprobe: Add ftrace based probe APIs

The fprobe is a wrapper API for ftrace function tracer. Unlike kprobes, this probes only supports the function entry, but it can probe multiple functions by one fprobe. The usage is almost same as the kprobe, user will specify the function names by fprobe::entries[].syms, the number of syms by fprobe::nentry, and the user handler by fprobe::entry_handler.

struct fprobe fp = { 0 };
struct fprobe_entry targets[] =

	{{.sym = "func1"}, {.sym = "func2"}, {.sym = "func3"}};

fp.handler = user_handler;
fp.nentry = ARRAY_SIZE(targets);

fp.entries = targets;

ret = register_fprobe(&fp);

Note that the fp::entries will be sorted by the converted function address.

---
 include/linux/fprobes.h |   71 +++++++++++++++++++++++++
 kernel/trace/Kconfig    |   10 ++++
 kernel/trace/Makefile   |    1 
 kernel/trace/fprobes.c  |  132 +++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 214 insertions(+)
 create mode 100644 include/linux/fprobes.h
 create mode 100644 kernel/trace/fprobes.c
include/linux/fprobes.h new file +71 -0
new file mode 100644
index 000000000000..fa85a2fc3ad1
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Simple ftrace probe wrapper */
+#ifndef _LINUX_FPROBES_H
+#define _LINUX_FPROBES_H
+
+#include <linux/compiler.h>
+#include <linux/ftrace.h>
+
+/*
+ * fprobe_entry - function entry for fprobe
+ * @sym: The symbol name of the function.
+ * @addr: The address of @sym.
+ * @data: per-entry data
+ *
+ * User must specify either @sym or @addr (not both). @data is optional.
+ */
+struct fprobe_entry {
+	const char	*sym;
+	unsigned long	addr;
+	void		*data;
+};
+
+struct fprobe {
+	struct fprobe_entry	*entries;
+	unsigned int		nentry;
+
+	struct ftrace_ops	ftrace;
+	unsigned long		nmissed;
+	unsigned int		flags;
+	void (*entry_handler) (struct fprobe *, unsigned long, struct pt_regs *);
+};
+
+#define FPROBE_FL_DISABLED	1
+
+static inline bool fprobe_disabled(struct fprobe *fp)
+{
+	return (fp) ? fp->flags & FPROBE_FL_DISABLED : false;
+}
+
+#ifdef CONFIG_FPROBES
+int register_fprobe(struct fprobe *fp);
+int unregister_fprobe(struct fprobe *fp);
+struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr);
+#else
+static inline int register_fprobe(struct fprobe *fp)
+{
+	return -ENOTSUPP;
+}
+static inline int unregister_fprobe(struct fprobe *fp)
+{
+	return -ENOTSUPP;
+}
+struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
+{
+	return NULL;
+}
+#endif
+
+static inline void disable_fprobe(struct fprobe *fp)
+{
+	if (fp)
+		fp->flags |= FPROBE_FL_DISABLED;
+}
+
+static inline void enable_fprobe(struct fprobe *fp)
+{
+	if (fp)
+		fp->flags &= ~FPROBE_FL_DISABLED;
+}
+
+#endif
kernel/trace/Kconfig +10 -0
index 420ff4bc67fd..45a3618a20a7 100644
@@ -223,6 +223,16 @@ config DYNAMIC_FTRACE_WITH_ARGS
 	depends on DYNAMIC_FTRACE
 	depends on HAVE_DYNAMIC_FTRACE_WITH_ARGS
 
+config FPROBES
+	bool "Kernel Function Probe (fprobe)"
+	depends on FUNCTION_TRACER
+	depends on DYNAMIC_FTRACE_WITH_REGS
+	default n
+	help
+	  This option enables kernel function probe feature, which is
+	  similar to kprobes, but probes only for kernel function entries
+	  and it can probe multiple functions by one fprobe.
+
 config FUNCTION_PROFILER
 	bool "Kernel function profiler"
 	depends on FUNCTION_TRACER
kernel/trace/Makefile +1 -0
index bedc5caceec7..47a37a3bb974 100644
@@ -97,6 +97,7 @@ obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o
 obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o
 obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o
 obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
+obj-$(CONFIG_FPROBES) += fprobes.o
 
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
kernel/trace/fprobes.c new file +132 -0

132 lines not on this page · raw message

maintainer
#38 lore raw

RFC PATCH 2/6 rethook: Add a generic return hook

In reply to Masami Hiramatsu

Add a return hook framework which hooks the function return. Most of the idea came from the kretprobe, but this is independent from kretprobe.
Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just a additional hook.

---
 include/linux/rethook.h |   74 +++++++++++++++
 include/linux/sched.h   |    3 +
 kernel/exit.c           |    2 
 kernel/fork.c           |    3 +
 kernel/trace/Kconfig    |   11 ++
 kernel/trace/Makefile   |    1 
 kernel/trace/rethook.c  |  226 +++++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 320 insertions(+)
 create mode 100644 include/linux/rethook.h
 create mode 100644 kernel/trace/rethook.c
include/linux/rethook.h new file +74 -0
new file mode 100644
index 000000000000..2622bcd5213a
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Return hooking with list-based shadow stack.
+ */
+#ifndef _LINUX_RETHOOK_H
+#define _LINUX_RETHOOK_H
+
+#include <linux/compiler.h>
+#include <linux/freelist.h>
+#include <linux/llist.h>
+#include <linux/rcupdate.h>
+#include <linux/refcount.h>
+
+struct rethook_node;
+
+typedef void (*rethook_handler_t) (struct rethook_node *, void *, struct pt_regs *);
+
+struct rethook {
+	void			*data;
+	rethook_handler_t	handler;
+	struct freelist_head	pool;
+	refcount_t		ref;
+	struct rcu_head		rcu;
+};
+
+struct rethook_node {
+	union {
+		struct freelist_node freelist;
+		struct rcu_head      rcu;
+	};
+	struct llist_node	llist;
+	struct rethook		*rethook;
+	unsigned long		ret_addr;
+	unsigned long		frame;
+};
+
+int rethook_node_init(struct rethook_node *node);
+
+struct rethook *rethook_alloc(void *data, rethook_handler_t handler);
+void rethook_free(struct rethook *rh);
+void rethook_add_node(struct rethook *rh, struct rethook_node *node);
+
+struct rethook_node *rethook_try_get(struct rethook *rh);
+void rethook_node_recycle(struct rethook_node *node);
+void rethook_hook_current(struct rethook_node *node, struct pt_regs *regs);
+
+unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame,
+				    struct llist_node **cur);
+
+/* Arch dependent code must implement this and trampoline code */
+void arch_rethook_prepare(struct rethook_node *node, struct pt_regs *regs);
+void arch_rethook_trampoline(void);
+
+static inline bool is_rethook_trampoline(unsigned long addr)
+{
+	return addr == (unsigned long)arch_rethook_trampoline;
+}
+
+/* If the architecture needs a fixup the return address, implement it. */
+void arch_rethook_fixup_return(struct pt_regs *regs,
+			       unsigned long correct_ret_addr);
+
+/* Generic trampoline handler, arch code must prepare asm stub */
+unsigned long rethook_trampoline_handler(struct pt_regs *regs,
+					 unsigned long frame);
+
+#ifdef CONFIG_RETHOOK
+void rethook_flush_task(struct task_struct *tk);
+#else
+#define rethook_flush_task(tsk)	do { } while (0)
+#endif
+
+#endif
+
include/linux/sched.h +3 -0
index 78c351e35fec..2bfabf5355b7 100644
@@ -1473,6 +1473,9 @@ struct task_struct {
 #ifdef CONFIG_KRETPROBES
 	struct llist_head               kretprobe_instances;
 #endif
+#ifdef CONFIG_RETHOOK
+	struct llist_head               rethooks;
+#endif
 
 #ifdef CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH
 	/*
kernel/exit.c +2 -0
index f702a6a63686..a39a321c1f37 100644
@@ -64,6 +64,7 @@
 #include <linux/compat.h>
 #include <linux/io_uring.h>
 #include <linux/kprobes.h>
+#include <linux/rethook.h>
 
 #include <linux/uaccess.h>
 #include <asm/unistd.h>
@@ -169,6 +170,7 @@ static void delayed_put_task_struct(struct rcu_head *rhp)
 	struct task_struct *tsk = container_of(rhp, struct task_struct, rcu);
 
 	kprobe_flush_task(tsk);
+	rethook_flush_task(tsk);
 	perf_event_delayed_put(tsk);
 	trace_sched_process_free(tsk);
 	put_task_struct(tsk);
kernel/fork.c +3 -0
index 3244cc56b697..ffae38be64c4 100644
@@ -2282,6 +2282,9 @@ static __latent_entropy struct task_struct *copy_process(
 #ifdef CONFIG_KRETPROBES
 	p->kretprobe_instances.first = NULL;
 #endif
+#ifdef CONFIG_RETHOOK
+	p->rethooks.first = NULL;
+#endif
 
 	/*
 	 * Ensure that the cgroup subsystem policies allow the new process to be
kernel/trace/Kconfig +11 -0
index 45a3618a20a7..9328724258dc 100644
@@ -10,6 +10,17 @@ config USER_STACKTRACE_SUPPORT
 config NOP_TRACER
 	bool
 
+config HAVE_RETHOOK
+	bool
+
+config RETHOOK
+	bool
+	depends on HAVE_RETHOOK
+	help
+	  Enable generic return hooking feature. This is an internal
+	  API, which will be used by other function-entry hooking
+	  feature like fprobe and kprobes.
+
 config HAVE_FUNCTION_TRACER
 	bool
 	help
kernel/trace/Makefile +1 -0
index 47a37a3bb974..c68fdacbf9ef 100644
@@ -98,6 +98,7 @@ obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o
 obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o
 obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
 obj-$(CONFIG_FPROBES) += fprobes.o
+obj-$(CONFIG_RETHOOK) += rethook.o
 
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
kernel/trace/rethook.c new file +226 -0

226 lines not on this page · raw message

maintainer
#39 lore raw

RFC PATCH 3/6 rethook: x86: Add rethook x86 implementation

In reply to Masami Hiramatsu

Add rethook for x86 implementation. Most of the code has been copied from kretprobes on x86.

---
 arch/x86/Kconfig          |    1 
 arch/x86/kernel/Makefile  |    1 
 arch/x86/kernel/rethook.c |  115 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 117 insertions(+)
 create mode 100644 arch/x86/kernel/rethook.c
arch/x86/Kconfig +1 -0
index 7399327d1eff..939c4c897e63 100644
@@ -219,6 +219,7 @@ config X86
 	select HAVE_KPROBES_ON_FTRACE
 	select HAVE_FUNCTION_ERROR_INJECTION
 	select HAVE_KRETPROBES
+	select HAVE_RETHOOK
 	select HAVE_KVM
 	select HAVE_LIVEPATCH			if X86_64
 	select HAVE_MIXED_BREAKPOINTS_REGS
arch/x86/kernel/Makefile +1 -0
index 2ff3e600f426..66593d8c4d74 100644
@@ -106,6 +106,7 @@ obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
 obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
 obj-$(CONFIG_X86_TSC)		+= trace_clock.o
 obj-$(CONFIG_TRACING)		+= trace.o
+obj-$(CONFIG_RETHOOK)		+= rethook.o
 obj-$(CONFIG_CRASH_CORE)	+= crash_core_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= relocate_kernel_$(BITS).o crash.o
arch/x86/kernel/rethook.c new file +115 -0

115 lines not on this page · raw message

maintainer
#40 lore raw

RFC PATCH 4/6 fprobe: Add exit_handler support

In reply to Masami Hiramatsu

Add exit_handler to fprobe. fprobe + rethook allows us to hook the kernel function return without fgraph tracer. Eventually, the fgraph tracer will be generic array based return hooking and fprobe may use it if user requests. Since both array-based approach and list-based approach have Pros and Cons, (e.g. memory consumption v.s. less missing events) it is better to keep both but fprobe will provide the same exit-handler interface.

---
 include/linux/fprobes.h |    4 +++
 kernel/trace/Kconfig    |    1 +
 kernel/trace/fprobes.c  |   59 +++++++++++++++++++++++++++++++++++++++++++++--
 3 files changed, 62 insertions(+), 2 deletions(-)
include/linux/fprobes.h +4 -0
index fa85a2fc3ad1..d2eb064c5b79 100644
@@ -5,6 +5,7 @@
 
 #include <linux/compiler.h>
 #include <linux/ftrace.h>
+#include <linux/rethook.h>
 
 /*
  * fprobe_entry - function entry for fprobe
@@ -27,7 +28,10 @@ struct fprobe {
 	struct ftrace_ops	ftrace;
 	unsigned long		nmissed;
 	unsigned int		flags;
+	struct rethook		*rethook;
+
 	void (*entry_handler) (struct fprobe *, unsigned long, struct pt_regs *);
+	void (*exit_handler) (struct fprobe *, unsigned long, struct pt_regs *);
 };
 
 #define FPROBE_FL_DISABLED	1
kernel/trace/Kconfig +1 -0
index 9328724258dc..59e227ade0b7 100644
@@ -238,6 +238,7 @@ config FPROBES
 	bool "Kernel Function Probe (fprobe)"
 	depends on FUNCTION_TRACER
 	depends on DYNAMIC_FTRACE_WITH_REGS
+	select RETHOOK
 	default n
 	help
 	  This option enables kernel function probe feature, which is
kernel/trace/fprobes.c +57 -2

101 lines not on this page · raw message

maintainer
#44 lore raw

RFC PATCH 5/6 fprobe: Add sample program for fprobe

In reply to Masami Hiramatsu

Add a sample program for the fprobe.

---
 samples/Kconfig                 |    6 ++
 samples/Makefile                |    1 
 samples/fprobe/Makefile         |    3 +
 samples/fprobe/fprobe_example.c |  103 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 113 insertions(+)
 create mode 100644 samples/fprobe/Makefile
 create mode 100644 samples/fprobe/fprobe_example.c
samples/Kconfig +6 -0
index 43d2e9aa557f..487b5d17f722 100644
@@ -73,6 +73,12 @@ config SAMPLE_HW_BREAKPOINT
 	help
 	  This builds kernel hardware breakpoint example modules.
 
+config SAMPLE_FPROBE
+	tristate "Build fprobe examples -- loadable modules only"
+	depends on FPROBES && m
+	help
+	  This build several fprobe example modules.
+
 config SAMPLE_KFIFO
 	tristate "Build kfifo examples -- loadable modules only"
 	depends on m
samples/Makefile +1 -0
index 4bcd6b93bffa..4f73fe7aa473 100644
@@ -32,3 +32,4 @@ obj-$(CONFIG_SAMPLE_INTEL_MEI)		+= mei/
 subdir-$(CONFIG_SAMPLE_WATCHDOG)	+= watchdog
 subdir-$(CONFIG_SAMPLE_WATCH_QUEUE)	+= watch_queue
 obj-$(CONFIG_DEBUG_KMEMLEAK_TEST)	+= kmemleak/
+obj-$(CONFIG_SAMPLE_FPROBE)		+= fprobe/
samples/fprobe/Makefile new file +3 -0
new file mode 100644
index 000000000000..ecccbfa6e99b
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+obj-$(CONFIG_SAMPLE_FPROBE) += fprobe_example.o
samples/fprobe/fprobe_example.c new file +103 -0

103 lines not on this page · raw message

maintainer
#45 lore raw

RFC PATCH 6/6 bpf: Add kprobe link for attaching raw kprobes

In reply to Masami Hiramatsu

From: Jiri Olsa <[email protected]>

Adding new link type BPF_LINK_TYPE_KPROBE to attach so called "kprobes" directly through fprobe API. Note that since the using kprobes with multiple same handler is not efficient, this uses the fprobe which natively support multiple probe points for one same handler, but limited on function entry and exit.

Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program.

The new link allows to create multiple kprobes link by using new link_create interface:

  struct {
    __aligned_u64   addrs;
    __u32           cnt;
    __u64           bpf_cookie;
  } kprobe;

Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe.

---
 include/linux/bpf_types.h      |    1 
 include/uapi/linux/bpf.h       |   12 ++
 kernel/bpf/syscall.c           |  199 +++++++++++++++++++++++++++++++++++++++-
 tools/include/uapi/linux/bpf.h |   12 ++
 4 files changed, 219 insertions(+), 5 deletions(-)
include/linux/bpf_types.h +1 -0
index 48a91c51c015..a9000feab34e 100644
@@ -140,3 +140,4 @@ BPF_LINK_TYPE(BPF_LINK_TYPE_XDP, xdp)
 #ifdef CONFIG_PERF_EVENTS
 BPF_LINK_TYPE(BPF_LINK_TYPE_PERF_EVENT, perf)
 #endif
+BPF_LINK_TYPE(BPF_LINK_TYPE_KPROBE, kprobe)
include/uapi/linux/bpf.h +12 -0
index ba5af15e25f5..10e9b56a074e 100644
@@ -995,6 +995,7 @@ enum bpf_attach_type {
 	BPF_SK_REUSEPORT_SELECT,
 	BPF_SK_REUSEPORT_SELECT_OR_MIGRATE,
 	BPF_PERF_EVENT,
+	BPF_TRACE_RAW_KPROBE,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1009,6 +1010,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_NETNS = 5,
 	BPF_LINK_TYPE_XDP = 6,
 	BPF_LINK_TYPE_PERF_EVENT = 7,
+	BPF_LINK_TYPE_KPROBE = 8,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE		(1U << 4)
 
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1463,6 +1470,11 @@ union bpf_attr {
 				 */
 				__u64		bpf_cookie;
 			} perf_event;
+			struct {
+				__aligned_u64	addrs;
+				__u32		cnt;
+				__u64		bpf_cookie;
+			} kprobe;
 		};
 	} link_create;
 
kernel/bpf/syscall.c +194 -5

241 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +12 -0
index ba5af15e25f5..10e9b56a074e 100644
@@ -995,6 +995,7 @@ enum bpf_attach_type {
 	BPF_SK_REUSEPORT_SELECT,
 	BPF_SK_REUSEPORT_SELECT_OR_MIGRATE,
 	BPF_PERF_EVENT,
+	BPF_TRACE_RAW_KPROBE,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1009,6 +1010,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_NETNS = 5,
 	BPF_LINK_TYPE_XDP = 6,
 	BPF_LINK_TYPE_PERF_EVENT = 7,
+	BPF_LINK_TYPE_KPROBE = 8,
 
 	MAX_BPF_LINK_TYPE,
 };
@@ -1111,6 +1113,11 @@ enum bpf_link_type {
  */
 #define BPF_F_SLEEPABLE		(1U << 4)
 
+/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE
+ * attach type.
+ */
+#define BPF_F_KPROBE_RETURN	(1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
@@ -1463,6 +1470,11 @@ union bpf_attr {
 				 */
 				__u64		bpf_cookie;
 			} perf_event;
+			struct {
+				__aligned_u64	addrs;
+				__u32		cnt;
+				__u64		bpf_cookie;
+			} kprobe;
 		};
 	} link_create;
 
#49 lore raw

Re: [RFC PATCH 0/6] fprobe: Introduce fprobe function entry/exit probe

In reply to Masami Hiramatsu
Quoted 13 lines · Masami Hiramatsu

Hi Jiri,

Here is a short series of patches, which shows what I replied to your series.

This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return, which I cloned from kretprobe.

I also rewrite your [08/13] bpf patch to use this fprobe instead of kprobes. I didn't tested that one, but the sample module seems to work. Please test bpf part with your libbpf updates.

Quoted 8 lines

BTW, while implementing the fprobe, I introduced the per-probe point private data, but I'm not sure why you need it. It seems that data is not used from bpf...

If this is good for you, I would like to proceed this with the rethook and rewrite the kretprobe to use the rethook to hook the functions. That should be much cleaner (and easy to prepare for the fgraph tracer integration)

What is the speed of attach/detach of thousands fprobes?

maintainer
#50 lore raw

Re: [RFC PATCH 0/6] fprobe: Introduce fprobe function entry/exit probe

Hi Alexei,

Quoted 18 lines · On Tue, 11 Jan 2022 14:39:44 -0800 Alexei Starovoitov…
Quoted 17 lines · Masami Hiramatsu

Hi Jiri,

Here is a short series of patches, which shows what I replied to your series.

This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return, which I cloned from kretprobe.

I also rewrite your [08/13] bpf patch to use this fprobe instead of kprobes. I didn't tested that one, but the sample module seems to work. Please test bpf part with your libbpf updates.

BTW, while implementing the fprobe, I introduced the per-probe point private data, but I'm not sure why you need it. It seems that data is not used from bpf...

Quoted 6 lines
Quoted 4 lines

If this is good for you, I would like to proceed this with the rethook and rewrite the kretprobe to use the rethook to hook the functions. That should be much cleaner (and easy to prepare for the fgraph tracer integration)

What is the speed of attach/detach of thousands fprobes?

I've treaked my example module and it shows below result;

/lib/modules/5.16.0-rc4+/kernel/samples/fprobe # time insmod ./fprobe_example.ko
symbol='btrfs_*'

[  187.095925] fprobe_init: 1028 symbols found
[  188.521694] fprobe_init: Planted fprobe at btrfs_*

real 0m 1.47s
user 0m 0.00s
sys 0m 1.36s

I think using ftrace_set_filter_ips() can make it faster. (maybe it needs to drop per-probe point private data, that prevents fprobe to use that interface)

Thank you,

Signature · 1 line
Masami Hiramatsu <[email protected]>
maintainer
#51 lore raw

Re: [RFC PATCH 0/6] fprobe: Introduce fprobe function entry/exit probe

Quoted 36 lines · On Wed, 12 Jan 2022 16:33:53 +0900 Masami Hiramatsu <[email protected]> wrote:

Hi Alexei,

Quoted 18 lines · On Tue, 11 Jan 2022 14:39:44 -0800 Alexei Starovoitov…
Quoted 17 lines · Masami Hiramatsu

Hi Jiri,

Here is a short series of patches, which shows what I replied to your series.

This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return, which I cloned from kretprobe.

I also rewrite your [08/13] bpf patch to use this fprobe instead of kprobes. I didn't tested that one, but the sample module seems to work. Please test bpf part with your libbpf updates.

BTW, while implementing the fprobe, I introduced the per-probe point private data, but I'm not sure why you need it. It seems that data is not used from bpf...

Quoted 6 lines
Quoted 4 lines

If this is good for you, I would like to proceed this with the rethook and rewrite the kretprobe to use the rethook to hook the functions. That should be much cleaner (and easy to prepare for the fgraph tracer integration)

What is the speed of attach/detach of thousands fprobes?

I've treaked my example module and it shows below result;

/lib/modules/5.16.0-rc4+/kernel/samples/fprobe # time insmod ./fprobe_example.ko
symbol='btrfs_*'

[  187.095925] fprobe_init: 1028 symbols found
[  188.521694] fprobe_init: Planted fprobe at btrfs_*
Quoted 7 lines

real 0m 1.47s
user 0m 0.00s
sys 0m 1.36s

I think using ftrace_set_filter_ips() can make it faster. (maybe it needs to drop per-probe point private data, that prevents fprobe to use that interface)

OK, I've updated fprobes to use the ftrace_set_filter_ips() and got below result.

/lib/modules/5.16.0-rc4+/kernel/samples/fprobe # time insmod fprobe_example.ko s ymbol='btrfs_*'

[   36.130947] fprobe_init: 1028 symbols found
[   36.177901] fprobe_init: Planted fprobe at btrfs_*

real 0m 0.08s
user 0m 0.00s
sys 0m 0.07s

Let me update the series :)

Thank you,

Signature · 1 line
Masami Hiramatsu <[email protected]>
#46 lore raw

Re: [RFC PATCH 6/6] bpf: Add kprobe link for attaching raw kprobes

In reply to Masami Hiramatsu
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi Masami,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on rostedt-trace/for-next]
[also build test WARNING on bpf-next/master bpf/master linus/master v5.16] [cannot apply to next-20220112]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: x86_64-randconfig-a011 (https://download.01.org/0day-ci/archive/20220113/202201130621.vPqexP9P-lkp(a)intel.com/config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0

reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/ba002e6556acd9e0af16442f4226ddb2a8759567
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050 git checkout ba002e6556acd9e0af16442f4226ddb2a8759567
# save the config file to linux build tree
mkdir build_dir
make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash kernel/bpf/

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

In file included from kernel/bpf/syscall.c:34:

>> include/linux/fprobes.h:57:22: warning: no previous prototype for 'fprobe_find_entry' [-Wmissing-prototypes]

57 | struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
| ^~~~~~~~~~~~~~~~~

vim +/fprobe_find_entry +57 include/linux/fprobes.h

7b8ed3653f3755 Masami Hiramatsu 2022-01-12  43  
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  44  #ifdef CONFIG_FPROBES
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  45  int register_fprobe(struct fprobe *fp);
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  46  int unregister_fprobe(struct fprobe *fp);
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  47  struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr);
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  48  #else
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  49  static inline int register_fprobe(struct fprobe *fp)
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  50  {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  51  	return -ENOTSUPP;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  52  }
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  53  static inline int unregister_fprobe(struct fprobe *fp)
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  54  {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  55  	return -ENOTSUPP;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  56  }
7b8ed3653f3755 Masami Hiramatsu 2022-01-12 @57  struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  58  {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  59  	return NULL;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  60  }
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  61  #endif

7b8ed3653f3755 Masami Hiramatsu 2022-01-12 62

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#41 lore raw

Re: [RFC PATCH 4/6] fprobe: Add exit_handler support

In reply to Masami Hiramatsu
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi Masami,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on rostedt-trace/for-next]
[also build test WARNING on bpf-next/master bpf/master linus/master v5.16 next-20220112] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20220113/202201130629.IyZ9vaDV-lkp(a)intel.com/config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/c0e0471b58c3c9122bbff7523f97a363558284eb
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050
        git checkout c0e0471b58c3c9122bbff7523f97a363558284eb
        # save the config file to linux build tree
        mkdir build_dir
        make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash arch/x86/kernel/

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> arch/x86/kernel/rethook.c:14:23: warning: no previous prototype for 'arch_rethook_trampoline_callback' [-Wmissing-prototypes]

14 | __used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +/arch_rethook_trampoline_callback +14 arch/x86/kernel/rethook.c

3727c0ee2be25c Masami Hiramatsu 2022-01-12  10  
3727c0ee2be25c Masami Hiramatsu 2022-01-12  11  /*
3727c0ee2be25c Masami Hiramatsu 2022-01-12  12   * Called from arch_rethook_trampoline
3727c0ee2be25c Masami Hiramatsu 2022-01-12  13   */
3727c0ee2be25c Masami Hiramatsu 2022-01-12 @14  __used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs)
3727c0ee2be25c Masami Hiramatsu 2022-01-12  15  {
3727c0ee2be25c Masami Hiramatsu 2022-01-12  16  	unsigned long *frame_pointer;
3727c0ee2be25c Masami Hiramatsu 2022-01-12  17  
3727c0ee2be25c Masami Hiramatsu 2022-01-12  18  	/* fixup registers */
3727c0ee2be25c Masami Hiramatsu 2022-01-12  19  	regs->cs = __KERNEL_CS;
3727c0ee2be25c Masami Hiramatsu 2022-01-12  20  #ifdef CONFIG_X86_32
3727c0ee2be25c Masami Hiramatsu 2022-01-12  21  	regs->gs = 0;
3727c0ee2be25c Masami Hiramatsu 2022-01-12  22  #endif
3727c0ee2be25c Masami Hiramatsu 2022-01-12  23  	regs->ip = (unsigned long)&arch_rethook_trampoline;
3727c0ee2be25c Masami Hiramatsu 2022-01-12  24  	regs->orig_ax = ~0UL;
3727c0ee2be25c Masami Hiramatsu 2022-01-12  25  	regs->sp += sizeof(long);
3727c0ee2be25c Masami Hiramatsu 2022-01-12  26  	frame_pointer = &regs->sp + 1;
3727c0ee2be25c Masami Hiramatsu 2022-01-12  27  
3727c0ee2be25c Masami Hiramatsu 2022-01-12  28  	/*
3727c0ee2be25c Masami Hiramatsu 2022-01-12  29  	 * The return address at 'frame_pointer' is recovered by the
3727c0ee2be25c Masami Hiramatsu 2022-01-12  30  	 * arch_rethook_fixup_return() which called from this
3727c0ee2be25c Masami Hiramatsu 2022-01-12  31  	 * rethook_trampoline_handler().
3727c0ee2be25c Masami Hiramatsu 2022-01-12  32  	 */
3727c0ee2be25c Masami Hiramatsu 2022-01-12  33  	rethook_trampoline_handler(regs, (unsigned long)frame_pointer);
3727c0ee2be25c Masami Hiramatsu 2022-01-12  34  
3727c0ee2be25c Masami Hiramatsu 2022-01-12  35  	/*
3727c0ee2be25c Masami Hiramatsu 2022-01-12  36  	 * Copy FLAGS to 'pt_regs::sp' so that arch_rethook_trapmoline()
3727c0ee2be25c Masami Hiramatsu 2022-01-12  37  	 * can do RET right after POPF.
3727c0ee2be25c Masami Hiramatsu 2022-01-12  38  	 */
3727c0ee2be25c Masami Hiramatsu 2022-01-12  39  	regs->sp = regs->flags;
3727c0ee2be25c Masami Hiramatsu 2022-01-12  40  }
3727c0ee2be25c Masami Hiramatsu 2022-01-12  41  NOKPROBE_SYMBOL(arch_rethook_trampoline_callback);
3727c0ee2be25c Masami Hiramatsu 2022-01-12  42  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#47 lore raw

Re: [RFC PATCH 6/6] bpf: Add kprobe link for attaching raw kprobes

In reply to Masami Hiramatsu
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi Masami,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on rostedt-trace/for-next]
[also build test WARNING on bpf-next/master bpf/master linus/master v5.16] [cannot apply to next-20220112]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: hexagon-randconfig-r001-20220112 (https://download.01.org/0day-ci/archive/20220113/202201130747.yNR0J8tO-lkp(a)intel.com/config)
compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 244dd2913a43a200f5a6544d424cdc37b771028b)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/ba002e6556acd9e0af16442f4226ddb2a8759567
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050 git checkout ba002e6556acd9e0af16442f4226ddb2a8759567
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=hexagon SHELL=/bin/bash kernel/bpf/

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

In file included from kernel/bpf/syscall.c:34:
include/linux/fprobes.h:28:20: error: field has incomplete type 'struct ftrace_ops'

           struct ftrace_ops       ftrace;
                                   ^
   include/linux/ftrace.h:332:8: note: forward declaration of 'struct ftrace_ops'
   struct ftrace_ops;
          ^
   In file included from kernel/bpf/syscall.c:34:
>> include/linux/fprobes.h:57:22: warning: no previous prototype for function 'fprobe_find_entry' [-Wmissing-prototypes]
   struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
                        ^

include/linux/fprobes.h:57:1: note: declare 'static' if the function is not intended to be used outside of this translation unit struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
^
static
1 warning and 1 error generated.

vim +/fprobe_find_entry +57 include/linux/fprobes.h

7b8ed3653f37559 Masami Hiramatsu 2022-01-12  43  
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  44  #ifdef CONFIG_FPROBES
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  45  int register_fprobe(struct fprobe *fp);
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  46  int unregister_fprobe(struct fprobe *fp);
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  47  struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr);
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  48  #else
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  49  static inline int register_fprobe(struct fprobe *fp)
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  50  {
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  51  	return -ENOTSUPP;
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  52  }
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  53  static inline int unregister_fprobe(struct fprobe *fp)
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  54  {
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  55  	return -ENOTSUPP;
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  56  }
7b8ed3653f37559 Masami Hiramatsu 2022-01-12 @57  struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  58  {
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  59  	return NULL;
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  60  }
7b8ed3653f37559 Masami Hiramatsu 2022-01-12  61  #endif

7b8ed3653f37559 Masami Hiramatsu 2022-01-12 62

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#48 lore raw

Re: [RFC PATCH 6/6] bpf: Add kprobe link for attaching raw kprobes

In reply to Masami Hiramatsu
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi Masami,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on rostedt-trace/for-next]
[also build test ERROR on bpf-next/master bpf/master linus/master v5.16] [cannot apply to next-20220112]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220113/202201130737.O0QNCKhe-lkp(a)intel.com/config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0

reproduce (this is a W=1 build):
# https://github.com/0day-ci/linux/commit/ba002e6556acd9e0af16442f4226ddb2a8759567
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050 git checkout ba002e6556acd9e0af16442f4226ddb2a8759567
# save the config file to linux build tree
mkdir build_dir
make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash kernel/bpf/

If you fix the issue, kindly add following tag as appropriate

All error/warnings (new ones prefixed by >>):

In file included from kernel/bpf/syscall.c:34:

>> include/linux/fprobes.h:28:20: error: field 'ftrace' has incomplete type
      28 |  struct ftrace_ops ftrace;
         |                    ^~~~~~
>> include/linux/fprobes.h:57:22: warning: no previous prototype for 'fprobe_find_entry' [-Wmissing-prototypes]

57 | struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
| ^~~~~~~~~~~~~~~~~

vim +/ftrace +28 include/linux/fprobes.h

7b8ed3653f3755 Masami Hiramatsu 2022-01-12  23  
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  24  struct fprobe {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  25  	struct fprobe_entry	*entries;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  26  	unsigned int		nentry;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  27  
7b8ed3653f3755 Masami Hiramatsu 2022-01-12 @28  	struct ftrace_ops	ftrace;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  29  	unsigned long		nmissed;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  30  	unsigned int		flags;
c0e0471b58c3c9 Masami Hiramatsu 2022-01-12  31  	struct rethook		*rethook;
c0e0471b58c3c9 Masami Hiramatsu 2022-01-12  32  
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  33  	void (*entry_handler) (struct fprobe *, unsigned long, struct pt_regs *);
c0e0471b58c3c9 Masami Hiramatsu 2022-01-12  34  	void (*exit_handler) (struct fprobe *, unsigned long, struct pt_regs *);
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  35  };
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  36  
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  37  #define FPROBE_FL_DISABLED	1
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  38  
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  39  static inline bool fprobe_disabled(struct fprobe *fp)
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  40  {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  41  	return (fp) ? fp->flags & FPROBE_FL_DISABLED : false;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  42  }
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  43  
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  44  #ifdef CONFIG_FPROBES
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  45  int register_fprobe(struct fprobe *fp);
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  46  int unregister_fprobe(struct fprobe *fp);
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  47  struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr);
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  48  #else
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  49  static inline int register_fprobe(struct fprobe *fp)
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  50  {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  51  	return -ENOTSUPP;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  52  }
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  53  static inline int unregister_fprobe(struct fprobe *fp)
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  54  {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  55  	return -ENOTSUPP;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  56  }
7b8ed3653f3755 Masami Hiramatsu 2022-01-12 @57  struct fprobe_entry *fprobe_find_entry(struct fprobe *fp, unsigned long addr)
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  58  {
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  59  	return NULL;
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  60  }
7b8ed3653f3755 Masami Hiramatsu 2022-01-12  61  #endif

7b8ed3653f3755 Masami Hiramatsu 2022-01-12 62

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#42 lore raw

Re: [RFC PATCH 4/6] fprobe: Add exit_handler support

In reply to Masami Hiramatsu
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi Masami,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on rostedt-trace/for-next]
[also build test WARNING on bpf-next/master bpf/master linus/master v5.16 next-20220112] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: i386-allmodconfig (https://download.01.org/0day-ci/archive/20220113/202201131009.oKKmEVVq-lkp(a)intel.com/config)
compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 244dd2913a43a200f5a6544d424cdc37b771028b)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c0e0471b58c3c9122bbff7523f97a363558284eb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050 git checkout c0e0471b58c3c9122bbff7523f97a363558284eb
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash arch/x86/kernel/

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> arch/x86/kernel/rethook.c:14:23: warning: no previous prototype for function 'arch_rethook_trampoline_callback' [-Wmissing-prototypes]
   __used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs)
                         ^
   arch/x86/kernel/rethook.c:14:18: note: declare 'static' if the function is not intended to be used outside of this translation unit
   __used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs)
                    ^
                    static 

1 warning generated.

vim +/arch_rethook_trampoline_callback +14 arch/x86/kernel/rethook.c

3727c0ee2be25cf Masami Hiramatsu 2022-01-12  10  
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  11  /*
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  12   * Called from arch_rethook_trampoline
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  13   */
3727c0ee2be25cf Masami Hiramatsu 2022-01-12 @14  __used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs)
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  15  {
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  16  	unsigned long *frame_pointer;
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  17  
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  18  	/* fixup registers */
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  19  	regs->cs = __KERNEL_CS;
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  20  #ifdef CONFIG_X86_32
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  21  	regs->gs = 0;
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  22  #endif
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  23  	regs->ip = (unsigned long)&arch_rethook_trampoline;
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  24  	regs->orig_ax = ~0UL;
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  25  	regs->sp += sizeof(long);
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  26  	frame_pointer = &regs->sp + 1;
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  27  
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  28  	/*
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  29  	 * The return address at 'frame_pointer' is recovered by the
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  30  	 * arch_rethook_fixup_return() which called from this
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  31  	 * rethook_trampoline_handler().
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  32  	 */
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  33  	rethook_trampoline_handler(regs, (unsigned long)frame_pointer);
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  34  
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  35  	/*
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  36  	 * Copy FLAGS to 'pt_regs::sp' so that arch_rethook_trapmoline()
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  37  	 * can do RET right after POPF.
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  38  	 */
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  39  	regs->sp = regs->flags;
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  40  }
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  41  NOKPROBE_SYMBOL(arch_rethook_trampoline_callback);
3727c0ee2be25cf Masami Hiramatsu 2022-01-12  42  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#43 lore raw

Re: [RFC PATCH 4/6] fprobe: Add exit_handler support

In reply to Masami Hiramatsu
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi Masami,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on rostedt-trace/for-next]
[also build test ERROR on bpf-next/master bpf/master linus/master v5.17-rc1 next-20220124] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git for-next
config: s390-allmodconfig (https://download.01.org/0day-ci/archive/20220125/202201250537.lwxse0qH-lkp(a)intel.com/config)
compiler: s390-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c0e0471b58c3c9122bbff7523f97a363558284eb
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Masami-Hiramatsu/fprobe-Add-ftrace-based-probe-APIs/20220112-000050 git checkout c0e0471b58c3c9122bbff7523f97a363558284eb
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=s390 SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

s390-linux-ld: kernel/dma/coherent.o: in function `dma_init_coherent_memory':
coherent.c:(.text+0x13e): undefined reference to `memremap'
s390-linux-ld: coherent.c:(.text+0x274): undefined reference to `memunmap'
s390-linux-ld: kernel/dma/coherent.o: in function `dma_declare_coherent_memory':
coherent.c:(.text+0x820): undefined reference to `memunmap'
s390-linux-ld: kernel/trace/rethook.o: in function `__rethook_find_ret_addr':

>> rethook.c:(.text+0xa8): undefined reference to `arch_rethook_trampoline'
   s390-linux-ld: kernel/trace/rethook.o: in function `rethook_hook_current':
>> rethook.c:(.text+0x1a7e): undefined reference to `arch_rethook_prepare'

s390-linux-ld: kernel/trace/rethook.o: in function `rethook_trampoline_handler':
rethook.c:(.text+0x1bb2): undefined reference to `arch_rethook_trampoline'
s390-linux-ld: drivers/irqchip/irq-al-fic.o: in function `al_fic_init_dt':
irq-al-fic.c:(.init.text+0x98): undefined reference to `of_iomap'
s390-linux-ld: irq-al-fic.c:(.init.text+0x5a8): undefined reference to `iounmap'
s390-linux-ld: drivers/clk/clk-fixed-mmio.o: in function `fixed_mmio_clk_setup':
clk-fixed-mmio.c:(.text+0x9a): undefined reference to `of_iomap'
s390-linux-ld: clk-fixed-mmio.c:(.text+0xe6): undefined reference to `iounmap'
s390-linux-ld: drivers/clocksource/timer-of.o: in function `timer_of_init':
timer-of.c:(.init.text+0x184): undefined reference to `of_iomap'
s390-linux-ld: timer-of.c:(.init.text+0x9bc): undefined reference to `iounmap'
s390-linux-ld: drivers/clocksource/timer-of.o: in function `timer_of_cleanup':
timer-of.c:(.init.text+0xc46): undefined reference to `iounmap'
s390-linux-ld: drivers/clocksource/timer-microchip-pit64b.o: in function `mchp_pit64b_dt_init_timer':
timer-microchip-pit64b.c:(.init.text+0x8a8): undefined reference to `of_iomap'
s390-linux-ld: timer-microchip-pit64b.c:(.init.text+0x1072): undefined reference to `iounmap'

Kconfig warnings: (for reference only)
WARNING: unmet direct dependencies detected for RETHOOK Depends on HAVE_RETHOOK
Selected by
- FPROBES && FTRACE && FUNCTION_TRACER && DYNAMIC_FTRACE_WITH_REGS

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org