Skip to contentSkip to conversation

fprobe: Introduce fprobe function entry/exit probe

PATCH Superseded Masami Hiramatsu 3 participants 19 messages Mar 8, 2022 – Mar 12, 2022

Subsystems (16)

v10 of 13 Superseded Latest is v13 →

lore
Share & tools
lore

Masami Hiramatsu

author maintainer
#0 lore raw

v10 cover fprobe: Introduce fprobe function entry/exit probe

Hi,

Here is the 10th version of fprobe. In this version, the rethook arm support ([7/12]) comes back :-D, add an internal rethook flag for identify the context ([3-8/12]), and simplifies the ftrace_location lookup loops([2/12]) according to Jiri's comment :)

The previous version (v9) is here[1];

[1] https://lore.kernel.org/all/164655933970.1674510.3809060481512713846.stgit@devnote2/T/#u

This series introduces the fprobe, the function entry/exit probe with multiple probe point support for x86, arm64 and powerpc64le. This also introduces the rethook for hooking function return as same as the kretprobe does. This abstraction will help us to generalize the fgraph tracer, because we can just switch to it from the rethook in fprobe, depending on the kernel configuration.

The patch [1/10] is from Jiri's series[2].

[2] https://lore.kernel.org/all/[email protected]/T/#u

And the patch [9/10] adds the FPROBE_FL_KPROBE_SHARED flag for the case if user wants to share the same code (or share a same resource) on the fprobe and the kprobes.

I forcibly updated my kprobes/fprobe branch, you can pull this series from:

https://git.kernel.org/pub/scm/linux/kernel/git/mhiramat/linux.git kprobes/fprobe

Thank you,

---

Jiri Olsa (1):
ftrace: Add ftrace_set_filter_ips function

Masami Hiramatsu (11):
fprobe: Add ftrace based probe APIs
rethook: Add a generic return hook
rethook: x86: Add rethook x86 implementation
arm64: rethook: Add arm64 rethook implementation
powerpc: Add rethook support
ARM: rethook: Add rethook arm implementation
fprobe: Add exit_handler support
fprobe: Add sample program for fprobe
fprobe: Introduce FPROBE_FL_KPROBE_SHARED flag for fprobe
docs: fprobe: Add fprobe description to ftrace-use.rst
fprobe: Add a selftest for fprobe

 Documentation/trace/fprobe.rst                |  171 +++++++++++++
 Documentation/trace/index.rst                 |    1 
 arch/arm/Kconfig                              |    1 
 arch/arm/include/asm/stacktrace.h             |    4 
 arch/arm/kernel/stacktrace.c                  |    6 
 arch/arm/probes/Makefile                      |    1 
 arch/arm/probes/rethook.c                     |  101 ++++++++
 arch/arm64/Kconfig                            |    1 
 arch/arm64/include/asm/stacktrace.h           |    2 
 arch/arm64/kernel/probes/Makefile             |    1 
 arch/arm64/kernel/probes/rethook.c            |   25 ++
 arch/arm64/kernel/probes/rethook_trampoline.S |   87 +++++++
 arch/arm64/kernel/stacktrace.c                |    7 -
 arch/powerpc/Kconfig                          |    1 
 arch/powerpc/kernel/Makefile                  |    1 
 arch/powerpc/kernel/rethook.c                 |   72 +++++
 arch/x86/Kconfig                              |    1 
 arch/x86/include/asm/unwind.h                 |    8 +
 arch/x86/kernel/Makefile                      |    1 
 arch/x86/kernel/kprobes/common.h              |    1 
 arch/x86/kernel/rethook.c                     |  115 +++++++++
 include/linux/fprobe.h                        |  105 ++++++++
 include/linux/ftrace.h                        |    3 
 include/linux/kprobes.h                       |    3 
 include/linux/rethook.h                       |  100 ++++++++
 include/linux/sched.h                         |    3 
 kernel/exit.c                                 |    2 
 kernel/fork.c                                 |    3 
 kernel/trace/Kconfig                          |   26 ++
 kernel/trace/Makefile                         |    2 
 kernel/trace/fprobe.c                         |  332 +++++++++++++++++++++++++
 kernel/trace/ftrace.c                         |   58 ++++
 kernel/trace/rethook.c                        |  317 ++++++++++++++++++++++++
 lib/Kconfig.debug                             |   12 +
 lib/Makefile                                  |    2 
 lib/test_fprobe.c                             |  174 +++++++++++++
 samples/Kconfig                               |    7 +
 samples/Makefile                              |    1 
 samples/fprobe/Makefile                       |    3 
 samples/fprobe/fprobe_example.c               |  120 +++++++++
 40 files changed, 1867 insertions(+), 14 deletions(-)
 create mode 100644 Documentation/trace/fprobe.rst
 create mode 100644 arch/arm/probes/rethook.c
 create mode 100644 arch/arm64/kernel/probes/rethook.c
 create mode 100644 arch/arm64/kernel/probes/rethook_trampoline.S
 create mode 100644 arch/powerpc/kernel/rethook.c
 create mode 100644 arch/x86/kernel/rethook.c
 create mode 100644 include/linux/fprobe.h
 create mode 100644 include/linux/rethook.h
 create mode 100644 kernel/trace/fprobe.c
 create mode 100644 kernel/trace/rethook.c
 create mode 100644 lib/test_fprobe.c
 create mode 100644 samples/fprobe/Makefile
 create mode 100644 samples/fprobe/fprobe_example.c

--
Masami Hiramatsu (Linaro) <[email protected]>

author maintainer
#1 lore raw

PATCH v10 1/12 ftrace: Add ftrace_set_filter_ips function

From: Jiri Olsa <[email protected]>

Adding ftrace_set_filter_ips function to be able to set filter on multiple ip addresses at once.

With the kprobe multi attach interface we have cases where we need to initialize ftrace_ops object with thousands of functions, so having single function diving into ftrace_hash_move_and_update_ops with ftrace_lock is faster.

The functions ips are passed as unsigned long array with count.

---
Changes in v6: [Masami]
- Fix a typo and add a comment.

---
 include/linux/ftrace.h |    3 ++
 kernel/trace/ftrace.c  |   58 +++++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 52 insertions(+), 9 deletions(-)
include/linux/ftrace.h +3 -0
index 9999e29187de..60847cbce0da 100644
@@ -512,6 +512,8 @@ struct dyn_ftrace {
 
 int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip,
 			 int remove, int reset);
+int ftrace_set_filter_ips(struct ftrace_ops *ops, unsigned long *ips,
+			  unsigned int cnt, int remove, int reset);
 int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
 		       int len, int reset);
 int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
@@ -802,6 +804,7 @@ static inline unsigned long ftrace_location(unsigned long ip)
 #define ftrace_regex_open(ops, flag, inod, file) ({ -ENODEV; })
 #define ftrace_set_early_filter(ops, buf, enable) do { } while (0)
 #define ftrace_set_filter_ip(ops, ip, remove, reset) ({ -ENODEV; })
+#define ftrace_set_filter_ips(ops, ips, cnt, remove, reset) ({ -ENODEV; })
 #define ftrace_set_filter(ops, buf, len, reset) ({ -ENODEV; })
 #define ftrace_set_notrace(ops, buf, len, reset) ({ -ENODEV; })
 #define ftrace_free_filter(ops) do { } while (0)
kernel/trace/ftrace.c +49 -9

100 lines not on this page · raw message

author maintainer
#2 lore raw

PATCH v10 2/12 fprobe: Add ftrace based probe APIs

In reply to Masami Hiramatsu

The fprobe is a wrapper API for ftrace function tracer. Unlike kprobes, this probes only supports the function entry, but this can probe multiple functions by one fprobe. The usage is similar, user will set their callback to fprobe::entry_handler and call register_fprobe*() with probed functions.
There are 3 registration interfaces,

- register_fprobe() takes filtering patterns of the functin names.
- register_fprobe_ips() takes an array of ftrace-location addresses.
- register_fprobe_syms() takes an array of function names.

The registered fprobes can be unregistered with unregister_fprobe(). e.g.

struct fprobe fp = { .entry_handler = user_handler }; const char *targets[] = { "func1", "func2", "func3"};
...

ret = register_fprobe_syms(&fp, targets, ARRAY_SIZE(targets));

...

unregister_fprobe(&fp);

---
Changes in v10:
- Simplify the loops in get_ftrace_locations() (Thanks Jiri!)
Changes in v9:
- Return an error if fprobe can not get the address or size of given symbol from kallsyms.
- Use register_fprobe_ips() from register_fprobe_syms().
Changes in v8:
- Remove ftrace_ops if CONFIG_FUNCTION_TRACER=n.
Changes in v7:
- Fix kerneldoc for the APIs.
Changes in v6:
- Remove syms, addrs, and nentry fields from struct fprobe.
- Introduce 3 variants of registration functions.
- Call ftrace_free_filter() at unregistration.

Changes in v4:
- Fix a memory leak when symbol lookup failed.
- Use ftrace location address instead of symbol address.
- Convert the given symbol address to ftrace location automatically.
- Rename fprobe::ftrace to fprobe::ops.
- Update the Kconfig description.

---
 include/linux/fprobe.h |   87 ++++++++++++++++++++
 kernel/trace/Kconfig   |   12 +++
 kernel/trace/Makefile  |    1 
 kernel/trace/fprobe.c  |  211 ++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 311 insertions(+)
 create mode 100644 include/linux/fprobe.h
 create mode 100644 kernel/trace/fprobe.c
include/linux/fprobe.h new file +87 -0
new file mode 100644
index 000000000000..2ba099aff041
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Simple ftrace probe wrapper */
+#ifndef _LINUX_FPROBE_H
+#define _LINUX_FPROBE_H
+
+#include <linux/compiler.h>
+#include <linux/ftrace.h>
+
+/**
+ * struct fprobe - ftrace based probe.
+ * @ops: The ftrace_ops.
+ * @nmissed: The counter for missing events.
+ * @flags: The status flag.
+ * @entry_handler: The callback function for function entry.
+ */
+struct fprobe {
+#ifdef CONFIG_FUNCTION_TRACER
+	/*
+	 * If CONFIG_FUNCTION_TRACER is not set, CONFIG_FPROBE is disabled too.
+	 * But user of fprobe may keep embedding the struct fprobe on their own
+	 * code. To avoid build error, this will keep the fprobe data structure
+	 * defined here, but remove ftrace_ops data structure.
+	 */
+	struct ftrace_ops	ops;
+#endif
+	unsigned long		nmissed;
+	unsigned int		flags;
+	void (*entry_handler)(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
+};
+
+#define FPROBE_FL_DISABLED	1
+
+static inline bool fprobe_disabled(struct fprobe *fp)
+{
+	return (fp) ? fp->flags & FPROBE_FL_DISABLED : false;
+}
+
+#ifdef CONFIG_FPROBE
+int register_fprobe(struct fprobe *fp, const char *filter, const char *notfilter);
+int register_fprobe_ips(struct fprobe *fp, unsigned long *addrs, int num);
+int register_fprobe_syms(struct fprobe *fp, const char **syms, int num);
+int unregister_fprobe(struct fprobe *fp);
+#else
+static inline int register_fprobe(struct fprobe *fp, const char *filter, const char *notfilter)
+{
+	return -EOPNOTSUPP;
+}
+static inline int register_fprobe_ips(struct fprobe *fp, unsigned long *addrs, int num)
+{
+	return -EOPNOTSUPP;
+}
+static inline int register_fprobe_syms(struct fprobe *fp, const char **syms, int num)
+{
+	return -EOPNOTSUPP;
+}
+static inline int unregister_fprobe(struct fprobe *fp)
+{
+	return -EOPNOTSUPP;
+}
+#endif
+
+/**
+ * disable_fprobe() - Disable fprobe
+ * @fp: The fprobe to be disabled.
+ *
+ * This will soft-disable @fp. Note that this doesn't remove the ftrace
+ * hooks from the function entry.
+ */
+static inline void disable_fprobe(struct fprobe *fp)
+{
+	if (fp)
+		fp->flags |= FPROBE_FL_DISABLED;
+}
+
+/**
+ * enable_fprobe() - Enable fprobe
+ * @fp: The fprobe to be enabled.
+ *
+ * This will soft-enable @fp.
+ */
+static inline void enable_fprobe(struct fprobe *fp)
+{
+	if (fp)
+		fp->flags &= ~FPROBE_FL_DISABLED;
+}
+
+#endif
kernel/trace/Kconfig +12 -0
index a5eb5e7fd624..7ce31abc542b 100644
@@ -236,6 +236,18 @@ config DYNAMIC_FTRACE_WITH_ARGS
 	depends on DYNAMIC_FTRACE
 	depends on HAVE_DYNAMIC_FTRACE_WITH_ARGS
 
+config FPROBE
+	bool "Kernel Function Probe (fprobe)"
+	depends on FUNCTION_TRACER
+	depends on DYNAMIC_FTRACE_WITH_REGS
+	default n
+	help
+	  This option enables kernel function probe (fprobe) based on ftrace,
+	  which is similar to kprobes, but probes only for kernel function
+	  entries and it can probe multiple functions by one fprobe.
+
+	  If unsure, say N.
+
 config FUNCTION_PROFILER
 	bool "Kernel function profiler"
 	depends on FUNCTION_TRACER
kernel/trace/Makefile +1 -0
index bedc5caceec7..79255f9de9a4 100644
@@ -97,6 +97,7 @@ obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o
 obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o
 obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o
 obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
+obj-$(CONFIG_FPROBE) += fprobe.o
 
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
kernel/trace/fprobe.c new file +211 -0

211 lines not on this page · raw message

author maintainer
#3 lore raw

PATCH v10 3/12 rethook: Add a generic return hook

In reply to Masami Hiramatsu

Add a return hook framework which hooks the function return. Most of the logic came from the kretprobe, but this is independent from kretprobe.

Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just an additional hook.

---
Changes in v10:
- Add @mcount parameter to rethook_hook() to identify the context.
Changes in v6:
- Fix typos.
- Use dereference_symbol_descriptor() to check the trampoline address.
- Shrink down the preempt-disabled section for recycling nodes.
- Reject stack searching if the task is not current and is running in rethook_find_ret_addr().
Changes in v4:
- Fix rethook_trampoline_handler() loops as same as
what currently kretprobe does. This will fix some stacktrace issue in the rethook handler.

---
 include/linux/rethook.h |  100 +++++++++++++++
 include/linux/sched.h   |    3 
 kernel/exit.c           |    2 
 kernel/fork.c           |    3 
 kernel/trace/Kconfig    |   11 ++
 kernel/trace/Makefile   |    1 
 kernel/trace/rethook.c  |  317 +++++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 437 insertions(+)
 create mode 100644 include/linux/rethook.h
 create mode 100644 kernel/trace/rethook.c
include/linux/rethook.h new file +100 -0
new file mode 100644
index 000000000000..c8ac1e5afcd1
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Return hooking with list-based shadow stack.
+ */
+#ifndef _LINUX_RETHOOK_H
+#define _LINUX_RETHOOK_H
+
+#include <linux/compiler.h>
+#include <linux/freelist.h>
+#include <linux/kallsyms.h>
+#include <linux/llist.h>
+#include <linux/rcupdate.h>
+#include <linux/refcount.h>
+
+struct rethook_node;
+
+typedef void (*rethook_handler_t) (struct rethook_node *, void *, struct pt_regs *);
+
+/**
+ * struct rethook - The rethook management data structure.
+ * @data: The user-defined data storage.
+ * @handler: The user-defined return hook handler.
+ * @pool: The pool of struct rethook_node.
+ * @ref: The reference counter.
+ * @rcu: The rcu_head for deferred freeing.
+ *
+ * Don't embed to another data structure, because this is a self-destructive
+ * data structure when all rethook_node are freed.
+ */
+struct rethook {
+	void			*data;
+	rethook_handler_t	handler;
+	struct freelist_head	pool;
+	refcount_t		ref;
+	struct rcu_head		rcu;
+};
+
+/**
+ * struct rethook_node - The rethook shadow-stack entry node.
+ * @freelist: The freelist, linked to struct rethook::pool.
+ * @rcu: The rcu_head for deferred freeing.
+ * @llist: The llist, linked to a struct task_struct::rethooks.
+ * @rethook: The pointer to the struct rethook.
+ * @ret_addr: The storage for the real return address.
+ * @frame: The storage for the frame pointer.
+ *
+ * You can embed this to your extended data structure to store any data
+ * on each entry of the shadow stack.
+ */
+struct rethook_node {
+	union {
+		struct freelist_node freelist;
+		struct rcu_head      rcu;
+	};
+	struct llist_node	llist;
+	struct rethook		*rethook;
+	unsigned long		ret_addr;
+	unsigned long		frame;
+};
+
+struct rethook *rethook_alloc(void *data, rethook_handler_t handler);
+void rethook_free(struct rethook *rh);
+void rethook_add_node(struct rethook *rh, struct rethook_node *node);
+struct rethook_node *rethook_try_get(struct rethook *rh);
+void rethook_recycle(struct rethook_node *node);
+void rethook_hook(struct rethook_node *node, struct pt_regs *regs, bool mcount);
+unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame,
+				    struct llist_node **cur);
+
+/* Arch dependent code must implement arch_* and trampoline code */
+void arch_rethook_prepare(struct rethook_node *node, struct pt_regs *regs, bool mcount);
+void arch_rethook_trampoline(void);
+
+/**
+ * is_rethook_trampoline() - Check whether the address is rethook trampoline
+ * @addr: The address to be checked
+ *
+ * Return true if the @addr is the rethook trampoline address.
+ */
+static inline bool is_rethook_trampoline(unsigned long addr)
+{
+	return addr == (unsigned long)dereference_symbol_descriptor(arch_rethook_trampoline);
+}
+
+/* If the architecture needs to fixup the return address, implement it. */
+void arch_rethook_fixup_return(struct pt_regs *regs,
+			       unsigned long correct_ret_addr);
+
+/* Generic trampoline handler, arch code must prepare asm stub */
+unsigned long rethook_trampoline_handler(struct pt_regs *regs,
+					 unsigned long frame);
+
+#ifdef CONFIG_RETHOOK
+void rethook_flush_task(struct task_struct *tk);
+#else
+#define rethook_flush_task(tsk)	do { } while (0)
+#endif
+
+#endif
+
include/linux/sched.h +3 -0
index 508b91d57470..efc0e2b6a772 100644
@@ -1485,6 +1485,9 @@ struct task_struct {
 #ifdef CONFIG_KRETPROBES
 	struct llist_head               kretprobe_instances;
 #endif
+#ifdef CONFIG_RETHOOK
+	struct llist_head               rethooks;
+#endif
 
 #ifdef CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH
 	/*
kernel/exit.c +2 -0
index b00a25bb4ab9..2d1803fa8fe6 100644
@@ -64,6 +64,7 @@
 #include <linux/compat.h>
 #include <linux/io_uring.h>
 #include <linux/kprobes.h>
+#include <linux/rethook.h>
 
 #include <linux/uaccess.h>
 #include <asm/unistd.h>
@@ -169,6 +170,7 @@ static void delayed_put_task_struct(struct rcu_head *rhp)
 	struct task_struct *tsk = container_of(rhp, struct task_struct, rcu);
 
 	kprobe_flush_task(tsk);
+	rethook_flush_task(tsk);
 	perf_event_delayed_put(tsk);
 	trace_sched_process_free(tsk);
 	put_task_struct(tsk);
kernel/fork.c +3 -0
index d75a528f7b21..0e29dd7b6cc2 100644
@@ -2255,6 +2255,9 @@ static __latent_entropy struct task_struct *copy_process(
 #ifdef CONFIG_KRETPROBES
 	p->kretprobe_instances.first = NULL;
 #endif
+#ifdef CONFIG_RETHOOK
+	p->rethooks.first = NULL;
+#endif
 
 	/*
 	 * Ensure that the cgroup subsystem policies allow the new process to be
kernel/trace/Kconfig +11 -0
index 7ce31abc542b..e75504e42ab8 100644
@@ -10,6 +10,17 @@ config USER_STACKTRACE_SUPPORT
 config NOP_TRACER
 	bool
 
+config HAVE_RETHOOK
+	bool
+
+config RETHOOK
+	bool
+	depends on HAVE_RETHOOK
+	help
+	  Enable generic return hooking feature. This is an internal
+	  API, which will be used by other function-entry hooking
+	  features like fprobe and kprobes.
+
 config HAVE_FUNCTION_TRACER
 	bool
 	help
kernel/trace/Makefile +1 -0
index 79255f9de9a4..c6f11a139eac 100644
@@ -98,6 +98,7 @@ obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o
 obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o
 obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
 obj-$(CONFIG_FPROBE) += fprobe.o
+obj-$(CONFIG_RETHOOK) += rethook.o
 
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
kernel/trace/rethook.c new file +317 -0

317 lines not on this page · raw message

author maintainer
#5 lore raw

PATCH v10 4/12 rethook: x86: Add rethook x86 implementation

In reply to Masami Hiramatsu

Add rethook for x86 implementation. Most of the code has been copied from kretprobes on x86.

---
Changes in v10:
- Add a dummy @mcount to arch_rethook_prepare().
Changes in v5:
- Fix a build error if !CONFIG_KRETPROBES and !CONFIG_RETHOOK.
Changes in v4:
- fix stack backtrace as same as kretprobe does.

---
 arch/x86/Kconfig                 |    1 
 arch/x86/include/asm/unwind.h    |    8 ++-
 arch/x86/kernel/Makefile         |    1 
 arch/x86/kernel/kprobes/common.h |    1 
 arch/x86/kernel/rethook.c        |  115 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 125 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/rethook.c
arch/x86/Kconfig +1 -0
index 9f5bd41bf660..a91373c56d9f 100644
@@ -220,6 +220,7 @@ config X86
 	select HAVE_KPROBES_ON_FTRACE
 	select HAVE_FUNCTION_ERROR_INJECTION
 	select HAVE_KRETPROBES
+	select HAVE_RETHOOK
 	select HAVE_KVM
 	select HAVE_LIVEPATCH			if X86_64
 	select HAVE_MIXED_BREAKPOINTS_REGS
arch/x86/include/asm/unwind.h +7 -1
index 2a1f8734416d..192df5b2094d 100644
@@ -5,6 +5,7 @@
 #include <linux/sched.h>
 #include <linux/ftrace.h>
 #include <linux/kprobes.h>
+#include <linux/rethook.h>
 #include <asm/ptrace.h>
 #include <asm/stacktrace.h>
 
@@ -16,7 +17,7 @@ struct unwind_state {
 	unsigned long stack_mask;
 	struct task_struct *task;
 	int graph_idx;
-#ifdef CONFIG_KRETPROBES
+#if defined(CONFIG_KRETPROBES) || defined(CONFIG_RETHOOK)
 	struct llist_node *kr_cur;
 #endif
 	bool error;
@@ -107,6 +108,11 @@ static inline
 unsigned long unwind_recover_kretprobe(struct unwind_state *state,
 				       unsigned long addr, unsigned long *addr_p)
 {
+#ifdef CONFIG_RETHOOK
+	if (is_rethook_trampoline(addr))
+		return rethook_find_ret_addr(state->task, (unsigned long)addr_p,
+					     &state->kr_cur);
+#endif
 #ifdef CONFIG_KRETPROBES
 	return is_kretprobe_trampoline(addr) ?
 		kretprobe_find_ret_addr(state->task, addr_p, &state->kr_cur) :
arch/x86/kernel/Makefile +1 -0
index 6aef9ee28a39..792a893a5cc5 100644
@@ -106,6 +106,7 @@ obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
 obj-$(CONFIG_FTRACE_SYSCALLS)	+= ftrace.o
 obj-$(CONFIG_X86_TSC)		+= trace_clock.o
 obj-$(CONFIG_TRACING)		+= trace.o
+obj-$(CONFIG_RETHOOK)		+= rethook.o
 obj-$(CONFIG_CRASH_CORE)	+= crash_core_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= relocate_kernel_$(BITS).o crash.o
arch/x86/kernel/kprobes/common.h +1 -0
index 7d3a2e2daf01..c993521d4933 100644
@@ -6,6 +6,7 @@
 
 #include <asm/asm.h>
 #include <asm/frame.h>
+#include <asm/insn.h>
 
 #ifdef CONFIG_X86_64
 
arch/x86/kernel/rethook.c new file +115 -0

115 lines not on this page · raw message

author maintainer
#6 lore raw

PATCH v10 5/12 arm64: rethook: Add arm64 rethook implementation

In reply to Masami Hiramatsu

Add rethook arm64 implementation. Most of the code has been copied from kretprobes on arm64.

---
Changes in v10:
- Add a dummy @mcount to arch_rethook_prepare().
Changes in v5:
- Add description.
- Fix build error if !CONFIG_KRETPROBES

---
 arch/arm64/Kconfig                            |    1 
 arch/arm64/include/asm/stacktrace.h           |    2 -
 arch/arm64/kernel/probes/Makefile             |    1 
 arch/arm64/kernel/probes/rethook.c            |   25 +++++++
 arch/arm64/kernel/probes/rethook_trampoline.S |   87 +++++++++++++++++++++++++
 arch/arm64/kernel/stacktrace.c                |    7 ++
 6 files changed, 121 insertions(+), 2 deletions(-)
 create mode 100644 arch/arm64/kernel/probes/rethook.c
 create mode 100644 arch/arm64/kernel/probes/rethook_trampoline.S
arch/arm64/Kconfig +1 -0
index 6978140edfa4..f098ff3b2273 100644
@@ -201,6 +201,7 @@ config ARM64
 	select HAVE_SYSCALL_TRACEPOINTS
 	select HAVE_KPROBES
 	select HAVE_KRETPROBES
+	select HAVE_RETHOOK
 	select HAVE_GENERIC_VDSO
 	select IOMMU_DMA if IOMMU_SUPPORT
 	select IRQ_DOMAIN
arch/arm64/include/asm/stacktrace.h +1 -1
index e77cdef9ca29..bf04107da97c 100644
@@ -58,7 +58,7 @@ struct stackframe {
 	DECLARE_BITMAP(stacks_done, __NR_STACK_TYPES);
 	unsigned long prev_fp;
 	enum stack_type prev_type;
-#ifdef CONFIG_KRETPROBES
+#if defined(CONFIG_KRETPROBES) || defined(CONFIG_RETHOOK)
 	struct llist_node *kr_cur;
 #endif
 };
arch/arm64/kernel/probes/Makefile +1 -0
index 8e4be92e25b1..24e689f44c32 100644
@@ -4,3 +4,4 @@ obj-$(CONFIG_KPROBES)		+= kprobes.o decode-insn.o	\
 				   simulate-insn.o
 obj-$(CONFIG_UPROBES)		+= uprobes.o decode-insn.o	\
 				   simulate-insn.o
+obj-$(CONFIG_RETHOOK)		+= rethook.o rethook_trampoline.o
arch/arm64/kernel/probes/rethook.c new file +25 -0
new file mode 100644
index 000000000000..edc6b804ad6a
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Generic return hook for arm64.
+ * Most of the code is copied from arch/arm64/kernel/probes/kprobes.c
+ */
+
+#include <linux/kprobes.h>
+#include <linux/rethook.h>
+
+/* This is called from arch_rethook_trampoline() */
+unsigned long __used arch_rethook_trampoline_callback(struct pt_regs *regs)
+{
+	return rethook_trampoline_handler(regs, regs->regs[29]);
+}
+NOKPROBE_SYMBOL(arch_rethook_trampoline_callback);
+
+void arch_rethook_prepare(struct rethook_node *rhn, struct pt_regs *regs, bool mcount)
+{
+	rhn->ret_addr = regs->regs[30];
+	rhn->frame = regs->regs[29];
+
+	/* replace return addr (x30) with trampoline */
+	regs->regs[30] = (u64)arch_rethook_trampoline;
+}
+NOKPROBE_SYMBOL(arch_rethook_prepare);
arch/arm64/kernel/probes/rethook_trampoline.S new file +87 -0

87 lines not on this page · raw message

arch/arm64/kernel/stacktrace.c +6 -1
index 0fb58fed54cb..821c4b90fe44 100644
@@ -8,6 +8,7 @@
 #include <linux/export.h>
 #include <linux/ftrace.h>
 #include <linux/kprobes.h>
+#include <linux/rethook.h>
 #include <linux/sched.h>
 #include <linux/sched/debug.h>
 #include <linux/sched/task_stack.h>
@@ -38,7 +39,7 @@ static void start_backtrace(struct stackframe *frame, unsigned long fp,
 {
 	frame->fp = fp;
 	frame->pc = pc;
-#ifdef CONFIG_KRETPROBES
+#if defined(CONFIG_KRETPROBES) || defined(CONFIG_RETHOOK)
 	frame->kr_cur = NULL;
 #endif
 
@@ -137,6 +138,10 @@ static int notrace unwind_frame(struct task_struct *tsk,
 	if (is_kretprobe_trampoline(frame->pc))
 		frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur);
 #endif
+#ifdef CONFIG_RETHOOK
+	if (is_rethook_trampoline(frame->pc))
+		frame->pc = rethook_find_ret_addr(tsk, frame->fp, &frame->kr_cur);
+#endif
 
 	return 0;
 }
author maintainer
#7 lore raw

PATCH v10 6/12 powerpc: Add rethook support

In reply to Masami Hiramatsu

Add rethook powerpc64 implementation. Most of the code has been copied from kretprobes on powerpc64.

---
Changes in v10:
- Add a dummy @mcount to arch_rethook_prepare().

---
 arch/powerpc/Kconfig          |    1 +
 arch/powerpc/kernel/Makefile  |    1 +
 arch/powerpc/kernel/rethook.c |   72 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 74 insertions(+)
 create mode 100644 arch/powerpc/kernel/rethook.c
arch/powerpc/Kconfig +1 -0
index b779603978e1..5feaa241fb56 100644
@@ -229,6 +229,7 @@ config PPC
 	select HAVE_PERF_EVENTS_NMI		if PPC64
 	select HAVE_PERF_REGS
 	select HAVE_PERF_USER_STACK_DUMP
+	select HAVE_RETHOOK			if KPROBES
 	select HAVE_REGS_AND_STACK_ACCESS_API
 	select HAVE_RELIABLE_STACKTRACE
 	select HAVE_RSEQ
arch/powerpc/kernel/Makefile +1 -0
index 4d7829399570..feb24ea83ca6 100644
@@ -115,6 +115,7 @@ obj-$(CONFIG_SMP)		+= smp.o
 obj-$(CONFIG_KPROBES)		+= kprobes.o
 obj-$(CONFIG_OPTPROBES)		+= optprobes.o optprobes_head.o
 obj-$(CONFIG_KPROBES_ON_FTRACE)	+= kprobes-ftrace.o
+obj-$(CONFIG_RETHOOK)		+= rethook.o
 obj-$(CONFIG_UPROBES)		+= uprobes.o
 obj-$(CONFIG_PPC_UDBG_16550)	+= legacy_serial.o udbg_16550.o
 obj-$(CONFIG_SWIOTLB)		+= dma-swiotlb.o
arch/powerpc/kernel/rethook.c new file +72 -0

72 lines not on this page · raw message

author maintainer
#9 lore raw

PATCH v10 8/12 fprobe: Add exit_handler support

In reply to Masami Hiramatsu

Add exit_handler to fprobe. fprobe + rethook allows us to hook the kernel function return. The rethook will be enabled only if the fprobe::exit_handler is set.

---
Changes in v10:
- Call rethook_hook() with mcount context flag.
Changes in v7:
- Fix unregister_fprobe() to ensure the rethook handlers are finished when it returns.
- Update Kconfig help.
Changes in v6:
- Update according to the fprobe update.
Changes in v5:
- Add dependency for HAVE_RETHOOK.
Changes in v4:
- Check fprobe is disabled in the exit handler.
Changes in v3:
- Make sure to clear rethook->data before free.
- Handler checks the data is not NULL.
- Free rethook only if the rethook is using.

---
 include/linux/fprobe.h |    6 ++
 kernel/trace/Kconfig   |    9 ++--
 kernel/trace/fprobe.c  |  116 ++++++++++++++++++++++++++++++++++++++++++++++--
 3 files changed, 122 insertions(+), 9 deletions(-)
include/linux/fprobe.h +6 -0
index 2ba099aff041..8eefec2b485e 100644
@@ -5,13 +5,16 @@
 
 #include <linux/compiler.h>
 #include <linux/ftrace.h>
+#include <linux/rethook.h>
 
 /**
  * struct fprobe - ftrace based probe.
  * @ops: The ftrace_ops.
  * @nmissed: The counter for missing events.
  * @flags: The status flag.
+ * @rethook: The rethook data structure. (internal data)
  * @entry_handler: The callback function for function entry.
+ * @exit_handler: The callback function for function exit.
  */
 struct fprobe {
 #ifdef CONFIG_FUNCTION_TRACER
@@ -25,7 +28,10 @@ struct fprobe {
 #endif
 	unsigned long		nmissed;
 	unsigned int		flags;
+	struct rethook		*rethook;
+
 	void (*entry_handler)(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
+	void (*exit_handler)(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
 };
 
 #define FPROBE_FL_DISABLED	1
kernel/trace/Kconfig +6 -3
index e75504e42ab8..99dd4ca63d68 100644
@@ -251,11 +251,14 @@ config FPROBE
 	bool "Kernel Function Probe (fprobe)"
 	depends on FUNCTION_TRACER
 	depends on DYNAMIC_FTRACE_WITH_REGS
+	depends on HAVE_RETHOOK
+	select RETHOOK
 	default n
 	help
-	  This option enables kernel function probe (fprobe) based on ftrace,
-	  which is similar to kprobes, but probes only for kernel function
-	  entries and it can probe multiple functions by one fprobe.
+	  This option enables kernel function probe (fprobe) based on ftrace.
+	  The fprobe is similar to kprobes, but probes only for kernel function
+	  entries and exits. This also can probe multiple functions by one
+	  fprobe.
 
 	  If unsure, say N.
 
kernel/trace/fprobe.c +110 -6

176 lines not on this page · raw message

author maintainer
#8 lore raw

PATCH v10 7/12 ARM: rethook: Add rethook arm implementation

In reply to Masami Hiramatsu

Add rethook arm implementation. Most of the code has been copied from kretprobes on arm.
Since the arm's ftrace implementation is a bit special, this needs a special care using from fprobe.

---
Changes in v10:
- Fix for the mcount entry.
Changes in v5:
- Fix build error when !CONFIG_KRETPROBES

---
 arch/arm/Kconfig                  |    1 
 arch/arm/include/asm/stacktrace.h |    4 +
 arch/arm/kernel/stacktrace.c      |    6 ++
 arch/arm/probes/Makefile          |    1 
 arch/arm/probes/rethook.c         |  101 +++++++++++++++++++++++++++++++++++++
 5 files changed, 111 insertions(+), 2 deletions(-)
 create mode 100644 arch/arm/probes/rethook.c
arch/arm/Kconfig +1 -0
index 4c97cb40eebb..440f69ee8af5 100644
@@ -107,6 +107,7 @@ config ARM
 	select HAVE_MOD_ARCH_SPECIFIC
 	select HAVE_NMI
 	select HAVE_OPTPROBES if !THUMB2_KERNEL
+	select HAVE_RETHOOK
 	select HAVE_PERF_EVENTS
 	select HAVE_PERF_REGS
 	select HAVE_PERF_USER_STACK_DUMP
arch/arm/include/asm/stacktrace.h +2 -2
index 8f54f9ad8a9b..babed1707ca8 100644
@@ -14,7 +14,7 @@ struct stackframe {
 	unsigned long sp;
 	unsigned long lr;
 	unsigned long pc;
-#ifdef CONFIG_KRETPROBES
+#if defined(CONFIG_KRETPROBES) || defined(CONFIG_RETHOOK)
 	struct llist_node *kr_cur;
 	struct task_struct *tsk;
 #endif
@@ -27,7 +27,7 @@ void arm_get_current_stackframe(struct pt_regs *regs, struct stackframe *frame)
 		frame->sp = regs->ARM_sp;
 		frame->lr = regs->ARM_lr;
 		frame->pc = regs->ARM_pc;
-#ifdef CONFIG_KRETPROBES
+#if defined(CONFIG_KRETPROBES) || defined(CONFIG_RETHOOK)
 		frame->kr_cur = NULL;
 		frame->tsk = current;
 #endif
arch/arm/kernel/stacktrace.c +6 -0
index 75e905508f27..f509c6be4f57 100644
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-only
 #include <linux/export.h>
 #include <linux/kprobes.h>
+#include <linux/rethook.h>
 #include <linux/sched.h>
 #include <linux/sched/debug.h>
 #include <linux/stacktrace.h>
@@ -66,6 +67,11 @@ int notrace unwind_frame(struct stackframe *frame)
 	frame->sp = *(unsigned long *)(fp - 8);
 	frame->pc = *(unsigned long *)(fp - 4);
 #endif
+#ifdef CONFIG_RETHOOK
+	if (is_rethook_trampoline(frame->pc))
+		frame->pc = rethook_find_ret_addr(frame->tsk, frame->fp,
+						  &frame->kr_cur);
+#endif
 #ifdef CONFIG_KRETPROBES
 	if (is_kretprobe_trampoline(frame->pc))
 		frame->pc = kretprobe_find_ret_addr(frame->tsk,
arch/arm/probes/Makefile +1 -0
index 8b0ea5ace100..10c083a22223 100644
@@ -6,3 +6,4 @@ obj-$(CONFIG_KPROBES)		+= decode-thumb.o
 else
 obj-$(CONFIG_KPROBES)		+= decode-arm.o
 endif
+obj-$(CONFIG_RETHOOK)		+= rethook.o
arch/arm/probes/rethook.c new file +101 -0

101 lines not on this page · raw message

author maintainer
#10 lore raw

PATCH v10 9/12 fprobe: Add sample program for fprobe

In reply to Masami Hiramatsu

Add a sample program for the fprobe. The sample_fprobe puts a fprobe on kernel_clone() by default. This dump stack and some called address info at the function entry and exit.

The sample_fprobe.ko gets 2 parameters.
- symbol: you can specify the comma separated symbols or wildcard symbol pattern (in this case you can not use comma)
- stackdump: a bool value to enable or disable stack dump in the fprobe handler.

---
Changes in v6:
- Dump stack on the handler as explained in the comment.
- Add "stackdump" option to enable/disable stackdump.
- Support wildcard filter.
Changes in v2:
- Fix infinit loop for multiple symbols.
- Fix memory leaks for copied string and entry array.
- Update for new fprobe APIs.
- Fix style issues.

---
 samples/Kconfig                 |    7 ++
 samples/Makefile                |    1 
 samples/fprobe/Makefile         |    3 +
 samples/fprobe/fprobe_example.c |  120 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 131 insertions(+)
 create mode 100644 samples/fprobe/Makefile
 create mode 100644 samples/fprobe/fprobe_example.c
samples/Kconfig +7 -0
index 22cc921ae291..8415d60ea5f4 100644
@@ -73,6 +73,13 @@ config SAMPLE_HW_BREAKPOINT
 	help
 	  This builds kernel hardware breakpoint example modules.
 
+config SAMPLE_FPROBE
+	tristate "Build fprobe examples -- loadable modules only"
+	depends on FPROBE && m
+	help
+	  This builds a fprobe example module. This module has an option 'symbol'.
+	  You can specify a probed symbol or symbols separated with ','.
+
 config SAMPLE_KFIFO
 	tristate "Build kfifo examples -- loadable modules only"
 	depends on m
samples/Makefile +1 -0
index 1ae4de99c983..6d662965be5b 100644
@@ -33,3 +33,4 @@ subdir-$(CONFIG_SAMPLE_WATCHDOG)	+= watchdog
 subdir-$(CONFIG_SAMPLE_WATCH_QUEUE)	+= watch_queue
 obj-$(CONFIG_DEBUG_KMEMLEAK_TEST)	+= kmemleak/
 obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG)	+= coresight/
+obj-$(CONFIG_SAMPLE_FPROBE)		+= fprobe/
samples/fprobe/Makefile new file +3 -0
new file mode 100644
index 000000000000..ecccbfa6e99b
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+obj-$(CONFIG_SAMPLE_FPROBE) += fprobe_example.o
samples/fprobe/fprobe_example.c new file +120 -0

120 lines not on this page · raw message

author maintainer
#11 lore raw

PATCH v10 10/12 fprobe: Introduce FPROBE_FL_KPROBE_SHARED flag for fprobe

In reply to Masami Hiramatsu

Introduce FPROBE_FL_KPROBE_SHARED flag for sharing fprobe callback with kprobes safely from the viewpoint of recursion.

Since the recursion safety of the fprobe (and ftrace) is a bit different from the kprobes, this may cause an issue if user wants to run the same code from the fprobe and the kprobes.

The kprobes has per-cpu 'current_kprobe' variable which protects the kprobe handler from recursion in any case. On the other hand, the fprobe uses only ftrace_test_recursion_trylock(), which will allow interrupt context calls another (or same) fprobe during the fprobe user handler is running.

This is not a matter in cases if the common callback shared among the kprobes and the fprobe has its own recursion detection, or it can handle the recursion in the different contexts (normal/interrupt/NMI.) But if it relies on the 'current_kprobe' recursion lock, it has to check kprobe_running() and use kprobe_busy_*() APIs.

Fprobe has FPROBE_FL_KPROBE_SHARED flag to do this. If your common callback code will be shared with kprobes, please set FPROBE_FL_KPROBE_SHARED *before* registering the fprobe, like;

fprobe.flags = FPROBE_FL_KPROBE_SHARED;

register_fprobe(&fprobe, "func*", NULL);

This will protect your common callback from the nested call.

---
 include/linux/fprobe.h  |   12 ++++++++++++
 include/linux/kprobes.h |    3 +++
 kernel/trace/fprobe.c   |   19 ++++++++++++++++++-
 3 files changed, 33 insertions(+), 1 deletion(-)
include/linux/fprobe.h +12 -0
index 8eefec2b485e..1c2bde0ead73 100644
@@ -34,13 +34,25 @@ struct fprobe {
 	void (*exit_handler)(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
 };
 
+/* This fprobe is soft-disabled. */
 #define FPROBE_FL_DISABLED	1
 
+/*
+ * This fprobe handler will be shared with kprobes.
+ * This flag must be set before registering.
+ */
+#define FPROBE_FL_KPROBE_SHARED	2
+
 static inline bool fprobe_disabled(struct fprobe *fp)
 {
 	return (fp) ? fp->flags & FPROBE_FL_DISABLED : false;
 }
 
+static inline bool fprobe_shared_with_kprobes(struct fprobe *fp)
+{
+	return (fp) ? fp->flags & FPROBE_FL_KPROBE_SHARED : false;
+}
+
 #ifdef CONFIG_FPROBE
 int register_fprobe(struct fprobe *fp, const char *filter, const char *notfilter);
 int register_fprobe_ips(struct fprobe *fp, unsigned long *addrs, int num);
include/linux/kprobes.h +3 -0
index 19b884353b15..5f1859836deb 100644
@@ -427,6 +427,9 @@ static inline struct kprobe *kprobe_running(void)
 {
 	return NULL;
 }
+#define kprobe_busy_begin()	do {} while (0)
+#define kprobe_busy_end()	do {} while (0)
+
 static inline int register_kprobe(struct kprobe *p)
 {
 	return -EOPNOTSUPP;
kernel/trace/fprobe.c +18 -1
index 38073632bfe4..8b2dd5b9dcd1 100644
@@ -56,6 +56,20 @@ static void fprobe_handler(unsigned long ip, unsigned long parent_ip,
 }
 NOKPROBE_SYMBOL(fprobe_handler);
 
+static void fprobe_kprobe_handler(unsigned long ip, unsigned long parent_ip,
+				  struct ftrace_ops *ops, struct ftrace_regs *fregs)
+{
+	struct fprobe *fp = container_of(ops, struct fprobe, ops);
+
+	if (unlikely(kprobe_running())) {
+		fp->nmissed++;
+		return;
+	}
+	kprobe_busy_begin();
+	fprobe_handler(ip, parent_ip, ops, fregs);
+	kprobe_busy_end();
+}
+
 static void fprobe_exit_handler(struct rethook_node *rh, void *data,
 				struct pt_regs *regs)
 {
@@ -110,7 +124,10 @@ static unsigned long *get_ftrace_locations(const char **syms, int num)
 static void fprobe_init(struct fprobe *fp)
 {
 	fp->nmissed = 0;
-	fp->ops.func = fprobe_handler;
+	if (fprobe_shared_with_kprobes(fp))
+		fp->ops.func = fprobe_kprobe_handler;
+	else
+		fp->ops.func = fprobe_handler;
 	fp->ops.flags |= FTRACE_OPS_FL_SAVE_REGS;
 }
 
author maintainer
#12 lore raw

PATCH v10 11/12 docs: fprobe: Add fprobe description to ftrace-use.rst

In reply to Masami Hiramatsu

Add a documentation of fprobe for the user who needs this interface.

---
Changes in v7:
- Clarify unregister_fprobe() guarantee the callbacks will no longer being called after that.
- Fix some wording.
Changes in v6:
- Update document according to the latest spec.

---
 Documentation/trace/fprobe.rst |  171 ++++++++++++++++++++++++++++++++++++++++
 Documentation/trace/index.rst  |    1 
 2 files changed, 172 insertions(+)
 create mode 100644 Documentation/trace/fprobe.rst
Documentation/trace/fprobe.rst new file +171 -0
new file mode 100644
index 000000000000..4275e95e16bc
@@ -0,0 +1,171 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==================================
+Fprobe - Function entry/exit probe
+==================================
+
+.. Author: Masami Hiramatsu <[email protected]>
+
+Introduction
+============
+
+Instead of using ftrace full feature, if you only want to attach callbacks
+on function entry and exit, similar to the kprobes and kretprobes, you can
+use fprobe. Compared with kprobes and kretprobes, fprobe gives faster
+instrumentation for multiple functions with single handler. This document
+describes how to use fprobe.
+
+The usage of fprobe
+===================
+
+The fprobe is a wrapper of ftrace (+ kretprobe-like return callback) to
+attach callbacks to multiple function entry and exit. User needs to set up
+the `struct fprobe` and pass it to `register_fprobe()`.
+
+Typically, `fprobe` data structure is initialized with the `entry_handler`
+and/or `exit_handler` as below.
+
+.. code-block:: c
+
+ struct fprobe fp = {
+        .entry_handler  = my_entry_callback,
+        .exit_handler   = my_exit_callback,
+ };
+
+To enable the fprobe, call one of register_fprobe(), register_fprobe_ips(), and
+register_fprobe_syms(). These register the fprobe with different type of
+parameters.
+
+The register_fprobe() enables a fprobe by function-name filters.
+E.g. this enables @fp on "func*()" function except "func2()".::
+
+  register_fprobe(&fp, "func*", "func2");
+
+The register_fprobe_ips() enables a fprobe by ftrace-location addresses.
+E.g.
+
+.. code-block:: c
+
+  unsigned long ips[] = { 0x.... };
+
+  register_fprobe_ips(&fp, ips, ARRAY_SIZE(ips));
+
+And the register_fprobe_syms() enables a fprobe by symbol names.
+E.g.
+
+.. code-block:: c
+
+  char syms[] = {"func1", "func2", "func3"};
+
+  register_fprobe_syms(&fp, syms, ARRAY_SIZE(syms));
+
+To disable (remove from functions) this fprobe, call::
+
+  unregister_fprobe(&fp);
+
+You can temporally (soft) disable the fprobe by::
+
+  disable_fprobe(&fp);
+
+and resume by::
+
+  enable_fprobe(&fp);
+
+The above is defined by including the header::
+
+  #include <linux/fprobe.h>
+
+Same as ftrace, the registered callback will start being called some time
+after the register_fprobe() is called and before it returns. See
+:file:`Documentation/trace/ftrace.rst`.
+
+Also, the unregister_fprobe() will guarantee that the both enter and exit
+handlers are no longer being called by functions after unregister_fprobe()
+returns as same as unregister_ftrace_function().
+
+The fprobe entry/exit handler
+=============================
+
+The prototype of the entry/exit callback function is as follows:
+
+.. code-block:: c
+
+ void callback_func(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
+
+Note that both entry and exit callback has same ptototype. The @entry_ip is
+saved at function entry and passed to exit handler.
+
+@fp
+        This is the address of `fprobe` data structure related to this handler.
+        You can embed the `fprobe` to your data structure and get it by
+        container_of() macro from @fp. The @fp must not be NULL.
+
+@entry_ip
+        This is the entry address of the traced function (both entry and exit).
+
+@regs
+        This is the `pt_regs` data structure at the entry and exit. Note that
+        the instruction pointer of @regs may be different from the @entry_ip
+        in the entry_handler. If you need traced instruction pointer, you need
+        to use @entry_ip. On the other hand, in the exit_handler, the instruction
+        pointer of @regs is set to the currect return address.
+
+Share the callbacks with kprobes
+================================
+
+Since the recursion safety of the fprobe (and ftrace) is a bit different
+from the kprobes, this may cause an issue if user wants to run the same
+code from the fprobe and the kprobes.
+
+The kprobes has per-cpu 'current_kprobe' variable which protects the
+kprobe handler from recursion in any case. On the other hand, the fprobe
+uses only ftrace_test_recursion_trylock(), which will allow interrupt
+context calls another (or same) fprobe during the fprobe user handler is
+running.
+
+This is not a matter in cases if the common callback shared among the
+kprobes and the fprobe has its own recursion detection, or it can handle
+the recursion in the different contexts (normal/interrupt/NMI.)
+But if it relies on the 'current_kprobe' recursion lock, it has to check
+kprobe_running() and use kprobe_busy_*() APIs.
+
+Fprobe has FPROBE_FL_KPROBE_SHARED flag to do this. If your common callback
+code will be shared with kprobes, please set FPROBE_FL_KPROBE_SHARED
+*before* registering the fprobe, like:
+
+.. code-block:: c
+
+ fprobe.flags = FPROBE_FL_KPROBE_SHARED;
+
+ register_fprobe(&fprobe, "func*", NULL);
+
+This will protect your common callback from the nested call.
+
+The missed counter
+==================
+
+The `fprobe` data structure has `fprobe::nmissed` counter field as same as
+kprobes.
+This counter counts up when;
+
+ - fprobe fails to take ftrace_recursion lock. This usually means that a function
+   which is traced by other ftrace users is called from the entry_handler.
+
+ - fprobe fails to setup the function exit because of the shortage of rethook
+   (the shadow stack for hooking the function return.)
+
+Since `fprobe::nmissed` field is counted up in both case, the former case
+will skip both of entry and exit callback, and the latter case will skip exit
+callback, but in both case the counter is just increased by 1.
+
+Note that if you set the FTRACE_OPS_FL_RECURSION and/or FTRACE_OPS_FL_RCU to
+`fprobe::ops::flags` (ftrace_ops::flags) when registering the fprobe, this
+counter may not work correctly, because those will skip fprobe's callback.
+
+
+Functions and structures
+========================
+
+.. kernel-doc:: include/linux/fprobe.h
+.. kernel-doc:: kernel/trace/fprobe.c
+
Documentation/trace/index.rst +1 -0
index 3769b9b7aed8..b9f3757f8269 100644
@@ -9,6 +9,7 @@ Linux Tracing Technologies
    tracepoint-analysis
    ftrace
    ftrace-uses
+   fprobe
    kprobes
    kprobetrace
    uprobetracer
author maintainer
#13 lore raw

PATCH v10 12/12 fprobe: Add a selftest for fprobe

In reply to Masami Hiramatsu

Add a KUnit based selftest for fprobe interface.

---
Changes in v9:
- Rename fprobe_target* to fprobe_selftest_target*.
- Find the correct expected ip by ftrace_location_range().
- Since the ftrace_location_range() is not exposed to module, make this test only for embedded.
- Add entry only test.
- Reset the fprobe structure before reuse it.

---
 lib/Kconfig.debug |   12 ++++
 lib/Makefile      |    2 +
 lib/test_fprobe.c |  174 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+)
 create mode 100644 lib/test_fprobe.c
lib/Kconfig.debug +12 -0
index 14b89aa37c5c..ffc469a12afc 100644
@@ -2100,6 +2100,18 @@ config KPROBES_SANITY_TEST
 
 	  Say N if you are unsure.
 
+config FPROBE_SANITY_TEST
+	bool "Self test for fprobe"
+	depends on DEBUG_KERNEL
+	depends on FPROBE
+	depends on KUNIT
+	help
+	  This option will enable testing the fprobe when the system boot.
+	  A series of tests are made to verify that the fprobe is functioning
+	  properly.
+
+	  Say N if you are unsure.
+
 config BACKTRACE_SELF_TEST
 	tristate "Self test for the backtrace code"
 	depends on DEBUG_KERNEL
lib/Makefile +2 -0
index 300f569c626b..154008764b16 100644
@@ -103,6 +103,8 @@ obj-$(CONFIG_TEST_HMM) += test_hmm.o
 obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o
 obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
 obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o
+CFLAGS_test_fprobe.o += $(CC_FLAGS_FTRACE)
+obj-$(CONFIG_FPROBE_SANITY_TEST) += test_fprobe.o
 #
 # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
 # off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
lib/test_fprobe.c new file +174 -0

174 lines not on this page · raw message

author maintainer
#14 lore raw

Re: [PATCH v10 12/12] fprobe: Add a selftest for fprobe

Hi,

Quoted 23 lines · On Tue, 8 Mar 2022 20:10:48 +0900 Masami Hiramatsu <[email protected]> wrote:

Add a KUnit based selftest for fprobe interface.

---
Changes in v9:
- Rename fprobe_target* to fprobe_selftest_target*.
- Find the correct expected ip by ftrace_location_range().
- Since the ftrace_location_range() is not exposed to module, make this test only for embedded.
- Add entry only test.
- Reset the fprobe structure before reuse it.

---
 lib/Kconfig.debug |   12 ++++
 lib/Makefile      |    2 +
 lib/test_fprobe.c |  174 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+)
 create mode 100644 lib/test_fprobe.c
lib/Kconfig.debug +0 -0
index 14b89aa37c5c..ffc469a12afc 100644

lib/Kconfig.debug config KPROBES_SANITY_TEST · line 2107

Quoted 8 lines
lib/Kconfig.debug +5 -0
@@ -2100,6 +2100,18 @@ config KPROBES_SANITY_TEST
 
 	  Say N if you are unsure.
 
+config FPROBE_SANITY_TEST
+	bool "Self test for fprobe"
+	depends on DEBUG_KERNEL
+	depends on FPROBE
+	depends on KUNIT

Hmm, this caused a build error with allmodconfig because KUNIT=m but FPROBE_SANITY_TEST=y. Let me fix this issue.

Thank you,

Quoted 204 lines
lib/Kconfig.debug +7 -0
+	help
+	  This option will enable testing the fprobe when the system boot.
+	  A series of tests are made to verify that the fprobe is functioning
+	  properly.
+
+	  Say N if you are unsure.
+
 config BACKTRACE_SELF_TEST
 	tristate "Self test for the backtrace code"
 	depends on DEBUG_KERNEL
lib/Makefile +2 -0
index 300f569c626b..154008764b16 100644
@@ -103,6 +103,8 @@ obj-$(CONFIG_TEST_HMM) += test_hmm.o
 obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o
 obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
 obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o
+CFLAGS_test_fprobe.o += $(CC_FLAGS_FTRACE)
+obj-$(CONFIG_FPROBE_SANITY_TEST) += test_fprobe.o
 #
 # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
 # off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
lib/test_fprobe.c new file +174 -0
new file mode 100644
index 000000000000..ed70637a2ffa
@@ -0,0 +1,174 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * test_fprobe.c - simple sanity test for fprobe
+ */
+
+#include <linux/kernel.h>
+#include <linux/fprobe.h>
+#include <linux/random.h>
+#include <kunit/test.h>
+
+#define div_factor 3
+
+static struct kunit *current_test;
+
+static u32 rand1, entry_val, exit_val;
+
+/* Use indirect calls to avoid inlining the target functions */
+static u32 (*target)(u32 value);
+static u32 (*target2)(u32 value);
+static unsigned long target_ip;
+static unsigned long target2_ip;
+
+static noinline u32 fprobe_selftest_target(u32 value)
+{
+	return (value / div_factor);
+}
+
+static noinline u32 fprobe_selftest_target2(u32 value)
+{
+	return (value / div_factor) + 1;
+}
+
+static notrace void fp_entry_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+	KUNIT_EXPECT_FALSE(current_test, preemptible());
+	/* This can be called on the fprobe_selftest_target and the fprobe_selftest_target2 */
+	if (ip != target_ip)
+		KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+	entry_val = (rand1 / div_factor);
+}
+
+static notrace void fp_exit_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+	unsigned long ret = regs_return_value(regs);
+
+	KUNIT_EXPECT_FALSE(current_test, preemptible());
+	if (ip != target_ip) {
+		KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+		KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor) + 1);
+	} else
+		KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor));
+	KUNIT_EXPECT_EQ(current_test, entry_val, (rand1 / div_factor));
+	exit_val = entry_val + div_factor;
+}
+
+/* Test entry only (no rethook) */
+static void test_fprobe_entry(struct kunit *test)
+{
+	struct fprobe fp_entry = {
+		.entry_handler = fp_entry_handler,
+	};
+
+	current_test = test;
+
+	/* Before register, unregister should be failed. */
+	KUNIT_EXPECT_NE(test, 0, unregister_fprobe(&fp_entry));
+	KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp_entry, "fprobe_selftest_target*", NULL));
+
+	entry_val = 0;
+	exit_val = 0;
+	target(rand1);
+	KUNIT_EXPECT_NE(test, 0, entry_val);
+	KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+	entry_val = 0;
+	exit_val = 0;
+	target2(rand1);
+	KUNIT_EXPECT_NE(test, 0, entry_val);
+	KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+	KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp_entry));
+}
+
+static void test_fprobe(struct kunit *test)
+{
+	struct fprobe fp = {
+		.entry_handler = fp_entry_handler,
+		.exit_handler = fp_exit_handler,
+	};
+
+	current_test = test;
+	KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp, "fprobe_selftest_target*", NULL));
+
+	entry_val = 0;
+	exit_val = 0;
+	target(rand1);
+	KUNIT_EXPECT_NE(test, 0, entry_val);
+	KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+	entry_val = 0;
+	exit_val = 0;
+	target2(rand1);
+	KUNIT_EXPECT_NE(test, 0, entry_val);
+	KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+	KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static void test_fprobe_syms(struct kunit *test)
+{
+	static const char *syms[] = {"fprobe_selftest_target", "fprobe_selftest_target2"};
+	struct fprobe fp = {
+		.entry_handler = fp_entry_handler,
+		.exit_handler = fp_exit_handler,
+	};
+
+	current_test = test;
+	KUNIT_EXPECT_EQ(test, 0, register_fprobe_syms(&fp, syms, 2));
+
+	entry_val = 0;
+	exit_val = 0;
+	target(rand1);
+	KUNIT_EXPECT_NE(test, 0, entry_val);
+	KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+	entry_val = 0;
+	exit_val = 0;
+	target2(rand1);
+	KUNIT_EXPECT_NE(test, 0, entry_val);
+	KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+	KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static unsigned long get_ftrace_location(void *func)
+{
+	unsigned long size, addr = (unsigned long)func;
+
+	if (!kallsyms_lookup_size_offset(addr, &size, NULL) || !size)
+		return 0;
+
+	return ftrace_location_range(addr, addr + size - 1);
+}
+
+static int fprobe_test_init(struct kunit *test)
+{
+	do {
+		rand1 = prandom_u32();
+	} while (rand1 <= div_factor);
+
+	target = fprobe_selftest_target;
+	target2 = fprobe_selftest_target2;
+	target_ip = get_ftrace_location(target);
+	target2_ip = get_ftrace_location(target2);
+
+	return 0;
+}
+
+static struct kunit_case fprobe_testcases[] = {
+	KUNIT_CASE(test_fprobe_entry),
+	KUNIT_CASE(test_fprobe),
+	KUNIT_CASE(test_fprobe_syms),
+	{}
+};
+
+static struct kunit_suite fprobe_test_suite = {
+	.name = "fprobe_test",
+	.init = fprobe_test_init,
+	.test_cases = fprobe_testcases,
+};
+
+kunit_test_suites(&fprobe_test_suite);
+
+MODULE_LICENSE("GPL");
Signature · 1 line
Masami Hiramatsu <[email protected]>
#15 lore raw

Re: [PATCH v10 12/12] fprobe: Add a selftest for fprobe

Quoted 32 lines · Masami Hiramatsu

Hi,

Quoted 26 lines · On Tue, 8 Mar 2022 20:10:48 +0900 Masami Hiramatsu <[email protected]> wrote:

Add a KUnit based selftest for fprobe interface.

---
Changes in v9:
- Rename fprobe_target* to fprobe_selftest_target*.
- Find the correct expected ip by ftrace_location_range().
- Since the ftrace_location_range() is not exposed to module, make this test only for embedded.
- Add entry only test.
- Reset the fprobe structure before reuse it.

---
 lib/Kconfig.debug |   12 ++++
 lib/Makefile      |    2 +
 lib/test_fprobe.c |  174 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+)
 create mode 100644 lib/test_fprobe.c
lib/Kconfig.debug +0 -0
index 14b89aa37c5c..ffc469a12afc 100644
@@ -2100,6 +2100,18 @@ config KPROBES_SANITY_TEST
 
        Say N if you are unsure.
 
Quoted 8 lines

lib/Kconfig.debug config KPROBES_SANITY_TEST · line 2107

Quoted 5 lines
lib/Kconfig.debug +5 -0
+config FPROBE_SANITY_TEST
+     bool "Self test for fprobe"
+     depends on DEBUG_KERNEL
+     depends on FPROBE
+     depends on KUNIT

Hmm, this caused a build error with allmodconfig because KUNIT=m but FPROBE_SANITY_TEST=y. Let me fix this issue.

Please base on top of bpf-next and add [PATCH v11 bpf-next] to subject.

Quoted 211 lines

Thank you,

Quoted 194 lines
lib/Kconfig.debug +7 -0
+     help
+       This option will enable testing the fprobe when the system boot.
+       A series of tests are made to verify that the fprobe is functioning
+       properly.
+
+       Say N if you are unsure.
+
 config BACKTRACE_SELF_TEST
      tristate "Self test for the backtrace code"
      depends on DEBUG_KERNEL
lib/Makefile +2 -0
index 300f569c626b..154008764b16 100644
@@ -103,6 +103,8 @@ obj-$(CONFIG_TEST_HMM) += test_hmm.o
 obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o
 obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
 obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o
+CFLAGS_test_fprobe.o += $(CC_FLAGS_FTRACE)
+obj-$(CONFIG_FPROBE_SANITY_TEST) += test_fprobe.o
 #
 # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
 # off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
lib/test_fprobe.c new file +164 -0
new file mode 100644
index 000000000000..ed70637a2ffa
@@ -0,0 +1,174 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * test_fprobe.c - simple sanity test for fprobe
+ */
+
+#include <linux/kernel.h>
+#include <linux/fprobe.h>
+#include <linux/random.h>
+#include <kunit/test.h>
+
+#define div_factor 3
+
+static struct kunit *current_test;
+
+static u32 rand1, entry_val, exit_val;
+
+/* Use indirect calls to avoid inlining the target functions */
+static u32 (*target)(u32 value);
+static u32 (*target2)(u32 value);
+static unsigned long target_ip;
+static unsigned long target2_ip;
+
+static noinline u32 fprobe_selftest_target(u32 value)
+{
+     return (value / div_factor);
+}
+
+static noinline u32 fprobe_selftest_target2(u32 value)
+{
+     return (value / div_factor) + 1;
+}
+
+static notrace void fp_entry_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     /* This can be called on the fprobe_selftest_target and the fprobe_selftest_target2 */
+     if (ip != target_ip)
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+     entry_val = (rand1 / div_factor);
+}
+
+static notrace void fp_exit_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     unsigned long ret = regs_return_value(regs);
+
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     if (ip != target_ip) {
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor) + 1);
+     } else
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor));
+     KUNIT_EXPECT_EQ(current_test, entry_val, (rand1 / div_factor));
+     exit_val = entry_val + div_factor;
+}
+
+/* Test entry only (no rethook) */
+static void test_fprobe_entry(struct kunit *test)
+{
+     struct fprobe fp_entry = {
+             .entry_handler = fp_entry_handler,
+     };
+
+     current_test = test;
+
+     /* Before register, unregister should be failed. */
+     KUNIT_EXPECT_NE(test, 0, unregister_fprobe(&fp_entry));
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp_entry, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp_entry));
+}
+
+static void test_fprobe(struct kunit *test)
+{
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static void test_fprobe_syms(struct kunit *test)
+{
+     static const char *syms[] = {"fprobe_selftest_target", "fprobe_selftest_target2"};
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe_syms(&fp, syms, 2));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static unsigned long get_ftrace_location(void *func)
+{
+     unsigned long size, addr = (unsigned long)func;
+
+     if (!kallsyms_lookup_size_offset(addr, &size, NULL) || !size)
+             return 0;
+
+     return ftrace_location_range(addr, addr + size - 1);
+}
+
+static int fprobe_test_init(struct kunit *test)
+{
+     do {
+             rand1 = prandom_u32();
+     } while (rand1 <= div_factor);
+
+     target = fprobe_selftest_target;
+     target2 = fprobe_selftest_target2;
+     target_ip = get_ftrace_location(target);
+     target2_ip = get_ftrace_location(target2);
+
+     return 0;
+}
+
+static struct kunit_case fprobe_testcases[] = {
+     KUNIT_CASE(test_fprobe_entry),
+     KUNIT_CASE(test_fprobe),
+     KUNIT_CASE(test_fprobe_syms),
+     {}
+};

lib/test_fprobe.c line 174

Quoted 10 lines
lib/test_fprobe.c new file +10 -0
+
+static struct kunit_suite fprobe_test_suite = {
+     .name = "fprobe_test",
+     .init = fprobe_test_init,
+     .test_cases = fprobe_testcases,
+};
+
+kunit_test_suites(&fprobe_test_suite);
+
+MODULE_LICENSE("GPL");

--
Masami Hiramatsu <[email protected]>

author maintainer
#16 lore raw

Re: [PATCH v10 12/12] fprobe: Add a selftest for fprobe

Quoted 42 lines · On Wed, 9 Mar 2022 16:40:00 -0800 Andrii Nakryiko <[email protected]>…
Quoted 32 lines · Masami Hiramatsu

Hi,

Quoted 26 lines · On Tue, 8 Mar 2022 20:10:48 +0900 Masami Hiramatsu <[email protected]> wrote:

Add a KUnit based selftest for fprobe interface.

---
Changes in v9:
- Rename fprobe_target* to fprobe_selftest_target*.
- Find the correct expected ip by ftrace_location_range().
- Since the ftrace_location_range() is not exposed to module, make this test only for embedded.
- Add entry only test.
- Reset the fprobe structure before reuse it.

---
 lib/Kconfig.debug |   12 ++++
 lib/Makefile      |    2 +
 lib/test_fprobe.c |  174 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+)
 create mode 100644 lib/test_fprobe.c
lib/Kconfig.debug +0 -0
index 14b89aa37c5c..ffc469a12afc 100644
@@ -2100,6 +2100,18 @@ config KPROBES_SANITY_TEST
 
        Say N if you are unsure.
 
Quoted 8 lines

lib/Kconfig.debug config KPROBES_SANITY_TEST · line 2107

Quoted 5 lines
lib/Kconfig.debug +5 -0
+config FPROBE_SANITY_TEST
+     bool "Self test for fprobe"
+     depends on DEBUG_KERNEL
+     depends on FPROBE
+     depends on KUNIT

Hmm, this caused a build error with allmodconfig because KUNIT=m but FPROBE_SANITY_TEST=y. Let me fix this issue.

Quoted 1 line

Please base on top of bpf-next and add [PATCH v11 bpf-next] to subject.

OK, let me rebase on it.
There are master and for-next branch, which one is better to use?

Thank you,

Quoted 212 lines
Quoted 212 lines

Thank you,

Quoted 194 lines
lib/Kconfig.debug +7 -0
+     help
+       This option will enable testing the fprobe when the system boot.
+       A series of tests are made to verify that the fprobe is functioning
+       properly.
+
+       Say N if you are unsure.
+
 config BACKTRACE_SELF_TEST
      tristate "Self test for the backtrace code"
      depends on DEBUG_KERNEL
lib/Makefile +2 -0
index 300f569c626b..154008764b16 100644
@@ -103,6 +103,8 @@ obj-$(CONFIG_TEST_HMM) += test_hmm.o
 obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o
 obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
 obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o
+CFLAGS_test_fprobe.o += $(CC_FLAGS_FTRACE)
+obj-$(CONFIG_FPROBE_SANITY_TEST) += test_fprobe.o
 #
 # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
 # off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
lib/test_fprobe.c new file +164 -0
new file mode 100644
index 000000000000..ed70637a2ffa
@@ -0,0 +1,174 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * test_fprobe.c - simple sanity test for fprobe
+ */
+
+#include <linux/kernel.h>
+#include <linux/fprobe.h>
+#include <linux/random.h>
+#include <kunit/test.h>
+
+#define div_factor 3
+
+static struct kunit *current_test;
+
+static u32 rand1, entry_val, exit_val;
+
+/* Use indirect calls to avoid inlining the target functions */
+static u32 (*target)(u32 value);
+static u32 (*target2)(u32 value);
+static unsigned long target_ip;
+static unsigned long target2_ip;
+
+static noinline u32 fprobe_selftest_target(u32 value)
+{
+     return (value / div_factor);
+}
+
+static noinline u32 fprobe_selftest_target2(u32 value)
+{
+     return (value / div_factor) + 1;
+}
+
+static notrace void fp_entry_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     /* This can be called on the fprobe_selftest_target and the fprobe_selftest_target2 */
+     if (ip != target_ip)
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+     entry_val = (rand1 / div_factor);
+}
+
+static notrace void fp_exit_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     unsigned long ret = regs_return_value(regs);
+
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     if (ip != target_ip) {
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor) + 1);
+     } else
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor));
+     KUNIT_EXPECT_EQ(current_test, entry_val, (rand1 / div_factor));
+     exit_val = entry_val + div_factor;
+}
+
+/* Test entry only (no rethook) */
+static void test_fprobe_entry(struct kunit *test)
+{
+     struct fprobe fp_entry = {
+             .entry_handler = fp_entry_handler,
+     };
+
+     current_test = test;
+
+     /* Before register, unregister should be failed. */
+     KUNIT_EXPECT_NE(test, 0, unregister_fprobe(&fp_entry));
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp_entry, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp_entry));
+}
+
+static void test_fprobe(struct kunit *test)
+{
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static void test_fprobe_syms(struct kunit *test)
+{
+     static const char *syms[] = {"fprobe_selftest_target", "fprobe_selftest_target2"};
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe_syms(&fp, syms, 2));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static unsigned long get_ftrace_location(void *func)
+{
+     unsigned long size, addr = (unsigned long)func;
+
+     if (!kallsyms_lookup_size_offset(addr, &size, NULL) || !size)
+             return 0;
+
+     return ftrace_location_range(addr, addr + size - 1);
+}
+
+static int fprobe_test_init(struct kunit *test)
+{
+     do {
+             rand1 = prandom_u32();
+     } while (rand1 <= div_factor);
+
+     target = fprobe_selftest_target;
+     target2 = fprobe_selftest_target2;
+     target_ip = get_ftrace_location(target);
+     target2_ip = get_ftrace_location(target2);
+
+     return 0;
+}
+
+static struct kunit_case fprobe_testcases[] = {
+     KUNIT_CASE(test_fprobe_entry),
+     KUNIT_CASE(test_fprobe),
+     KUNIT_CASE(test_fprobe_syms),
+     {}
+};

lib/test_fprobe.c line 174

Quoted 10 lines
lib/test_fprobe.c new file +10 -0
+
+static struct kunit_suite fprobe_test_suite = {
+     .name = "fprobe_test",
+     .init = fprobe_test_init,
+     .test_cases = fprobe_testcases,
+};
+
+kunit_test_suites(&fprobe_test_suite);
+
+MODULE_LICENSE("GPL");

--
Masami Hiramatsu <[email protected]>

Signature · 1 line
Masami Hiramatsu <[email protected]>
#4 lore raw

Re: [PATCH v10 03/12] rethook: Add a generic return hook

In reply to Masami Hiramatsu

Hi,

Quoted 7 lines · Masami Hiramatsu

Add a return hook framework which hooks the function return. Most of the logic came from the kretprobe, but this is independent from kretprobe.

Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just an additional hook.

If my understanding of rethook is correct,  @regs for exit_handler is pt_regs when the traced function is returned instead of entered. Could we provide a similar functionality as fexit in bpf trampoline in which function parameters and return value are passed to exit_handler ? So for arch without bpf trampoline support there is another alternative.

Regards,
Tao

Quoted 539 lines

---
Changes in v10:
- Add @mcount parameter to rethook_hook() to identify the context.
Changes in v6:
- Fix typos.
- Use dereference_symbol_descriptor() to check the trampoline address.
- Shrink down the preempt-disabled section for recycling nodes.
- Reject stack searching if the task is not current and is running in rethook_find_ret_addr().
Changes in v4:
- Fix rethook_trampoline_handler() loops as same as
what currently kretprobe does. This will fix some stacktrace issue in the rethook handler.

---
 include/linux/rethook.h |  100 +++++++++++++++
 include/linux/sched.h   |    3 
 kernel/exit.c           |    2 
 kernel/fork.c           |    3 
 kernel/trace/Kconfig    |   11 ++
 kernel/trace/Makefile   |    1 
 kernel/trace/rethook.c  |  317 +++++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 437 insertions(+)
 create mode 100644 include/linux/rethook.h
 create mode 100644 kernel/trace/rethook.c
include/linux/rethook.h new file +100 -0
new file mode 100644
index 000000000000..c8ac1e5afcd1
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Return hooking with list-based shadow stack.
+ */
+#ifndef _LINUX_RETHOOK_H
+#define _LINUX_RETHOOK_H
+
+#include <linux/compiler.h>
+#include <linux/freelist.h>
+#include <linux/kallsyms.h>
+#include <linux/llist.h>
+#include <linux/rcupdate.h>
+#include <linux/refcount.h>
+
+struct rethook_node;
+
+typedef void (*rethook_handler_t) (struct rethook_node *, void *, struct pt_regs *);
+
+/**
+ * struct rethook - The rethook management data structure.
+ * @data: The user-defined data storage.
+ * @handler: The user-defined return hook handler.
+ * @pool: The pool of struct rethook_node.
+ * @ref: The reference counter.
+ * @rcu: The rcu_head for deferred freeing.
+ *
+ * Don't embed to another data structure, because this is a self-destructive
+ * data structure when all rethook_node are freed.
+ */
+struct rethook {
+	void			*data;
+	rethook_handler_t	handler;
+	struct freelist_head	pool;
+	refcount_t		ref;
+	struct rcu_head		rcu;
+};
+
+/**
+ * struct rethook_node - The rethook shadow-stack entry node.
+ * @freelist: The freelist, linked to struct rethook::pool.
+ * @rcu: The rcu_head for deferred freeing.
+ * @llist: The llist, linked to a struct task_struct::rethooks.
+ * @rethook: The pointer to the struct rethook.
+ * @ret_addr: The storage for the real return address.
+ * @frame: The storage for the frame pointer.
+ *
+ * You can embed this to your extended data structure to store any data
+ * on each entry of the shadow stack.
+ */
+struct rethook_node {
+	union {
+		struct freelist_node freelist;
+		struct rcu_head      rcu;
+	};
+	struct llist_node	llist;
+	struct rethook		*rethook;
+	unsigned long		ret_addr;
+	unsigned long		frame;
+};
+
+struct rethook *rethook_alloc(void *data, rethook_handler_t handler);
+void rethook_free(struct rethook *rh);
+void rethook_add_node(struct rethook *rh, struct rethook_node *node);
+struct rethook_node *rethook_try_get(struct rethook *rh);
+void rethook_recycle(struct rethook_node *node);
+void rethook_hook(struct rethook_node *node, struct pt_regs *regs, bool mcount);
+unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame,
+				    struct llist_node **cur);
+
+/* Arch dependent code must implement arch_* and trampoline code */
+void arch_rethook_prepare(struct rethook_node *node, struct pt_regs *regs, bool mcount);
+void arch_rethook_trampoline(void);
+
+/**
+ * is_rethook_trampoline() - Check whether the address is rethook trampoline
+ * @addr: The address to be checked
+ *
+ * Return true if the @addr is the rethook trampoline address.
+ */
+static inline bool is_rethook_trampoline(unsigned long addr)
+{
+	return addr == (unsigned long)dereference_symbol_descriptor(arch_rethook_trampoline);
+}
+
+/* If the architecture needs to fixup the return address, implement it. */
+void arch_rethook_fixup_return(struct pt_regs *regs,
+			       unsigned long correct_ret_addr);
+
+/* Generic trampoline handler, arch code must prepare asm stub */
+unsigned long rethook_trampoline_handler(struct pt_regs *regs,
+					 unsigned long frame);
+
+#ifdef CONFIG_RETHOOK
+void rethook_flush_task(struct task_struct *tk);
+#else
+#define rethook_flush_task(tsk)	do { } while (0)
+#endif
+
+#endif
+
include/linux/sched.h +3 -0
index 508b91d57470..efc0e2b6a772 100644
@@ -1485,6 +1485,9 @@ struct task_struct {
 #ifdef CONFIG_KRETPROBES
 	struct llist_head               kretprobe_instances;
 #endif
+#ifdef CONFIG_RETHOOK
+	struct llist_head               rethooks;
+#endif
 
 #ifdef CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH
 	/*
kernel/exit.c +2 -0
index b00a25bb4ab9..2d1803fa8fe6 100644
@@ -64,6 +64,7 @@
 #include <linux/compat.h>
 #include <linux/io_uring.h>
 #include <linux/kprobes.h>
+#include <linux/rethook.h>
 
 #include <linux/uaccess.h>
 #include <asm/unistd.h>
@@ -169,6 +170,7 @@ static void delayed_put_task_struct(struct rcu_head *rhp)
 	struct task_struct *tsk = container_of(rhp, struct task_struct, rcu);
 
 	kprobe_flush_task(tsk);
+	rethook_flush_task(tsk);
 	perf_event_delayed_put(tsk);
 	trace_sched_process_free(tsk);
 	put_task_struct(tsk);
kernel/fork.c +3 -0
index d75a528f7b21..0e29dd7b6cc2 100644
@@ -2255,6 +2255,9 @@ static __latent_entropy struct task_struct *copy_process(
 #ifdef CONFIG_KRETPROBES
 	p->kretprobe_instances.first = NULL;
 #endif
+#ifdef CONFIG_RETHOOK
+	p->rethooks.first = NULL;
+#endif
 
 	/*
 	 * Ensure that the cgroup subsystem policies allow the new process to be
kernel/trace/Kconfig +11 -0
index 7ce31abc542b..e75504e42ab8 100644
@@ -10,6 +10,17 @@ config USER_STACKTRACE_SUPPORT
 config NOP_TRACER
 	bool
 
+config HAVE_RETHOOK
+	bool
+
+config RETHOOK
+	bool
+	depends on HAVE_RETHOOK
+	help
+	  Enable generic return hooking feature. This is an internal
+	  API, which will be used by other function-entry hooking
+	  features like fprobe and kprobes.
+
 config HAVE_FUNCTION_TRACER
 	bool
 	help
kernel/trace/Makefile +1 -0
index 79255f9de9a4..c6f11a139eac 100644
@@ -98,6 +98,7 @@ obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o
 obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o
 obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
 obj-$(CONFIG_FPROBE) += fprobe.o
+obj-$(CONFIG_RETHOOK) += rethook.o
 
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 
kernel/trace/rethook.c new file +317 -0
new file mode 100644
index 000000000000..ab463a4d2b23
@@ -0,0 +1,317 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define pr_fmt(fmt) "rethook: " fmt
+
+#include <linux/bug.h>
+#include <linux/kallsyms.h>
+#include <linux/kprobes.h>
+#include <linux/preempt.h>
+#include <linux/rethook.h>
+#include <linux/slab.h>
+#include <linux/sort.h>
+
+/* Return hook list (shadow stack by list) */
+
+/*
+ * This function is called from delayed_put_task_struct() when a task is
+ * dead and cleaned up to recycle any kretprobe instances associated with
+ * this task. These left over instances represent probed functions that
+ * have been called but will never return.
+ */
+void rethook_flush_task(struct task_struct *tk)
+{
+	struct rethook_node *rhn;
+	struct llist_node *node;
+
+	node = __llist_del_all(&tk->rethooks);
+	while (node) {
+		rhn = container_of(node, struct rethook_node, llist);
+		node = node->next;
+		preempt_disable();
+		rethook_recycle(rhn);
+		preempt_enable();
+	}
+}
+
+static void rethook_free_rcu(struct rcu_head *head)
+{
+	struct rethook *rh = container_of(head, struct rethook, rcu);
+	struct rethook_node *rhn;
+	struct freelist_node *node;
+	int count = 1;
+
+	node = rh->pool.head;
+	while (node) {
+		rhn = container_of(node, struct rethook_node, freelist);
+		node = node->next;
+		kfree(rhn);
+		count++;
+	}
+
+	/* The rh->ref is the number of pooled node + 1 */
+	if (refcount_sub_and_test(count, &rh->ref))
+		kfree(rh);
+}
+
+/**
+ * rethook_free() - Free struct rethook.
+ * @rh: the struct rethook to be freed.
+ *
+ * Free the rethook. Before calling this function, user must ensure the
+ * @rh::data is cleaned if needed (or, the handler can access it after
+ * calling this function.) This function will set the @rh to be freed
+ * after all rethook_node are freed (not soon). And the caller must
+ * not touch @rh after calling this.
+ */
+void rethook_free(struct rethook *rh)
+{
+	rcu_assign_pointer(rh->handler, NULL);
+
+	call_rcu(&rh->rcu, rethook_free_rcu);
+}
+
+/**
+ * rethook_alloc() - Allocate struct rethook.
+ * @data: a data to pass the @handler when hooking the return.
+ * @handler: the return hook callback function.
+ *
+ * Allocate and initialize a new rethook with @data and @handler.
+ * Return NULL if memory allocation fails or @handler is NULL.
+ * Note that @handler == NULL means this rethook is going to be freed.
+ */
+struct rethook *rethook_alloc(void *data, rethook_handler_t handler)
+{
+	struct rethook *rh = kzalloc(sizeof(struct rethook), GFP_KERNEL);
+
+	if (!rh || !handler)
+		return NULL;
+
+	rh->data = data;
+	rh->handler = handler;
+	rh->pool.head = NULL;
+	refcount_set(&rh->ref, 1);
+
+	return rh;
+}
+
+/**
+ * rethook_add_node() - Add a new node to the rethook.
+ * @rh: the struct rethook.
+ * @node: the struct rethook_node to be added.
+ *
+ * Add @node to @rh. User must allocate @node (as a part of user's
+ * data structure.) The @node fields are initialized in this function.
+ */
+void rethook_add_node(struct rethook *rh, struct rethook_node *node)
+{
+	node->rethook = rh;
+	freelist_add(&node->freelist, &rh->pool);
+	refcount_inc(&rh->ref);
+}
+
+static void free_rethook_node_rcu(struct rcu_head *head)
+{
+	struct rethook_node *node = container_of(head, struct rethook_node, rcu);
+
+	if (refcount_dec_and_test(&node->rethook->ref))
+		kfree(node->rethook);
+	kfree(node);
+}
+
+/**
+ * rethook_recycle() - return the node to rethook.
+ * @node: The struct rethook_node to be returned.
+ *
+ * Return back the @node to @node::rethook. If the @node::rethook is already
+ * marked as freed, this will free the @node.
+ */
+void rethook_recycle(struct rethook_node *node)
+{
+	lockdep_assert_preemption_disabled();
+
+	if (likely(READ_ONCE(node->rethook->handler)))
+		freelist_add(&node->freelist, &node->rethook->pool);
+	else
+		call_rcu(&node->rcu, free_rethook_node_rcu);
+}
+NOKPROBE_SYMBOL(rethook_recycle);
+
+/**
+ * rethook_try_get() - get an unused rethook node.
+ * @rh: The struct rethook which pools the nodes.
+ *
+ * Get an unused rethook node from @rh. If the node pool is empty, this
+ * will return NULL. Caller must disable preemption.
+ */
+struct rethook_node *rethook_try_get(struct rethook *rh)
+{
+	rethook_handler_t handler = READ_ONCE(rh->handler);
+	struct freelist_node *fn;
+
+	lockdep_assert_preemption_disabled();
+
+	/* Check whether @rh is going to be freed. */
+	if (unlikely(!handler))
+		return NULL;
+
+	fn = freelist_try_get(&rh->pool);
+	if (!fn)
+		return NULL;
+
+	return container_of(fn, struct rethook_node, freelist);
+}
+NOKPROBE_SYMBOL(rethook_try_get);
+
+/**
+ * rethook_hook() - Hook the current function return.
+ * @node: The struct rethook node to hook the function return.
+ * @regs: The struct pt_regs for the function entry.
+ * @mcount: True if this is called from mcount(ftrace) context.
+ *
+ * Hook the current running function return. This must be called when the
+ * function entry (or at least @regs must be the registers of the function
+ * entry.) @mcount is used for identifying the context. If this is called
+ * from ftrace (mcount) callback, @mcount must be set true. If this is called
+ * from the real function entry (e.g. kprobes) @mcount must be set false.
+ * This is because the way to hook the function return depends on the context.
+ */
+void rethook_hook(struct rethook_node *node, struct pt_regs *regs, bool mcount)
+{
+	arch_rethook_prepare(node, regs, mcount);
+	__llist_add(&node->llist, &current->rethooks);
+}
+NOKPROBE_SYMBOL(rethook_hook);
+
+/* This assumes the 'tsk' is the current task or is not running. */
+static unsigned long __rethook_find_ret_addr(struct task_struct *tsk,
+					     struct llist_node **cur)
+{
+	struct rethook_node *rh = NULL;
+	struct llist_node *node = *cur;
+
+	if (!node)
+		node = tsk->rethooks.first;
+	else
+		node = node->next;
+
+	while (node) {
+		rh = container_of(node, struct rethook_node, llist);
+		if (rh->ret_addr != (unsigned long)arch_rethook_trampoline) {
+			*cur = node;
+			return rh->ret_addr;
+		}
+		node = node->next;
+	}
+	return 0;
+}
+NOKPROBE_SYMBOL(__rethook_find_ret_addr);
+
+/**
+ * rethook_find_ret_addr -- Find correct return address modified by rethook
+ * @tsk: Target task
+ * @frame: A frame pointer
+ * @cur: a storage of the loop cursor llist_node pointer for next call
+ *
+ * Find the correct return address modified by a rethook on @tsk in unsigned
+ * long type.
+ * The @tsk must be 'current' or a task which is not running. @frame is a hint
+ * to get the currect return address - which is compared with the
+ * rethook::frame field. The @cur is a loop cursor for searching the
+ * kretprobe return addresses on the @tsk. The '*@cur' should be NULL at the
+ * first call, but '@cur' itself must NOT NULL.
+ *
+ * Returns found address value or zero if not found.
+ */
+unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame,
+				    struct llist_node **cur)
+{
+	struct rethook_node *rhn = NULL;
+	unsigned long ret;
+
+	if (WARN_ON_ONCE(!cur))
+		return 0;
+
+	if (WARN_ON_ONCE(tsk != current && task_is_running(tsk)))
+		return 0;
+
+	do {
+		ret = __rethook_find_ret_addr(tsk, cur);
+		if (!ret)
+			break;
+		rhn = container_of(*cur, struct rethook_node, llist);
+	} while (rhn->frame != frame);
+
+	return ret;
+}
+NOKPROBE_SYMBOL(rethook_find_ret_addr);
+
+void __weak arch_rethook_fixup_return(struct pt_regs *regs,
+				      unsigned long correct_ret_addr)
+{
+	/*
+	 * Do nothing by default. If the architecture which uses a
+	 * frame pointer to record real return address on the stack,
+	 * it should fill this function to fixup the return address
+	 * so that stacktrace works from the rethook handler.
+	 */
+}
+
+/* This function will be called from each arch-defined trampoline. */
+unsigned long rethook_trampoline_handler(struct pt_regs *regs,
+					 unsigned long frame)
+{
+	struct llist_node *first, *node = NULL;
+	unsigned long correct_ret_addr;
+	rethook_handler_t handler;
+	struct rethook_node *rhn;
+
+	correct_ret_addr = __rethook_find_ret_addr(current, &node);
+	if (!correct_ret_addr) {
+		pr_err("rethook: Return address not found! Maybe there is a bug in the kernel\n");
+		BUG_ON(1);
+	}
+
+	instruction_pointer_set(regs, correct_ret_addr);
+
+	/*
+	 * These loops must be protected from rethook_free_rcu() because those
+	 * are accessing 'rhn->rethook'.
+	 */
+	preempt_disable();
+
+	/*
+	 * Run the handler on the shadow stack. Do not unlink the list here because
+	 * stackdump inside the handlers needs to decode it.
+	 */
+	first = current->rethooks.first;
+	while (first) {
+		rhn = container_of(first, struct rethook_node, llist);
+		if (WARN_ON_ONCE(rhn->frame != frame))
+			break;
+		handler = READ_ONCE(rhn->rethook->handler);
+		if (handler)
+			handler(rhn, rhn->rethook->data, regs);
+
+		if (first == node)
+			break;
+		first = first->next;
+	}
+
+	/* Fixup registers for returning to correct address. */
+	arch_rethook_fixup_return(regs, correct_ret_addr);
+
+	/* Unlink used shadow stack */
+	first = current->rethooks.first;
+	current->rethooks.first = node->next;
+	node->next = NULL;
+
+	while (first) {
+		rhn = container_of(first, struct rethook_node, llist);
+		first = first->next;
+		rethook_recycle(rhn);
+	}
+	preempt_enable();
+
+	return correct_ret_addr;
+}
+NOKPROBE_SYMBOL(rethook_trampoline_handler);

.

#17 lore raw

Re: [PATCH v10 12/12] fprobe: Add a selftest for fprobe

In reply to Masami Hiramatsu
Quoted 46 lines · Masami Hiramatsu
Quoted 42 lines · On Wed, 9 Mar 2022 16:40:00 -0800 Andrii Nakryiko <[email protected]>…
Quoted 40 lines · Masami Hiramatsu

Hi,

Quoted 23 lines · On Tue, 8 Mar 2022 20:10:48 +0900 Masami Hiramatsu <[email protected]> wrote:

Add a KUnit based selftest for fprobe interface.

---
Changes in v9:
- Rename fprobe_target* to fprobe_selftest_target*.
- Find the correct expected ip by ftrace_location_range().
- Since the ftrace_location_range() is not exposed to module, make this test only for embedded.
- Add entry only test.
- Reset the fprobe structure before reuse it.

---
 lib/Kconfig.debug |   12 ++++
 lib/Makefile      |    2 +
 lib/test_fprobe.c |  174 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+)
 create mode 100644 lib/test_fprobe.c
lib/Kconfig.debug +0 -0
index 14b89aa37c5c..ffc469a12afc 100644

lib/Kconfig.debug config KPROBES_SANITY_TEST · line 2107

Quoted 8 lines
lib/Kconfig.debug +5 -0
@@ -2100,6 +2100,18 @@ config KPROBES_SANITY_TEST
 
        Say N if you are unsure.
 
+config FPROBE_SANITY_TEST
+     bool "Self test for fprobe"
+     depends on DEBUG_KERNEL
+     depends on FPROBE
+     depends on KUNIT

Hmm, this caused a build error with allmodconfig because KUNIT=m but FPROBE_SANITY_TEST=y. Let me fix this issue.

Quoted 5 lines
Quoted 1 line

Please base on top of bpf-next and add [PATCH v11 bpf-next] to subject.

OK, let me rebase on it.
There are master and for-next branch, which one is better to use?

Sorry, missed your reply earlier. Always rebase against master.

You forgot to add "bpf-next" into [PATCH] prefix, so I had to manually mark it in patchworks as delegated to bpf queue (this is necessary for our CI to properly pick it up). For future submissions to bpf-next, please don't forget to add "bpf-next" marker.

Quoted 218 lines

Thank you,

Quoted 204 lines
Quoted 204 lines

Thank you,

Quoted 200 lines
lib/Kconfig.debug +7 -0
+     help
+       This option will enable testing the fprobe when the system boot.
+       A series of tests are made to verify that the fprobe is functioning
+       properly.
+
+       Say N if you are unsure.
+
 config BACKTRACE_SELF_TEST
      tristate "Self test for the backtrace code"
      depends on DEBUG_KERNEL
lib/Makefile +2 -0
index 300f569c626b..154008764b16 100644
@@ -103,6 +103,8 @@ obj-$(CONFIG_TEST_HMM) += test_hmm.o
 obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o
 obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
 obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o
+CFLAGS_test_fprobe.o += $(CC_FLAGS_FTRACE)
+obj-$(CONFIG_FPROBE_SANITY_TEST) += test_fprobe.o
 #
 # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
 # off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
lib/test_fprobe.c new file +171 -0
new file mode 100644
index 000000000000..ed70637a2ffa
@@ -0,0 +1,174 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * test_fprobe.c - simple sanity test for fprobe
+ */
+
+#include <linux/kernel.h>
+#include <linux/fprobe.h>
+#include <linux/random.h>
+#include <kunit/test.h>
+
+#define div_factor 3
+
+static struct kunit *current_test;
+
+static u32 rand1, entry_val, exit_val;
+
+/* Use indirect calls to avoid inlining the target functions */
+static u32 (*target)(u32 value);
+static u32 (*target2)(u32 value);
+static unsigned long target_ip;
+static unsigned long target2_ip;
+
+static noinline u32 fprobe_selftest_target(u32 value)
+{
+     return (value / div_factor);
+}
+
+static noinline u32 fprobe_selftest_target2(u32 value)
+{
+     return (value / div_factor) + 1;
+}
+
+static notrace void fp_entry_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     /* This can be called on the fprobe_selftest_target and the fprobe_selftest_target2 */
+     if (ip != target_ip)
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+     entry_val = (rand1 / div_factor);
+}
+
+static notrace void fp_exit_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     unsigned long ret = regs_return_value(regs);
+
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     if (ip != target_ip) {
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor) + 1);
+     } else
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor));
+     KUNIT_EXPECT_EQ(current_test, entry_val, (rand1 / div_factor));
+     exit_val = entry_val + div_factor;
+}
+
+/* Test entry only (no rethook) */
+static void test_fprobe_entry(struct kunit *test)
+{
+     struct fprobe fp_entry = {
+             .entry_handler = fp_entry_handler,
+     };
+
+     current_test = test;
+
+     /* Before register, unregister should be failed. */
+     KUNIT_EXPECT_NE(test, 0, unregister_fprobe(&fp_entry));
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp_entry, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp_entry));
+}
+
+static void test_fprobe(struct kunit *test)
+{
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static void test_fprobe_syms(struct kunit *test)
+{
+     static const char *syms[] = {"fprobe_selftest_target", "fprobe_selftest_target2"};
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe_syms(&fp, syms, 2));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static unsigned long get_ftrace_location(void *func)
+{
+     unsigned long size, addr = (unsigned long)func;
+
+     if (!kallsyms_lookup_size_offset(addr, &size, NULL) || !size)
+             return 0;
+
+     return ftrace_location_range(addr, addr + size - 1);
+}
+
+static int fprobe_test_init(struct kunit *test)
+{
+     do {
+             rand1 = prandom_u32();
+     } while (rand1 <= div_factor);
+
+     target = fprobe_selftest_target;
+     target2 = fprobe_selftest_target2;
+     target_ip = get_ftrace_location(target);
+     target2_ip = get_ftrace_location(target2);
+
+     return 0;
+}
+
+static struct kunit_case fprobe_testcases[] = {
+     KUNIT_CASE(test_fprobe_entry),
+     KUNIT_CASE(test_fprobe),
+     KUNIT_CASE(test_fprobe_syms),
+     {}
+};
+
+static struct kunit_suite fprobe_test_suite = {
+     .name = "fprobe_test",
+     .init = fprobe_test_init,
+     .test_cases = fprobe_testcases,
+};
+
Quoted 8 lines
Quoted 8 lines

lib/test_fprobe.c line 174

Quoted 4 lines
lib/test_fprobe.c new file +3 -0
+kunit_test_suites(&fprobe_test_suite);
+
+MODULE_LICENSE("GPL");

--
Masami Hiramatsu <[email protected]>

--
Masami Hiramatsu <[email protected]>

author maintainer
#18 lore raw

Re: [PATCH v10 12/12] fprobe: Add a selftest for fprobe

Quoted 53 lines · On Fri, 11 Mar 2022 10:35:30 -0800 Andrii Nakryiko <[email protected]>…
Quoted 46 lines · Masami Hiramatsu
Quoted 42 lines · On Wed, 9 Mar 2022 16:40:00 -0800 Andrii Nakryiko <[email protected]>…
Quoted 40 lines · Masami Hiramatsu

Hi,

Quoted 23 lines · On Tue, 8 Mar 2022 20:10:48 +0900 Masami Hiramatsu <[email protected]> wrote:

Add a KUnit based selftest for fprobe interface.

---
Changes in v9:
- Rename fprobe_target* to fprobe_selftest_target*.
- Find the correct expected ip by ftrace_location_range().
- Since the ftrace_location_range() is not exposed to module, make this test only for embedded.
- Add entry only test.
- Reset the fprobe structure before reuse it.

---
 lib/Kconfig.debug |   12 ++++
 lib/Makefile      |    2 +
 lib/test_fprobe.c |  174 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+)
 create mode 100644 lib/test_fprobe.c
lib/Kconfig.debug +0 -0
index 14b89aa37c5c..ffc469a12afc 100644

lib/Kconfig.debug config KPROBES_SANITY_TEST · line 2107

Quoted 8 lines
lib/Kconfig.debug +5 -0
@@ -2100,6 +2100,18 @@ config KPROBES_SANITY_TEST
 
        Say N if you are unsure.
 
+config FPROBE_SANITY_TEST
+     bool "Self test for fprobe"
+     depends on DEBUG_KERNEL
+     depends on FPROBE
+     depends on KUNIT

Hmm, this caused a build error with allmodconfig because KUNIT=m but FPROBE_SANITY_TEST=y. Let me fix this issue.

Quoted 5 lines
Quoted 1 line

Please base on top of bpf-next and add [PATCH v11 bpf-next] to subject.

OK, let me rebase on it.
There are master and for-next branch, which one is better to use?

Quoted 7 lines

Sorry, missed your reply earlier. Always rebase against master.

You forgot to add "bpf-next" into [PATCH] prefix, so I had to manually mark it in patchworks as delegated to bpf queue (this is necessary for our CI to properly pick it up). For future submissions to bpf-next, please don't forget to add "bpf-next" marker.

Oops, sorry, I forgot that. Should I resend with that tag?

Thank you,

Quoted 218 lines
Quoted 218 lines

Thank you,

Quoted 204 lines
Quoted 204 lines

Thank you,

Quoted 200 lines
lib/Kconfig.debug +7 -0
+     help
+       This option will enable testing the fprobe when the system boot.
+       A series of tests are made to verify that the fprobe is functioning
+       properly.
+
+       Say N if you are unsure.
+
 config BACKTRACE_SELF_TEST
      tristate "Self test for the backtrace code"
      depends on DEBUG_KERNEL
lib/Makefile +2 -0
index 300f569c626b..154008764b16 100644
@@ -103,6 +103,8 @@ obj-$(CONFIG_TEST_HMM) += test_hmm.o
 obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o
 obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
 obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o
+CFLAGS_test_fprobe.o += $(CC_FLAGS_FTRACE)
+obj-$(CONFIG_FPROBE_SANITY_TEST) += test_fprobe.o
 #
 # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
 # off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
lib/test_fprobe.c new file +171 -0
new file mode 100644
index 000000000000..ed70637a2ffa
@@ -0,0 +1,174 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * test_fprobe.c - simple sanity test for fprobe
+ */
+
+#include <linux/kernel.h>
+#include <linux/fprobe.h>
+#include <linux/random.h>
+#include <kunit/test.h>
+
+#define div_factor 3
+
+static struct kunit *current_test;
+
+static u32 rand1, entry_val, exit_val;
+
+/* Use indirect calls to avoid inlining the target functions */
+static u32 (*target)(u32 value);
+static u32 (*target2)(u32 value);
+static unsigned long target_ip;
+static unsigned long target2_ip;
+
+static noinline u32 fprobe_selftest_target(u32 value)
+{
+     return (value / div_factor);
+}
+
+static noinline u32 fprobe_selftest_target2(u32 value)
+{
+     return (value / div_factor) + 1;
+}
+
+static notrace void fp_entry_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     /* This can be called on the fprobe_selftest_target and the fprobe_selftest_target2 */
+     if (ip != target_ip)
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+     entry_val = (rand1 / div_factor);
+}
+
+static notrace void fp_exit_handler(struct fprobe *fp, unsigned long ip, struct pt_regs *regs)
+{
+     unsigned long ret = regs_return_value(regs);
+
+     KUNIT_EXPECT_FALSE(current_test, preemptible());
+     if (ip != target_ip) {
+             KUNIT_EXPECT_EQ(current_test, ip, target2_ip);
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor) + 1);
+     } else
+             KUNIT_EXPECT_EQ(current_test, ret, (rand1 / div_factor));
+     KUNIT_EXPECT_EQ(current_test, entry_val, (rand1 / div_factor));
+     exit_val = entry_val + div_factor;
+}
+
+/* Test entry only (no rethook) */
+static void test_fprobe_entry(struct kunit *test)
+{
+     struct fprobe fp_entry = {
+             .entry_handler = fp_entry_handler,
+     };
+
+     current_test = test;
+
+     /* Before register, unregister should be failed. */
+     KUNIT_EXPECT_NE(test, 0, unregister_fprobe(&fp_entry));
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp_entry, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, 0, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp_entry));
+}
+
+static void test_fprobe(struct kunit *test)
+{
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe(&fp, "fprobe_selftest_target*", NULL));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static void test_fprobe_syms(struct kunit *test)
+{
+     static const char *syms[] = {"fprobe_selftest_target", "fprobe_selftest_target2"};
+     struct fprobe fp = {
+             .entry_handler = fp_entry_handler,
+             .exit_handler = fp_exit_handler,
+     };
+
+     current_test = test;
+     KUNIT_EXPECT_EQ(test, 0, register_fprobe_syms(&fp, syms, 2));
+
+     entry_val = 0;
+     exit_val = 0;
+     target(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     entry_val = 0;
+     exit_val = 0;
+     target2(rand1);
+     KUNIT_EXPECT_NE(test, 0, entry_val);
+     KUNIT_EXPECT_EQ(test, entry_val + div_factor, exit_val);
+
+     KUNIT_EXPECT_EQ(test, 0, unregister_fprobe(&fp));
+}
+
+static unsigned long get_ftrace_location(void *func)
+{
+     unsigned long size, addr = (unsigned long)func;
+
+     if (!kallsyms_lookup_size_offset(addr, &size, NULL) || !size)
+             return 0;
+
+     return ftrace_location_range(addr, addr + size - 1);
+}
+
+static int fprobe_test_init(struct kunit *test)
+{
+     do {
+             rand1 = prandom_u32();
+     } while (rand1 <= div_factor);
+
+     target = fprobe_selftest_target;
+     target2 = fprobe_selftest_target2;
+     target_ip = get_ftrace_location(target);
+     target2_ip = get_ftrace_location(target2);
+
+     return 0;
+}
+
+static struct kunit_case fprobe_testcases[] = {
+     KUNIT_CASE(test_fprobe_entry),
+     KUNIT_CASE(test_fprobe),
+     KUNIT_CASE(test_fprobe_syms),
+     {}
+};
+
+static struct kunit_suite fprobe_test_suite = {
+     .name = "fprobe_test",
+     .init = fprobe_test_init,
+     .test_cases = fprobe_testcases,
+};
+
Quoted 8 lines
Quoted 8 lines

lib/test_fprobe.c line 174

Quoted 4 lines
lib/test_fprobe.c new file +3 -0
+kunit_test_suites(&fprobe_test_suite);
+
+MODULE_LICENSE("GPL");

--
Masami Hiramatsu <[email protected]>

--
Masami Hiramatsu <[email protected]>

Signature · 1 line
Masami Hiramatsu <[email protected]>