tracing: Updates for 5.18
Subsystems (13)
Steven Rostedt
maintainercover tracing: Updates for 5.18
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
for-next-core
Head SHA1: 03cd03b5c3f9836e625a99fa603b4683dc1ebb47
Beau Belgrave (1):
user_events: Add trace event call as root for low permission cases
Jiri Olsa (1):
ftrace: Add ftrace_set_filter_ips function
Masami Hiramatsu (11):
fprobe: Add ftrace based probe APIs
rethook: Add a generic return hook
rethook: x86: Add rethook x86 implementation
arm64: rethook: Add arm64 rethook implementation
powerpc: Add rethook support
ARM: rethook: Add rethook arm implementation
fprobe: Add exit_handler support
fprobe: Add sample program for fprobe
fprobe: Introduce FPROBE_FL_KPROBE_SHARED flag for fprobe
docs: fprobe: Add fprobe description to ftrace-use.rst
fprobe: Add a selftest for fprobe
----
Documentation/trace/fprobe.rst | 173 ++++++++++++++ Documentation/trace/index.rst | 1 + arch/arm/Kconfig | 1 + arch/arm/include/asm/stacktrace.h | 4 +- arch/arm/kernel/stacktrace.c | 6 + arch/arm/probes/Makefile | 1 + arch/arm/probes/rethook.c | 103 ++++++++ arch/arm64/Kconfig | 1 + arch/arm64/include/asm/stacktrace.h | 2 +- arch/arm64/kernel/probes/Makefile | 1 + arch/arm64/kernel/probes/rethook.c | 25 ++ arch/arm64/kernel/probes/rethook_trampoline.S | 87 +++++++ arch/arm64/kernel/stacktrace.c | 7 +- arch/powerpc/Kconfig | 1 + arch/powerpc/kernel/Makefile | 1 + arch/powerpc/kernel/rethook.c | 72 ++++++ arch/x86/Kconfig | 1 + arch/x86/include/asm/unwind.h | 8 +- arch/x86/kernel/Makefile | 1 + arch/x86/kernel/kprobes/common.h | 1 + arch/x86/kernel/rethook.c | 119 +++++++++ include/linux/fprobe.h | 105 ++++++++ include/linux/ftrace.h | 3 + include/linux/kprobes.h | 3 + include/linux/rethook.h | 99 ++++++++ include/linux/sched.h | 3 + kernel/exit.c | 2 + kernel/fork.c | 3 + kernel/trace/Kconfig | 26 ++ kernel/trace/Makefile | 2 + kernel/trace/fprobe.c | 332 ++++++++++++++++++++++++++ kernel/trace/ftrace.c | 58 ++++- kernel/trace/rethook.c | 317 ++++++++++++++++++++++++ kernel/trace/trace_events_user.c | 39 ++- lib/Kconfig.debug | 12 + lib/Makefile | 2 + lib/test_fprobe.c | 174 ++++++++++++++ samples/Kconfig | 7 + samples/Makefile | 1 + samples/fprobe/Makefile | 3 + samples/fprobe/fprobe_example.c | 120 ++++++++++ 41 files changed, 1911 insertions(+), 16 deletions(-) create mode 100644 Documentation/trace/fprobe.rst create mode 100644 arch/arm/probes/rethook.c create mode 100644 arch/arm64/kernel/probes/rethook.c create mode 100644 arch/arm64/kernel/probes/rethook_trampoline.S create mode 100644 arch/powerpc/kernel/rethook.c create mode 100644 arch/x86/kernel/rethook.c create mode 100644 include/linux/fprobe.h create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/fprobe.c create mode 100644 kernel/trace/rethook.c create mode 100644 lib/test_fprobe.c create mode 100644 samples/fprobe/Makefile create mode 100644 samples/fprobe/fprobe_example.c
Steven Rostedt
maintainerPATCH 1/13 user_events: Add trace event call as root for low permission cases
From: Beau Belgrave <[email protected]>
Tracefs by default is locked down heavily. System operators can open up some files, such as user_events to a broader set of users. These users do not have access within tracefs beyond just the user_event files. Due to this restriction the trace_add_event_call/remove calls will silently fail since the caller does not have permissions to create directories.
To fix this trace_add_event_call/remove calls will be issued with override creds of the global root UID. Creds are reverted immediately afterward.
--- kernel/trace/trace_events_user.c | 39 ++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-)
kernel/trace/trace_events_user.c +37 -2
@@ -562,6 +562,41 @@ static struct trace_event_functions user_event_funcs = { .trace = user_event_print_trace, }; +static int user_event_set_call_visible (struct user_event *user,bool visible) +{ +int ret; +const struct cred *old_cred; +struct cred *cred; + + cred =prepare_creds (); + +if (!cred) +return -ENOMEM ; + +/* +* While by default tracefs is locked down, systems can be configured +* to allow user_event files to be less locked down. The extreme case +* being "other" has read/write access to user_events_data/status. +* +* When not locked down, processes may not have have permissions to +* add/remove calls themselves to tracefs. We need to temporarily +* switch to root file permission to allow for this scenario. +*/ + cred->fsuid =GLOBAL_ROOT_UID ; + + old_cred =override_creds (cred); + +if (visible) + ret =trace_add_event_call (&user->call); +else + ret =trace_remove_event_call (&user->call); + +revert_creds (old_cred); +put_cred (cred); + +return ret; +} +static int destroy_user_event (struct user_event *user) {int ret =0 ;
@@ -569,7 +604,7 @@ static int destroy_user_event(struct user_event *user)/* Must destroy fields before call removal */ user_event_destroy_fields (user); - ret =trace_remove_event_call (&user->call); + ret =user_event_set_call_visible (user, false);if (ret)return ret;
@@ -1049,7 +1084,7 @@ static int user_event_trace_register(struct user_event *user)if (!ret)return -ENODEV ; - ret =trace_add_event_call (&user->call); + ret =user_event_set_call_visible (user, true);if (ret)unregister_trace_event (&user->call.event);
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 2/13 ftrace: Add ftrace_set_filter_ips function
In reply to Steven RostedtFrom: Jiri Olsa <[email protected]>
Adding ftrace_set_filter_ips function to be able to set filter on multiple ip addresses at once.
With the kprobe multi attach interface we have cases where we need to initialize ftrace_ops object with thousands of functions, so having single function diving into ftrace_hash_move_and_update_ops with ftrace_lock is faster.
The functions ips are passed as unsigned long array with count.
--- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 58 +++++++++++++++++++++++++++++++++++------- 2 files changed, 52 insertions(+), 9 deletions(-)
include/linux/ftrace.h +3 -0
@@ -521,6 +521,8 @@ struct dyn_ftrace {int ftrace_set_filter_ip (struct ftrace_ops *ops,unsigned long ip,int remove,int reset); +int ftrace_set_filter_ips (struct ftrace_ops *ops,unsigned long *ips, +unsigned int cnt,int remove,int reset);int ftrace_set_filter (struct ftrace_ops *ops,unsigned char *buf,int len,int reset); int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
@@ -811,6 +813,7 @@ static inline unsigned long ftrace_location(unsigned long ip)#define ftrace_regex_open (ops, flag, inod, file) ({ -ENODEV; })#define ftrace_set_early_filter (ops, buf, enable) do { } while (0)#define ftrace_set_filter_ip (ops, ip, remove, reset) ({ -ENODEV; }) +#define ftrace_set_filter_ips (ops, ips, cnt, remove, reset) ({ -ENODEV; })#define ftrace_set_filter (ops, buf, len, reset) ({ -ENODEV; })#define ftrace_set_notrace (ops, buf, len, reset) ({ -ENODEV; })#define ftrace_free_filter (ops) do { } while (0)
kernel/trace/ftrace.c +49 -9
100 lines not on this page · raw message
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 3/13 fprobe: Add ftrace based probe APIs
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
The fprobe is a wrapper API for ftrace function tracer. Unlike kprobes, this probes only supports the function entry, but this can probe multiple functions by one fprobe. The usage is similar, user will set their callback to fprobe::entry_handler and call register_fprobe*() with probed functions.
There are 3 registration interfaces,
- register_fprobe() takes filtering patterns of the functin names.
- register_fprobe_ips() takes an array of ftrace-location addresses.
- register_fprobe_syms() takes an array of function names.
The registered fprobes can be unregistered with unregister_fprobe(). e.g.
struct fprobe fp = { .entry_handler = user_handler }; const char *targets[] = { "func1", "func2", "func3"};
...
ret = register_fprobe_syms(&fp, targets, ARRAY_SIZE(targets));
...
unregister_fprobe(&fp);
--- include/linux/fprobe.h | 87 +++++++++++++++++ kernel/trace/Kconfig | 12 +++ kernel/trace/Makefile | 1 + kernel/trace/fprobe.c | 211 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 311 insertions(+) create mode 100644 include/linux/fprobe.h create mode 100644 kernel/trace/fprobe.c
include/linux/fprobe.h new file +87 -0
@@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Simple ftrace probe wrapper */ +#ifndef _LINUX_FPROBE_H +#define _LINUX_FPROBE_H + +#include <linux/compiler.h> +#include <linux/ftrace.h> + +/** +* struct fprobe - ftrace based probe. +* @ops: The ftrace_ops. +* @nmissed: The counter for missing events. +* @flags: The status flag. +* @entry_handler: The callback function for function entry. +*/ +struct fprobe { +#ifdef CONFIG_FUNCTION_TRACER +/* +* If CONFIG_FUNCTION_TRACER is not set, CONFIG_FPROBE is disabled too. +* But user of fprobe may keep embedding the struct fprobe on their own +* code. To avoid build error, this will keep the fprobe data structure +* defined here, but remove ftrace_ops data structure. +*/ +struct ftrace_ops ops; +#endif +unsigned long nmissed; +unsigned int flags; +void (*entry_handler)(struct fprobe *fp,unsigned long entry_ip,struct pt_regs *regs); +}; + +#define FPROBE_FL_DISABLED 1 + +static inline bool fprobe_disabled (struct fprobe *fp) +{ +return (fp) ? fp->flags &FPROBE_FL_DISABLED : false; +} + +#ifdef CONFIG_FPROBE +int register_fprobe (struct fprobe *fp,const char *filter,const char *notfilter); +int register_fprobe_ips (struct fprobe *fp,unsigned long *addrs,int num); +int register_fprobe_syms (struct fprobe *fp,const char **syms,int num); +int unregister_fprobe (struct fprobe *fp); +#else +static inline int register_fprobe (struct fprobe *fp,const char *filter,const char *notfilter) +{ +return -EOPNOTSUPP ; +} +static inline int register_fprobe_ips (struct fprobe *fp,unsigned long *addrs,int num) +{ +return -EOPNOTSUPP ; +} +static inline int register_fprobe_syms (struct fprobe *fp,const char **syms,int num) +{ +return -EOPNOTSUPP ; +} +static inline int unregister_fprobe (struct fprobe *fp) +{ +return -EOPNOTSUPP ; +} +#endif + +/** +* disable_fprobe() - Disable fprobe +* @fp: The fprobe to be disabled. +* +* This will soft-disable @fp. Note that this doesn't remove the ftrace +* hooks from the function entry. +*/ +static inline void disable_fprobe (struct fprobe *fp) +{ +if (fp) + fp->flags |=FPROBE_FL_DISABLED ; +} + +/** +* enable_fprobe() - Enable fprobe +* @fp: The fprobe to be enabled. +* +* This will soft-enable @fp. +*/ +static inline void enable_fprobe (struct fprobe *fp) +{ +if (fp) + fp->flags &= ~FPROBE_FL_DISABLED ; +} + +#endif
kernel/trace/Kconfig +12 -0
@@ -236,6 +236,18 @@ config DYNAMIC_FTRACE_WITH_ARGS depends on DYNAMIC_FTRACE depends on HAVE_DYNAMIC_FTRACE_WITH_ARGS +config FPROBE + bool "Kernel Function Probe (fprobe)" + depends on FUNCTION_TRACER + depends on DYNAMIC_FTRACE_WITH_REGS + default n + help + This option enables kernel function probe (fprobe) based on ftrace, + which is similar to kprobes, but probes only for kernel function + entries and it can probe multiple functions by one fprobe. + + If unsure, say N. + config FUNCTION_PROFILER bool "Kernel function profiler" depends on FUNCTION_TRACER
kernel/trace/Makefile +1 -0
@@ -98,6 +98,7 @@ obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o +obj-$(CONFIG_FPROBE) += fprobe.o obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
kernel/trace/fprobe.c new file +211 -0
211 lines not on this page · raw message
Signature · 1 line
2.35.1
Re: [for-next][PATCH 03/13] fprobe: Add ftrace based probe APIs
Quoted 44 lines · Steven Rostedt
From: Masami Hiramatsu <[email protected]>
The fprobe is a wrapper API for ftrace function tracer. Unlike kprobes, this probes only supports the function entry, but this can probe multiple functions by one fprobe. The usage is similar, user will set their callback to
fprobe::entry_handlerand call register_fprobe*() with probed functions.
There are 3 registration interfaces,-
register_fprobe()takes filtering patterns of the functin names.
-register_fprobe_ips()takes an array of ftrace-location addresses.
-register_fprobe_syms()takes an array of function names.The registered fprobes can be unregistered with
unregister_fprobe(). e.g.struct fprobe fp = { .entry_handler =
user_handler}; const char *targets[] = { "func1", "func2", "func3"};
...ret =
register_fprobe_syms(&fp, targets, ARRAY_SIZE(targets));...
unregister_fprobe(&fp);Cc Jiri Olsa <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Andrii Nakryiko <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Andrii Nakryiko <[email protected]>Cc Martin KaFai Lau <[email protected]>Cc Song Liu <[email protected]>Cc Yonghong Song <[email protected]>Cc John Fastabend <[email protected]>Cc KP Singh <[email protected]>Cc "Naveen N . Rao" <[email protected]>Cc Anil S Keshavamurthy <[email protected]>Cc "David S . Miller" <[email protected]>Signed-off-by Masami Hiramatsu <[email protected]>Signed-off-by Steven Rostedt (Google) <[email protected]>
Quoted 1 line
---
Hey Steven!
Do I understand correctly that this patch set was applied in your tree? I was under the impression that we agreed to route this through the bpf-next tree earlier (see [0]), but I might have misunderstood something, sorry.
Either way, the reason it matters is because Jiri's multi-attach kprobe patch set ([1]) is depending on Masami's patches and having fprobe patches in bpf-next tree would simplify logistics significantly.
So I wonder if it's still possible to route it through bpf-next?
If not, we'd need a way to get these changes into the bpf-next tree somehow. Having it in a separate branch that we can merge would be a way to go about this, I presume? But it's certainly a more complicated way, so it would be preferable to back it out and land through bpf-next.
Please let me know how we should proceed. Thanks!
[0] https://lore.kernel.org/bpf/CAEf4BzaugZWf6f_0JzA-mqaGfp52tCwEp5dWdhpeVt6GjDLQ3Q@mail.gmail.com/
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 8 lines
include/linux/fprobe.h | 87 +++++++++++++++++ kernel/trace/Kconfig | 12 +++ kernel/trace/Makefile | 1 + kernel/trace/fprobe.c | 211 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 311 insertions(+) create mode 100644 include/linux/fprobe.h create mode 100644 kernel/trace/fprobe.c
[...]
Steven Rostedt
maintainerRe: [for-next][PATCH 03/13] fprobe: Add ftrace based probe APIs
Quoted 9 lines · On Thu, 17 Mar 2022 15:03:33 -0700 Andrii Nakryiko <[email protected]>…
Do I understand correctly that this patch set was applied in your tree? I was under the impression that we agreed to route this through the bpf-next tree earlier (see [0]), but I might have misunderstood something, sorry.
Either way, the reason it matters is because Jiri's multi-attach kprobe patch set ([1]) is depending on Masami's patches and having fprobe patches in bpf-next tree would simplify logistics significantly.
I knew Jiri's patches were to go through the bpf tree, but I missed that those were dependent on this and you wanted these to go through as well.
I had just finished my automated tests that ran these patches. I haven't pushed them to my next branch yet so I can hold them off. I don't have anything dependent on them.
Would you be able to take these for-next patches directly (as they all have been tested) and you can switch my signed-off-by to Reviewed-by.
The first of the series is unrelated and will go through my tree. That's the user_events patch.
-- Steve
Quoted 8 lines
So I wonder if it's still possible to route it through bpf-next?
If not, we'd need a way to get these changes into the bpf-next tree somehow. Having it in a separate branch that we can merge would be a way to go about this, I presume? But it's certainly a more complicated way, so it would be preferable to back it out and land through bpf-next.
Re: [for-next][PATCH 03/13] fprobe: Add ftrace based probe APIs
Quoted 14 lines · Steven Rostedt
Quoted 9 lines · On Thu, 17 Mar 2022 15:03:33 -0700 Andrii Nakryiko <[email protected]>…
Do I understand correctly that this patch set was applied in your tree? I was under the impression that we agreed to route this through the bpf-next tree earlier (see [0]), but I might have misunderstood something, sorry.
Either way, the reason it matters is because Jiri's multi-attach kprobe patch set ([1]) is depending on Masami's patches and having fprobe patches in bpf-next tree would simplify logistics significantly.
Quoted 6 lines
I knew Jiri's patches were to go through the bpf tree, but I missed that those were dependent on this and you wanted these to go through as well.
I had just finished my automated tests that ran these patches. I haven't pushed them to my next branch yet so I can hold them off. I don't have anything dependent on them.
Excellent. Thanks for testing.
Quoted 5 lines
Would you be able to take these for-next patches directly (as they all have been tested) and you can switch my signed-off-by to Reviewed-by.
The first of the series is unrelated and will go through my tree. That's the
user_eventspatch.
Right.
We're talking about this set:
https://patchwork.kernel.org/project/netdevbpf/cover/164735281449.1084943.12438881786173547153.stgit@devnote2/
I believe it has a small doc difference vs what you've tested. Looks like our CI is green on it too.
We'll do additional testing and add your SOB and Tested-by before pushing.
Thanks!
Masami Hiramatsu
maintainerRe: [for-next][PATCH 03/13] fprobe: Add ftrace based probe APIs
Quoted 30 lines · On Thu, 17 Mar 2022 17:26:23 -0700 Alexei Starovoitov…
Quoted 14 lines · Steven Rostedt
Quoted 9 lines · On Thu, 17 Mar 2022 15:03:33 -0700 Andrii Nakryiko <[email protected]>…
Do I understand correctly that this patch set was applied in your tree? I was under the impression that we agreed to route this through the bpf-next tree earlier (see [0]), but I might have misunderstood something, sorry.
Either way, the reason it matters is because Jiri's multi-attach kprobe patch set ([1]) is depending on Masami's patches and having fprobe patches in bpf-next tree would simplify logistics significantly.
Quoted 6 lines
I knew Jiri's patches were to go through the bpf tree, but I missed that those were dependent on this and you wanted these to go through as well.
I had just finished my automated tests that ran these patches. I haven't pushed them to my next branch yet so I can hold them off. I don't have anything dependent on them.
Excellent. Thanks for testing.
Quoted 5 lines
Would you be able to take these for-next patches directly (as they all have been tested) and you can switch my signed-off-by to Reviewed-by.
The first of the series is unrelated and will go through my tree. That's the
user_eventspatch.
Quoted 8 lines
Right.
We're talking about this set:
https://patchwork.kernel.org/project/netdevbpf/cover/164735281449.1084943.12438881786173547153.stgit@devnote2/I believe it has a small doc difference vs what you've tested. Looks like our CI is green on it too.
We'll do additional testing and add your SOB and Tested-by before pushing.
Thanks Alexei and Steve, I think this fprobe series is good to go through bpf-next tree because it is currently used only from bpf with Jiri's patch. Sorry Steve for confusion. I think I might better move you to CC when I tagged it 'bpf-next'.
Thank you,
Quoted 2 lines
Thanks!
Signature · 1 line
Masami Hiramatsu <[email protected]>
Steven Rostedt
maintainerPATCH 4/13 rethook: Add a generic return hook
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add a return hook framework which hooks the function return. Most of the logic came from the kretprobe, but this is independent from kretprobe.
Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just an additional hook.
--- include/linux/rethook.h | 99 +++++++++++++ include/linux/sched.h | 3 + kernel/exit.c | 2 + kernel/fork.c | 3 + kernel/trace/Kconfig | 11 ++ kernel/trace/Makefile | 1 + kernel/trace/rethook.c | 317 ++++++++++++++++++++++++++++++++++++++++ 7 files changed, 436 insertions(+) create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/rethook.c
include/linux/rethook.h new file +99 -0
@@ -0,0 +1,99 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Return hooking with list-based shadow stack. +*/ +#ifndef _LINUX_RETHOOK_H +#define _LINUX_RETHOOK_H + +#include <linux/compiler.h> +#include <linux/freelist.h> +#include <linux/kallsyms.h> +#include <linux/llist.h> +#include <linux/rcupdate.h> +#include <linux/refcount.h> + +struct rethook_node ; + +typedef void (*rethook_handler_t ) (struct rethook_node *,void *,struct pt_regs *); + +/** +* struct rethook - The rethook management data structure. +* @data: The user-defined data storage. +* @handler: The user-defined return hook handler. +* @pool: The pool of struct rethook_node. +* @ref: The reference counter. +* @rcu: The rcu_head for deferred freeing. +* +* Don't embed to another data structure, because this is a self-destructive +* data structure when all rethook_node are freed. +*/ +struct rethook { +void *data; +rethook_handler_t handler; +struct freelist_head pool; +refcount_t ref; +struct rcu_head rcu; +}; + +/** +* struct rethook_node - The rethook shadow-stack entry node. +* @freelist: The freelist, linked to struct rethook::pool. +* @rcu: The rcu_head for deferred freeing. +* @llist: The llist, linked to a struct task_struct::rethooks. +* @rethook: The pointer to the struct rethook. +* @ret_addr: The storage for the real return address. +* @frame: The storage for the frame pointer. +* +* You can embed this to your extended data structure to store any data +* on each entry of the shadow stack. +*/ +struct rethook_node { +union { +struct freelist_node freelist; +struct rcu_head rcu; + }; +struct llist_node llist; +struct rethook *rethook; +unsigned long ret_addr; +unsigned long frame; +}; + +struct rethook *rethook_alloc (void *data,rethook_handler_t handler); +void rethook_free (struct rethook *rh); +void rethook_add_node (struct rethook *rh,struct rethook_node *node); +struct rethook_node *rethook_try_get (struct rethook *rh); +void rethook_recycle (struct rethook_node *node); +void rethook_hook (struct rethook_node *node,struct pt_regs *regs,bool mcount); +unsigned long rethook_find_ret_addr (struct task_struct *tsk,unsigned long frame, +struct llist_node **cur); + +/* Arch dependent code must implement arch_* and trampoline code */ +void arch_rethook_prepare (struct rethook_node *node,struct pt_regs *regs,bool mcount); +void arch_rethook_trampoline (void ); + +/** +* is_rethook_trampoline() - Check whether the address is rethook trampoline +* @addr: The address to be checked +* +* Return true if the @addr is the rethook trampoline address. +*/ +static inline bool is_rethook_trampoline (unsigned long addr) +{ +return addr == (unsigned long )dereference_symbol_descriptor (arch_rethook_trampoline); +} + +/* If the architecture needs to fixup the return address, implement it. */ +void arch_rethook_fixup_return (struct pt_regs *regs, +unsigned long correct_ret_addr); + +/* Generic trampoline handler, arch code must prepare asm stub */ +unsigned long rethook_trampoline_handler (struct pt_regs *regs, +unsigned long frame); + +#ifdef CONFIG_RETHOOK +void rethook_flush_task (struct task_struct *tk); +#else +#define rethook_flush_task (tsk) do { } while (0) +#endif + +#endif
include/linux/sched.h +3 -0
@@ -1485,6 +1485,9 @@ struct task_struct {#ifdef CONFIG_KRETPROBES struct llist_head kretprobe_instances;#endif +#ifdef CONFIG_RETHOOK +struct llist_head rethooks; +#endif #ifdef CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH /*
kernel/exit.c +2 -0
@@ -64,6 +64,7 @@#include <linux/compat.h> #include <linux/io_uring.h> #include <linux/kprobes.h> +#include <linux/rethook.h> #include <linux/uaccess.h> #include <asm/unistd.h>
@@ -169,6 +170,7 @@ static void delayed_put_task_struct(struct rcu_head *rhp) struct task_struct *tsk = container_of(rhp, struct task_struct, rcu);kprobe_flush_task (tsk); +rethook_flush_task (tsk);perf_event_delayed_put (tsk);trace_sched_process_free (tsk);put_task_struct (tsk);
kernel/fork.c +3 -0
@@ -2255,6 +2255,9 @@ static __latent_entropy struct task_struct *copy_process(#ifdef CONFIG_KRETPROBES p->kretprobe_instances.first =NULL ;#endif +#ifdef CONFIG_RETHOOK + p->rethooks.first =NULL ; +#endif /* * Ensure that the cgroup subsystem policies allow the new process to be
kernel/trace/Kconfig +11 -0
@@ -10,6 +10,17 @@ config USER_STACKTRACE_SUPPORT config NOP_TRACER bool +config HAVE_RETHOOK + bool + +config RETHOOK + bool + depends on HAVE_RETHOOK + help + Enable generic return hooking feature. This is an internal + API, which will be used by other function-entry hooking + features like fprobe and kprobes. + config HAVE_FUNCTION_TRACER bool help
kernel/trace/Makefile +1 -0
@@ -99,6 +99,7 @@ obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o obj-$(CONFIG_FPROBE) += fprobe.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
kernel/trace/rethook.c new file +317 -0
317 lines not on this page · raw message
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 5/13 rethook: x86: Add rethook x86 implementation
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add rethook for x86 implementation. Most of the code has been copied from kretprobes on x86.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/unwind.h | 8 ++- arch/x86/kernel/Makefile | 1 + arch/x86/kernel/kprobes/common.h | 1 + arch/x86/kernel/rethook.c | 119 +++++++++++++++++++++++++++++++ 5 files changed, 129 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kernel/rethook.c
arch/x86/Kconfig +1 -0
@@ -220,6 +220,7 @@ config X86 select HAVE_KPROBES_ON_FTRACE select HAVE_FUNCTION_ERROR_INJECTION select HAVE_KRETPROBES + select HAVE_RETHOOK select HAVE_KVM select HAVE_LIVEPATCH if X86_64 select HAVE_MIXED_BREAKPOINTS_REGS
arch/x86/include/asm/unwind.h +7 -1
@@ -5,6 +5,7 @@#include <linux/sched.h> #include <linux/ftrace.h> #include <linux/kprobes.h> +#include <linux/rethook.h> #include <asm/ptrace.h> #include <asm/stacktrace.h>
@@ -16,7 +17,7 @@ struct unwind_state {unsigned long stack_mask;struct task_struct *task;int graph_idx; -#ifdef CONFIG_KRETPROBES +#if defined(CONFIG_KRETPROBES ) || defined(CONFIG_RETHOOK )struct llist_node *kr_cur;#endif bool error;
@@ -107,6 +108,11 @@ static inlineunsigned long unwind_recover_kretprobe (struct unwind_state *state,unsigned long addr,unsigned long *addr_p) { +#ifdef CONFIG_RETHOOK +if (is_rethook_trampoline (addr)) +return rethook_find_ret_addr (state->task, (unsigned long )addr_p, + &state->kr_cur); +#endif #ifdef CONFIG_KRETPROBES return is_kretprobe_trampoline (addr) ? kretprobe_find_ret_addr(state->task, addr_p, &state->kr_cur) :
arch/x86/kernel/Makefile +1 -0
@@ -106,6 +106,7 @@ obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o obj-$(CONFIG_FTRACE_SYSCALLS) += ftrace.o obj-$(CONFIG_X86_TSC) += trace_clock.o obj-$(CONFIG_TRACING) += trace.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_CRASH_CORE) += crash_core_$(BITS).o obj-$(CONFIG_KEXEC_CORE) += machine_kexec_$(BITS).o obj-$(CONFIG_KEXEC_CORE) += relocate_kernel_$(BITS).o crash.o
arch/x86/kernel/kprobes/common.h +1 -0
@@ -6,6 +6,7 @@#include <asm/asm.h> #include <asm/frame.h> +#include <asm/insn.h> #ifdef CONFIG_X86_64
arch/x86/kernel/rethook.c new file +119 -0
119 lines not on this page · raw message
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 6/13 arm64: rethook: Add arm64 rethook implementation
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add rethook arm64 implementation. Most of the code has been copied from kretprobes on arm64.
--- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/stacktrace.h | 2 +- arch/arm64/kernel/probes/Makefile | 1 + arch/arm64/kernel/probes/rethook.c | 25 ++++++ arch/arm64/kernel/probes/rethook_trampoline.S | 87 +++++++++++++++++++ arch/arm64/kernel/stacktrace.c | 7 +- 6 files changed, 121 insertions(+), 2 deletions(-) create mode 100644 arch/arm64/kernel/probes/rethook.c create mode 100644 arch/arm64/kernel/probes/rethook_trampoline.S
arch/arm64/Kconfig +1 -0
@@ -201,6 +201,7 @@ config ARM64 select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES select HAVE_KRETPROBES + select HAVE_RETHOOK select HAVE_GENERIC_VDSO select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN
arch/arm64/include/asm/stacktrace.h +1 -1
@@ -58,7 +58,7 @@ struct stackframe {DECLARE_BITMAP (stacks_done, __NR_STACK_TYPES);unsigned long prev_fp;enum stack_type prev_type; -#ifdef CONFIG_KRETPROBES +#if defined(CONFIG_KRETPROBES ) || defined(CONFIG_RETHOOK )struct llist_node *kr_cur;#endif };
arch/arm64/kernel/probes/Makefile +1 -0
@@ -4,3 +4,4 @@ obj-$(CONFIG_KPROBES) += kprobes.o decode-insn.o \ simulate-insn.o obj-$(CONFIG_UPROBES) += uprobes.o decode-insn.o \ simulate-insn.o +obj-$(CONFIG_RETHOOK) += rethook.o rethook_trampoline.o
arch/arm64/kernel/probes/rethook.c new file +25 -0
@@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Generic return hook for arm64. +* Most of the code is copied from arch/arm64/kernel/probes/kprobes.c +*/ + +#include <linux/kprobes.h> +#include <linux/rethook.h> + +/* This is called from arch_rethook_trampoline() */ +unsigned long __used arch_rethook_trampoline_callback (struct pt_regs *regs) +{ +return rethook_trampoline_handler (regs, regs->regs[29 ]); +} +NOKPROBE_SYMBOL (arch_rethook_trampoline_callback); + +void arch_rethook_prepare (struct rethook_node *rhn,struct pt_regs *regs,bool mcount) +{ + rhn->ret_addr = regs->regs[30 ]; + rhn->frame = regs->regs[29 ]; + +/* replace return addr (x30) with trampoline */ + regs->regs[30 ] = (u64 )arch_rethook_trampoline; +} +NOKPROBE_SYMBOL (arch_rethook_prepare);
arch/arm64/kernel/probes/rethook_trampoline.S new file +87 -0
87 lines not on this page · raw message
arch/arm64/kernel/stacktrace.c +6 -1
@@ -8,6 +8,7 @@#include <linux/export.h> #include <linux/ftrace.h> #include <linux/kprobes.h> +#include <linux/rethook.h> #include <linux/sched.h> #include <linux/sched/debug.h> #include <linux/sched/task_stack.h>
@@ -38,7 +39,7 @@ static void start_backtrace(struct stackframe *frame, unsigned long fp, { frame->fp = fp; frame->pc = pc; -#ifdef CONFIG_KRETPROBES +#if defined(CONFIG_KRETPROBES ) || defined(CONFIG_RETHOOK ) frame->kr_cur =NULL ;#endif
@@ -137,6 +138,10 @@ static int notrace unwind_frame(struct task_struct *tsk,if (is_kretprobe_trampoline (frame->pc)) frame->pc =kretprobe_find_ret_addr (tsk, (void *)frame->fp, &frame->kr_cur);#endif +#ifdef CONFIG_RETHOOK +if (is_rethook_trampoline (frame->pc)) + frame->pc =rethook_find_ret_addr (tsk, frame->fp, &frame->kr_cur); +#endif return 0 ; }
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 7/13 powerpc: Add rethook support
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add rethook powerpc64 implementation. Most of the code has been copied from kretprobes on powerpc64.
--- arch/powerpc/Kconfig | 1 + arch/powerpc/kernel/Makefile | 1 + arch/powerpc/kernel/rethook.c | 72 +++++++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+) create mode 100644 arch/powerpc/kernel/rethook.c
arch/powerpc/Kconfig +1 -0
@@ -229,6 +229,7 @@ config PPC select HAVE_PERF_EVENTS_NMI if PPC64 select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP + select HAVE_RETHOOK if KPROBES select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE select HAVE_RSEQ
arch/powerpc/kernel/Makefile +1 -0
@@ -115,6 +115,7 @@ obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_KPROBES) += kprobes.o obj-$(CONFIG_OPTPROBES) += optprobes.o optprobes_head.o obj-$(CONFIG_KPROBES_ON_FTRACE) += kprobes-ftrace.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_UPROBES) += uprobes.o obj-$(CONFIG_PPC_UDBG_16550) += legacy_serial.o udbg_16550.o obj-$(CONFIG_SWIOTLB) += dma-swiotlb.o
arch/powerpc/kernel/rethook.c new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 8/13 ARM: rethook: Add rethook arm implementation
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add rethook arm implementation. Most of the code has been copied from kretprobes on arm.
Since the arm's ftrace implementation is a bit special, this needs a special care using from fprobe.
--- arch/arm/Kconfig | 1 + arch/arm/include/asm/stacktrace.h | 4 +- arch/arm/kernel/stacktrace.c | 6 ++ arch/arm/probes/Makefile | 1 + arch/arm/probes/rethook.c | 103 ++++++++++++++++++++++++++++++ 5 files changed, 113 insertions(+), 2 deletions(-) create mode 100644 arch/arm/probes/rethook.c
arch/arm/Kconfig +1 -0
@@ -107,6 +107,7 @@ config ARM select HAVE_MOD_ARCH_SPECIFIC select HAVE_NMI select HAVE_OPTPROBES if !THUMB2_KERNEL + select HAVE_RETHOOK select HAVE_PERF_EVENTS select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP
arch/arm/include/asm/stacktrace.h +2 -2
@@ -14,7 +14,7 @@ struct stackframe {unsigned long sp;unsigned long lr;unsigned long pc; -#ifdef CONFIG_KRETPROBES +#if defined(CONFIG_KRETPROBES ) || defined(CONFIG_RETHOOK )struct llist_node *kr_cur;struct task_struct *tsk;#endif
@@ -27,7 +27,7 @@ void arm_get_current_stackframe(struct pt_regs *regs, struct stackframe *frame) frame->sp = regs->ARM_sp; frame->lr = regs->ARM_lr; frame->pc = regs->ARM_pc; -#ifdef CONFIG_KRETPROBES +#if defined(CONFIG_KRETPROBES ) || defined(CONFIG_RETHOOK ) frame->kr_cur =NULL ; frame->tsk = current;#endif
arch/arm/kernel/stacktrace.c +6 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0-only #include <linux/export.h> #include <linux/kprobes.h> +#include <linux/rethook.h> #include <linux/sched.h> #include <linux/sched/debug.h> #include <linux/stacktrace.h>
@@ -66,6 +67,11 @@ int notrace unwind_frame(struct stackframe *frame) frame->sp = *(unsigned long *)(fp -8 ); frame->pc = *(unsigned long *)(fp -4 );#endif +#ifdef CONFIG_RETHOOK +if (is_rethook_trampoline (frame->pc)) + frame->pc =rethook_find_ret_addr (frame->tsk, frame->fp, + &frame->kr_cur); +#endif #ifdef CONFIG_KRETPROBES if (is_kretprobe_trampoline (frame->pc)) frame->pc = kretprobe_find_ret_addr(frame->tsk,
arch/arm/probes/Makefile +1 -0
@@ -6,3 +6,4 @@ obj-$(CONFIG_KPROBES) += decode-thumb.o else obj-$(CONFIG_KPROBES) += decode-arm.o endif +obj-$(CONFIG_RETHOOK) += rethook.o
arch/arm/probes/rethook.c new file +103 -0
103 lines not on this page · raw message
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 9/13 fprobe: Add exit_handler support
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add exit_handler to fprobe. fprobe + rethook allows us to hook the kernel function return. The rethook will be enabled only if the fprobe::exit_handler is set.
--- include/linux/fprobe.h | 6 +++ kernel/trace/Kconfig | 9 ++-- kernel/trace/fprobe.c | 116 ++++++++++++++++++++++++++++++++++++++--- 3 files changed, 122 insertions(+), 9 deletions(-)
include/linux/fprobe.h +6 -0
@@ -5,13 +5,16 @@#include <linux/compiler.h> #include <linux/ftrace.h> +#include <linux/rethook.h> /** * struct fprobe - ftrace based probe. * @ops: The ftrace_ops. * @nmissed: The counter for missing events. * @flags: The status flag. +* @rethook: The rethook data structure. (internal data) * @entry_handler: The callback function for function entry. +* @exit_handler: The callback function for function exit. */ struct fprobe {#ifdef CONFIG_FUNCTION_TRACER
@@ -25,7 +28,10 @@ struct fprobe {#endif unsigned long nmissed;unsigned int flags; +struct rethook *rethook; +void (*entry_handler)(struct fprobe *fp,unsigned long entry_ip,struct pt_regs *regs); +void (*exit_handler)(struct fprobe *fp,unsigned long entry_ip,struct pt_regs *regs); };#define FPROBE_FL_DISABLED 1
kernel/trace/Kconfig +6 -3
@@ -251,11 +251,14 @@ config FPROBE bool "Kernel Function Probe (fprobe)" depends on FUNCTION_TRACER depends on DYNAMIC_FTRACE_WITH_REGS + depends on HAVE_RETHOOK + select RETHOOK default n help - This option enables kernel function probe (fprobe) based on ftrace, - which is similar to kprobes, but probes only for kernel function - entries and it can probe multiple functions by one fprobe. + This option enables kernel function probe (fprobe) based on ftrace. + The fprobe is similar to kprobes, but probes only for kernel function + entries and exits. This also can probe multiple functions by one + fprobe. If unsure, say N.
kernel/trace/fprobe.c +110 -6
176 lines not on this page · raw message
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 10/13 fprobe: Add sample program for fprobe
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add a sample program for the fprobe. The sample_fprobe puts a fprobe on kernel_clone() by default. This dump stack and some called address info at the function entry and exit.
The sample_fprobe.ko gets 2 parameters.
- symbol: you can specify the comma separated symbols or wildcard symbol pattern (in this case you can not use comma)
- stackdump: a bool value to enable or disable stack dump in the fprobe handler.
--- samples/Kconfig | 7 ++ samples/Makefile | 1 + samples/fprobe/Makefile | 3 + samples/fprobe/fprobe_example.c | 120 ++++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+) create mode 100644 samples/fprobe/Makefile create mode 100644 samples/fprobe/fprobe_example.c
samples/Kconfig +7 -0
@@ -79,6 +79,13 @@ config SAMPLE_HW_BREAKPOINT help This builds kernel hardware breakpoint example modules. +config SAMPLE_FPROBE + tristate "Build fprobe examples -- loadable modules only" + depends on FPROBE && m + help + This builds a fprobe example module. This module has an option 'symbol'. + You can specify a probed symbol or symbols separated with ','. + config SAMPLE_KFIFO tristate "Build kfifo examples -- loadable modules only" depends on m
samples/Makefile +1 -0
@@ -34,3 +34,4 @@ subdir-$(CONFIG_SAMPLE_WATCHDOG) += watchdog subdir-$(CONFIG_SAMPLE_WATCH_QUEUE) += watch_queue obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak/ obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG) += coresight/ +obj-$(CONFIG_SAMPLE_FPROBE) += fprobe/
samples/fprobe/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only + +obj-$(CONFIG_SAMPLE_FPROBE) += fprobe_example.o
samples/fprobe/fprobe_example.c new file +120 -0
120 lines not on this page · raw message
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 11/13 fprobe: Introduce FPROBE_FL_KPROBE_SHARED flag for fprobe
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Introduce FPROBE_FL_KPROBE_SHARED flag for sharing fprobe callback with kprobes safely from the viewpoint of recursion.
Since the recursion safety of the fprobe (and ftrace) is a bit different from the kprobes, this may cause an issue if user wants to run the same code from the fprobe and the kprobes.
The kprobes has per-cpu 'current_kprobe' variable which protects the kprobe handler from recursion in any case. On the other hand, the fprobe uses only ftrace_test_recursion_trylock(), which will allow interrupt context calls another (or same) fprobe during the fprobe user handler is running.
This is not a matter in cases if the common callback shared among the kprobes and the fprobe has its own recursion detection, or it can handle the recursion in the different contexts (normal/interrupt/NMI.) But if it relies on the 'current_kprobe' recursion lock, it has to check kprobe_running() and use kprobe_busy_*() APIs.
Fprobe has FPROBE_FL_KPROBE_SHARED flag to do this. If your common callback code will be shared with kprobes, please set FPROBE_FL_KPROBE_SHARED *before* registering the fprobe, like;
fprobe.flags = FPROBE_FL_KPROBE_SHARED;
register_fprobe(&fprobe, "func*", NULL);
This will protect your common callback from the nested call.
--- include/linux/fprobe.h | 12 ++++++++++++ include/linux/kprobes.h | 3 +++ kernel/trace/fprobe.c | 19 ++++++++++++++++++- 3 files changed, 33 insertions(+), 1 deletion(-)
include/linux/fprobe.h +12 -0
@@ -34,13 +34,25 @@ struct fprobe {void (*exit_handler)(struct fprobe *fp,unsigned long entry_ip,struct pt_regs *regs); }; +/* This fprobe is soft-disabled. */ #define FPROBE_FL_DISABLED 1 +/* +* This fprobe handler will be shared with kprobes. +* This flag must be set before registering. +*/ +#define FPROBE_FL_KPROBE_SHARED 2 +static inline bool fprobe_disabled (struct fprobe *fp) {return (fp) ? fp->flags &FPROBE_FL_DISABLED : false; } +static inline bool fprobe_shared_with_kprobes (struct fprobe *fp) +{ +return (fp) ? fp->flags &FPROBE_FL_KPROBE_SHARED : false; +} +#ifdef CONFIG_FPROBE int register_fprobe (struct fprobe *fp,const char *filter,const char *notfilter);int register_fprobe_ips (struct fprobe *fp,unsigned long *addrs,int num);
include/linux/kprobes.h +3 -0
@@ -427,6 +427,9 @@ static inline struct kprobe *kprobe_running(void) {return NULL ; } +#define kprobe_busy_begin () do {} while (0) +#define kprobe_busy_end () do {} while (0) +static inline int register_kprobe (struct kprobe *p) {return -EOPNOTSUPP ;
kernel/trace/fprobe.c +18 -1
@@ -56,6 +56,20 @@ static void fprobe_handler(unsigned long ip, unsigned long parent_ip, }NOKPROBE_SYMBOL (fprobe_handler); +static void fprobe_kprobe_handler (unsigned long ip,unsigned long parent_ip, +struct ftrace_ops *ops,struct ftrace_regs *fregs) +{ + struct fprobe *fp = container_of(ops, struct fprobe, ops); + +if (unlikely (kprobe_running ())) { + fp->nmissed++; +return ; + } +kprobe_busy_begin (); +fprobe_handler (ip, parent_ip, ops, fregs); +kprobe_busy_end (); +} +static void fprobe_exit_handler (struct rethook_node *rh,void *data,struct pt_regs *regs) {
@@ -110,7 +124,10 @@ static unsigned long *get_ftrace_locations(const char **syms, int num)static void fprobe_init (struct fprobe *fp) { fp->nmissed =0 ; - fp->ops.func = fprobe_handler; +if (fprobe_shared_with_kprobes (fp)) + fp->ops.func = fprobe_kprobe_handler; +else + fp->ops.func = fprobe_handler; fp->ops.flags |=FTRACE_OPS_FL_SAVE_REGS ; }
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 12/13 docs: fprobe: Add fprobe description to ftrace-use.rst
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add a documentation of fprobe for the user who needs this interface.
--- Documentation/trace/fprobe.rst | 173 +++++++++++++++++++++++++++++++++ Documentation/trace/index.rst | 1 + 2 files changed, 174 insertions(+) create mode 100644 Documentation/trace/fprobe.rst
Documentation/trace/fprobe.rst new file +173 -0
@@ -0,0 +1,173 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================================== +Fprobe - Function entry/exit probe +================================== + +.. Author: Masami Hiramatsu <[email protected]> + +Introduction +============ + +Fprobe is a function entry/exit probe mechanism based on ftrace. +Instead of using ftrace full feature, if you only want to attach callbacks +on function entry and exit, similar to the kprobes and kretprobes, you can +use fprobe. Compared with kprobes and kretprobes, fprobe gives faster +instrumentation for multiple functions with single handler. This document +describes how to use fprobe. + +The usage of fprobe +=================== + +The fprobe is a wrapper of ftrace (+ kretprobe-like return callback) to +attach callbacks to multiple function entry and exit. User needs to set up +the `struct fprobe` and pass it to `register_fprobe()`. + +Typically, `fprobe` data structure is initialized with the `entry_handler` +and/or `exit_handler` as below. + +.. code-block:: c + + struct fprobe fp = { + .entry_handler = my_entry_callback, + .exit_handler = my_exit_callback, + }; + +To enable the fprobe, call one of register_fprobe(), register_fprobe_ips(), and +register_fprobe_syms(). These functions register the fprobe with different types +of parameters. + +The register_fprobe() enables a fprobe by function-name filters. +E.g. this enables @fp on "func*()" function except "func2()".:: + + register_fprobe(&fp, "func*", "func2"); + +The register_fprobe_ips() enables a fprobe by ftrace-location addresses. +E.g. + +.. code-block:: c + + unsigned long ips[] = { 0x.... }; + + register_fprobe_ips(&fp, ips, ARRAY_SIZE(ips)); + +And the register_fprobe_syms() enables a fprobe by symbol names. +E.g. + +.. code-block:: c + + char syms[] = {"func1", "func2", "func3"}; + + register_fprobe_syms(&fp, syms, ARRAY_SIZE(syms)); + +To disable (remove from functions) this fprobe, call:: + + unregister_fprobe(&fp); + +You can temporally (soft) disable the fprobe by:: + + disable_fprobe(&fp); + +and resume by:: + + enable_fprobe(&fp); + +The above is defined by including the header:: + + #include <linux/fprobe.h> + +Same as ftrace, the registered callbacks will start being called some time +after the register_fprobe() is called and before it returns. See +:file:`Documentation/trace/ftrace.rst`. + +Also, the unregister_fprobe() will guarantee that the both enter and exit +handlers are no longer being called by functions after unregister_fprobe() +returns as same as unregister_ftrace_function(). + +The fprobe entry/exit handler +============================= + +The prototype of the entry/exit callback function is as follows: + +.. code-block:: c + + void callback_func(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs); + +Note that both entry and exit callbacks have same ptototype. The @entry_ip is +saved at function entry and passed to exit handler. + +@fp + This is the address of `fprobe` data structure related to this handler. + You can embed the `fprobe` to your data structure and get it by + container_of() macro from @fp. The @fp must not be NULL. + +@entry_ip + This is the ftrace address of the traced function (both entry and exit). + Note that this may not be the actual entry address of the function but + the address where the ftrace is instrumented. + +@regs + This is the `pt_regs` data structure at the entry and exit. Note that + the instruction pointer of @regs may be different from the @entry_ip + in the entry_handler. If you need traced instruction pointer, you need + to use @entry_ip. On the other hand, in the exit_handler, the instruction + pointer of @regs is set to the currect return address. + +Share the callbacks with kprobes +================================ + +Since the recursion safeness of the fprobe (and ftrace) is a bit different +from the kprobes, this may cause an issue if user wants to run the same +code from the fprobe and the kprobes. + +Kprobes has per-cpu 'current_kprobe' variable which protects the kprobe +handler from recursion in all cases. On the other hand, fprobe uses +only ftrace_test_recursion_trylock(). This allows interrupt context to +call another (or same) fprobe while the fprobe user handler is running. + +This is not a matter if the common callback code has its own recursion +detection, or it can handle the recursion in the different contexts +(normal/interrupt/NMI.) +But if it relies on the 'current_kprobe' recursion lock, it has to check +kprobe_running() and use kprobe_busy_*() APIs. + +Fprobe has FPROBE_FL_KPROBE_SHARED flag to do this. If your common callback +code will be shared with kprobes, please set FPROBE_FL_KPROBE_SHARED +*before* registering the fprobe, like: + +.. code-block:: c + + fprobe.flags = FPROBE_FL_KPROBE_SHARED; + + register_fprobe(&fprobe, "func*", NULL); + +This will protect your common callback from the nested call. + +The missed counter +================== + +The `fprobe` data structure has `fprobe::nmissed` counter field as same as +kprobes. +This counter counts up when; + + - fprobe fails to take ftrace_recursion lock. This usually means that a function + which is traced by other ftrace users is called from the entry_handler. + + - fprobe fails to setup the function exit because of the shortage of rethook + (the shadow stack for hooking the function return.) + +The `fprobe::nmissed` field counts up in both cases. Therefore, the former +skips both of entry and exit callback and the latter skips the exit +callback, but in both case the counter will increase by 1. + +Note that if you set the FTRACE_OPS_FL_RECURSION and/or FTRACE_OPS_FL_RCU to +`fprobe::ops::flags` (ftrace_ops::flags) when registering the fprobe, this +counter may not work correctly, because ftrace skips the fprobe function which +increase the counter. + + +Functions and structures +======================== + +.. kernel-doc:: include/linux/fprobe.h +.. kernel-doc:: kernel/trace/fprobe.c
Documentation/trace/index.rst +1 -0
@@ -9,6 +9,7 @@ Linux Tracing Technologies tracepoint-analysis ftrace ftrace-uses + fprobe kprobes kprobetrace uprobetracer
Signature · 1 line
2.35.1
Steven Rostedt
maintainerPATCH 13/13 fprobe: Add a selftest for fprobe
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add a KUnit based selftest for fprobe interface.
--- lib/Kconfig.debug | 12 ++++ lib/Makefile | 2 + lib/test_fprobe.c | 174 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 188 insertions(+) create mode 100644 lib/test_fprobe.c
lib/Kconfig.debug +12 -0
@@ -2100,6 +2100,18 @@ config KPROBES_SANITY_TEST Say N if you are unsure. +config FPROBE_SANITY_TEST + bool "Self test for fprobe" + depends on DEBUG_KERNEL + depends on FPROBE + depends on KUNIT=y + help + This option will enable testing the fprobe when the system boot. + A series of tests are made to verify that the fprobe is functioning + properly. + + Say N if you are unsure. + config BACKTRACE_SELF_TEST tristate "Self test for the backtrace code" depends on DEBUG_KERNEL
lib/Makefile +2 -0
@@ -103,6 +103,8 @@ obj-$(CONFIG_TEST_HMM) += test_hmm.o obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o obj-$(CONFIG_TEST_REF_TRACKER) += test_ref_tracker.o +CFLAGS_test_fprobe.o += $(CC_FLAGS_FTRACE) +obj-$(CONFIG_FPROBE_SANITY_TEST) += test_fprobe.o # # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns # off the generation of FPU/SSE* instructions for kernel proper but FPU_FLAGS
lib/test_fprobe.c new file +174 -0
174 lines not on this page · raw message
Signature · 1 line
2.35.1