fprobe: Introduce fprobe function entry/exit probe
Subsystems (10)
Masami Hiramatsu
maintainerv2 cover fprobe: Introduce fprobe function entry/exit probe
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the ftrace_set_filter_ips() for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.
This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
[1] https://lore.kernel.org/all/[email protected]/T/#u
I also added an out-of-tree (just for testing) patch at the end of this series ([8/8]) for adding a wildcard support to the sample program. With that patch, it shows how long the registration will take;
# time insmod fprobe_example.ko symbol='btrfs_*' [ 36.130947] fprobe_init: 1028 symbols found [ 36.177901] fprobe_init: Planted fprobe at btrfs_* real 0m 0.08s user 0m 0.00s sys 0m 0.07s
Thank you,
---
Jiri Olsa (2):
ftrace: Add ftrace_set_filter_ips function
bpf: Add kprobe link for attaching raw kprobes
Masami Hiramatsu (6):
fprobe: Add ftrace based probe APIs
rethook: Add a generic return hook
rethook: x86: Add rethook x86 implementation
fprobe: Add exit_handler support
fprobe: Add sample program for fprobe
[DO NOT MERGE] Out-of-tree: Support wildcard symbol option to sample
arch/x86/Kconfig | 1 arch/x86/kernel/Makefile | 1 arch/x86/kernel/rethook.c | 115 ++++++++++++++++++++ include/linux/bpf_types.h | 1 include/linux/fprobe.h | 57 ++++++++++ include/linux/ftrace.h | 3 + include/linux/rethook.h | 74 +++++++++++++ include/linux/sched.h | 3 + include/uapi/linux/bpf.h | 12 ++ kernel/bpf/syscall.c | 195 +++++++++++++++++++++++++++++++++- kernel/exit.c | 2 kernel/fork.c | 3 + kernel/kallsyms.c | 1 kernel/trace/Kconfig | 22 ++++ kernel/trace/Makefile | 2 kernel/trace/fprobe.c | 168 +++++++++++++++++++++++++++++ kernel/trace/ftrace.c | 54 ++++++++- kernel/trace/rethook.c | 226 +++++++++++++++++++++++++++++++++++++++ samples/Kconfig | 7 + samples/Makefile | 1 samples/fprobe/Makefile | 3 + samples/fprobe/fprobe_example.c | 154 +++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 12 ++ 23 files changed, 1103 insertions(+), 14 deletions(-) create mode 100644 arch/x86/kernel/rethook.c create mode 100644 include/linux/fprobe.h create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/fprobe.c create mode 100644 kernel/trace/rethook.c create mode 100644 samples/fprobe/Makefile create mode 100644 samples/fprobe/fprobe_example.c
--
Masami Hiramatsu (Linaro) <[email protected]>
Masami Hiramatsu
maintainerRFC PATCH v2 1/8 ftrace: Add ftrace_set_filter_ips function
From: Jiri Olsa <[email protected]>
Adding ftrace_set_filter_ips function to be able to set filter on multiple ip addresses at once.
With the kprobe multi attach interface we have cases where we need to initialize ftrace_ops object with thousands of functions, so having single function diving into ftrace_hash_move_and_update_ops with ftrace_lock is faster.
The functions ips are passed as unsigned long array with count.
--- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 53 ++++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 47 insertions(+), 9 deletions(-)
include/linux/ftrace.h +3 -0
@@ -512,6 +512,8 @@ struct dyn_ftrace {int ftrace_set_filter_ip (struct ftrace_ops *ops,unsigned long ip,int remove,int reset); +int ftrace_set_filter_ips (struct ftrace_ops *ops,unsigned long *ips, +unsigned int cnt,int remove,int reset);int ftrace_set_filter (struct ftrace_ops *ops,unsigned char *buf,int len,int reset); int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
@@ -802,6 +804,7 @@ static inline unsigned long ftrace_location(unsigned long ip)#define ftrace_regex_open (ops, flag, inod, file) ({ -ENODEV; })#define ftrace_set_early_filter (ops, buf, enable) do { } while (0)#define ftrace_set_filter_ip (ops, ip, remove, reset) ({ -ENODEV; }) +#define ftrace_set_filter_ips (ops, ips, cnt, remove, reset) ({ -ENODEV; })#define ftrace_set_filter (ops, buf, len, reset) ({ -ENODEV; })#define ftrace_set_notrace (ops, buf, len, reset) ({ -ENODEV; })#define ftrace_free_filter (ops) do { } while (0)
kernel/trace/ftrace.c +44 -9
95 lines not on this page · raw message
Masami Hiramatsu
maintainerRFC PATCH v2 2/8 fprobe: Add ftrace based probe APIs
In reply to Masami HiramatsuThe fprobe is a wrapper API for ftrace function tracer. Unlike kprobes, this probes only supports the function entry, but it can probe multiple functions by one fprobe. The usage is almost same as the kprobe, user will specify the function names by fprobe::syms, the number of syms by fprobe::nentry,
and the user handler by fprobe::entry_handler.
struct fprobe fp = { 0 };
const char *targets[] = { "func1", "func2", "func3"};
fp.handler = user_handler;
fp.nentry = ARRAY_SIZE(targets);
fp.syms = targets;
ret = register_fprobe(&fp);
---
Changes in v2:
- Remove fprobe_entry.
- Do not sort the address array since there is no user private data.
- Since there is only "fprobe", make filename and config name "fprobe".
- Use ftrace_set_filter_ips()
- Fix style warnings.
--- include/linux/fprobe.h | 53 +++++++++++++++++++++++ kernel/trace/Kconfig | 10 ++++ kernel/trace/Makefile | 1 kernel/trace/fprobe.c | 113 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 177 insertions(+) create mode 100644 include/linux/fprobe.h create mode 100644 kernel/trace/fprobe.c
include/linux/fprobe.h new file +53 -0
@@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Simple ftrace probe wrapper */ +#ifndef _LINUX_FPROBE_H +#define _LINUX_FPROBE_H + +#include <linux/compiler.h> +#include <linux/ftrace.h> + +struct fprobe { +const char **syms; +unsigned long *addrs; +unsigned int nentry; + +struct ftrace_ops ftrace; +unsigned long nmissed; +unsigned int flags; +void (*entry_handler)(struct fprobe *fp,unsigned long entry_ip,struct pt_regs *regs); +}; + +#define FPROBE_FL_DISABLED 1 + +static inline bool fprobe_disabled (struct fprobe *fp) +{ +return (fp) ? fp->flags &FPROBE_FL_DISABLED : false; +} + +#ifdef CONFIG_FPROBE +int register_fprobe (struct fprobe *fp); +int unregister_fprobe (struct fprobe *fp); +#else +static inline int register_fprobe (struct fprobe *fp) +{ +return -EOPNOTSUPP ; +} +static inline int unregister_fprobe (struct fprobe *fp) +{ +return -EOPNOTSUPP ; +} +#endif + +static inline void disable_fprobe (struct fprobe *fp) +{ +if (fp) + fp->flags |=FPROBE_FL_DISABLED ; +} + +static inline void enable_fprobe (struct fprobe *fp) +{ +if (fp) + fp->flags &= ~FPROBE_FL_DISABLED ; +} + +#endif
kernel/trace/Kconfig +10 -0
@@ -223,6 +223,16 @@ config DYNAMIC_FTRACE_WITH_ARGS depends on DYNAMIC_FTRACE depends on HAVE_DYNAMIC_FTRACE_WITH_ARGS +config FPROBE + bool "Kernel Function Probe (fprobe)" + depends on FUNCTION_TRACER + depends on DYNAMIC_FTRACE_WITH_REGS + default n + help + This option enables kernel function probe feature, which is + similar to kprobes, but probes only for kernel function entries + and it can probe multiple functions by one fprobe. + config FUNCTION_PROFILER bool "Kernel function profiler" depends on FUNCTION_TRACER
kernel/trace/Makefile +1 -0
@@ -97,6 +97,7 @@ obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o +obj-$(CONFIG_FPROBE) += fprobe.o obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
kernel/trace/fprobe.c new file +113 -0
113 lines not on this page · raw message
Masami Hiramatsu
maintainerRFC PATCH v2 3/8 rethook: Add a generic return hook
In reply to Masami HiramatsuAdd a return hook framework which hooks the function return. Most of the idea came from the kretprobe, but this is independent from kretprobe.
Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just a additional hook.
--- include/linux/rethook.h | 74 +++++++++++++++ include/linux/sched.h | 3 + kernel/exit.c | 2 kernel/fork.c | 3 + kernel/trace/Kconfig | 11 ++ kernel/trace/Makefile | 1 kernel/trace/rethook.c | 226 +++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 320 insertions(+) create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/rethook.c
include/linux/rethook.h new file +74 -0
@@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Return hooking with list-based shadow stack. +*/ +#ifndef _LINUX_RETHOOK_H +#define _LINUX_RETHOOK_H + +#include <linux/compiler.h> +#include <linux/freelist.h> +#include <linux/llist.h> +#include <linux/rcupdate.h> +#include <linux/refcount.h> + +struct rethook_node ; + +typedef void (*rethook_handler_t ) (struct rethook_node *,void *,struct pt_regs *); + +struct rethook { +void *data; +rethook_handler_t handler; +struct freelist_head pool; +refcount_t ref; +struct rcu_head rcu; +}; + +struct rethook_node { +union { +struct freelist_node freelist; +struct rcu_head rcu; + }; +struct llist_node llist; +struct rethook *rethook; +unsigned long ret_addr; +unsigned long frame; +}; + +int rethook_node_init (struct rethook_node *node); + +struct rethook *rethook_alloc (void *data,rethook_handler_t handler); +void rethook_free (struct rethook *rh); +void rethook_add_node (struct rethook *rh,struct rethook_node *node); + +struct rethook_node *rethook_try_get (struct rethook *rh); +void rethook_node_recycle (struct rethook_node *node); +void rethook_hook_current (struct rethook_node *node,struct pt_regs *regs); + +unsigned long rethook_find_ret_addr (struct task_struct *tsk,unsigned long frame, +struct llist_node **cur); + +/* Arch dependent code must implement this and trampoline code */ +void arch_rethook_prepare (struct rethook_node *node,struct pt_regs *regs); +void arch_rethook_trampoline (void ); + +static inline bool is_rethook_trampoline (unsigned long addr) +{ +return addr == (unsigned long )arch_rethook_trampoline; +} + +/* If the architecture needs a fixup the return address, implement it. */ +void arch_rethook_fixup_return (struct pt_regs *regs, +unsigned long correct_ret_addr); + +/* Generic trampoline handler, arch code must prepare asm stub */ +unsigned long rethook_trampoline_handler (struct pt_regs *regs, +unsigned long frame); + +#ifdef CONFIG_RETHOOK +void rethook_flush_task (struct task_struct *tk); +#else +#define rethook_flush_task (tsk) do { } while (0) +#endif + +#endif +
include/linux/sched.h +3 -0
@@ -1473,6 +1473,9 @@ struct task_struct {#ifdef CONFIG_KRETPROBES struct llist_head kretprobe_instances;#endif +#ifdef CONFIG_RETHOOK +struct llist_head rethooks; +#endif #ifdef CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH /*
kernel/exit.c +2 -0
@@ -64,6 +64,7 @@#include <linux/compat.h> #include <linux/io_uring.h> #include <linux/kprobes.h> +#include <linux/rethook.h> #include <linux/uaccess.h> #include <asm/unistd.h>
@@ -169,6 +170,7 @@ static void delayed_put_task_struct(struct rcu_head *rhp) struct task_struct *tsk = container_of(rhp, struct task_struct, rcu);kprobe_flush_task (tsk); +rethook_flush_task (tsk);perf_event_delayed_put (tsk);trace_sched_process_free (tsk);put_task_struct (tsk);
kernel/fork.c +3 -0
@@ -2282,6 +2282,9 @@ static __latent_entropy struct task_struct *copy_process(#ifdef CONFIG_KRETPROBES p->kretprobe_instances.first =NULL ;#endif +#ifdef CONFIG_RETHOOK + p->rethooks.first =NULL ; +#endif /* * Ensure that the cgroup subsystem policies allow the new process to be
kernel/trace/Kconfig +11 -0
@@ -10,6 +10,17 @@ config USER_STACKTRACE_SUPPORT config NOP_TRACER bool +config HAVE_RETHOOK + bool + +config RETHOOK + bool + depends on HAVE_RETHOOK + help + Enable generic return hooking feature. This is an internal + API, which will be used by other function-entry hooking + feature like fprobe and kprobes. + config HAVE_FUNCTION_TRACER bool help
kernel/trace/Makefile +1 -0
@@ -98,6 +98,7 @@ obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o obj-$(CONFIG_FPROBE) += fprobe.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
kernel/trace/rethook.c new file +226 -0
226 lines not on this page · raw message
Masami Hiramatsu
maintainerRFC PATCH v2 4/8 rethook: x86: Add rethook x86 implementation
In reply to Masami HiramatsuAdd rethook for x86 implementation. Most of the code has been copied from kretprobes on x86.
--- arch/x86/Kconfig | 1 arch/x86/kernel/Makefile | 1 arch/x86/kernel/rethook.c | 115 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 117 insertions(+) create mode 100644 arch/x86/kernel/rethook.c
arch/x86/Kconfig +1 -0
@@ -219,6 +219,7 @@ config X86 select HAVE_KPROBES_ON_FTRACE select HAVE_FUNCTION_ERROR_INJECTION select HAVE_KRETPROBES + select HAVE_RETHOOK select HAVE_KVM select HAVE_LIVEPATCH if X86_64 select HAVE_MIXED_BREAKPOINTS_REGS
arch/x86/kernel/Makefile +1 -0
@@ -106,6 +106,7 @@ obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o obj-$(CONFIG_FTRACE_SYSCALLS) += ftrace.o obj-$(CONFIG_X86_TSC) += trace_clock.o obj-$(CONFIG_TRACING) += trace.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_CRASH_CORE) += crash_core_$(BITS).o obj-$(CONFIG_KEXEC_CORE) += machine_kexec_$(BITS).o obj-$(CONFIG_KEXEC_CORE) += relocate_kernel_$(BITS).o crash.o
arch/x86/kernel/rethook.c new file +115 -0
115 lines not on this page · raw message
Masami Hiramatsu
maintainerRFC PATCH v2 5/8 fprobe: Add exit_handler support
In reply to Masami HiramatsuAdd exit_handler to fprobe. fprobe + rethook allows us to hook the kernel function return without fgraph tracer. Eventually, the fgraph tracer will be generic array based return hooking and fprobe may use it if user requests. Since both array-based approach and list-based approach have Pros and Cons, (e.g. memory consumption v.s. less missing events) it is better to keep both but fprobe will provide the same exit-handler interface.
--- include/linux/fprobe.h | 4 +++ kernel/trace/Kconfig | 1 + kernel/trace/fprobe.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 63 insertions(+), 3 deletions(-)
include/linux/fprobe.h +4 -0
@@ -5,6 +5,7 @@#include <linux/compiler.h> #include <linux/ftrace.h> +#include <linux/rethook.h> struct fprobe {const char **syms;
@@ -14,7 +15,10 @@ struct fprobe {struct ftrace_ops ftrace;unsigned long nmissed;unsigned int flags; +struct rethook *rethook; +void (*entry_handler)(struct fprobe *fp,unsigned long entry_ip,struct pt_regs *regs); +void (*exit_handler)(struct fprobe *fp,unsigned long entry_ip,struct pt_regs *regs); };#define FPROBE_FL_DISABLED 1
kernel/trace/Kconfig +1 -0
@@ -238,6 +238,7 @@ config FPROBE bool "Kernel Function Probe (fprobe)" depends on FUNCTION_TRACER depends on DYNAMIC_FTRACE_WITH_REGS + select RETHOOK default n help This option enables kernel function probe feature, which is
kernel/trace/fprobe.c +58 -3
105 lines not on this page · raw message
Masami Hiramatsu
maintainerRFC PATCH v2 6/8 fprobe: Add sample program for fprobe
In reply to Masami HiramatsuAdd a sample program for the fprobe.
---
Changes in v2:
- Fix infinit loop for multiple symbols.
- Fix memory leaks for copied string and entry array.
- Update for new fprobe APIs.
- Fix style issues.
--- samples/Kconfig | 7 +++ samples/Makefile | 1 samples/fprobe/Makefile | 3 + samples/fprobe/fprobe_example.c | 103 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 114 insertions(+) create mode 100644 samples/fprobe/Makefile create mode 100644 samples/fprobe/fprobe_example.c
samples/Kconfig +7 -0
@@ -73,6 +73,13 @@ config SAMPLE_HW_BREAKPOINT help This builds kernel hardware breakpoint example modules. +config SAMPLE_FPROBE + tristate "Build fprobe examples -- loadable modules only" + depends on FPROBE && m + help + This builds a fprobe example module. This module has an option 'symbol'. + You can specify a probed symbol or symbols separated with ','. + config SAMPLE_KFIFO tristate "Build kfifo examples -- loadable modules only" depends on m
samples/Makefile +1 -0
@@ -32,3 +32,4 @@ obj-$(CONFIG_SAMPLE_INTEL_MEI) += mei/ subdir-$(CONFIG_SAMPLE_WATCHDOG) += watchdog subdir-$(CONFIG_SAMPLE_WATCH_QUEUE) += watch_queue obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak/ +obj-$(CONFIG_SAMPLE_FPROBE) += fprobe/
samples/fprobe/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only + +obj-$(CONFIG_SAMPLE_FPROBE) += fprobe_example.o
samples/fprobe/fprobe_example.c new file +103 -0
103 lines not on this page · raw message
Masami Hiramatsu
maintainerRFC PATCH v2 7/8 bpf: Add kprobe link for attaching raw kprobes
In reply to Masami HiramatsuFrom: Jiri Olsa <[email protected]>
Adding new link type BPF_LINK_TYPE_KPROBE to attach so called "kprobes" directly through fprobe API. Note that since the using kprobes with multiple same handler is not efficient, this uses the fprobe which natively support multiple probe points for one same handler, but limited on function entry and exit.
Adding new attach type BPF_TRACE_RAW_KPROBE that enables such link for kprobe program.
The new link allows to create multiple kprobes link by using new link_create interface:
struct {
__aligned_u64 addrs;
__u32 cnt;
__u64 bpf_cookie;
} kprobe;Plus new flag BPF_F_KPROBE_RETURN for link_create.flags to create return probe.
--- include/linux/bpf_types.h | 1 include/uapi/linux/bpf.h | 12 ++ kernel/bpf/syscall.c | 195 +++++++++++++++++++++++++++++++++++++++- tools/include/uapi/linux/bpf.h | 12 ++ 4 files changed, 215 insertions(+), 5 deletions(-)
include/linux/bpf_types.h +1 -0
@@ -140,3 +140,4 @@ BPF_LINK_TYPE(BPF_LINK_TYPE_XDP, xdp)#ifdef CONFIG_PERF_EVENTS BPF_LINK_TYPE(BPF_LINK_TYPE_PERF_EVENT, perf)#endif +BPF_LINK_TYPE (BPF_LINK_TYPE_KPROBE , kprobe)
include/uapi/linux/bpf.h +12 -0
@@ -995,6 +995,7 @@ enum bpf_attach_type {BPF_SK_REUSEPORT_SELECT ,BPF_SK_REUSEPORT_SELECT_OR_MIGRATE ,BPF_PERF_EVENT , +BPF_TRACE_RAW_KPROBE , __MAX_BPF_ATTACH_TYPE };
@@ -1009,6 +1010,7 @@ enum bpf_link_type {BPF_LINK_TYPE_NETNS =5 ,BPF_LINK_TYPE_XDP =6 ,BPF_LINK_TYPE_PERF_EVENT =7 , +BPF_LINK_TYPE_KPROBE =8 , MAX_BPF_LINK_TYPE, };
@@ -1111,6 +1113,11 @@ enum bpf_link_type {*/ #define BPF_F_SLEEPABLE (1U << 4) +/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE +* attach type. +*/ +#define BPF_F_KPROBE_RETURN (1U << 0) +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: *
@@ -1463,6 +1470,11 @@ union bpf_attr {*/ __u64 bpf_cookie; } perf_event; +struct { +__aligned_u64 addrs; +__u32 cnt; +__u64 bpf_cookie; + } kprobe; }; } link_create;
kernel/bpf/syscall.c +190 -5
237 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +12 -0
@@ -995,6 +995,7 @@ enum bpf_attach_type {BPF_SK_REUSEPORT_SELECT ,BPF_SK_REUSEPORT_SELECT_OR_MIGRATE ,BPF_PERF_EVENT , +BPF_TRACE_RAW_KPROBE , __MAX_BPF_ATTACH_TYPE };
@@ -1009,6 +1010,7 @@ enum bpf_link_type {BPF_LINK_TYPE_NETNS =5 ,BPF_LINK_TYPE_XDP =6 ,BPF_LINK_TYPE_PERF_EVENT =7 , +BPF_LINK_TYPE_KPROBE =8 , MAX_BPF_LINK_TYPE, };
@@ -1111,6 +1113,11 @@ enum bpf_link_type {*/ #define BPF_F_SLEEPABLE (1U << 4) +/* link_create flags used in LINK_CREATE command for BPF_TRACE_RAW_KPROBE +* attach type. +*/ +#define BPF_F_KPROBE_RETURN (1U << 0) +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: *
@@ -1463,6 +1470,11 @@ union bpf_attr {*/ __u64 bpf_cookie; } perf_event; +struct { +__aligned_u64 addrs; +__u32 cnt; +__u64 bpf_cookie; + } kprobe; }; } link_create;
Masami Hiramatsu
maintainerRFC PATCH v2 8/8 Out-of-tree: Support wildcard symbol option to sample
In reply to Masami HiramatsuThis is not intended to be merged to upstream code (since this expose some kernel internal functions just for an example.)
But this is good to show how the fprobe is time-efficient for registering a probe on thousands of functions.
# time insmod fprobe_example.ko symbol='btrfs_*' [ 36.130947] fprobe_init: 1028 symbols found [ 36.177901] fprobe_init: Planted fprobe at btrfs_* real 0m 0.08s user 0m 0.00s sys 0m 0.07s
--- kernel/kallsyms.c | 1 + kernel/trace/ftrace.c | 1 + samples/fprobe/fprobe_example.c | 69 ++++++++++++++++++++++++++++++++++----- 3 files changed, 62 insertions(+), 9 deletions(-)
kernel/kallsyms.c +1 -0
@@ -246,6 +246,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, }return 0 ; } +EXPORT_SYMBOL_GPL (kallsyms_on_each_symbol);#endif /* CONFIG_LIVEPATCH */ static unsigned long get_symbol_pos(unsigned long addr,
kernel/trace/ftrace.c +1 -0
@@ -1580,6 +1580,7 @@ unsigned long ftrace_location(unsigned long ip) {return ftrace_location_range (ip, ip); } +EXPORT_SYMBOL_GPL (ftrace_location);/** * ftrace_text_reserved - return true if range contains an ftrace location
samples/fprobe/fprobe_example.c +60 -9
113 lines not on this page · raw message
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 6 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
Quoted 8 lines
[1] https://lore.kernel.org/all/[email protected]/T/#u
I also added an out-of-tree (just for testing) patch at the end of this series ([8/8]) for adding a wildcard support to the sample program. With that patch, it shows how long the registration will take;
Quoted 6 lines
# time insmod fprobe_example.ko symbol='btrfs_*' [ 36.130947] fprobe_init: 1028 symbols found [ 36.177901] fprobe_init: Planted fprobe at btrfs_* real 0m 0.08s user 0m 0.00s sys 0m 0.07s
I'll run my bpftrace tests on top of that
thanks,
jirka
Quoted 53 lines
Thank you,
---
Jiri Olsa (2):
ftrace: Addftrace_set_filter_ipsfunction
bpf: Add kprobe link for attaching raw kprobesMasami Hiramatsu (6):
fprobe: Add ftrace based probe APIs
rethook: Add a generic return hook
rethook: x86: Add rethook x86 implementation
fprobe: Addexit_handlersupport
fprobe: Add sample program for fprobe
[DO NOT MERGE] Out-of-tree: Support wildcard symbol option to samplearch/x86/Kconfig | 1 arch/x86/kernel/Makefile | 1 arch/x86/kernel/rethook.c | 115 ++++++++++++++++++++ include/linux/bpf_types.h | 1 include/linux/fprobe.h | 57 ++++++++++ include/linux/ftrace.h | 3 + include/linux/rethook.h | 74 +++++++++++++ include/linux/sched.h | 3 + include/uapi/linux/bpf.h | 12 ++ kernel/bpf/syscall.c | 195 +++++++++++++++++++++++++++++++++- kernel/exit.c | 2 kernel/fork.c | 3 + kernel/kallsyms.c | 1 kernel/trace/Kconfig | 22 ++++ kernel/trace/Makefile | 2 kernel/trace/fprobe.c | 168 +++++++++++++++++++++++++++++ kernel/trace/ftrace.c | 54 ++++++++- kernel/trace/rethook.c | 226 +++++++++++++++++++++++++++++++++++++++ samples/Kconfig | 7 + samples/Makefile | 1 samples/fprobe/Makefile | 3 + samples/fprobe/fprobe_example.c | 154 +++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 12 ++ 23 files changed, 1103 insertions(+), 14 deletions(-) create mode 100644 arch/x86/kernel/rethook.c create mode 100644 include/linux/fprobe.h create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/fprobe.c create mode 100644 kernel/trace/rethook.c create mode 100644 samples/fprobe/Makefile create mode 100644 samples/fprobe/fprobe_example.c--
Masami Hiramatsu (Linaro) <[email protected]>
Re: [RFC PATCH v2 3/8] rethook: Add a generic return hook
In reply to Masami HiramatsuQuoted 8 lines · Masami Hiramatsu
Add a return hook framework which hooks the function return. Most of the idea came from the kretprobe, but this is independent from kretprobe.
Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just a additional hook.
this looks similar to the code kretprobe is using now
would it make sense to incrementaly change current code to provide this rethook interface? instead of big switch of current kretprobe to kprobe + new rethook interface in future?
thanks,
jirka
Quoted 409 lines
Signed-off-by Masami Hiramatsu <[email protected]>--- include/linux/rethook.h | 74 +++++++++++++++ include/linux/sched.h | 3 + kernel/exit.c | 2 kernel/fork.c | 3 + kernel/trace/Kconfig | 11 ++ kernel/trace/Makefile | 1 kernel/trace/rethook.c | 226 +++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 320 insertions(+) create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/rethook.cinclude/linux/rethook.h new file +74
-0@@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Return hooking with list-based shadow stack. +*/ +#ifndef _LINUX_RETHOOK_H +#define _LINUX_RETHOOK_H + +#include <linux/compiler.h> +#include <linux/freelist.h> +#include <linux/llist.h> +#include <linux/rcupdate.h> +#include <linux/refcount.h> + +struct rethook_node ; + +typedef void (*rethook_handler_t ) (struct rethook_node *,void *,struct pt_regs *); + +struct rethook { +void *data; +rethook_handler_t handler; +struct freelist_head pool; +refcount_t ref; +struct rcu_head rcu; +}; + +struct rethook_node { +union { +struct freelist_node freelist; +struct rcu_head rcu; + }; +struct llist_node llist; +struct rethook *rethook; +unsigned long ret_addr; +unsigned long frame; +}; + +int rethook_node_init (struct rethook_node *node); + +struct rethook *rethook_alloc (void *data,rethook_handler_t handler); +void rethook_free (struct rethook *rh); +void rethook_add_node (struct rethook *rh,struct rethook_node *node); + +struct rethook_node *rethook_try_get (struct rethook *rh); +void rethook_node_recycle (struct rethook_node *node); +void rethook_hook_current (struct rethook_node *node,struct pt_regs *regs); + +unsigned long rethook_find_ret_addr (struct task_struct *tsk,unsigned long frame, +struct llist_node **cur); + +/* Arch dependent code must implement this and trampoline code */ +void arch_rethook_prepare (struct rethook_node *node,struct pt_regs *regs); +void arch_rethook_trampoline (void ); + +static inline bool is_rethook_trampoline (unsigned long addr) +{ +return addr == (unsigned long )arch_rethook_trampoline; +} + +/* If the architecture needs a fixup the return address, implement it. */ +void arch_rethook_fixup_return (struct pt_regs *regs, +unsigned long correct_ret_addr); + +/* Generic trampoline handler, arch code must prepare asm stub */ +unsigned long rethook_trampoline_handler (struct pt_regs *regs, +unsigned long frame); + +#ifdef CONFIG_RETHOOK +void rethook_flush_task (struct task_struct *tk); +#else +#define rethook_flush_task (tsk) do { } while (0) +#endif + +#endif +include/linux/sched.h +3
-0@@ -1473,6 +1473,9 @@ struct task_struct {#ifdef CONFIG_KRETPROBES struct llist_head kretprobe_instances;#endif +#ifdef CONFIG_RETHOOK +struct llist_head rethooks; +#endif #ifdef CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH /* kernel/exit.c +2
-0@@ -64,6 +64,7 @@#include <linux/compat.h> #include <linux/io_uring.h> #include <linux/kprobes.h> +#include <linux/rethook.h> #include <linux/uaccess.h> #include <asm/unistd.h> @@ -169,6 +170,7 @@ static void delayed_put_task_struct(struct rcu_head *rhp) struct task_struct *tsk = container_of(rhp, struct task_struct, rcu);kprobe_flush_task (tsk); +rethook_flush_task (tsk);perf_event_delayed_put (tsk);trace_sched_process_free (tsk);put_task_struct (tsk);kernel/fork.c +3
-0@@ -2282,6 +2282,9 @@ static __latent_entropy struct task_struct *copy_process(#ifdef CONFIG_KRETPROBES p->kretprobe_instances.first =NULL ;#endif +#ifdef CONFIG_RETHOOK + p->rethooks.first =NULL ; +#endif /* * Ensure that the cgroup subsystem policies allow the new process to be kernel/trace/Kconfig +11
-0@@ -10,6 +10,17 @@ config USER_STACKTRACE_SUPPORT config NOP_TRACER bool +config HAVE_RETHOOK + bool + +config RETHOOK + bool + depends on HAVE_RETHOOK + help + Enable generic return hooking feature. This is an internal + API, which will be used by other function-entry hooking + feature like fprobe and kprobes. + config HAVE_FUNCTION_TRACER bool helpkernel/trace/Makefile +1
-0@@ -98,6 +98,7 @@ obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o obj-$(CONFIG_FPROBE) += fprobe.o +obj-$(CONFIG_RETHOOK) += rethook.o obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.okernel/trace/rethook.c new file +226
-0@@ -0,0 +1,226 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define pr_fmt (fmt) "rethook: " fmt + +#include <linux/bug.h> +#include <linux/kallsyms.h> +#include <linux/kprobes.h> +#include <linux/preempt.h> +#include <linux/rethook.h> +#include <linux/slab.h> +#include <linux/sort.h> + +/* Return hook list (shadow stack by list) */ + +void rethook_flush_task (struct task_struct *tk) +{ +struct rethook_node *rhn; +struct llist_node *node; + +preempt_disable (); + + node =__llist_del_all (&tk->rethooks); +while (node) { + rhn = container_of(node, struct rethook_node, llist); + node = node->next; +rethook_node_recycle (rhn); + } + +preempt_enable (); +} + +static void rethook_free_rcu (struct rcu_head *head) +{ + struct rethook *rh = container_of(head, struct rethook, rcu); +struct rethook_node *rhn; +struct freelist_node *node; +int count =1 ; + + node = rh->pool.head; +while (node) { + rhn = container_of(node, struct rethook_node, freelist); + node = node->next; +kfree (rhn); + count++; + } + +/* The rh->ref is the number of pooled node + 1 */ +if (refcount_sub_and_test (count, &rh->ref)) +kfree (rh); +} + +void rethook_free (struct rethook *rh) +{ + rh->handler =NULL ; + rh->data =NULL ; + +call_rcu (&rh->rcu, rethook_free_rcu); +} + +/* +* @handler must not NULL. @handler == NULL means this rethook is +* going to be freed. +*/ +struct rethook *rethook_alloc (void *data,rethook_handler_t handler) +{ +struct rethook *rh =kzalloc (sizeof (struct rethook ),GFP_KERNEL ); + +if (!rh || !handler) +return NULL ; + + rh->data = data; + rh->handler = handler; + rh->pool.head =NULL ; +refcount_set (&rh->ref,1 ); + +return rh; +} + +void rethook_add_node (struct rethook *rh,struct rethook_node *node) +{ + node->rethook = rh; +freelist_add (&node->freelist, &rh->pool); +refcount_inc (&rh->ref); +} + +static void free_rethook_node_rcu (struct rcu_head *head) +{ + struct rethook_node *node = container_of(head, struct rethook_node, rcu); + +if (refcount_dec_and_test (&node->rethook->ref)) +kfree (node->rethook); +kfree (node); +} + +void rethook_node_recycle (struct rethook_node *node) +{ +if (likely (READ_ONCE (node->rethook->handler))) +freelist_add (&node->freelist, &node->rethook->pool); +else +call_rcu (&node->rcu, free_rethook_node_rcu); +} + +struct rethook_node *rethook_try_get (struct rethook *rh) +{ +struct freelist_node *fn; + +/* Check whether @rh is going to be freed. */ +if (unlikely (!READ_ONCE (rh->handler))) +return NULL ; + + fn =freelist_try_get (&rh->pool); +if (!fn) +return NULL ; + + return container_of(fn, struct rethook_node, freelist); +} + +void rethook_hook_current (struct rethook_node *node,struct pt_regs *regs) +{ +arch_rethook_prepare (node, regs); +__llist_add (&node->llist, ¤t->rethooks); +} + +/* This assumes the 'tsk' is the current task or the is not running. */ +static unsigned long __rethook_find_ret_addr (struct task_struct *tsk, +struct llist_node **cur) +{ +struct rethook_node *rh =NULL ; +struct llist_node *node = *cur; + +if (!node) + node = tsk->rethooks.first; +else + node = node->next; + +while (node) { + rh = container_of(node, struct rethook_node, llist); +if (rh->ret_addr != (unsigned long )arch_rethook_trampoline) { + *cur = node; +return rh->ret_addr; + } + node = node->next; + } +return 0 ; +} +NOKPROBE_SYMBOL (__rethook_find_ret_addr); + +/** +* rethook_find_ret_addr -- Find correct return address modified by rethook +* @tsk: Target task +* @frame: A frame pointer +* @cur: a storage of the loop cursor llist_node pointer for next call +* +* Find the correct return address modified by a rethook on @tsk in unsigned +* long type. If it finds the return address, this returns that address value, +* or this returns 0. +* The @tsk must be 'current' or a task which is not running. @frame is a hint +* to get the currect return address - which is compared with the +* rethook::frame field. The @cur is a loop cursor for searching the +* kretprobe return addresses on the @tsk. The '*@cur' should be NULL at the +* first call, but '@cur' itself must NOT NULL. +*/ +unsigned long rethook_find_ret_addr (struct task_struct *tsk,unsigned long frame, +struct llist_node **cur) +{ +struct rethook_node *rhn =NULL ; +unsigned long ret; + +if (WARN_ON_ONCE (!cur)) +return 0 ; + +do { + ret =__rethook_find_ret_addr (tsk, cur); +if (!ret) +break ; + rhn = container_of(*cur, struct rethook_node, llist); + }while (rhn->frame != frame); + +return ret; +} +NOKPROBE_SYMBOL (rethook_find_ret_addr); + +void __weak arch_rethook_fixup_return (struct pt_regs *regs, +unsigned long correct_ret_addr) +{ +/* +* Do nothing by default. If the architecture which uses a +* frame pointer to record real return address on the stack, +* it should fill this function to fixup the return address +* so that stacktrace works from the rethook handler. +*/ +} + +unsigned long rethook_trampoline_handler (struct pt_regs *regs, +unsigned long frame) +{ +struct rethook_node *rhn; +struct llist_node *first, *node =NULL ; +unsigned long correct_ret_addr =__rethook_find_ret_addr (current, &node); + +if (!correct_ret_addr) { +pr_err ("rethook: Return address not found! Maybe there is a bug in the kernel\n" ); +BUG_ON (1 ); + } + +instruction_pointer_set (regs, correct_ret_addr); +arch_rethook_fixup_return (regs, correct_ret_addr); + + first = current->rethooks.first; + current->rethooks.first = node->next; + node->next =NULL ; + +while (first) { + rhn = container_of(first, struct rethook_node, llist); +if (WARN_ON_ONCE (rhn->frame != frame)) +break ; +if (rhn->rethook->handler) + rhn->rethook->handler (rhn, rhn->rethook->data, regs); + + first = first->next; +rethook_node_recycle (rhn); + } + +return correct_ret_addr; +} +
Quoted 15 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 8 lines
Quoted 6 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
jirka
Masami Hiramatsu
maintainerRe: [RFC PATCH v2 3/8] rethook: Add a generic return hook
In reply to Jiri OlsaQuoted 11 lines · On Thu, 13 Jan 2022 13:25:52 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 8 lines · Masami Hiramatsu
Add a return hook framework which hooks the function return. Most of the idea came from the kretprobe, but this is independent from kretprobe.
Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just a additional hook.this looks similar to the code kretprobe is using now
Yes, I've mostly re-typed the code :)
Quoted 3 lines
would it make sense to incrementaly change current code to provide this rethook interface? instead of big switch of current kretprobe to kprobe + new rethook interface in future?
Would you mean modifying the kretprobe instance code to provide similar one, and rename it at some point?
My original idea is to keep the current kretprobe code and build up the similar one, and switch to it at some point. Actually, I don't want to change the current kretprobe interface itself, but the backend will be changed. For example, current kretprobe has below interface.
struct kretprobe {
struct kprobe kp;
kretprobe_handler_t handler;
kretprobe_handler_t entry_handler;
int maxactive;
int nmissed;
size_t data_size;
struct freelist_head freelist;
struct kretprobe_holder *rph;
};My idea is switching it to below.
struct kretprobe {
struct kprobe kp;
kretprobe_handler_t handler;
kretprobe_handler_t entry_handler;
int maxactive;
int nmissed;
size_t data_size;
struct rethook *rethook;
};Of course 'kretprobe_instance' may need to be changed...
struct kretprobe_instance {
struct rethook_node;
char data[];
};But even though, since there is 'get_kretprobe(ri)' wrapper, user will be able to access the 'struct kretprobe' from kretprobe_instance transparently.
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerQuoted 20 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 19 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 4 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
Hmm, would you mean tools/testing/selftests/bpf/prog_tests/ ? Let me check it too.
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerHi Jiri and Alexei,
Quoted 20 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 19 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 4 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
I've tried to build tools/testing/selftests/bpf on my machine, but I got below errors. Would you know how I can setup to build the bpf selftests correctly? (I tried "make M=samples/bpf", but same result)
~/ksrc/linux/tools/testing/selftests/bpf$ make
[...]
CLANG /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool/pid_iter.bpf.oskeleton/pid_iter.bpf.c:35:10: error: incomplete definition of type 'struct bpf_link'
return BPF_CORE_READ((struct bpf_link *)ent, id);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:403:2: note: expanded from macro 'BPF_CORE_READ'
___type((src), a, ##__VA_ARGS__) __r; \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:274:29: note: expanded from macro '___type'
#define ___type(...) typeof(___arrow(__VA_ARGS__))
^~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:272:23: note: expanded from macro '___arrow'
#define ___arrow(...) ___apply(___arrow, ___narg(__VA_ARGS__))(__VA_ARGS__)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:223:25: note: expanded from macro '___concat'
#define ___concat(a, b) a ## b
^<scratch space>:16:1: note: expanded from here___arrow2
^/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:263:26: note: expanded from macro '___arrow2'
#define ___arrow2(a, b) a->b
~^
skeleton/pid_iter.bpf.c:35:32: note: forward declaration of 'struct bpf_link'
return BPF_CORE_READ((struct bpf_link *)ent, id);
^
skeleton/pid_iter.bpf.c:35:10: error: incomplete definition of type 'struct bpf_link'
return BPF_CORE_READ((struct bpf_link *)ent, id);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:404:2: note: expanded from macro 'BPF_CORE_READ'
BPF_CORE_READ_INTO(&__r, (src), a, ##__VA_ARGS__); \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:311:2: note: expanded from macro 'BPF_CORE_READ_INTO'
___core_read(bpf_core_read, bpf_core_read, \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:302:2: note: expanded from macro '___core_read'
___apply(___core_read, ___empty(__VA_ARGS__))(fn, fn_ptr, dst, \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:296:2: note: expanded from macro '___core_read0'
___read(fn, dst, ___type(src), src, a);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:277:59: note: expanded from macro '___read'
read_fn((void *)(dst), sizeof(*(dst)), &((src_type)(src))->accessor)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:206:79: note: expanded from macro 'bpf_core_read'
bpf_probe_read_kernel(dst, sz, (const void *)__builtin_preserve_access_index(src))
^~~skeleton/pid_iter.bpf.c:35:32: note: forward declaration of 'struct bpf_link'
return BPF_CORE_READ((struct bpf_link *)ent, id);
^
skeleton/pid_iter.bpf.c:35:10: error: returning 'void' from a function with incompatible result type '__u32' (aka 'unsigned int')
return BPF_CORE_READ((struct bpf_link *)ent, id);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:402:36: note: expanded from macro 'BPF_CORE_READ'
#define BPF_CORE_READ(src, a, ...) ({ \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
skeleton/pid_iter.bpf.c:42:17: warning: declaration of 'struct bpf_iter__task_file' will not be visible outside of this function [-Wvisibility]
int iter(struct bpf_iter__task_file *ctx)
^
skeleton/pid_iter.bpf.c:44:25: error: incomplete definition of type 'struct bpf_iter__task_file'
struct file *file = ctx->file;
~~~^
skeleton/pid_iter.bpf.c:42:17: note: forward declaration of 'struct bpf_iter__task_file'
int iter(struct bpf_iter__task_file *ctx)
^
skeleton/pid_iter.bpf.c:45:32: error: incomplete definition of type 'struct bpf_iter__task_file'
struct task_struct *task = ctx->task;
~~~^
skeleton/pid_iter.bpf.c:42:17: note: forward declaration of 'struct bpf_iter__task_file'
int iter(struct bpf_iter__task_file *ctx)
^
skeleton/pid_iter.bpf.c:76:19: error: incomplete definition of type 'struct bpf_iter__task_file'
bpf_seq_write(ctx->meta->seq, &e, sizeof(e));
~~~^
skeleton/pid_iter.bpf.c:42:17: note: forward declaration of 'struct bpf_iter__task_file'
int iter(struct bpf_iter__task_file *ctx)
^1 warning and 6 errors generated.
make[1]: *** [Makefile:188: /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool/pid_iter.bpf.o] Error 1
make: *** [Makefile:219: /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/sbin/bpftool] Error 2
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 28 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Quoted 23 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 22 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 6 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
I'll try the bpf selftests on top of this
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
I've tried to build
tools/testing/selftests/bpfon my machine, but I got below errors. Would you know how I can setup to build the bpf selftests correctly? (I tried "make M=samples/bpf", but same result)
what's your clang version? your distro might be behind, I'm using clang 14 compiled from sources:
$ /opt/clang/bin/clang --version clang version 14.0.0 (https://github.com/llvm/llvm-project.git 9f8ffaaa0bddcefeec15a3df9858fd50b05fcbae) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /opt/clang/bin
and compiling bpf selftests with:
$ CLANG=/opt/clang/bin/clang make
jirka
Quoted 91 lines
~/ksrc/linux/tools/testing/selftests/bpf$ make
[...]
CLANG/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool/pid_iter.bpf.oskeleton/pid_iter.bpf.c:35:10: error: incomplete definition of type 'structbpf_link'return BPF_CORE_READ((struct bpf_link *)ent, id); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:403:2: note: expanded from macro 'BPF_CORE_READ' ___type((src), a, ##__VA_ARGS__) __r; \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:274:29: note: expanded from macro '___type' #define ___type(...) typeof(___arrow(__VA_ARGS__)) ^~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:272:23: note: expanded from macro '___arrow' #define ___arrow(...) ___apply(___arrow, ___narg(__VA_ARGS__))(__VA_ARGS__) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ note: (skipping 1 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:223:25: note: expanded from macro '___concat' #define ___concat(a, b) a ## b ^<scratch space>:16:1: note: expanded from here
___arrow2
^/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:263:26: note: expanded from macro '___arrow2'#define ___arrow2(a, b) a->b ~^ skeleton/pid_iter.bpf.c:35:32: note: forward declaration of 'struct bpf_link' return BPF_CORE_READ((struct bpf_link *)ent, id); ^ skeleton/pid_iter.bpf.c:35:10: error: incomplete definition of type 'struct bpf_link' return BPF_CORE_READ((struct bpf_link *)ent, id); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:404:2: note: expanded from macro 'BPF_CORE_READ' BPF_CORE_READ_INTO(&__r, (src), a, ##__VA_ARGS__); \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:311:2: note: expanded from macro 'BPF_CORE_READ_INTO' ___core_read(bpf_core_read, bpf_core_read, \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:302:2: note: expanded from macro '___core_read' ___apply(___core_read, ___empty(__VA_ARGS__))(fn, fn_ptr, dst, \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all) /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:296:2: note: expanded from macro '___core_read0' ___read(fn, dst, ___type(src), src, a); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:277:59: note: expanded from macro '___read' read_fn((void *)(dst), sizeof(*(dst)), &((src_type)(src))->accessor) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~
/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:206:79: note: expanded from macro 'bpf_core_read'
bpf_probe_read_kernel(dst, sz, (const void *)__builtin_preserve_access_index(src))
^~~skeleton/pid_iter.bpf.c:35:32: note: forward declaration of 'structbpf_link'return BPF_CORE_READ((struct bpf_link *)ent, id); ^ skeleton/pid_iter.bpf.c:35:10: error: returning 'void' from a function with incompatible result type '__u32' (aka 'unsigned int') return BPF_CORE_READ((struct bpf_link *)ent, id); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool//bootstrap/libbpf//include/bpf/bpf_core_read.h:402:36: note: expanded from macro 'BPF_CORE_READ' #define BPF_CORE_READ(src, a, ...) ({ \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ skeleton/pid_iter.bpf.c:42:17: warning: declaration of 'struct bpf_iter__task_file' will not be visible outside of this function [-Wvisibility] int iter(struct bpf_iter__task_file *ctx) ^ skeleton/pid_iter.bpf.c:44:25: error: incomplete definition of type 'struct bpf_iter__task_file' struct file *file = ctx->file; ~~~^ skeleton/pid_iter.bpf.c:42:17: note: forward declaration of 'struct bpf_iter__task_file' int iter(struct bpf_iter__task_file *ctx) ^ skeleton/pid_iter.bpf.c:45:32: error: incomplete definition of type 'struct bpf_iter__task_file' struct task_struct *task = ctx->task; ~~~^ skeleton/pid_iter.bpf.c:42:17: note: forward declaration of 'struct bpf_iter__task_file' int iter(struct bpf_iter__task_file *ctx) ^ skeleton/pid_iter.bpf.c:76:19: error: incomplete definition of type 'struct bpf_iter__task_file' bpf_seq_write(ctx->meta->seq, &e, sizeof(e)); ~~~^ skeleton/pid_iter.bpf.c:42:17: note: forward declaration of 'struct bpf_iter__task_file' int iter(struct bpf_iter__task_file *ctx) ^1 warning and 6 errors generated.
make[1]: *** [Makefile:188:/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/build/bpftool/pid_iter.bpf.o] Error 1
make: *** [Makefile:219:/home/mhiramat/ksrc/linux/tools/testing/selftests/bpf/tools/sbin/bpftool] Error 2Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Re: [RFC PATCH v2 3/8] rethook: Add a generic return hook
In reply to Masami HiramatsuQuoted 42 lines · Masami Hiramatsu
Quoted 11 lines · On Thu, 13 Jan 2022 13:25:52 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 8 lines · Masami Hiramatsu
Add a return hook framework which hooks the function return. Most of the idea came from the kretprobe, but this is independent from kretprobe.
Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just a additional hook.this looks similar to the code kretprobe is using now
Yes, I've mostly re-typed the code :)
Quoted 3 lines
would it make sense to incrementaly change current code to provide this rethook interface? instead of big switch of current kretprobe to kprobe + new rethook interface in future?
Would you mean modifying the kretprobe instance code to provide similar one, and rename it at some point?
My original idea is to keep the current kretprobe code and build up the similar one, and switch to it at some point. Actually, I don't want to change the current kretprobe interface itself, but the backend will be changed. For example, current kretprobe has below interface.struct kretprobe { struct kprobe kp; kretprobe_handler_t handler; kretprobe_handler_t entry_handler; int maxactive; int nmissed; size_t data_size; struct freelist_head freelist; struct kretprobe_holder *rph; };My idea is switching it to below.
Quoted 9 lines
struct kretprobe {
struct kprobe kp;
kretprobe_handler_t handler;
kretprobe_handler_t entry_handler;
int maxactive;
int nmissed;
size_t data_size;
struct rethook *rethook;
};looks good, will this be a lot of changes?
could you include it in the patchset?
thanks,
jirka
Quoted 18 lines
Of course '
kretprobe_instance' may need to be changed...struct kretprobe_instance { struct rethook_node; char data[]; };But even though, since there is '
get_kretprobe(ri)' wrapper, user will be able to access the 'struct kretprobe' fromkretprobe_instancetransparently.Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 32 lines · Jiri Olsa
Quoted 30 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Quoted 25 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 23 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 6 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
I've tried to build
tools/testing/selftests/bpfon my machine, but I got below errors. Would you know how I can setup to build the bpf selftests correctly? (I tried "make M=samples/bpf", but same result)what's your clang version? your distro might be behind,
If you have very recent Clang, decently recent pahole, and qemu, try using vmtest.sh. That should build the kernel with all the necessary kernel config options and start qemu image with that latest image and build selftests. And even run selftests automatically.
Quoted 16 lines
I'm using clang 14 compiled from sources:
$ /opt/clang/bin/clang --version clang version 14.0.0 (https://github.com/llvm/llvm-project.git 9f8ffaaa0bddcefeec15a3df9858fd50b05fcbae) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /opt/clang/binand compiling bpf selftests with:
$ CLANG=/opt/clang/bin/clang makejirka
Quoted 1 line
[...]
Quoted 7 lines
Quoted 6 lines
Thank you,
--
Masami Hiramatsu <[email protected]>
Quoted 7 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.
This per-probe private data is necessary for the feature called BPF cookie, in which each attachment has a unique user-provided u64 value associated to it, accessible at runtime through bpf_get_attach_cookie() helper. One way or another we'll need to support this to make these multi-attach BPF programs really useful for generic tracing applications.
Jiri,
We've discussed with Alexei this week how cookies can be supported for multi-attach fentry (where it seems even more challenging than in kprobe case), and agreed on rather simple solution, which roughly goes like this. When multi-attaching either fentry/fexit program, save sorted array of IP addresses and then sorted in the same order as IPs list of u64 cookies. At runtime, bpf_get_attach_cookie() helper should somehow get access to these two arrays and functions IP (that we already have with bpf_get_func_ip()), perform binary search and locate necessary cookie. This offloads the overhead of finding this cookie to actual call site of bpf_get_attach_cookie() (and it's a log(N), so not bad at all, especially if BPF program can be optimized to call this helper just once).
I think something like that should be doable for Masami's fprobe-based multi-attach kprobes, right? That would allow to have super-fast attachment, but still support BPF cookie per each individual IP/kernel function attachment. I haven't looked at code thoroughly, though, please let me know if I'm missing something fundamental.
Quoted 76 lines
This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
[1] https://lore.kernel.org/all/[email protected]/T/#u
I also added an out-of-tree (just for testing) patch at the end of this series ([8/8]) for adding a wildcard support to the sample program. With that patch, it shows how long the registration will take;
# time insmod fprobe_example.ko symbol='btrfs_*' [ 36.130947] fprobe_init: 1028 symbols found [ 36.177901] fprobe_init: Planted fprobe at btrfs_* real 0m 0.08s user 0m 0.00s sys 0m 0.07sThank you,
---
Jiri Olsa (2):
ftrace: Addftrace_set_filter_ipsfunction
bpf: Add kprobe link for attaching raw kprobesMasami Hiramatsu (6):
fprobe: Add ftrace based probe APIs
rethook: Add a generic return hook
rethook: x86: Add rethook x86 implementation
fprobe: Addexit_handlersupport
fprobe: Add sample program for fprobe
[DO NOT MERGE] Out-of-tree: Support wildcard symbol option to samplearch/x86/Kconfig | 1 arch/x86/kernel/Makefile | 1 arch/x86/kernel/rethook.c | 115 ++++++++++++++++++++ include/linux/bpf_types.h | 1 include/linux/fprobe.h | 57 ++++++++++ include/linux/ftrace.h | 3 + include/linux/rethook.h | 74 +++++++++++++ include/linux/sched.h | 3 + include/uapi/linux/bpf.h | 12 ++ kernel/bpf/syscall.c | 195 +++++++++++++++++++++++++++++++++- kernel/exit.c | 2 kernel/fork.c | 3 + kernel/kallsyms.c | 1 kernel/trace/Kconfig | 22 ++++ kernel/trace/Makefile | 2 kernel/trace/fprobe.c | 168 +++++++++++++++++++++++++++++ kernel/trace/ftrace.c | 54 ++++++++- kernel/trace/rethook.c | 226 +++++++++++++++++++++++++++++++++++++++ samples/Kconfig | 7 + samples/Makefile | 1 samples/fprobe/Makefile | 3 + samples/fprobe/fprobe_example.c | 154 +++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 12 ++ 23 files changed, 1103 insertions(+), 14 deletions(-) create mode 100644 arch/x86/kernel/rethook.c create mode 100644 include/linux/fprobe.h create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/fprobe.c create mode 100644 kernel/trace/rethook.c create mode 100644 samples/fprobe/Makefile create mode 100644 samples/fprobe/fprobe_example.c--
Masami Hiramatsu (Linaro) <[email protected]>
Masami Hiramatsu
maintainerHi Andrii,
Quoted 33 lines · On Fri, 14 Jan 2022 17:02:31 -0800 Andrii Nakryiko <[email protected]>…
Quoted 32 lines · Jiri Olsa
Quoted 30 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Quoted 25 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 23 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 6 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
I've tried to build
tools/testing/selftests/bpfon my machine, but I got below errors. Would you know how I can setup to build the bpf selftests correctly? (I tried "make M=samples/bpf", but same result)what's your clang version? your distro might be behind,
I'm using clang 13.0.0.
$ clang -v clang version 13.0.0 (https://github.com/llvm/llvm-project.git d7b669b3a30345cfcdb2fde2af6f48aa4b94845d) Target: x86_64-unknown-linux-gnu
Quoted 5 lines
If you have very recent Clang, decently recent pahole, and qemu, try using
vmtest.sh. That should build the kernel with all the necessary kernel config options and start qemu image with that latest image and build selftests. And even run selftests automatically.
OK, vmtest.sh works! :)
So I got the vmtest.sh runs out with some failures. Jiri, did you talked about these failures, or real crash?
Summary: 212/1033 PASSED, 12 SKIPPED, 14 FAILED
Thanks!
Quoted 27 lines
Quoted 17 lines
I'm using clang 14 compiled from sources:
$ /opt/clang/bin/clang --version clang version 14.0.0 (https://github.com/llvm/llvm-project.git 9f8ffaaa0bddcefeec15a3df9858fd50b05fcbae) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /opt/clang/binand compiling bpf selftests with:
$ CLANG=/opt/clang/bin/clang makejirka
Quoted 1 line
[...]
Quoted 7 lines
Quoted 6 lines
Thank you,
--
Masami Hiramatsu <[email protected]>
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerRe: [RFC PATCH v2 3/8] rethook: Add a generic return hook
In reply to Jiri OlsaQuoted 53 lines · On Fri, 14 Jan 2022 16:18:13 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 42 lines · Masami Hiramatsu
Quoted 11 lines · On Thu, 13 Jan 2022 13:25:52 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 8 lines · Masami Hiramatsu
Add a return hook framework which hooks the function return. Most of the idea came from the kretprobe, but this is independent from kretprobe.
Note that this is expected to be used with other function entry hooking feature, like ftrace, fprobe, adn kprobes. Eventually this will replace the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment, this is just a additional hook.this looks similar to the code kretprobe is using now
Yes, I've mostly re-typed the code :)
Quoted 3 lines
would it make sense to incrementaly change current code to provide this rethook interface? instead of big switch of current kretprobe to kprobe + new rethook interface in future?
Would you mean modifying the kretprobe instance code to provide similar one, and rename it at some point?
My original idea is to keep the current kretprobe code and build up the similar one, and switch to it at some point. Actually, I don't want to change the current kretprobe interface itself, but the backend will be changed. For example, current kretprobe has below interface.struct kretprobe { struct kprobe kp; kretprobe_handler_t handler; kretprobe_handler_t entry_handler; int maxactive; int nmissed; size_t data_size; struct freelist_head freelist; struct kretprobe_holder *rph; };My idea is switching it to below.
Quoted 9 lines
struct kretprobe { struct kprobe kp; kretprobe_handler_t handler; kretprobe_handler_t entry_handler; int maxactive; int nmissed; size_t data_size; struct rethook *rethook; };
Quoted 1 line
looks good, will this be a lot of changes?
Yes and no, we can easily replace the kretprobe generic trampoline callback (since it almost same, and have same feature), but it also needs to update per-arch kretprobe trampoline to rethook trampoline.
Quoted 1 line
could you include it in the patchset?
Let me try, but since it involves many archs (which support kretprobes) it may take a time to be merged.
Thank you,
Quoted 23 lines
thanks,
jirkaQuoted 18 lines
Of course '
kretprobe_instance' may need to be changed...struct kretprobe_instance { struct rethook_node; char data[]; };But even though, since there is '
get_kretprobe(ri)' wrapper, user will be able to access the 'struct kretprobe' fromkretprobe_instancetransparently.Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerQuoted 20 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 19 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 4 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
OK, I got a kernel stall. I missed to enable CONFIG_FPROBE. I think vmtest.sh should support menuconfig option.
#6 bind_perm:OK
#7 bloom_filter_map:OK
[ 107.282403] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'tsc' as unstable because the skew is too large: [ 107.283240] clocksource: 'hpet' wd_nsec: 496216090 wd_now: 7ddc7120 wd_last: 7ae746b7 mask: ffffffff [ 107.284045] clocksource: 'tsc' cs_nsec: 495996979 cs_now: 31fdb69b39 cs_last: 31c2d29219 mask: ffffffffffffffff [ 107.284926] clocksource: 'tsc' is current clocksource. [ 107.285487] tsc: Marking TSC unstable due to clocksource watchdog [ 107.285973] TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. [ 107.286616] sched_clock: Marking unstable (107240582544, 45390230)<-(107291410145, -5437339) [ 107.290408] clocksource: Not enough CPUs to check clocksource 'tsc'. [ 107.290879] clocksource: Switched to clocksource hpet [ 604.210415] INFO: rcu_tasks detected stalls on tasks: [ 604.210830] (____ptrval____): .. nvcsw: 86/86 holdout: 1 idle_cpu: -1/0 [ 604.211314] task:test_progs state:R running task stack: 0 pid: 87 ppid: 85 flags:0x00004000 [ 604.212058] Call Trace: [ 604.212246] <TASK> [ 604.212452] __schedule+0x362/0xbb0 [ 604.212723] ? preempt_schedule_notrace_thunk+0x16/0x18 [ 604.213107] preempt_schedule_notrace+0x48/0x80 [ 604.217403] ? asm_sysvec_apic_timer_interrupt+0x12/0x20 [ 604.217790] ? ftrace_regs_call+0xd/0x52 [ 604.218087] ? bpf_test_finish.isra.0+0x190/0x190 [ 604.218461] ? bpf_fentry_test1+0x5/0x10 [ 604.218750] ? trace_clock_x86_tsc+0x10/0x10 [ 604.219064] ? __sys_bpf+0x8b1/0x2970 [ 604.219337] ? lock_is_held_type+0xd7/0x130 [ 604.219680] ? __x64_sys_bpf+0x1c/0x20 [ 604.219957] ? do_syscall_64+0x35/0x80 [ 604.220237] ? entry_SYSCALL_64_after_hwframe+0x44/0xae [ 604.220653] </TASK>
Jiri, is that what you had seen?
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 65 lines · Masami Hiramatsu
Quoted 20 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 19 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 4 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
OK, I got a kernel stall. I missed to enable
CONFIG_FPROBE. I thinkvmtest.shshould support menuconfig option.#6 bind_perm:OK
#7 bloom_filter_map:OK[ 107.282403] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'tsc' as unstable because the skew is too large: [ 107.283240] clocksource: 'hpet' wd_nsec: 496216090 wd_now: 7ddc7120 wd_last: 7ae746b7 mask: ffffffff [ 107.284045] clocksource: 'tsc' cs_nsec: 495996979 cs_now: 31fdb69b39 cs_last: 31c2d29219 mask: ffffffffffffffff [ 107.284926] clocksource: 'tsc' is current clocksource. [ 107.285487] tsc: Marking TSC unstable due to clocksource watchdog [ 107.285973] TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. [ 107.286616] sched_clock: Marking unstable (107240582544, 45390230)<-(107291410145, -5437339) [ 107.290408] clocksource: Not enough CPUs to check clocksource 'tsc'. [ 107.290879] clocksource: Switched to clocksource hpet [ 604.210415] INFO: rcu_tasks detected stalls on tasks: [ 604.210830] (____ptrval____): .. nvcsw: 86/86 holdout: 1 idle_cpu: -1/0 [ 604.211314] task:test_progs state:R running task stack: 0 pid: 87 ppid: 85 flags:0x00004000 [ 604.212058] Call Trace: [ 604.212246] <TASK> [ 604.212452] __schedule+0x362/0xbb0 [ 604.212723] ? preempt_schedule_notrace_thunk+0x16/0x18 [ 604.213107] preempt_schedule_notrace+0x48/0x80 [ 604.217403] ? asm_sysvec_apic_timer_interrupt+0x12/0x20 [ 604.217790] ? ftrace_regs_call+0xd/0x52 [ 604.218087] ? bpf_test_finish.isra.0+0x190/0x190 [ 604.218461] ? bpf_fentry_test1+0x5/0x10 [ 604.218750] ? trace_clock_x86_tsc+0x10/0x10 [ 604.219064] ? __sys_bpf+0x8b1/0x2970 [ 604.219337] ? lock_is_held_type+0xd7/0x130 [ 604.219680] ? __x64_sys_bpf+0x1c/0x20 [ 604.219957] ? do_syscall_64+0x35/0x80 [ 604.220237] ? entry_SYSCALL_64_after_hwframe+0x44/0xae [ 604.220653] </TASK>
Quoted 1 line
Jiri, is that what you had seen?
hi,
sorry for late response
I did not get any backtrace for the stall, debugging showed that the first probed function was called over and over for some reason
as for the crash I used the small fix below
do you have any newer version I could play with?
jirka
---
kernel/trace/fprobe.c +2 -1
@@ -157,7 +157,8 @@ int unregister_fprobe(struct fprobe *fp) ret =unregister_ftrace_function (&fp->ftrace);if (!ret) { -rethook_free (fp->rethook); +if (fp->rethook) +rethook_free (fp->rethook);if (fp->syms) {kfree (fp->addrs); fp->addrs =NULL ;
Quoted 31 lines · Andrii Nakryiko
Quoted 7 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This per-probe private data is necessary for the feature called BPF cookie, in which each attachment has a unique user-provided u64 value associated to it, accessible at runtime through
bpf_get_attach_cookie()helper. One way or another we'll need to support this to make these multi-attach BPF programs really useful for generic tracing applications.Jiri,
We've discussed with Alexei this week how cookies can be supported for multi-attach fentry (where it seems even more challenging than in kprobe case), and agreed on rather simple solution, which roughly goes like this. When multi-attaching either fentry/fexit program, save sorted array of IP addresses and then sorted in the same order as IPs list of u64 cookies. At runtime,
bpf_get_attach_cookie()helper should somehow get access to these two arrays and functions IP (that we already have withbpf_get_func_ip()), perform binary search and locate necessary cookie. This offloads the overhead of finding this cookie to actual call site ofbpf_get_attach_cookie()(and it's a log(N), so not bad at all, especially if BPF program can be optimized to call this helper just once).
Quoted 5 lines
I think something like that should be doable for Masami's fprobe-based multi-attach kprobes, right? That would allow to have super-fast attachment, but still support BPF cookie per each individual IP/kernel function attachment. I haven't looked at code thoroughly, though, please let me know if I'm missing something fundamental.
ok, that seems doable, we should be able to get the link struct in bpf_get_attach_cookie_trace and reach both ips and cookies
jirka
Quoted 78 lines
Quoted 77 lines
This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
[1] https://lore.kernel.org/all/[email protected]/T/#u
I also added an out-of-tree (just for testing) patch at the end of this series ([8/8]) for adding a wildcard support to the sample program. With that patch, it shows how long the registration will take;
# time insmod fprobe_example.ko symbol='btrfs_*' [ 36.130947] fprobe_init: 1028 symbols found [ 36.177901] fprobe_init: Planted fprobe at btrfs_* real 0m 0.08s user 0m 0.00s sys 0m 0.07sThank you,
---
Jiri Olsa (2):
ftrace: Addftrace_set_filter_ipsfunction
bpf: Add kprobe link for attaching raw kprobesMasami Hiramatsu (6):
fprobe: Add ftrace based probe APIs
rethook: Add a generic return hook
rethook: x86: Add rethook x86 implementation
fprobe: Addexit_handlersupport
fprobe: Add sample program for fprobe
[DO NOT MERGE] Out-of-tree: Support wildcard symbol option to samplearch/x86/Kconfig | 1 arch/x86/kernel/Makefile | 1 arch/x86/kernel/rethook.c | 115 ++++++++++++++++++++ include/linux/bpf_types.h | 1 include/linux/fprobe.h | 57 ++++++++++ include/linux/ftrace.h | 3 + include/linux/rethook.h | 74 +++++++++++++ include/linux/sched.h | 3 + include/uapi/linux/bpf.h | 12 ++ kernel/bpf/syscall.c | 195 +++++++++++++++++++++++++++++++++- kernel/exit.c | 2 kernel/fork.c | 3 + kernel/kallsyms.c | 1 kernel/trace/Kconfig | 22 ++++ kernel/trace/Makefile | 2 kernel/trace/fprobe.c | 168 +++++++++++++++++++++++++++++ kernel/trace/ftrace.c | 54 ++++++++- kernel/trace/rethook.c | 226 +++++++++++++++++++++++++++++++++++++++ samples/Kconfig | 7 + samples/Makefile | 1 samples/fprobe/Makefile | 3 + samples/fprobe/fprobe_example.c | 154 +++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 12 ++ 23 files changed, 1103 insertions(+), 14 deletions(-) create mode 100644 arch/x86/kernel/rethook.c create mode 100644 include/linux/fprobe.h create mode 100644 include/linux/rethook.h create mode 100644 kernel/trace/fprobe.c create mode 100644 kernel/trace/rethook.c create mode 100644 samples/fprobe/Makefile create mode 100644 samples/fprobe/fprobe_example.c--
Masami Hiramatsu (Linaro) <[email protected]>
Masami Hiramatsu
maintainerQuoted 68 lines · On Tue, 18 Jan 2022 15:25:25 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 65 lines · Masami Hiramatsu
Quoted 20 lines · On Thu, 13 Jan 2022 13:27:34 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 19 lines · Jiri Olsa
Quoted 10 lines · Masami Hiramatsu
Hi Jiri and Alexei,
Here is the 2nd version of fprobe. This version uses the
ftrace_set_filter_ips()for reducing the registering overhead. Note that this also drops per-probe point private data, which is not used anyway.This introduces the fprobe, the function entry/exit probe with multiple probe point support. This also introduces the rethook for hooking function return as same as kretprobe does. This
nice, I was going through the multi-user-graph support and was wondering that this might be a better way
Quoted 4 lines
abstraction will help us to generalize the fgraph tracer, because we can just switch it from rethook in fprobe, depending on the kernel configuration.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
The patch [1/8] and [7/8] are from your series[1]. Other libbpf patches will not be affected by this change.
I'll try the bpf selftests on top of this
I'm getting crash and stall when running bpf selftests, the fprobe sample module works fine, I'll check on that
OK, I got a kernel stall. I missed to enable
CONFIG_FPROBE. I thinkvmtest.shshould support menuconfig option.#6 bind_perm:OK
#7 bloom_filter_map:OK[ 107.282403] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'tsc' as unstable because the skew is too large: [ 107.283240] clocksource: 'hpet' wd_nsec: 496216090 wd_now: 7ddc7120 wd_last: 7ae746b7 mask: ffffffff [ 107.284045] clocksource: 'tsc' cs_nsec: 495996979 cs_now: 31fdb69b39 cs_last: 31c2d29219 mask: ffffffffffffffff [ 107.284926] clocksource: 'tsc' is current clocksource. [ 107.285487] tsc: Marking TSC unstable due to clocksource watchdog [ 107.285973] TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. [ 107.286616] sched_clock: Marking unstable (107240582544, 45390230)<-(107291410145, -5437339) [ 107.290408] clocksource: Not enough CPUs to check clocksource 'tsc'. [ 107.290879] clocksource: Switched to clocksource hpet [ 604.210415] INFO: rcu_tasks detected stalls on tasks: [ 604.210830] (____ptrval____): .. nvcsw: 86/86 holdout: 1 idle_cpu: -1/0 [ 604.211314] task:test_progs state:R running task stack: 0 pid: 87 ppid: 85 flags:0x00004000 [ 604.212058] Call Trace: [ 604.212246] <TASK> [ 604.212452] __schedule+0x362/0xbb0 [ 604.212723] ? preempt_schedule_notrace_thunk+0x16/0x18 [ 604.213107] preempt_schedule_notrace+0x48/0x80 [ 604.217403] ? asm_sysvec_apic_timer_interrupt+0x12/0x20 [ 604.217790] ? ftrace_regs_call+0xd/0x52 [ 604.218087] ? bpf_test_finish.isra.0+0x190/0x190 [ 604.218461] ? bpf_fentry_test1+0x5/0x10 [ 604.218750] ? trace_clock_x86_tsc+0x10/0x10 [ 604.219064] ? __sys_bpf+0x8b1/0x2970 [ 604.219337] ? lock_is_held_type+0xd7/0x130 [ 604.219680] ? __x64_sys_bpf+0x1c/0x20 [ 604.219957] ? do_syscall_64+0x35/0x80 [ 604.220237] ? entry_SYSCALL_64_after_hwframe+0x44/0xae [ 604.220653] </TASK>Quoted 1 line
Jiri, is that what you had seen?
Quoted 8 lines
hi,
sorry for late responseI did not get any backtrace for the stall, debugging showed that the first probed function was called over and over for some reason
as for the crash I used the small fix below
Oops, good catch!
Quoted 2 lines
do you have any newer version I could play with?
Let me update the fprobe and rethook. I'm now trying to integrate the rethook with kretprobes and find some issues.
Thank you!
Quoted 20 lines
jirka
---
kernel/trace/fprobe.c +2
-1@@ -157,7 +157,8 @@ int unregister_fprobe(struct fprobe *fp) ret =unregister_ftrace_function (&fp->ftrace);if (!ret) { -rethook_free (fp->rethook); +if (fp->rethook) +rethook_free (fp->rethook);if (fp->syms) {kfree (fp->addrs); fp->addrs =NULL ;
Signature · 1 line
Masami Hiramatsu <[email protected]>