tracing: attach eBPF programs to tracepoints/syscalls/kprobe
Subsystems (10)
v3 cover tracing: attach eBPF programs to tracepoints/syscalls/kprobe
Hi Steven,
This patch set is for linux-trace/for-next
It adds ability to attach eBPF programs to tracepoints, syscalls and kprobes. Obviously too late for 3.20, but please review. I'll rebase and repost when merge window closes.
Main difference in V3 is different attaching mechanism:
- load program via bpf() syscall and receive prog_fd
- event_fd = perf_event_open()
- ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd) to attach program to event
- close(event_fd) will destroy event and detach the program kernel diff became smaller and in general this approach is cleaner (thanks to Masami and Namhyung for suggesting it)
The programs are run before ring buffer is allocated to have minimal impact on a system, which can be demonstrated by
'dd if=/dev/zero of=/dev/null count=20000000' test:
4.80074 s, 2.1 GB/s - no tracing (raw base line)
5.62705 s, 1.8 GB/s - attached bpf program does 'map[log2(count)]++' without JIT 5.05963 s, 2.0 GB/s - attached bpf program does 'map[log2(count)]++' with JIT 4.91715 s, 2.1 GB/s - attached bpf program does 'return 0'
perf record -e skb:sys_write dd if=/dev/zero of=/dev/null count=20000000 8.75686 s, 1.2 GB/s
Warning: Processed 20355236 events and lost 44 chunks!
perf record -e skb:sys_write --filter cnt==1234 dd if=/dev/zero of=/dev/null count=20000000 5.69732 s, 1.8 GB/s
6.13730 s, 1.7 GB/s - echo 1 > /sys/../events/skb/sys_write/enable 6.50091 s, 1.6 GB/s - echo 'cnt == 1234' > /sys/../events/skb/sys_write/filter
(skb:sys_write is a temporary tracepoint in write() syscall)
So the overhead of realistic bpf program is 5.05963/4.80074 = ~5% which is faster than perf_event filtering: 5.69732/4.80074 = ~18% or ftrace filtering: 6.50091/4.80074 = ~35%
V2->V3:
- changed program attach interface from tracefs into perf_event ioctl
- rewrote user space helpers to use perf_events
- rewrote tracex1 example to use mmap-ed ring_buffer instead of trace_pipe
- as suggested by Arnaldo renamed bpf_memcmp to bpf_probe_memcmp to better indicate function logic
- added ifdefs to make bpf check a nop when CONFIG_BPF_SYSCALL is not set
V1->V2:
- dropped bpf_dump_stack() and bpf_printk() helpers
- disabled running programs in_nmi
- other minor cleanups
Program attach point and input arguments:
- programs attached to kprobes receive 'struct pt_regs *' as an input. See tracex4_kern.c that demonstrates how users can write a C program like:
SEC("events/kprobes/sys_write")
int bpf_prog4(struct pt_regs *regs)
{
long write_size = regs->dx;
// here user need to know the proto of sys_write() from kernel
// sources and x64 calling convention to know that register $rdx
// contains 3rd argument to sys_write() which is 'size_t count'it's obviously architecture dependent, but allows building sophisticated user tools on top, that can see from debug info of vmlinux which variables are in which registers or stack locations and fetch it from there. 'perf probe' can potentialy use this hook to generate programs in user space and insert them instead of letting kernel parse string during kprobe creation.
- programs attached to tracepoints and syscalls receive 'struct bpf_context *':
u64 arg1, arg2, ..., arg6;
for syscalls they match syscall arguments.
for tracepoints these args match arguments passed to tracepoint. For example:
trace_sched_migrate_task(p, new_cpu); from sched/core.c
arg1 <- p which is 'struct task_struct *'
arg2 <- new_cpu which is 'unsigned int'
arg3..arg6 = 0
the program can use bpf_fetch_u8/16/32/64/ptr() helpers to walk 'task_struct' or any other kernel data structures.
These helpers are using probe_kernel_read() similar to 'perf probe' which is not 100% safe in both cases, but good enough.
To access task_struct's pid inside 'sched_migrate_task' tracepoint the program can do:
struct task_struct *task = (struct task_struct *)ctx->arg1;
u32 pid = bpf_fetch_u32(&task->pid);
Since struct layout is kernel configuration specific such programs are not portable and require access to kernel headers to be compiled, but in this case we don't need debug info.
llvm with bpf backend will statically compute task->pid offset as a constant based on kernel headers only.
The example of this arbitrary pointer walking is tracex1_kern.c which does skb->dev->name == "lo" filtering.
In all cases the programs are called before ring buffer is allocated to minimize the overhead, since we want to filter huge number of events, but perf_trace_buf_prepare/submit and argument copy for every event is too costly.
Note, tracepoint/syscall and kprobe programs are two different types:BPF_PROG_TYPE_TRACEPOINT and BPF_PROG_TYPE_KPROBE,
since they expect different input.
Both use the same set of helper functions:
- map access (lookup/update/delete)
- fetch (probe_kernel_read wrappers)
- probe_memcmp (probe_kernel_read + memcmp)
Portability:
- kprobe programs are architecture dependent and need user scripting language like ktap/stap/dtrace/perf that will dynamically generate them based on debug info in vmlinux
- tracepoint programs are architecture independent, but if arbitrary pointer walking (with fetch() helpers) is used, they need data struct layout to match. Debug info is not necessary
- for networking use case we need to access 'struct sk_buff' fields in portable way (user space needs to fetch packet length without knowing layout of sk_buff), so for some frequently used data structures there will be a way to access them effeciently without bpf_fetch* helpers. Once it's ready tracepoint programs that access common data structs will be kernel independent.
Program return value:
- programs return 0 to discard an event
- and return non-zero to proceed with event (get ring buffer, copy arguments there and pass to user space via mmap-ed area)
Examples:
- dropmon.c - simple kfree_skb() accounting in eBPF assembler, similar to dropmon tool
- tracex1_kern.c - does net/netif_receive_skb event filtering for dev->skb->name == "lo" condition
trace1_user.c - receives PERF_SAMPLE_RAW events into mmap-ed buffer and prints them
- tracex2_kern.c - same kfree_skb() accounting like dropmon, but now in C plus computes histogram of all write sizes from sys_write syscall and prints the histogram in userspace
- tracex3_kern.c - most sophisticated example that computes IO latency between block/block_rq_issue and block/block_rq_complete events and prints 'heatmap' using gray shades of text terminal. Useful to analyze disk performance.
- tracex4_kern.c - computes histogram of write sizes from sys_write syscall using kprobe mechanism instead of syscall. Since kprobe is optimized into ftrace the overhead of instrumentation is smaller than in example 2.
The user space tools like ktap/dtrace/systemptap/perf that has access to debug info would probably want to use kprobe attachment point, since kprobe can be inserted anywhere and all registers are avaiable in the program. tracepoint attachments are useful without debug info, so standalone tools like iosnoop will use them.
The main difference vs existing perf_probe/ftrace infra is in kernel aggregation and conditional walking of arbitrary data structures.
Thanks!
Alexei Starovoitov (8):
tracing: attach eBPF programs to tracepoints and syscalls
tracing: allow eBPF programs to call ktime_get_ns()
samples: bpf: simple tracing example in eBPF assembler
samples: bpf: simple tracing example in C
samples: bpf: counting example for kfree_skb tracepoint and write
syscall
samples: bpf: IO latency analysis (iosnoop/heatmap)
tracing: attach eBPF programs to kprobe/kretprobe
samples: bpf: simple kprobe example
include/linux/bpf.h | 6 +- include/linux/ftrace_event.h | 14 +++ include/trace/bpf_trace.h | 25 +++++ include/trace/ftrace.h | 31 +++++++ include/uapi/linux/bpf.h | 9 ++ include/uapi/linux/perf_event.h | 1 + kernel/events/core.c | 58 ++++++++++++ kernel/trace/Makefile | 1 + kernel/trace/bpf_trace.c | 194 +++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_kprobe.c | 10 +- kernel/trace/trace_syscalls.c | 35 +++++++ samples/bpf/Makefile | 18 ++++ samples/bpf/bpf_helpers.h | 14 +++ samples/bpf/bpf_load.c | 136 +++++++++++++++++++++++++-- samples/bpf/bpf_load.h | 12 +++ samples/bpf/dropmon.c | 143 +++++++++++++++++++++++++++++ samples/bpf/libbpf.c | 7 ++ samples/bpf/libbpf.h | 4 + samples/bpf/tracex1_kern.c | 28 ++++++ samples/bpf/tracex1_user.c | 50 ++++++++++ samples/bpf/tracex2_kern.c | 71 ++++++++++++++ samples/bpf/tracex2_user.c | 95 +++++++++++++++++++ samples/bpf/tracex3_kern.c | 98 ++++++++++++++++++++ samples/bpf/tracex3_user.c | 152 ++++++++++++++++++++++++++++++ samples/bpf/tracex4_kern.c | 36 ++++++++ samples/bpf/tracex4_user.c | 83 +++++++++++++++++ 26 files changed, 1321 insertions(+), 10 deletions(-) create mode 100644 include/trace/bpf_trace.h create mode 100644 kernel/trace/bpf_trace.c create mode 100644 samples/bpf/dropmon.c create mode 100644 samples/bpf/tracex1_kern.c create mode 100644 samples/bpf/tracex1_user.c create mode 100644 samples/bpf/tracex2_kern.c create mode 100644 samples/bpf/tracex2_user.c create mode 100644 samples/bpf/tracex3_kern.c create mode 100644 samples/bpf/tracex3_user.c create mode 100644 samples/bpf/tracex4_kern.c create mode 100644 samples/bpf/tracex4_user.c
Signature · 1 line
1.7.9.5
PATCH v3 1/8 tracing: attach eBPF programs to tracepoints and syscalls
User interface:
struct perf_event_attr attr = {.type = PERF_TYPE_TRACEPOINT, .config = event_id, ...}; event_fd = perf_event_open(&attr,...);ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd);
prog_fd is a file descriptor associated with eBPF program previously loaded. event_id is an ID of static tracepoint event or syscall.
(kprobe support is in next patch)
close(event_fd) - automatically detaches eBPF program from it
eBPF programs can call in-kernel helper functions to:
- lookup/update/delete elements in maps
- fetch_ptr/u64/u32/u16/u8 values from unsafe address via probe_kernel_read(), so that eBPF program can walk any kernel data structures
- probe_memcmp - combination of probe_kernel_read() and memcmp()
--- include/linux/bpf.h | 6 +- include/linux/ftrace_event.h | 11 +++ include/trace/bpf_trace.h | 25 +++++++ include/trace/ftrace.h | 31 +++++++++ include/uapi/linux/bpf.h | 7 ++ include/uapi/linux/perf_event.h | 1 + kernel/events/core.c | 55 +++++++++++++++ kernel/trace/Makefile | 1 + kernel/trace/bpf_trace.c | 145 +++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_syscalls.c | 35 ++++++++++ 10 files changed, 316 insertions(+), 1 deletion(-) create mode 100644 include/trace/bpf_trace.h create mode 100644 kernel/trace/bpf_trace.c
include/linux/bpf.h +5 -1
@@ -130,10 +130,14 @@ struct bpf_prog_aux {#ifdef CONFIG_BPF_SYSCALL void bpf_prog_put (struct bpf_prog *prog); +struct bpf_prog *bpf_prog_get (u32 ufd);#else static inline void bpf_prog_put (struct bpf_prog *prog) {} +static inline struct bpf_prog *bpf_prog_get (u32 ufd) +{ +return ERR_PTR (-ENOENT ); +}#endif -struct bpf_prog *bpf_prog_get (u32 ufd);/* verify correctness of eBPF program */ int bpf_check (struct bpf_prog *fp,union bpf_attr *attr);
include/linux/ftrace_event.h +11 -0
@@ -13,6 +13,7 @@ struct trace_array;struct trace_buffer ;struct tracer ;struct dentry ; +struct bpf_prog ;struct trace_print_flags {unsigned long mask;
@@ -299,6 +300,7 @@ struct ftrace_event_call {#ifdef CONFIG_PERF_EVENTS int perf_refcount;struct hlist_head __percpu *perf_events; +struct bpf_prog *prog;int (*perf_perm)(struct ftrace_event_call *,struct perf_event *);
@@ -544,6 +546,15 @@ event_trigger_unlock_commit_regs(struct ftrace_event_file *file,event_triggers_post_call (file, tt); } +#ifdef CONFIG_BPF_SYSCALL +unsigned int trace_call_bpf (struct bpf_prog *prog,void *ctx); +#else +static inline unsigned int trace_call_bpf (struct bpf_prog *prog,void *ctx) +{ +return 1 ; +} +#endif +enum {FILTER_OTHER =0 ,FILTER_STATIC_STRING ,
include/trace/bpf_trace.h new file +25 -0
@@ -0,0 +1,25 @@ +/* Copyright (c) 2011-2015 PLUMgrid, http://plumgrid.com +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of version 2 of the GNU General Public +* License as published by the Free Software Foundation. +*/ +#ifndef _LINUX_KERNEL_BPF_TRACE_H +#define _LINUX_KERNEL_BPF_TRACE_H + +/* For tracepoint filters argN fields match one to one to arguments +* passed to tracepoint events +* +* For syscall entry filters argN fields match syscall arguments +* For syscall exit filters arg1 is a return value +*/ +struct bpf_context { +u64 arg1; +u64 arg2; +u64 arg3; +u64 arg4; +u64 arg5; +u64 arg6; +}; + +#endif /* _LINUX_KERNEL_BPF_TRACE_H */
include/trace/ftrace.h +31 -0
@@ -17,6 +17,7 @@*/ #include <linux/ftrace_event.h> +#include <trace/bpf_trace.h> /* * DECLARE_EVENT_CLASS can be used to add a generic function
@@ -755,12 +756,32 @@ __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call#undef __perf_task#define __perf_task (t) (__task = (t)) +/* zero extend integer, pointer or aggregate type to u64 without warnings */ +#define __CAST_TO_U64 (EXPR ) ({ \ + u64 ret = 0; \ + typeof(EXPR) expr = EXPR; \ + switch (sizeof(expr)) { \ + case 8: ret = *(u64 *) &expr; break; \ + case 4: ret = *(u32 *) &expr; break; \ + case 2: ret = *(u16 *) &expr; break; \ + case 1: ret = *(u8 *) &expr; break; \ + } \ + ret; }) + +#define __BPF_CAST1 (a,...) __CAST_TO_U64(a) +#define __BPF_CAST2 (a,...) __CAST_TO_U64(a), __BPF_CAST1(__VA_ARGS__) +#define __BPF_CAST3 (a,...) __CAST_TO_U64(a), __BPF_CAST2(__VA_ARGS__) +#define __BPF_CAST4 (a,...) __CAST_TO_U64(a), __BPF_CAST3(__VA_ARGS__) +#define __BPF_CAST5 (a,...) __CAST_TO_U64(a), __BPF_CAST4(__VA_ARGS__) +#define __BPF_CAST6 (a,...) __CAST_TO_U64(a), __BPF_CAST5(__VA_ARGS__) +#undef DECLARE_EVENT_CLASS#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ static notrace void \ perf_trace_##call(void *__data, proto) \ { \ struct ftrace_event_call *event_call = __data; \ + struct bpf_prog *prog = event_call->prog; \ struct ftrace_data_offsets_##call __maybe_unused __data_offsets;\ struct ftrace_raw_##call *entry; \ struct pt_regs __regs; \
@@ -771,6 +792,16 @@ perf_trace_##call(void *__data, proto) \int __data_size; \int rctx; \ \ +if (prog) { \ + __maybe_unused const u64 z = 0; \ +struct bpf_context __ctx = ((struct bpf_context ) { \ +__BPF_CAST6 (args, z, z, z, z, z) \ + }); \ + \ +if (!trace_call_bpf (prog, &__ctx)) \ +return ; \ + } \ + \ __data_size = ftrace_get_offsets_##call(&__data_offsets, args); \ \ head = this_cpu_ptr(event_call->perf_events); \
include/uapi/linux/bpf.h +7 -0
@@ -118,6 +118,7 @@ enum bpf_map_type {enum bpf_prog_type {BPF_PROG_TYPE_UNSPEC ,BPF_PROG_TYPE_SOCKET_FILTER , +BPF_PROG_TYPE_TRACEPOINT , };/* flags for BPF_MAP_UPDATE_ELEM command */
@@ -162,6 +163,12 @@ enum bpf_func_id { BPF_FUNC_map_lookup_elem,/* void *map_lookup_elem(&map, &key) */ BPF_FUNC_map_update_elem,/* int map_update_elem(&map, &key, &value, flags) */ BPF_FUNC_map_delete_elem,/* int map_delete_elem(&map, &key) */ + BPF_FUNC_fetch_ptr,/* void *bpf_fetch_ptr(void *unsafe_ptr) */ + BPF_FUNC_fetch_u64,/* u64 bpf_fetch_u64(void *unsafe_ptr) */ + BPF_FUNC_fetch_u32,/* u32 bpf_fetch_u32(void *unsafe_ptr) */ + BPF_FUNC_fetch_u16,/* u16 bpf_fetch_u16(void *unsafe_ptr) */ + BPF_FUNC_fetch_u8,/* u8 bpf_fetch_u8(void *unsafe_ptr) */ + BPF_FUNC_probe_memcmp,/* int bpf_probe_memcmp(unsafe_ptr, safe_ptr, size) */ __BPF_FUNC_MAX_ID, };
include/uapi/linux/perf_event.h +1 -0
@@ -360,6 +360,7 @@ struct perf_event_attr {#define PERF_EVENT_IOC_SET_OUTPUT _IO ('$', 5)#define PERF_EVENT_IOC_SET_FILTER _IOW('$', 6, char *)#define PERF_EVENT_IOC_ID _IOR('$', 7, __u64 *) +#define PERF_EVENT_IOC_SET_BPF _IOW('$', 8, __u32)enum perf_event_ioc_flags {PERF_IOC_FLAG_GROUP =1U <<0 ,
kernel/events/core.c +55 -0
97 lines not on this page · raw message
kernel/trace/Makefile +1 -0
@@ -53,6 +53,7 @@ obj-$(CONFIG_EVENT_TRACING) += trace_event_perf.o endif obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o obj-$(CONFIG_EVENT_TRACING) += trace_events_trigger.o +obj-$(CONFIG_BPF_SYSCALL) += bpf_trace.o obj-$(CONFIG_KPROBE_EVENT) += trace_kprobe.o obj-$(CONFIG_TRACEPOINTS) += power-traces.o ifeq ($(CONFIG_PM),y)
kernel/trace/bpf_trace.c new file +145 -0
145 lines not on this page · raw message
kernel/trace/trace_syscalls.c +35 -0
@@ -7,6 +7,7 @@#include <linux/ftrace.h> #include <linux/perf_event.h> #include <asm/syscall.h> +#include <trace/bpf_trace.h> #include "trace_output.h" #include "trace.h"
@@ -545,11 +546,26 @@ static DECLARE_BITMAP(enabled_perf_exit_syscalls, NR_syscalls);static int sys_perf_refcount_enter;static int sys_perf_refcount_exit; +static void populate_bpf_ctx (struct bpf_context *ctx,struct pt_regs *regs) +{ +struct task_struct *task = current; +unsigned long args[6 ]; + +syscall_get_arguments (task, regs,0 ,6 , args); + ctx->arg1 = args[0 ]; + ctx->arg2 = args[1 ]; + ctx->arg3 = args[2 ]; + ctx->arg4 = args[3 ]; + ctx->arg5 = args[4 ]; + ctx->arg6 = args[5 ]; +} +static void perf_syscall_enter (void *ignore,struct pt_regs *regs,long id) {struct syscall_metadata *sys_data;struct syscall_trace_enter *rec;struct hlist_head *head; +struct bpf_prog *prog;int syscall_nr;int rctx;int size;
@@ -564,6 +580,15 @@ static void perf_syscall_enter(void *ignore, struct pt_regs *regs, long id)if (!sys_data)return ; + prog = sys_data->enter_event->prog; +if (prog) { +struct bpf_context ctx; + +populate_bpf_ctx (&ctx, regs); +if (!trace_call_bpf (prog, &ctx)) +return ; + } + head =this_cpu_ptr (sys_data->enter_event->perf_events);if (hlist_empty (head))return ;
@@ -624,6 +649,7 @@ static void perf_syscall_exit(void *ignore, struct pt_regs *regs, long ret)struct syscall_metadata *sys_data;struct syscall_trace_exit *rec;struct hlist_head *head; +struct bpf_prog *prog;int syscall_nr;int rctx;int size;
@@ -638,6 +664,15 @@ static void perf_syscall_exit(void *ignore, struct pt_regs *regs, long ret)if (!sys_data)return ; + prog = sys_data->exit_event->prog; +if (prog) { +struct bpf_context ctx = {}; + + ctx.arg1 =syscall_get_return_value (current, regs); +if (!trace_call_bpf (prog, &ctx)) +return ; + } + head =this_cpu_ptr (sys_data->exit_event->perf_events);if (hlist_empty (head))return ;
Signature · 1 line
1.7.9.5
Steven Rostedt
maintainerRe: [PATCH v3 linux-trace 1/8] tracing: attach eBPF programs to tracepoints and syscalls
Quoted 33 lines · On Mon, 9 Feb 2015 19:45:54 -0800 Alexei Starovoitov <[email protected]> wrote:
include/trace/bpf_trace.h new file +1
-0+#endif /* _LINUX_KERNEL_BPF_TRACE_H */ include/trace/ftrace.h +17
-0@@ -17,6 +17,7 @@*/ #include <linux/ftrace_event.h> +#include <trace/bpf_trace.h> /* * DECLARE_EVENT_CLASS can be used to add a generic function @@ -755,12 +756,32 @@ __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call#undef __perf_task#define __perf_task (t) (__task = (t)) +/* zero extend integer, pointer or aggregate type to u64 without warnings */ +#define __CAST_TO_U64 (EXPR ) ({ \ + u64 ret = 0; \ + typeof(EXPR) expr = EXPR; \ + switch (sizeof(expr)) { \ + case 8: ret = *(u64 *) &expr; break; \ + case 4: ret = *(u32 *) &expr; break; \ + case 2: ret = *(u16 *) &expr; break; \ + case 1: ret = *(u8 *) &expr; break; \ + } \ + ret; }) + +#define __BPF_CAST1 (a,...) __CAST_TO_U64(a) +#define __BPF_CAST2 (a,...) __CAST_TO_U64(a), __BPF_CAST1(__VA_ARGS__) +#define __BPF_CAST3 (a,...) __CAST_TO_U64(a), __BPF_CAST2(__VA_ARGS__) +#define __BPF_CAST4 (a,...) __CAST_TO_U64(a), __BPF_CAST3(__VA_ARGS__)
include/trace/ftrace.h __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call · line 784
Quoted 10 lines
include/trace/ftrace.h +4
-0+#define __BPF_CAST5 (a,...) __CAST_TO_U64(a), __BPF_CAST4(__VA_ARGS__) +#define __BPF_CAST6 (a,...) __CAST_TO_U64(a), __BPF_CAST5(__VA_ARGS__) +#undef DECLARE_EVENT_CLASS#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ static notrace void \ perf_trace_##call(void *__data, proto) \ { \ struct ftrace_event_call *event_call = __data; \ + struct bpf_prog *prog = event_call->prog; \
Looks like this is entirely perf based and does not interact with ftrace at all. In other words, it's perf not tracing.
It makes more sense to go through tip than the tracing tree.
But I still do not want any hard coded event structures. All access to data from the binary code must be parsed by looking at the event/format files. Otherwise you will lock internals of the kernel as userspace ABI, because eBPF programs will break if those internals change, and that could severely limit progress in the future.
-- Steve
Quoted 20 lines
include/trace/ftrace.h +10
-0struct ftrace_data_offsets_##call __maybe_unused __data_offsets;\ struct ftrace_raw_##call *entry; \ struct pt_regs __regs; \@@ -771,6 +792,16 @@ perf_trace_##call(void *__data, proto) \int __data_size; \int rctx; \ \ +if (prog) { \ + __maybe_unused const u64 z = 0; \ +struct bpf_context __ctx = ((struct bpf_context ) { \ +__BPF_CAST6 (args, z, z, z, z, z) \ + }); \ + \ +if (!trace_call_bpf (prog, &__ctx)) \ +return ; \ + } \ + \ __data_size = ftrace_get_offsets_##call(&__data_offsets, args); \ \ head = this_cpu_ptr(event_call->perf_events); \
Steven Rostedt
maintainerRe: [PATCH v3 linux-trace 1/8] tracing: attach eBPF programs to tracepoints and syscalls
In reply to Alexei StarovoitovQuoted 62 lines · On Mon, 9 Feb 2015 19:45:54 -0800 Alexei Starovoitov <[email protected]> wrote:
include/trace/bpf_trace.h new file +16
-0+/* For tracepoint filters argN fields match one to one to arguments +* passed to tracepoint events +* +* For syscall entry filters argN fields match syscall arguments +* For syscall exit filters arg1 is a return value +*/ +struct bpf_context { +u64 arg1; +u64 arg2; +u64 arg3; +u64 arg4; +u64 arg5; +u64 arg6; +}; + +#endif /* _LINUX_KERNEL_BPF_TRACE_H */ include/trace/ftrace.h +21
-0@@ -17,6 +17,7 @@*/ #include <linux/ftrace_event.h> +#include <trace/bpf_trace.h> /* * DECLARE_EVENT_CLASS can be used to add a generic function @@ -755,12 +756,32 @@ __attribute__((section("_ftrace_events"))) *__event_##call = &event_##call#undef __perf_task#define __perf_task (t) (__task = (t)) +/* zero extend integer, pointer or aggregate type to u64 without warnings */ +#define __CAST_TO_U64 (EXPR ) ({ \ + u64 ret = 0; \ + typeof(EXPR) expr = EXPR; \ + switch (sizeof(expr)) { \ + case 8: ret = *(u64 *) &expr; break; \ + case 4: ret = *(u32 *) &expr; break; \ + case 2: ret = *(u16 *) &expr; break; \ + case 1: ret = *(u8 *) &expr; break; \ + } \ + ret; }) + +#define __BPF_CAST1 (a,...) __CAST_TO_U64(a) +#define __BPF_CAST2 (a,...) __CAST_TO_U64(a), __BPF_CAST1(__VA_ARGS__) +#define __BPF_CAST3 (a,...) __CAST_TO_U64(a), __BPF_CAST2(__VA_ARGS__) +#define __BPF_CAST4 (a,...) __CAST_TO_U64(a), __BPF_CAST3(__VA_ARGS__) +#define __BPF_CAST5 (a,...) __CAST_TO_U64(a), __BPF_CAST4(__VA_ARGS__) +#define __BPF_CAST6 (a,...) __CAST_TO_U64(a), __BPF_CAST5(__VA_ARGS__) +#undef DECLARE_EVENT_CLASS#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ static notrace void \ perf_trace_##call(void *__data, proto) \ { \ struct ftrace_event_call *event_call = __data; \ + struct bpf_prog *prog = event_call->prog; \ struct ftrace_data_offsets_##call __maybe_unused __data_offsets;\ struct ftrace_raw_##call *entry; \ struct pt_regs __regs; \
include/trace/ftrace.h perf_trace_##call(void *__data, proto) \ · line 798
Quoted 7 lines
include/trace/ftrace.h +4
-0@@ -771,6 +792,16 @@ perf_trace_##call(void *__data, proto) \int __data_size; \int rctx; \ \ +if (prog) { \ + __maybe_unused const u64 z = 0; \ + struct bpf_context __ctx = ((struct bpf_context) { \ + __BPF_CAST6(args, z, z, z, z, z) \
Note, there is no guarantee that args is at most 6. For example, in drivers/net/wireless/brcm80211/brcmsmac/brcms_trace_events.h, the trace_event brcms_txstatus has 8 args.
But I guess that's OK if you do not need those last args, right?
Also, there's no interface the lets us know what the args are. I may be able to come up with something. That's the reason I never filtered before tracing. Because we had no way of knowing what to filter on, because the args were never visible.
I'm nervous about showing args of tracepoints too, because we don't want that to become a strict ABI either.
-- Steve
Quoted 7 lines
include/trace/ftrace.h +6
-0+ }); \ + \ +if (!trace_call_bpf (prog, &__ctx)) \ +return ; \ + } \ + \ __data_size = ftrace_get_offsets_##call(&__data_offsets, args); \
PATCH v3 2/8 tracing: allow eBPF programs to call ktime_get_ns()
In reply to Alexei Starovoitovbpf_ktime_get_ns() is used by programs to compue time delta between events or as a timestamp
--- include/uapi/linux/bpf.h | 1 + kernel/trace/bpf_trace.c | 10 ++++++++++ 2 files changed, 11 insertions(+)
include/uapi/linux/bpf.h +1 -0
@@ -169,6 +169,7 @@ enum bpf_func_id { BPF_FUNC_fetch_u16,/* u16 bpf_fetch_u16(void *unsafe_ptr) */ BPF_FUNC_fetch_u8,/* u8 bpf_fetch_u8(void *unsafe_ptr) */ BPF_FUNC_probe_memcmp,/* int bpf_probe_memcmp(unsafe_ptr, safe_ptr, size) */ + BPF_FUNC_ktime_get_ns,/* u64 bpf_ktime_get_ns(void) */ __BPF_FUNC_MAX_ID, };
kernel/trace/bpf_trace.c +10 -0
@@ -69,6 +69,11 @@ static u64 bpf_probe_memcmp(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5)return -1 ; } +static u64 bpf_ktime_get_ns (u64 r1,u64 r2,u64 r3,u64 r4,u64 r5) +{ +return ktime_get_ns (); +} +static struct bpf_func_proto tp_prog_funcs[] = { #define FETCH(SIZE) \ [BPF_FUNC_fetch_##SIZE] = { \
@@ -90,6 +95,11 @@ static struct bpf_func_proto tp_prog_funcs[] = { .arg2_type = ARG_PTR_TO_STACK, .arg3_type = ARG_CONST_STACK_SIZE, }, + [BPF_FUNC_ktime_get_ns] = { + .func = bpf_ktime_get_ns, + .gpl_only = true, + .ret_type = RET_INTEGER, + }, };static const struct bpf_func_proto *tp_prog_func_proto (enum bpf_func_id func_id)
Signature · 1 line
1.7.9.5
PATCH v3 3/8 samples: bpf: simple tracing example in eBPF assembler
In reply to Alexei Starovoitovsimple packet drop monitor:
- in-kernel eBPF program attaches to skb:kfree_skb event and records number of packet drops at given location
- userspace iterates over the map every second and prints stats
Usage:
$ sudo dropmon location 0xffffffff81695995 count 1 location 0xffffffff816d0da9 count 2
location 0xffffffff81695995 count 2
location 0xffffffff816d0da9 count 2
location 0xffffffff81695995 count 3
location 0xffffffff816d0da9 count 2
$ addr2line -ape ./bld_x64/vmlinux 0xffffffff81695995 0xffffffff816d0da9 0xffffffff81695995: ./bld_x64/../net/ipv4/icmp.c:1038 0xffffffff816d0da9: ./bld_x64/../net/unix/af_unix.c:1231
--- samples/bpf/Makefile | 2 + samples/bpf/dropmon.c | 143 +++++++++++++++++++++++++++++++++++++++++++++++++ samples/bpf/libbpf.c | 7 +++ samples/bpf/libbpf.h | 4 ++ 4 files changed, 156 insertions(+) create mode 100644 samples/bpf/dropmon.c
samples/bpf/Makefile +2 -0
@@ -6,7 +6,9 @@ hostprogs-y := test_verifier test_maps hostprogs-y += sock_example hostprogs-y += sockex1 hostprogs-y += sockex2 +hostprogs-y += dropmon +dropmon-objs := dropmon.o libbpf.o test_verifier-objs := test_verifier.o libbpf.o test_maps-objs := test_maps.o libbpf.o sock_example-objs := sock_example.o libbpf.o
samples/bpf/dropmon.c new file +143 -0
143 lines not on this page · raw message
samples/bpf/libbpf.c +7 -0
@@ -121,3 +121,10 @@ int open_raw_sock(const char *name)return sock; } + +int perf_event_open (struct perf_event_attr *attr,int pid,int cpu, +int group_fd,unsigned long flags) +{ +return syscall (__NR_perf_event_open, attr, pid, cpu, + group_fd, flags); +}
samples/bpf/libbpf.h +4 -0
@@ -182,4 +182,8 @@ extern char bpf_log_buf[LOG_BUF_SIZE];/* create RAW socket and bind to interface 'name' */ int open_raw_sock (const char *name); +struct perf_event_attr ; +int perf_event_open (struct perf_event_attr *attr,int pid,int cpu, +int group_fd,unsigned long flags); +#endif
Signature · 1 line
1.7.9.5
PATCH v3 4/8 samples: bpf: simple tracing example in C
In reply to Alexei Starovoitovtracex1_kern.c - C program which will be compiled into eBPF to filter netif_receive_skb events on skb->dev->name == "lo" The programs returns 1 to store an event into ring_buffer and returns 0 - to discard an event.
tracex1_user.c - corresponding user space component that:
- loads bpf program via bpf() syscall
- opens net:netif_receive_skb event via perf_event_open() syscall
- attaches the program to event via ioctl(event_fd, PERF_EVENT_IOC_SET_BPF, prog_fd);
- mmaps event_fd
- polls event_fd, walks ring_buffer and prints events
Usage:
$ sudo tracex1 pid 29241 skb 0xffff88045e58c500 len 84 dev lo pid 29241 skb 0xffff88045e58cd00 len 84 dev lo pid 29241 skb 0xffff880074c35000 len 84 dev lo pid 29241 skb 0xffff880074c35200 len 84 dev lo
--- samples/bpf/Makefile | 4 ++ samples/bpf/bpf_helpers.h | 14 +++++ samples/bpf/bpf_load.c | 129 +++++++++++++++++++++++++++++++++++++++++--- samples/bpf/bpf_load.h | 12 +++++ samples/bpf/tracex1_kern.c | 28 ++++++++++ samples/bpf/tracex1_user.c | 50 +++++++++++++++++ 6 files changed, 231 insertions(+), 6 deletions(-) create mode 100644 samples/bpf/tracex1_kern.c create mode 100644 samples/bpf/tracex1_user.c
samples/bpf/Makefile +4 -0
@@ -7,6 +7,7 @@ hostprogs-y += sock_example hostprogs-y += sockex1 hostprogs-y += sockex2 hostprogs-y += dropmon +hostprogs-y += tracex1 dropmon-objs := dropmon.o libbpf.o test_verifier-objs := test_verifier.o libbpf.o
@@ -14,17 +15,20 @@ test_maps-objs := test_maps.o libbpf.o sock_example-objs := sock_example.o libbpf.o sockex1-objs := bpf_load.o libbpf.o sockex1_user.o sockex2-objs := bpf_load.o libbpf.o sockex2_user.o +tracex1-objs := bpf_load.o libbpf.o tracex1_user.o # Tell kbuild to always build the programs always := $(hostprogs-y) always += sockex1_kern.o always += sockex2_kern.o +always += tracex1_kern.o HOSTCFLAGS += -I$(objtree)/usr/include HOSTCFLAGS_bpf_load.o += -I$(objtree)/usr/include -Wno-unused-variable HOSTLOADLIBES_sockex1 += -lelf HOSTLOADLIBES_sockex2 += -lelf +HOSTLOADLIBES_tracex1 += -lelf # point this to your LLVM backend with bpf support LLC=$(srctree)/tools/bpf/llvm/bld/Debug+Asserts/bin/llc
samples/bpf/bpf_helpers.h +14 -0
@@ -15,6 +15,20 @@ static int (*bpf_map_update_elem)(void *map, void *key, void *value, (void *) BPF_FUNC_map_update_elem;static int (*bpf_map_delete_elem)(void *map,void *key) = (void *) BPF_FUNC_map_delete_elem; +static void *(*bpf_fetch_ptr)(void *unsafe_ptr) = + (void *) BPF_FUNC_fetch_ptr; +static unsigned long long (*bpf_fetch_u64)(void *unsafe_ptr) = + (void *) BPF_FUNC_fetch_u64; +static unsigned int (*bpf_fetch_u32)(void *unsafe_ptr) = + (void *) BPF_FUNC_fetch_u32; +static unsigned short (*bpf_fetch_u16)(void *unsafe_ptr) = + (void *) BPF_FUNC_fetch_u16; +static unsigned char (*bpf_fetch_u8)(void *unsafe_ptr) = + (void *) BPF_FUNC_fetch_u8; +static int (*bpf_probe_memcmp)(void *unsafe_ptr,void *safe_ptr,int size) = + (void *) BPF_FUNC_probe_memcmp; +static unsigned long long (*bpf_ktime_get_ns)(void ) = + (void *) BPF_FUNC_ktime_get_ns;/* llvm builtin functions that eBPF C program may use to * emit BPF_LD_ABS and BPF_LD_IND instructions
samples/bpf/bpf_load.c +123 -6
161 lines not on this page · raw message
samples/bpf/bpf_load.h +12 -0
@@ -6,6 +6,7 @@extern int map_fd[MAX_MAPS ];extern int prog_fd[MAX_PROGS ]; +extern int event_fd[MAX_PROGS ];/* parses elf file compiled by llvm .c->.o * . parses 'maps' section and creates maps via BPF syscall
@@ -21,4 +22,15 @@ extern int prog_fd[MAX_PROGS];*/ int load_bpf_file (char *path); +int perf_event_mmap (int fd); +int perf_event_poll (int fd); +typedef void (*print_fn )(void *data,int size); +void perf_event_read (print_fn fn); +struct trace_entry { +unsigned short type; +unsigned char flags; +unsigned char preempt_count; +int pid; +}; +#endif
samples/bpf/tracex1_kern.c new file +28 -0
@@ -0,0 +1,28 @@ +#include <linux/skbuff.h> +#include <linux/netdevice.h> +#include <uapi/linux/bpf.h> +#include <trace/bpf_trace.h> +#include "bpf_helpers.h" + +SEC("events/net/netif_receive_skb") +int bpf_prog1 (struct bpf_context *ctx) +{ +/* +* attaches to net:netif_receive_skb +* and filters events for loobpack device only +*/ +char devname[] ="lo" ; +struct net_device *dev; +struct sk_buff *skb =0 ; + + skb = (struct sk_buff *) ctx->arg1; + dev =bpf_fetch_ptr (&skb->dev); +if (bpf_probe_memcmp (dev->name, devname,2 ) ==0 ) +/* pass event to userspace via perf ring_buffer */ +return 1 ; + +/* drop event */ +return 0 ; +} + +char _license[] SEC("license") = "GPL";
samples/bpf/tracex1_user.c new file +50 -0
50 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Steven Rostedt
maintainerRe: [PATCH v3 linux-trace 4/8] samples: bpf: simple tracing example in C
samples/bpf/bpf_load.h extern int prog_fd[MAX_PROGS]; · line 35
Quoted 11 lines · On Mon, 9 Feb 2015 19:45:57 -0800 Alexei Starovoitov <[email protected]> wrote:
samples/bpf/bpf_load.h +11
-0+int perf_event_mmap (int fd); +int perf_event_poll (int fd); +typedef void (*print_fn )(void *data,int size); +void perf_event_read (print_fn fn); +struct trace_entry { +unsigned short type; +unsigned char flags; +unsigned char preempt_count; +int pid; +}; +
Please do not hard code any structures. This is not a stable ABI, and it may not even match if you are running 32 bit userspace on top of a 64 bit kernel.
Please parse the format files. libtraceevent does this for you. If need be, link to that. But if you look at the event format files you'll see the offsets and sizes in the binary code:
field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1;
I don't want to get stuck with pinned kernel data structures again. We had 4 blank bytes of data for every event, because latency top hard coded the field. Luckily, the 64 bit / 32 bit interface caused latency top to have to use the event_parse code to work, and we were able to remove that field after it was converted.
-- Steve
Steven Rostedt
maintainerRe: [PATCH v3 linux-trace 4/8] samples: bpf: simple tracing example in C
Quoted 5 lines · On Mon, 9 Feb 2015 23:08:36 -0500 Steven Rostedt <[email protected]> wrote:
I don't want to get stuck with pinned kernel data structures again. We had 4 blank bytes of data for every event, because latency top hard coded the field. Luckily, the 64 bit / 32 bit interface caused latency top to have to use the
event_parsecode to work, and we were able to remove that field after it was converted.
I'm wondering if we should label eBPF programs as "modules". That is, they have no guarantee of working from one kernel to the next. They execute in the kernel, thus they are very similar to modules.
If we can get Linus to say that eBPF programs are not user space, and that they are treated the same as modules (no internal ABI), then I think we can be a bit more free at what we allow.
-- Steve
Re: [PATCH v3 linux-trace 4/8] samples: bpf: simple tracing example in C
Quoted 8 lines · Steven Rostedt
Quoted 5 lines · On Mon, 9 Feb 2015 23:08:36 -0500 Steven Rostedt <[email protected]> wrote:
I don't want to get stuck with pinned kernel data structures again. We had 4 blank bytes of data for every event, because latency top hard coded the field. Luckily, the 64 bit / 32 bit interface caused latency top to have to use the
event_parsecode to work, and we were able to remove that field after it was converted.
I think your main point boils down to:
Quoted 5 lines
But I still do not want any hard coded event structures. All access to data from the binary code must be parsed by looking at the event/format files. Otherwise you will lock internals of the kernel as userspace ABI, because eBPF programs will break if those internals change, and that could severely limit progress in the future.
and I completely agree.
the patch 4 is an example. It doesn't mean in any way that structs defined here is an ABI.
To be compatible across kernels the user space must read format file as you mentioned in your other reply.
Quoted 7 lines
I'm wondering if we should label eBPF programs as "modules". That is, they have no guarantee of working from one kernel to the next. They execute in the kernel, thus they are very similar to modules.
If we can get Linus to say that eBPF programs are not user space, and that they are treated the same as modules (no internal ABI), then I think we can be a bit more free at what we allow.
I thought we already stated that.
Here is the quote from perf_event.h:
* # The RAW record below is opaque data wrt the ABI
* #
* # That is, the ABI doesn't make any promises wrt to
* # the stability of its content, it may vary depending
* # on event, hardware, kernel version and phase of
* # the moon.
* #
* # In other words, PERF_SAMPLE_RAW contents are not an ABI.and this example is reading PERF_SAMPLE_RAW events and uses locally defined structs to print them for simplicity.
Re: [PATCH v3 linux-trace 4/8] samples: bpf: simple tracing example in C
Quoted 11 lines · Alexei Starovoitov
I thought we already stated that.
Here is the quote fromperf_event.h:* # The RAW record below is opaque data wrt the ABI * # * # That is, the ABI doesn't make any promises wrt to * # the stability of its content, it may vary depending * # on event, hardware, kernel version and phase of * # the moon. * # * # In other words, PERF_SAMPLE_RAW contents are not an ABI.
Quoted 2 lines
and this example is reading
PERF_SAMPLE_RAWevents and uses locally defined structs to print them for simplicity.
to underline my point once more:
addition of bpf doesn't change at all what PERF_SAMPLE_RAW already delivers to user space.
so no new ABIs anywhere.
Steven Rostedt
maintainerRe: [PATCH v3 linux-trace 4/8] samples: bpf: simple tracing example in C
Quoted 12 lines · On Mon, 9 Feb 2015 21:47:42 -0800 Alexei Starovoitov <[email protected]> wrote:
Quoted 11 lines · Alexei Starovoitov
I thought we already stated that.
Here is the quote fromperf_event.h:* # The RAW record below is opaque data wrt the ABI * # * # That is, the ABI doesn't make any promises wrt to * # the stability of its content, it may vary depending * # on event, hardware, kernel version and phase of * # the moon. * # * # In other words, PERF_SAMPLE_RAW contents are not an ABI.
Quoted 7 lines
Quoted 2 lines
and this example is reading
PERF_SAMPLE_RAWevents and uses locally defined structs to print them for simplicity.to underline my point once more:
addition of bpf doesn't change at all whatPERF_SAMPLE_RAWalready delivers to user space.
so no new ABIs anywhere.
Again, it we give an example of how to hard code the data, definitely expect this to show up in user space. Users are going to look at this code to learn how to use eBPF. I really want it to do it the correct way instead of the 'easy' way. Because whatever way we have it here, will be the way we will see it out in the wild.
-- Steve
Steven Rostedt
maintainerRe: [PATCH v3 linux-trace 4/8] samples: bpf: simple tracing example in C
In reply to Alexei StarovoitovAdded Linus because he's the one that would revert changes on breakage.
Quoted 18 lines · On Mon, 9 Feb 2015 21:45:21 -0800 Alexei Starovoitov <[email protected]> wrote:
Quoted 8 lines · Steven Rostedt
Quoted 5 lines · On Mon, 9 Feb 2015 23:08:36 -0500 Steven Rostedt <[email protected]> wrote:
I don't want to get stuck with pinned kernel data structures again. We had 4 blank bytes of data for every event, because latency top hard coded the field. Luckily, the 64 bit / 32 bit interface caused latency top to have to use the
event_parsecode to work, and we were able to remove that field after it was converted.I think your main point boils down to:
Quoted 5 lines
But I still do not want any hard coded event structures. All access to data from the binary code must be parsed by looking at the event/format files. Otherwise you will lock internals of the kernel as userspace ABI, because eBPF programs will break if those internals change, and that could severely limit progress in the future.
Quoted 6 lines
and I completely agree.
the patch 4 is an example. It doesn't mean in any way that structs defined here is an ABI.
To be compatible across kernels the user space must read format file as you mentioned in your other reply.
The thing is, this is a sample. Which means it will be cut and pasted into other programs. If the sample does not follow the way we want users to use this, then how can we complain if they hard code it as well?
Quoted 20 lines
Quoted 8 lines
I'm wondering if we should label eBPF programs as "modules". That is, they have no guarantee of working from one kernel to the next. They execute in the kernel, thus they are very similar to modules.
If we can get Linus to say that eBPF programs are not user space, and that they are treated the same as modules (no internal ABI), then I think we can be a bit more free at what we allow.
I thought we already stated that.
Here is the quote fromperf_event.h:* # The RAW record below is opaque data wrt the ABI * # * # That is, the ABI doesn't make any promises wrt to * # the stability of its content, it may vary depending * # on event, hardware, kernel version and phase of * # the moon. * # * # In other words, PERF_SAMPLE_RAW contents are not an ABI.
Quoted 2 lines
and this example is reading
PERF_SAMPLE_RAWevents and uses locally defined structs to print them for simplicity.
As we found out the hard way with latencytop, comments like this does not matter. If an application does something like this, it's our fault if it breaks later. We can't say "hey you were suppose to do it this way". That argument breaks down even more if our own examples do not follow the way we want others to do things.
-- Steve
Steven Rostedt
maintainerRe: [PATCH v3 linux-trace 4/8] samples: bpf: simple tracing example in C
In reply to Alexei Starovoitovsamples/bpf/tracex1_user.c line 20
Quoted 8 lines · On Mon, 9 Feb 2015 19:45:57 -0800 Alexei Starovoitov <[email protected]> wrote:
samples/bpf/tracex1_user.c new file +8
-0+static void print_netif_receive_skb (void *data,int size) +{ +struct ftrace_raw_netif_receive_skb { +struct trace_entry t; +void *skb; +__u32 len; +__u32 name; + } *e = data;
Same here, there's no guarantee that this structure will always match.
cat /sys/kernel/debug/tracing/events/net/netif_receive_skb/format
name: netif_receive_skb
ID: 975
format:
field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1; field:void * skbaddr; offset:8; size:8; signed:0; field:unsigned int len; offset:16; size:4; signed:0; field:__data_loc char[] name; offset:20; size:4; signed:1;
print fmt: "dev=%s skbaddr=%p len=%u", __get_str(name), REC->skbaddr, REC->len
-- Steve
PATCH v3 5/8 samples: bpf: counting example for kfree_skb tracepoint and write syscall
In reply to Alexei Starovoitovthis example has two probes in one C file that attach to different tracepoints and use two different maps.
1st probe is the similar to dropmon.c. It attaches to kfree_skb tracepoint and count number of packet drops at different locations
2nd probe attaches to syscalls/sys_enter_write and computes a histogram of different write sizes
Usage:
$ sudo tracex2 location 0xffffffff816959a5 count 1
location 0xffffffff816959a5 count 2
557145+0 records in
557145+0 records out
285258240 bytes (285 MB) copied, 1.02379 s, 279 MB/s
syscall write() stats
byte_size : count distribution
1 -> 1 : 3 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 2 | |
32 -> 63 : 3 | |
64 -> 127 : 1 | |
128 -> 255 : 1 | |
256 -> 511 : 0 | |
512 -> 1023 : 1118968 |************************************* |Ctrl-C at any time. Kernel will auto cleanup maps and programs
--- samples/bpf/Makefile | 4 ++ samples/bpf/tracex2_kern.c | 71 +++++++++++++++++++++++++++++++++ samples/bpf/tracex2_user.c | 95 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 170 insertions(+) create mode 100644 samples/bpf/tracex2_kern.c create mode 100644 samples/bpf/tracex2_user.c
samples/bpf/Makefile +4 -0
@@ -8,6 +8,7 @@ hostprogs-y += sockex1 hostprogs-y += sockex2 hostprogs-y += dropmon hostprogs-y += tracex1 +hostprogs-y += tracex2 dropmon-objs := dropmon.o libbpf.o test_verifier-objs := test_verifier.o libbpf.o
@@ -16,12 +17,14 @@ sock_example-objs := sock_example.o libbpf.o sockex1-objs := bpf_load.o libbpf.o sockex1_user.o sockex2-objs := bpf_load.o libbpf.o sockex2_user.o tracex1-objs := bpf_load.o libbpf.o tracex1_user.o +tracex2-objs := bpf_load.o libbpf.o tracex2_user.o # Tell kbuild to always build the programs always := $(hostprogs-y) always += sockex1_kern.o always += sockex2_kern.o always += tracex1_kern.o +always += tracex2_kern.o HOSTCFLAGS += -I$(objtree)/usr/include
@@ -29,6 +32,7 @@ HOSTCFLAGS_bpf_load.o += -I$(objtree)/usr/include -Wno-unused-variable HOSTLOADLIBES_sockex1 += -lelf HOSTLOADLIBES_sockex2 += -lelf HOSTLOADLIBES_tracex1 += -lelf +HOSTLOADLIBES_tracex2 += -lelf # point this to your LLVM backend with bpf support LLC=$(srctree)/tools/bpf/llvm/bld/Debug+Asserts/bin/llc
samples/bpf/tracex2_kern.c new file +71 -0
71 lines not on this page · raw message
samples/bpf/tracex2_user.c new file +95 -0
95 lines not on this page · raw message
Signature · 1 line
1.7.9.5
PATCH v3 6/8 samples: bpf: IO latency analysis (iosnoop/heatmap)
In reply to Alexei StarovoitoveBPF C program attaches to block_rq_issue/block_rq_complete events to calculate IO latency. Then it waits for the first 100 events to compute average latency and uses range [0 .. ave_lat * 2] to record histogram of events in this latency range.
User space reads this histogram map every 2 seconds and prints it as a 'heatmap' using gray shades of text terminal. Black spaces have many events and white spaces have very few events. Left most space is the smallest latency, right most space is the largest latency in the range.
If kernel sees too many events that fall out of histogram range, user space adjusts the range up, so heatmap for next 2 seconds will be more accurate.
Usage:
$ sudo ./tracex3 and do 'sudo dd if=/dev/sda of=/dev/null' in other terminal. Observe IO latencies and how different activity (like 'make kernel') affects it.
Similar experiments can be done for network transmit latencies, syscalls, etc
--- samples/bpf/Makefile | 4 ++ samples/bpf/tracex3_kern.c | 98 ++++++++++++++++++++++++++++ samples/bpf/tracex3_user.c | 152 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 254 insertions(+) create mode 100644 samples/bpf/tracex3_kern.c create mode 100644 samples/bpf/tracex3_user.c
samples/bpf/Makefile +4 -0
@@ -9,6 +9,7 @@ hostprogs-y += sockex2 hostprogs-y += dropmon hostprogs-y += tracex1 hostprogs-y += tracex2 +hostprogs-y += tracex3 dropmon-objs := dropmon.o libbpf.o test_verifier-objs := test_verifier.o libbpf.o
@@ -18,6 +19,7 @@ sockex1-objs := bpf_load.o libbpf.o sockex1_user.o sockex2-objs := bpf_load.o libbpf.o sockex2_user.o tracex1-objs := bpf_load.o libbpf.o tracex1_user.o tracex2-objs := bpf_load.o libbpf.o tracex2_user.o +tracex3-objs := bpf_load.o libbpf.o tracex3_user.o # Tell kbuild to always build the programs always := $(hostprogs-y)
@@ -25,6 +27,7 @@ always += sockex1_kern.o always += sockex2_kern.o always += tracex1_kern.o always += tracex2_kern.o +always += tracex3_kern.o HOSTCFLAGS += -I$(objtree)/usr/include
@@ -33,6 +36,7 @@ HOSTLOADLIBES_sockex1 += -lelf HOSTLOADLIBES_sockex2 += -lelf HOSTLOADLIBES_tracex1 += -lelf HOSTLOADLIBES_tracex2 += -lelf +HOSTLOADLIBES_tracex3 += -lelf # point this to your LLVM backend with bpf support LLC=$(srctree)/tools/bpf/llvm/bld/Debug+Asserts/bin/llc
samples/bpf/tracex3_kern.c new file +98 -0
98 lines not on this page · raw message
samples/bpf/tracex3_user.c new file +152 -0
152 lines not on this page · raw message
Signature · 1 line
1.7.9.5
PATCH v3 7/8 tracing: attach eBPF programs to kprobe/kretprobe
In reply to Alexei Starovoitovintroduce new type of eBPF programs BPF_PROG_TYPE_KPROBE. Such programs are allowed to call the same helper functions as tracing filters, but bpf_context is different:
For tracing filters bpf_context is 6 arguments of tracepoints or syscalls For kprobe filters bpf_context == pt_regs
--- include/linux/ftrace_event.h | 3 +++ include/uapi/linux/bpf.h | 1 + kernel/events/core.c | 5 ++++- kernel/trace/bpf_trace.c | 39 +++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_kprobe.c | 10 +++++++++- 5 files changed, 56 insertions(+), 2 deletions(-)
include/linux/ftrace_event.h +3 -0
@@ -249,6 +249,7 @@ enum {TRACE_EVENT_FL_WAS_ENABLED_BIT ,TRACE_EVENT_FL_USE_CALL_FILTER_BIT ,TRACE_EVENT_FL_TRACEPOINT_BIT , + TRACE_EVENT_FL_KPROBE_BIT, };/*
@@ -262,6 +263,7 @@ enum {* it is best to clear the buffers that used it). * USE_CALL_FILTER - For ftrace internal events, don't use file filter * TRACEPOINT - Event is a tracepoint +* KPROBE - Event is a kprobe */ enum {TRACE_EVENT_FL_FILTERED = (1 <<TRACE_EVENT_FL_FILTERED_BIT ),
@@ -271,6 +273,7 @@ enum {TRACE_EVENT_FL_WAS_ENABLED = (1 <<TRACE_EVENT_FL_WAS_ENABLED_BIT ),TRACE_EVENT_FL_USE_CALL_FILTER = (1 <<TRACE_EVENT_FL_USE_CALL_FILTER_BIT ),TRACE_EVENT_FL_TRACEPOINT = (1 <<TRACE_EVENT_FL_TRACEPOINT_BIT ), + TRACE_EVENT_FL_KPROBE = (1 << TRACE_EVENT_FL_KPROBE_BIT), }; struct ftrace_event_call {
include/uapi/linux/bpf.h +1 -0
@@ -119,6 +119,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_UNSPEC ,BPF_PROG_TYPE_SOCKET_FILTER ,BPF_PROG_TYPE_TRACEPOINT , + BPF_PROG_TYPE_KPROBE, };/* flags for BPF_MAP_UPDATE_ELEM command */
kernel/events/core.c +4 -1
@@ -6288,7 +6288,10 @@ static int perf_event_set_bpf_prog(struct perf_event *event, u32 prog_fd)if (IS_ERR (prog))return PTR_ERR (prog); -if (prog->aux->prog_type !=BPF_PROG_TYPE_TRACEPOINT ) { +if (((event->tp_event->flags &TRACE_EVENT_FL_KPROBE ) && + prog->aux->prog_type !=BPF_PROG_TYPE_KPROBE ) || + (!(event->tp_event->flags &TRACE_EVENT_FL_KPROBE ) && + prog->aux->prog_type !=BPF_PROG_TYPE_TRACEPOINT )) {/* valid fd, but invalid bpf program type */ bpf_prog_put (prog);return -EINVAL ;
kernel/trace/bpf_trace.c +39 -0
@@ -153,3 +153,42 @@ static int __init register_tp_prog_ops(void)return 0 ; }late_initcall (register_tp_prog_ops); + +/* check access to fields of 'struct pt_regs' from BPF program */ +static bool kprobe_prog_is_valid_access (int off,int size,enum bpf_access_type type) +{ +/* check bounds */ +if (off <0 || off >=sizeof (struct pt_regs )) +return false; + +/* only read is allowed */ +if (type !=BPF_READ ) +return false; + +/* disallow misaligned access */ +if (off % size !=0 ) +return false; + +return true; +} +/* kprobe filter programs are allowed to call the same helper functions +* as tracing filters, but bpf_context is different: +* For tracing filters bpf_context is 6 arguments of tracepoints or syscalls +* For kprobe filters bpf_context == pt_regs +*/ +static struct bpf_verifier_ops kprobe_prog_ops = { + .get_func_proto = tp_prog_func_proto, + .is_valid_access = kprobe_prog_is_valid_access, +}; + +static struct bpf_prog_type_list kprobe_tl = { + .ops = &kprobe_prog_ops, + .type =BPF_PROG_TYPE_KPROBE , +}; + +static int __init register_kprobe_prog_ops (void ) +{ +bpf_register_prog_type (&kprobe_tl); +return 0 ; +} +late_initcall (register_kprobe_prog_ops);
kernel/trace/trace_kprobe.c +9 -1
@@ -1134,11 +1134,15 @@ static void kprobe_perf_func(struct trace_kprobe *tk, struct pt_regs *regs) { struct ftrace_event_call *call = &tk->tp.call; +struct bpf_prog *prog = call->prog;struct kprobe_trace_entry_head *entry;struct hlist_head *head;int size, __size, dsize;int rctx; +if (prog && !trace_call_bpf (prog, regs)) +return ; + head =this_cpu_ptr (call->perf_events);if (hlist_empty (head))return ;
@@ -1165,11 +1169,15 @@ kretprobe_perf_func(struct trace_kprobe *tk, struct kretprobe_instance *ri, struct pt_regs *regs) {struct ftrace_event_call *call = &tk->tp.call; +struct bpf_prog *prog = call->prog;struct kretprobe_trace_entry_head *entry;struct hlist_head *head;int size, __size, dsize;int rctx; +if (prog && !trace_call_bpf (prog, regs)) +return ; + head =this_cpu_ptr (call->perf_events);if (hlist_empty (head))return ;
@@ -1286,7 +1294,7 @@ static int register_kprobe_event(struct trace_kprobe *tk)kfree (call->print_fmt);return -ENODEV ; } - call->flags =0 ; + call->flags =TRACE_EVENT_FL_KPROBE ; call->class->reg = kprobe_register; call->data = tk; ret =trace_add_event_call (call);
Signature · 1 line
1.7.9.5
PATCH v3 8/8 samples: bpf: simple kprobe example
In reply to Alexei Starovoitovthe logic of the example is similar to tracex2, but syscall 'write' statistics is capturead from kprobe placed at sys_write function instead of through syscall instrumentation.
Also tracex4_kern.c has a different way of doing log2 in C.
Note, unlike tracepoint and syscall programs, kprobe programs receive 'struct pt_regs' as an input. It's responsibility of the program author or higher level dynamic tracing tool to match registers to function arguments. Since pt_regs are architecture dependent, programs are also arch dependent, unlike tracepoint/syscalls programs which are universal.
Usage:
$ sudo tracex4 2216443+0 records in 2216442+0 records out 1134818304 bytes (1.1 GB) copied, 2.00746 s, 565 MB/s
kprobe sys_write() stats
byte_size : count distribution
1 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 1 | |
128 -> 255 : 0 | |
256 -> 511 : 0 | |
512 -> 1023 : 2214734 |************************************* |--- samples/bpf/Makefile | 4 +++ samples/bpf/bpf_load.c | 3 ++ samples/bpf/tracex4_kern.c | 36 +++++++++++++++++++ samples/bpf/tracex4_user.c | 83 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 126 insertions(+) create mode 100644 samples/bpf/tracex4_kern.c create mode 100644 samples/bpf/tracex4_user.c
samples/bpf/Makefile +4 -0
@@ -10,6 +10,7 @@ hostprogs-y += dropmon hostprogs-y += tracex1 hostprogs-y += tracex2 hostprogs-y += tracex3 +hostprogs-y += tracex4 dropmon-objs := dropmon.o libbpf.o test_verifier-objs := test_verifier.o libbpf.o
@@ -20,6 +21,7 @@ sockex2-objs := bpf_load.o libbpf.o sockex2_user.o tracex1-objs := bpf_load.o libbpf.o tracex1_user.o tracex2-objs := bpf_load.o libbpf.o tracex2_user.o tracex3-objs := bpf_load.o libbpf.o tracex3_user.o +tracex4-objs := bpf_load.o libbpf.o tracex4_user.o # Tell kbuild to always build the programs always := $(hostprogs-y)
@@ -28,6 +30,7 @@ always += sockex2_kern.o always += tracex1_kern.o always += tracex2_kern.o always += tracex3_kern.o +always += tracex4_kern.o HOSTCFLAGS += -I$(objtree)/usr/include
@@ -37,6 +40,7 @@ HOSTLOADLIBES_sockex2 += -lelf HOSTLOADLIBES_tracex1 += -lelf HOSTLOADLIBES_tracex2 += -lelf HOSTLOADLIBES_tracex3 += -lelf +HOSTLOADLIBES_tracex4 += -lelf # point this to your LLVM backend with bpf support LLC=$(srctree)/tools/bpf/llvm/bld/Debug+Asserts/bin/llc
samples/bpf/bpf_load.c +3 -0
@@ -32,6 +32,7 @@ int prog_cnt;static int load_and_attach (const char *event,struct bpf_insn *prog,int size) {bool is_socket =strncmp (event,"socket" ,6 ) ==0 ; +bool is_kprobe =strncmp (event,"events/kprobes/" ,15 ) ==0 ;enum bpf_prog_type prog_type;char path[256 ] =DEBUGFS ;char buf[32 ];
@@ -45,6 +46,8 @@ static int load_and_attach(const char *event, struct bpf_insn *prog, int size)if (is_socket) prog_type =BPF_PROG_TYPE_SOCKET_FILTER ; +else if (is_kprobe) + prog_type =BPF_PROG_TYPE_KPROBE ;else prog_type =BPF_PROG_TYPE_TRACEPOINT ;
samples/bpf/tracex4_kern.c new file +36 -0
@@ -0,0 +1,36 @@ +#include <linux/skbuff.h> +#include <linux/netdevice.h> +#include <uapi/linux/bpf.h> +#include <trace/bpf_trace.h> +#include "bpf_helpers.h" + +static unsigned int log2l (unsigned long long n) +{ +#define S (k) if (n >= (1ull << k)) { i += k; n >>= k; } +int i = -(n ==0 ); +S (32 );S (16 );S (8 );S (4 );S (2 );S (1 ); +return i; +#undef S +} + +struct bpf_map_def SEC("maps") my_hist_map = { + .type = BPF_MAP_TYPE_ARRAY, + .key_size = sizeof(u32), + .value_size = sizeof(long), + .max_entries = 64, +}; + +SEC("events/kprobes/sys_write") +int bpf_prog4 (struct pt_regs *regs) +{ +long write_size = regs->dx;/* $rdx contains 3rd argument to a function */ +long init_val =1 ; +void *value; +u32 index =log2l (write_size); + + value =bpf_map_lookup_elem (&my_hist_map, &index); +if (value) +__sync_fetch_and_add ((long *)value,1 ); +return 0 ; +} +char _license[] SEC("license") = "GPL";
samples/bpf/tracex4_user.c new file +83 -0
83 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Steven Rostedt
maintainerQuoted 3 lines · On Mon, 9 Feb 2015 19:45:53 -0800 Alexei Starovoitov <[email protected]> wrote:
So the overhead of realistic bpf program is 5.05963/4.80074 = ~5% which is faster than
perf_eventfiltering: 5.69732/4.80074 = ~18% or ftrace filtering: 6.50091/4.80074 = ~35%
Come to think of it, this is comparing apples to oranges, as you move the filtering before the recording. It would be interesting to see the ftrace speed up, if it were to use eBPF instead of its own filtering. Maybe that 35% is the filter part, and not the discard part.
I just tried the dd test with count==1234 and count!=1234 and the one that drops events is only slightly slower. In this case it does seem that the most overhead is in the filter logic.
But by moving it before the recording, we can not use the fields defined in the format files, as the parameters and the fields do not match in most trace points. And to use the parameters, as I have stated, there's no interface to know what those parameters are, then filtering on them is a one shot deal. Might as well write a module and hook directly to the tracepoint and do the filtering natively. That would be faster than BPF too.
My point is, what's the use case? If you filter before recording, you can not use the fields of the tracepoint. That limits you to filtering only syscalls, and perhaps kprobes.
-- Steve