64-bit BPF insn set and tracing filters
Subsystems (9)
v2 cover 64-bit BPF insn set and tracing filters
Hi All,
this patch set addresses main sticking points of the previous discussion:
http://thread.gmane.org/gmane.linux.kernel/1605783
Main difference:
. all components are now in one place
tools/bpf/llvm - standalone LLVM backend for extended BPF instruction set
. regs.si, regs.di accessors are replaced with arg1, arg2
. compiler enforces presence of 'license' string in source C code
kernel enforces GPL compatibility of BPF program
Why bother with it?
Current 32-bit BPF is safe, but limited.
kernel modules are 'all-goes', but not-safe.
Extended 64-bit BPF provides safe and restricted kernel modules.
Just like the first two, extended BPF can be used for all sorts of things. Initially for tracing/debugging/[ks]tap-like without vmlinux around, then for networking, security, etc
To make exising kernel modules safe the x86 disassembler and code analyzer are needed. We've tried to follow that path. Disassembler was straight forward, but x86 analyzer was becoming unbearably complex due to variety of addressing modes, so we started to hack GCC to reduce output x86 insns and facing the headache of redoing disasm/analyzer for arm and other arhcs. Plus there is old 32-bit bpf insn set already.
On one side extended BPF is a 64-bit extension to current BPF.
On the other side it's a common subset of x86-64/aarch64/... ISAs:
a generic 64-bit insn set that can be JITed to native HW one to one.
Tested on x86-64 and i386.
BPF core was tested on arm-v7.
V2 vs V1 details:
0001-Extended-BPF-core-framework:
no difference to instruction set
new bpf image format to include license string and enforcement during load
0002-Extended-BPF-JIT-for-x86-64: no changes
0003-Extended-BPF-64-bit-BPF-design-document: no changes
0004-Revert-x86-ptrace-Remove-unused-regs_get_argument:
restoring Masami's get_Nth_argument accessor to simplify kprobe filters
0005-use-BPF-in-tracing-filters: minor changes to switch from si/di to argN
0006-LLVM-BPF-backend: standalone BPF backend for LLVM
requires: apt-get install llvm-3.2-dev clang compiles in 7 seconds, links with the rest of llvm infra compatible with llvm 3.2, 3.3 and just released 3.4 Written in llvm coding style and llvm license, so it can be upstreamed into llvm tree
0007-tracing-filter-examples-in-BPF:
tools/bpf/filter_check: userspace pre-checker of BPF filter runs the same bpf_check() code as kernel does
tools/bpf/examples/netif_rcv.c:
-----
#define DESC(NAME) __attribute__((section(NAME), used)) void my_filter(struct bpf_context *ctx)
{
char devname[4] = "lo";
struct net_device *dev;
struct sk_buff *skb = 0;
/*
* for tracepoints arg1 is the 1st arg of TP_ARGS() macro
* defined in include/trace/events/.h
* for kprobe events arg1 is the 1st arg of probed function
*/
skb = (struct sk_buff *)ctx->arg1;
dev = bpf_load_pointer(&skb->dev);
if (bpf_memcmp(dev->name, devname, 2) == 0) {
char fmt[] = "skb %p dev %p \n";
bpf_trace_printk(fmt, sizeof(fmt), (long)skb, (long)dev, 0);
}
}/* filter code license: */
char license[] DESC("license") = "GPL";
-----
$cd tools/bpf/examples
$make
compile it using clang+llvm_bpf
$make check
check safety
$make try
attach this filter to net:netif_receive_skb and kprobe __netif_receive_skb and try ping
dropmon.c is a demo of faster version of net_dropmonitor:
-----
/* attaches to /sys/kernel/debug/tracing/events/skb/kfree_skb */ void dropmon(struct bpf_context *ctx)
{
void *loc;
uint64_t *drop_cnt;
/*
* skb:kfree_skb is defined as:
* TRACE_EVENT(kfree_skb,
* TP_PROTO(struct sk_buff *skb, void *location),
* so ctx->arg2 is 'location'
*/
loc = (void *)ctx->arg2;
drop_cnt = bpf_table_lookup(ctx, 0, &loc);
if (drop_cnt) {
__sync_fetch_and_add(drop_cnt, 1);
} else {
uint64_t init = 0;
bpf_table_update(ctx, 0, &loc, &init);
}
}
struct bpf_table t[] DESC("bpftables") = {
{BPF_TABLE_HASH, sizeof(void *), sizeof(uint64_t), 4096, 0}
};/* filter code license: */
char l[] DESC("license") = "GPL v2";
-----
It's not fully functional yet. Minimal work remaining to implement bpf_table_lookup()/bpf_table_update() in kernel
and userspace access to filter's table.
This example demonstrates that some interesting events don't have to be always fed into userspace, but can be pre-processed in kernel. tools/perf/scripts/python/net_dropmonitor.py would need to read bpf table from kernel (via debugfs or netlink) and print it in a nice format.
Same as in V1 BPF filters are called before tracepoints store the TP_STRUCT fields, since performance advantage is significant.
TODO:
- complete 'dropmonitor': finish bpf hashtable and userspace access to it
- add multi-probe support, so that one C program can specify multiple functions for different probe points (similar to [ks]tap)
- add 'lsmod' like facility to list all loaded BPF filters
- add -m32 flag to llvm, so that C pointers are 32-bit, but emitted BPF is still 64-bit.
Useful for kernel struct walking in BPF program on 32-bit archs
- finish testing on arm
- teach llvm to store line numbers in BPF image, so that bpf_check() can print nice errors when program is not safe
- allow read-only "strings" in C code
today analyzer can only verify safety of: char s[] = "string"; bpf_print(s); but bpf_print("string"); cannot be proven yet
- write JIT from BPF to aarch64
If direction is ok, I would like to commit this part to a branch of tip tree or staging tree and continue working there.
Future deltas will be easier to review.
Thanks
Alexei Starovoitov (7):
Extended BPF core framework
Extended BPF JIT for x86-64
Extended BPF (64-bit BPF) design document
Revert "x86/ptrace: Remove unused regs_get_argument_nth API" use BPF in tracing filters
LLVM BPF backend
tracing filter examples in BPF
Documentation/bpf_jit.txt | 204 ++++ arch/x86/Kconfig | 1 + arch/x86/include/asm/ptrace.h | 3 + arch/x86/kernel/ptrace.c | 24 + arch/x86/net/Makefile | 1 + arch/x86/net/bpf64_jit_comp.c | 625 ++++++++++++ arch/x86/net/bpf_jit_comp.c | 23 +- arch/x86/net/bpf_jit_comp.h | 35 + include/linux/bpf.h | 149 +++ include/linux/bpf_jit.h | 134 +++ include/linux/ftrace_event.h | 5 + include/trace/bpf_trace.h | 41 + include/trace/ftrace.h | 17 + kernel/Makefile | 1 + kernel/bpf_jit/Makefile | 3 + kernel/bpf_jit/bpf_check.c | 1054 ++++++++++++++++++++ kernel/bpf_jit/bpf_run.c | 511 ++++++++++ kernel/trace/Kconfig | 1 + kernel/trace/Makefile | 1 + kernel/trace/bpf_trace_callbacks.c | 193 ++++ kernel/trace/trace.c | 7 + kernel/trace/trace.h | 11 +- kernel/trace/trace_events.c | 9 +- kernel/trace/trace_events_filter.c | 61 +- kernel/trace/trace_kprobe.c | 15 +- lib/Kconfig.debug | 15 + tools/bpf/examples/Makefile | 71 ++ tools/bpf/examples/README.txt | 59 ++ tools/bpf/examples/dropmon.c | 40 + tools/bpf/examples/netif_rcv.c | 34 + tools/bpf/filter_check/Makefile | 32 + tools/bpf/filter_check/README.txt | 3 + tools/bpf/filter_check/trace_filter_check.c | 115 +++ tools/bpf/llvm/LICENSE.TXT | 70 ++ tools/bpf/llvm/Makefile.rules | 641 ++++++++++++ tools/bpf/llvm/README.txt | 23 + tools/bpf/llvm/bld/.gitignore | 2 + tools/bpf/llvm/bld/Makefile | 27 + tools/bpf/llvm/bld/Makefile.common | 14 + tools/bpf/llvm/bld/Makefile.config | 124 +++ .../llvm/bld/include/llvm/Config/AsmParsers.def | 8 + .../llvm/bld/include/llvm/Config/AsmPrinters.def | 9 + .../llvm/bld/include/llvm/Config/Disassemblers.def | 8 + tools/bpf/llvm/bld/include/llvm/Config/Targets.def | 9 + .../bpf/llvm/bld/include/llvm/Support/DataTypes.h | 96 ++ tools/bpf/llvm/bld/lib/Makefile | 11 + .../llvm/bld/lib/Target/BPF/InstPrinter/Makefile | 10 + .../llvm/bld/lib/Target/BPF/MCTargetDesc/Makefile | 11 + tools/bpf/llvm/bld/lib/Target/BPF/Makefile | 17 + .../llvm/bld/lib/Target/BPF/TargetInfo/Makefile | 10 + tools/bpf/llvm/bld/lib/Target/Makefile | 11 + tools/bpf/llvm/bld/tools/Makefile | 12 + tools/bpf/llvm/bld/tools/llc/Makefile | 15 + tools/bpf/llvm/lib/Target/BPF/BPF.h | 30 + tools/bpf/llvm/lib/Target/BPF/BPF.td | 29 + tools/bpf/llvm/lib/Target/BPF/BPFAsmPrinter.cpp | 100 ++ tools/bpf/llvm/lib/Target/BPF/BPFCFGFixup.cpp | 62 ++ tools/bpf/llvm/lib/Target/BPF/BPFCallingConv.td | 24 + tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.cpp | 36 + tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.h | 35 + tools/bpf/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp | 182 ++++ tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.cpp | 676 +++++++++++++ tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.h | 105 ++ tools/bpf/llvm/lib/Target/BPF/BPFInstrFormats.td | 29 + tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.cpp | 162 +++ tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.h | 53 + tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.td | 455 +++++++++ tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.cpp | 77 ++ tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.h | 40 + tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.cpp | 122 +++ tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.h | 65 ++ tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.td | 39 + tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.cpp | 23 + tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.h | 33 + tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.cpp | 72 ++ tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.h | 69 ++ .../lib/Target/BPF/InstPrinter/BPFInstPrinter.cpp | 79 ++ .../lib/Target/BPF/InstPrinter/BPFInstPrinter.h | 34 + .../lib/Target/BPF/MCTargetDesc/BPFAsmBackend.cpp | 85 ++ .../llvm/lib/Target/BPF/MCTargetDesc/BPFBaseInfo.h | 33 + .../Target/BPF/MCTargetDesc/BPFELFObjectWriter.cpp | 119 +++ .../lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h | 34 + .../Target/BPF/MCTargetDesc/BPFMCCodeEmitter.cpp | 120 +++ .../lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.h | 67 ++ .../Target/BPF/MCTargetDesc/BPFMCTargetDesc.cpp | 115 +++ .../lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.h | 56 ++ .../lib/Target/BPF/TargetInfo/BPFTargetInfo.cpp | 13 + tools/bpf/llvm/tools/llc/llc.cpp | 381 +++++++ 88 files changed, 8255 insertions(+), 25 deletions(-) create mode 100644 Documentation/bpf_jit.txt create mode 100644 arch/x86/net/bpf64_jit_comp.c create mode 100644 arch/x86/net/bpf_jit_comp.h create mode 100644 include/linux/bpf.h create mode 100644 include/linux/bpf_jit.h create mode 100644 include/trace/bpf_trace.h create mode 100644 kernel/bpf_jit/Makefile create mode 100644 kernel/bpf_jit/bpf_check.c create mode 100644 kernel/bpf_jit/bpf_run.c create mode 100644 kernel/trace/bpf_trace_callbacks.c create mode 100644 tools/bpf/examples/Makefile create mode 100644 tools/bpf/examples/README.txt create mode 100644 tools/bpf/examples/dropmon.c create mode 100644 tools/bpf/examples/netif_rcv.c create mode 100644 tools/bpf/filter_check/Makefile create mode 100644 tools/bpf/filter_check/README.txt create mode 100644 tools/bpf/filter_check/trace_filter_check.c create mode 100644 tools/bpf/llvm/LICENSE.TXT create mode 100644 tools/bpf/llvm/Makefile.rules create mode 100644 tools/bpf/llvm/README.txt create mode 100644 tools/bpf/llvm/bld/.gitignore create mode 100644 tools/bpf/llvm/bld/Makefile create mode 100644 tools/bpf/llvm/bld/Makefile.common create mode 100644 tools/bpf/llvm/bld/Makefile.config create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/AsmParsers.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/AsmPrinters.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/Disassemblers.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/Targets.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Support/DataTypes.h create mode 100644 tools/bpf/llvm/bld/lib/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/InstPrinter/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/MCTargetDesc/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/TargetInfo/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/Makefile create mode 100644 tools/bpf/llvm/bld/tools/Makefile create mode 100644 tools/bpf/llvm/bld/tools/llc/Makefile create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPF.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPF.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFAsmPrinter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFCFGFixup.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFCallingConv.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrFormats.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/InstPrinter/BPFInstPrinter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/InstPrinter/BPFInstPrinter.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFAsmBackend.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFBaseInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFELFObjectWriter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/TargetInfo/BPFTargetInfo.cpp create mode 100644 tools/bpf/llvm/tools/llc/llc.cpp
Signature · 1 line
1.7.9.5
RFC PATCH v2 1/7 Extended BPF core framework
Extended BPF (or 64-bit BPF) is an instruction set to
create safe dynamically loadable filters that can call fixed set of kernel functions and take generic bpf_context as an input. BPF filter is a glue between kernel functions and bpf_context. Different kernel subsystems can define their own set of available functions and alter BPF machinery for specific use case.
include/linux/bpf.h - instruction set definition kernel/bpf_jit/bpf_check.c - code safety checker/static analyzer kernel/bpf_jit/bpf_run.c - emulator for archs without BPF64_JIT
Extended BPF instruction set is designed for efficient mapping to native instructions on 64-bit CPUs
--- include/linux/bpf.h | 149 +++++++ include/linux/bpf_jit.h | 134 ++++++ kernel/Makefile | 1 + kernel/bpf_jit/Makefile | 3 + kernel/bpf_jit/bpf_check.c | 1054 ++++++++++++++++++++++++++++++++++++++++++++ kernel/bpf_jit/bpf_run.c | 511 +++++++++++++++++++++ lib/Kconfig.debug | 15 + 7 files changed, 1867 insertions(+) create mode 100644 include/linux/bpf.h create mode 100644 include/linux/bpf_jit.h create mode 100644 kernel/bpf_jit/Makefile create mode 100644 kernel/bpf_jit/bpf_check.c create mode 100644 kernel/bpf_jit/bpf_run.c
include/linux/bpf.h new file +149 -0
@@ -0,0 +1,149 @@ +/* 64-bit BPF is Copyright (c) 2011-2014, PLUMgrid, http://plumgrid.com */ + +#ifndef __LINUX_BPF_H__ +#define __LINUX_BPF_H__ + +#include <linux/types.h> + +struct bpf_insn { +__u8 code;/* opcode */ +__u8 a_reg:4 ;/* dest register*/ +__u8 x_reg:4 ;/* source register */ +__s16 off;/* signed offset */ +__s32 imm;/* signed immediate constant */ +}; + +struct bpf_table { +__u32 type; +__u32 key_size; +__u32 elem_size; +__u32 max_entries; +__u32 param1;/* meaning is table-dependent */ +}; + +enum bpf_table_type { +BPF_TABLE_HASH =1 , +BPF_TABLE_LPM +}; + +/* maximum number of insns and tables in a BPF program */ +#define MAX_BPF_INSNS 4096 +#define MAX_BPF_TABLES 64 +#define MAX_BPF_STRTAB_SIZE 1024 + +/* pointer to bpf_context is the first and only argument to BPF program +* its definition is use-case specific */ +struct bpf_context ; + +/* bpf_add|sub|...: a += x +* bpf_mov: a = x +* bpf_bswap: bswap a */ +#define BPF_INSN_ALU (op, a, x) \ + (struct bpf_insn){BPF_ALU|BPF_OP(op)|BPF_X, a, x, 0, 0} + +/* bpf_add|sub|...: a += imm +* bpf_mov: a = imm */ +#define BPF_INSN_ALU_IMM (op, a, imm) \ + (struct bpf_insn){BPF_ALU|BPF_OP(op)|BPF_K, a, 0, 0, imm} + +/* a = *(uint *) (x + off) */ +#define BPF_INSN_LD (size, a, x, off) \ + (struct bpf_insn){BPF_LDX|BPF_SIZE(size)|BPF_REL, a, x, off, 0} + +/* *(uint *) (a + off) = x */ +#define BPF_INSN_ST (size, a, off, x) \ + (struct bpf_insn){BPF_STX|BPF_SIZE(size)|BPF_REL, a, x, off, 0} + +/* *(uint *) (a + off) = imm */ +#define BPF_INSN_ST_IMM (size, a, off, imm) \ + (struct bpf_insn){BPF_ST|BPF_SIZE(size)|BPF_REL, a, 0, off, imm} + +/* lock *(uint *) (a + off) += x */ +#define BPF_INSN_XADD (size, a, off, x) \ + (struct bpf_insn){BPF_STX|BPF_SIZE(size)|BPF_XADD, a, x, off, 0} + +/* if (a 'op' x) pc += off else fall through */ +#define BPF_INSN_JUMP (op, a, x, off) \ + (struct bpf_insn){BPF_JMP|BPF_OP(op)|BPF_X, a, x, off, 0} + +/* if (a 'op' imm) pc += off else fall through */ +#define BPF_INSN_JUMP_IMM (op, a, imm, off) \ + (struct bpf_insn){BPF_JMP|BPF_OP(op)|BPF_K, a, 0, off, imm} + +#define BPF_INSN_RET () \ + (struct bpf_insn){BPF_RET|BPF_K, 0, 0, 0, 0} + +#define BPF_INSN_CALL (fn_code) \ + (struct bpf_insn){BPF_JMP|BPF_CALL, 0, 0, 0, fn_code} + +/* Instruction classes */ +#define BPF_CLASS (code) ((code) & 0x07) +#define BPF_LD 0x00 +#define BPF_LDX 0x01 +#define BPF_ST 0x02 +#define BPF_STX 0x03 +#define BPF_ALU 0x04 +#define BPF_JMP 0x05 +#define BPF_RET 0x06 + +/* ld/ldx fields */ +#define BPF_SIZE (code) ((code) & 0x18) +#define BPF_W 0x00 +#define BPF_H 0x08 +#define BPF_B 0x10 +#define BPF_DW 0x18 +#define BPF_MODE (code) ((code) & 0xe0) +#define BPF_IMM 0x00 +#define BPF_ABS 0x20 +#define BPF_IND 0x40 +#define BPF_MEM 0x60 +#define BPF_LEN 0x80 +#define BPF_MSH 0xa0 +#define BPF_REL 0xc0 +#define BPF_XADD 0xe0/* exclusive add */ + +/* alu/jmp fields */ +#define BPF_OP (code) ((code) & 0xf0) +#define BPF_ADD 0x00 +#define BPF_SUB 0x10 +#define BPF_MUL 0x20 +#define BPF_DIV 0x30 +#define BPF_OR 0x40 +#define BPF_AND 0x50 +#define BPF_LSH 0x60 +#define BPF_RSH 0x70/* logical shift right */ +#define BPF_NEG 0x80 +#define BPF_MOD 0x90 +#define BPF_XOR 0xa0 +#define BPF_MOV 0xb0/* mov reg to reg */ +#define BPF_ARSH 0xc0/* sign extending arithmetic shift right */ +#define BPF_BSWAP32 0xd0/* swap lower 4 bytes of 64-bit register */ +#define BPF_BSWAP64 0xe0/* swap all 8 bytes of 64-bit register */ + +#define BPF_JA 0x00 +#define BPF_JEQ 0x10/* jump == */ +#define BPF_JGT 0x20/* GT is unsigned '>', JA in x86 */ +#define BPF_JGE 0x30/* GE is unsigned '>=', JAE in x86 */ +#define BPF_JSET 0x40 +#define BPF_JNE 0x50/* jump != */ +#define BPF_JSGT 0x60/* SGT is signed '>', GT in x86 */ +#define BPF_JSGE 0x70/* SGE is signed '>=', GE in x86 */ +#define BPF_CALL 0x80/* function call */ +#define BPF_SRC (code) ((code) & 0x08) +#define BPF_K 0x00 +#define BPF_X 0x08 + +/* 64-bit registers */ +#define R0 0 +#define R1 1 +#define R2 2 +#define R3 3 +#define R4 4 +#define R5 5 +#define R6 6 +#define R7 7 +#define R8 8 +#define R9 9 +#define __fp__ 10 + +#endif /* __LINUX_BPF_H__ */
include/linux/bpf_jit.h new file +134 -0
134 lines not on this page · raw message
kernel/Makefile +1 -0
@@ -83,6 +83,7 @@ obj-$(CONFIG_TRACING) += trace/ obj-$(CONFIG_TRACE_CLOCK) += trace/ obj-$(CONFIG_RING_BUFFER) += trace/ obj-$(CONFIG_TRACEPOINTS) += trace/ +obj-$(CONFIG_BPF64) += bpf_jit/ obj-$(CONFIG_IRQ_WORK) += irq_work.o obj-$(CONFIG_CPU_PM) += cpu_pm.o
kernel/bpf_jit/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +obj-$(CONFIG_BPF64) += bpf_check.o +obj-$(CONFIG_BPF64) += bpf_run.o +
kernel/bpf_jit/bpf_check.c new file +1054 -0
1,054 lines not on this page · raw message
kernel/bpf_jit/bpf_run.c new file +511 -0
511 lines not on this page · raw message
lib/Kconfig.debug +15 -0
@@ -1615,3 +1615,18 @@ source "samples/Kconfig" source "lib/Kconfig.kgdb" +# Used by archs to tell that they support 64-bit BPF JIT +config HAVE_BPF64_JIT + bool + +config BPF64 + bool "Enable 64-bit BPF instruction set support" + help + Enable this option to support 64-bit BPF programs + +config BPF64_JIT + bool "Enable 64-bit BPF JIT compiler" + depends on BPF64 && HAVE_BPF64_JIT + help + Enable Just-In-Time compiler for 64-bit BPF programs +
Signature · 1 line
1.7.9.5
RFC PATCH v2 2/7 Extended BPF JIT for x86-64
In reply to Alexei StarovoitovJust-In-Time compiler that maps 64-bit BPF instructions to x86-64 instructions.
Most BPF instructions have one to one mapping.
Every BPF register maps to one x86-64 register:
R0 -> rax
R1 -> rdi
R2 -> rsi
R3 -> rdx
R4 -> rcx
R5 -> r8
R6 -> rbx
R7 -> r13
R8 -> r14
R9 -> r15
FP -> rbp
BPF calling convention is defined as:
R0 - return value from in-kernel function
R1-R5 - arguments from BPF program to in-kernel function R6-R9 - callee saved registers that in-kernel function will preserve R10 - read-only frame pointer to access stack
so BPF calling convention maps directly to x86-64 calling convention.
Allowing zero-overhead calls between BPF filter and safe kernel functions
--- arch/x86/Kconfig | 1 + arch/x86/net/Makefile | 1 + arch/x86/net/bpf64_jit_comp.c | 625 +++++++++++++++++++++++++++++++++++++++++ arch/x86/net/bpf_jit_comp.c | 23 +- arch/x86/net/bpf_jit_comp.h | 35 +++ 5 files changed, 665 insertions(+), 20 deletions(-) create mode 100644 arch/x86/net/bpf64_jit_comp.c create mode 100644 arch/x86/net/bpf_jit_comp.h
arch/x86/Kconfig +1 -0
@@ -94,6 +94,7 @@ config X86 select GENERIC_CLOCKEVENTS_MIN_ADJUST select IRQ_FORCED_THREADING select HAVE_BPF_JIT if X86_64 + select HAVE_BPF64_JIT if X86_64 select HAVE_ARCH_TRANSPARENT_HUGEPAGE select CLKEVT_I8253 select ARCH_HAVE_NMI_SAFE_CMPXCHG
arch/x86/net/Makefile +1 -0
@@ -2,3 +2,4 @@ # Arch-specific network modules # obj-$(CONFIG_BPF_JIT) += bpf_jit.o bpf_jit_comp.o +obj-$(CONFIG_BPF64_JIT) += bpf64_jit_comp.o
arch/x86/net/bpf64_jit_comp.c new file +625 -0
625 lines not on this page · raw message
arch/x86/net/bpf_jit_comp.c +3 -20
@@ -13,6 +13,7 @@#include <linux/filter.h> #include <linux/if_vlan.h> #include <linux/random.h> +#include "bpf_jit_comp.h" /* * Conventions :
@@ -112,16 +113,6 @@ do { \#define SEEN_XREG 2/* ebx is used */ #define SEEN_MEM 4/* use mem[] for temporary storage */ -static inline void bpf_flush_icache (void *start,void *end) -{ -mm_segment_t old_fs =get_fs (); - -set_fs (KERNEL_DS ); -smp_wmb (); -flush_icache_range ((unsigned long )start, (unsigned long )end); -set_fs (old_fs); -} -#define CHOOSE_LOAD_FUNC (K , func) \ ((int)K < 0 ? ((int)K >= SKF_LL_OFF ? func##_negative_offset : func) : func##_positive_offset)
@@ -145,16 +136,8 @@ static int pkt_type_offset(void)return -1 ; } -struct bpf_binary_header { -unsigned int pages; -/* Note : for security reasons, bpf code will follow a randomly -* sized amount of int3 instructions -*/ -u8 image[]; -}; - -static struct bpf_binary_header *bpf_alloc_binary (unsigned int proglen, -u8 **image_ptr) +struct bpf_binary_header *bpf_alloc_binary (unsigned int proglen, +u8 **image_ptr) {unsigned int sz, hole;struct bpf_binary_header *header;
arch/x86/net/bpf_jit_comp.h new file +35 -0
@@ -0,0 +1,35 @@ +/* bpf_jit_comp.h : BPF filter alloc/free routines +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License +* as published by the Free Software Foundation; version 2 +* of the License. +*/ +#ifndef __BPF_JIT_COMP_H +#define __BPF_JIT_COMP_H + +#include <linux/uaccess.h> +#include <asm/cacheflush.h> + +struct bpf_binary_header { +unsigned int pages; +/* Note : for security reasons, bpf code will follow a randomly +* sized amount of int3 instructions +*/ +u8 image[]; +}; + +static inline void bpf_flush_icache (void *start,void *end) +{ +mm_segment_t old_fs =get_fs (); + +set_fs (KERNEL_DS ); +smp_wmb (); +flush_icache_range ((unsigned long )start, (unsigned long )end); +set_fs (old_fs); +} + +struct bpf_binary_header *bpf_alloc_binary (unsigned int proglen, +u8 **image_ptr); + +#endif
Signature · 1 line
1.7.9.5
RFC PATCH v2 3/7 Extended BPF (64-bit BPF) design document
In reply to Alexei Starovoitov--- Documentation/bpf_jit.txt | 204 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 204 insertions(+) create mode 100644 Documentation/bpf_jit.txt
Documentation/bpf_jit.txt new file +204 -0
@@ -0,0 +1,204 @@ +Subject: extended BPF or 64-bit BPF + +Q: What is BPF? +A: Safe dynamically loadable 32-bit program that can access skb->data via +sk_load_byte/half/word calls or seccomp_data. Can be attached to sockets, +to netfilter xtables, seccomp. In case of sockets/xtables input is skb. +In case of seccomp input is struct seccomp_data. + +Q: What is extended BPF? +A: Safe dynamically loadable 64-bit program that can call fixed set +of kernel functions and takes generic bpf_context as an input. +BPF program is a glue between kernel functions and bpf_context. +Different kernel subsystems can define their own set of available functions +and alter BPF machinery for specific use case. + +Example 1: +when function set is {bpf_load_byte/half/word} and bpf_context=skb +the extended BPF is equivalent to original BPF (w/o negative offset extensions), +since any such extended BPF program will only be able to load data from skb +and interpret it. + +Example 2: +when function set is {empty} and bpf_context=seccomp_data, +the extended BPF is equivalent to original seccomp BPF with simpler programs +and can immediately take advantage of extended BPF-JIT. +(original BPF-JIT doesn't work for seccomp) + +Example 3: +when function set is {bpf_load_xxx + bpf_table_lookup} and bpf_context=skb +the extended BPF can be used to implement network analytics in tcpdump. +Like counting all tcp flows through the dev or filtering for specific +set of IP addresses. + +Example 4: +when function set is {load_xxx + table_lookup + trace_printk} and +bpf_context=pt_regs, the extended BPF is used to implement systemtap-like +tracing filters + +Extended Instruction Set was designed with these goals: +- write programs in restricted C and compile into BPF with GCC/LLVM +- just-in-time map to modern 64-bit CPU with minimal performance overhead + over two steps: C -> BPF -> native code +- guarantee termination and safety of BPF program in kernel + with simple algorithm + +Writing filters in tcpdump syntax or in systemtap language is difficult. +Same filter done in C is easier to understand. +GCC/LLVM-bpf backend is optional. +Extended BPF can be coded with macroses from bpf.h just like original BPF. + +Minimal performance overhead is achieved by having one to one mapping +between BPF insns and native insns, and one to one mapping between BPF +registers and native registers on 64-bit CPUs + +Extended BPF allows jump forward and backward for two reasons: +to reduce branch mispredict penalty compiler moves cold basic blocks out of +fall-through path and to reduce code duplication that would be unavoidable +if only jump forward was available. +To guarantee termination simple non-recursive depth-first-search verifies +that there are no back-edges (no loops in the program), program is a DAG +with root at the first insn, all branches end at the last RET insn and +all instructions are reachable. +(Original BPF actually allows unreachable insns, but that's a bug) + +Original BPF has two registers (A and X) and hidden frame pointer. +Extended BPF has ten registers and read-only frame pointer. +Since 64-bit CPUs are passing arguments to the functions via registers +the number of args from BPF program to in-kernel function is restricted to 5 +and one register is used to accept return value from in-kernel function. +x86_64 passes first 6 arguments in registers. +aarch64/sparcv9/mips64 have 7-8 registers for arguments. +x86_64 has 6 callee saved registers. +aarch64/sparcv9/mips64 have 11 or more callee saved registers. + +Therefore extended BPF calling convention is defined as: +R0 - return value from in-kernel function +R1-R5 - arguments from BPF program to in-kernel function +R6-R9 - callee saved registers that in-kernel function will preserve +R10 - read-only frame pointer to access stack + +so that all BPF registers map one to one to HW registers on x86_64,aarch64,etc +and BPF calling convention maps directly to ABIs used by kernel on 64-bit +architectures. + +R0-R5 are scratch registers and BPF program needs spill/fill them if necessary +across calls. +Note that there is only one BPF program == one BPF function and it cannot call +other BPF functions. It can only call predefined in-kernel functions. + +All BPF registers are 64-bit without subregs, which makes JITed x86 code +less optimal, but matches sparc/mips architectures. +Adding 32-bit subregs was considered, since JIT can map them to x86 and aarch64 +nicely, but read-modify-write overhead for sparc/mips is not worth the gains. + +Original BPF and extended BPF are two operand instructions, which helps +to do one-to-one mapping between BPF insn and x86 insn during JIT. + +Extended BPF doesn't have pre-defined endianness not to favor one +architecture vs another. Therefore bswap insn was introduced. +Original BPF doesn't have such insn and does bswap as part of sk_load_word call +which is often unnecessary if we want to compare the value with the constant. +Restricted C code might be written differently depending on endianness +and GCC/LLVM-bpf will take an endianness flag. + +32-bit architectures run 64-bit extended BPF programs via interpreter + +Q: Why extended BPF is 64-bit? Cannot we live with 32-bit? +A: On 64-bit architectures, pointers are 64-bit and we want to pass 64-bit +values in/out kernel functions, so 32-bit BPF registers would require to define +register-pair ABI, there won't be a direct BPF register to HW register +mapping and JIT would need to do combine/split/move operations for every +register in and out of the function, which is complex, bug prone and slow. +Another reason is counters. To use 64-bit counter BPF program would need to do +a complex math. Again bug prone and not atomic. + +Q: Original BPF is safe, deterministic and kernel can easily prove that. + Does extended BPF keep these properties? +A: Yes. The safety of the program is determined in two steps. +First step does depth-first-search to disallow loops and other CFG validation. +Second step starts from the first insn and descends all possible paths. +It simulates execution of every insn and observes the state change of +registers and stack. +At the start of the program the register R1 contains a pointer to bpf_context +and has type PTR_TO_CTX. If checker sees an insn that does R2=R1, then R2 has +now type PTR_TO_CTX as well and can be used on right hand side of expression. +If R1=PTR_TO_CTX and insn is R2=R1+1, then R2=INVALID_PTR and it is readable. +If register was never written to, it's not readable. +After kernel function call, R1-R5 are reset to unreadable and R0 has a return +type of the function. Since R6-R9 are callee saved, their state is preserved +across the call. +load/store instructions are allowed only with registers of valid types, which +are PTR_TO_CTX, PTR_TO_TABLE, PTR_TO_STACK. They are bounds and alginment +checked. + +bpf_context structure is generic. Its contents are defined by specific use case. +For seccomp it can be seccomp_data and through get_context_access callback +BPF checker is customized, so that BPF program can only access certain fields +of bpf_context with specified size and alignment. +For example, the following insn: + BPF_INSN_LD(BPF_W, R0, R6, 8) +intends to load word from address R6 + 8 and store it into R0 +If R6=PTR_TO_CTX, then get_context_access callback should let the checker know +that offset 8 of size 4 bytes can be accessed for reading, otherwise the checker +will reject the program. +If R6=PTR_TO_STACK, then access should be aligned and be within stack bounds, +which are hard coded to [-480, 0]. In this example offset is 8, so it will fail +verification. +The checker will allow BPF program to read data from stack only after it wrote +into it. +Pointer register spill/fill is tracked as well, since four (R6-R9) callee saved +registers may not be enough for some programs. + +Allowed function calls are customized via get_func_proto callback. +For example: + u64 bpf_load_byte(struct bpf_context *ctx, u32 offset); +function will have the following definition: + struct bpf_func_proto proto = {RET_INTEGER, PTR_TO_CTX}; +and BPF checker will verify that bpf_load_byte is always called with first +argument being a valid pointer to bpf_context. After the call BPF register R0 +will be set to readable state, so that BPF program can access it. + +One of the useful functions that can be made available to BPF program +are bpf_table_lookup/bpf_table_update. +Using them a tracing filter can collect any type of statistics. + +Therefore extended BPF program consists of instructions and tables. +From BPF program the table is identified by constant table_id +and access to a table in C looks like: +elem = bpf_table_lookup(ctx, table_id, key); + +BPF checker matches 'table_id' against known tables, verifies that 'key' points +to stack and table->key_size bytes are initialized. +From there on bpf_table_lookup() is a normal kernel function. It needs to do +a lookup by whatever means and return either valid pointer to the element +or NULL. BPF checker will verify that the program accesses the pointer only +after comparing it to NULL. That's the meaning of PTR_TO_TABLE_CONDITIONAL and +PTR_TO_TABLE register types in bpf_check.c + +If a kernel subsystem wants to use this BPF framework and decides to implement +bpf_table_lookup, the checker will guarantee that argument 'ctx' is a valid +pointer to bpf_context, 'table_id' is valid table_id and table->key_size bytes +can be read from the pointer 'key'. It's up to implementation to decide how it +wants to do the lookup and what is the key. + +Going back to the example BPF insn: + BPF_INSN_LD(BPF_W, R0, R6, 8) +if R6=PTR_TO_TABLE, then offset and size of access must be within +[0, table->elem_size] which is determined by constant table_id that was passed +into bpf_table_lookup call prior to this insn. + +Just like original, extended BPF is limited to 4096 insns, which means that any +program will terminate quickly and will call fixed number of kernel functions. +Earlier implementation of the checker had a precise calculation of worst case +number of insns, but it was removed to simplify the code, since the worst number +is always less then number of insns in a program anyway (because it's a DAG). + +Since register/stack state tracking simulates execution of all insns in all +possible branches, it will explode if not bounded. There are two bounds. +verifier_state stack is limited to 1k, therefore BPF program cannot have +more than 1k jump insns. +Total number of insns to be analyzed is limited to 32k, which means that +checker will either prove correctness or reject the program in few +milliseconds on average x86 cpu. Valid programs take microseconds to verify. +
Signature · 1 line
1.7.9.5
RFC PATCH v2 4/7 Revert "x86/ptrace: Remove unused regs_get_argument_nth API"
In reply to Alexei StarovoitovThis reverts commit aa5add93e92019018e905146f8c3d3f8e3c08300.
--- arch/x86/include/asm/ptrace.h | 3 +++ arch/x86/kernel/ptrace.c | 24 ++++++++++++++++++++++++ 2 files changed, 27 insertions(+)
arch/x86/include/asm/ptrace.h +3 -0
@@ -222,6 +222,9 @@ static inline unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs,return 0 ; } +/* Get Nth argument at function call */ +unsigned long regs_get_argument_nth (struct pt_regs *regs,unsigned int n); +#define arch_has_single_step () (1)#ifdef CONFIG_X86_DEBUGCTLMSR #define arch_has_block_step () (1)
arch/x86/kernel/ptrace.c +24 -0
@@ -141,6 +141,30 @@ static const int arg_offs_table[] = {#endif }; +/** +* regs_get_argument_nth() - get Nth argument at function call +* @regs: pt_regs which contains registers at function entry. +* @n: argument number. +* +* regs_get_argument_nth() returns @n th argument of a function call. +* Since usually the kernel stack will be changed right after function entry, +* you must use this at function entry. If the @n th entry is NOT in the +* kernel stack or pt_regs, this returns 0. +*/ +unsigned long regs_get_argument_nth (struct pt_regs *regs,unsigned int n) +{ +if (n <ARRAY_SIZE (arg_offs_table)) +return *(unsigned long *)((char *)regs + arg_offs_table[n]); +else { +/* +* The typical case: arg n is on the stack. +* (Note: stack[0] = return address, so skip it) +*/ + n -=ARRAY_SIZE (arg_offs_table); +return regs_get_kernel_stack_nth (regs,1 + n); + } +} +/* * does not yet catch signals sent when the child dies. * in exit.c or in signal.c.
Signature · 1 line
1.7.9.5
RFC PATCH v2 5/7 use BPF in tracing filters
In reply to Alexei StarovoitovSuch filters can be written in C and allow safe read-only access to any kernel data structure.
Like systemtap but with safety guaranteed by kernel.
The user can do:
cat bpf_program > /sys/kernel/debug/tracing/.../filter if tracing event is either static or dynamic via kprobe_events.
The program can be anything as long as bpf_check() can verify its safety. For example, the user can create kprobe_event on dst_discard() and use logically following code inside BPF filter:
skb = (struct sk_buff *)ctx->arg1;
dev = bpf_load_pointer(&skb->dev);to access 'struct net_device'
Since its prototype is 'int dst_discard(struct sk_buff *skb);' bpf_load_pointer() will try to fetch 'dev' field of 'sk_buff' structure and will suppress page-fault if pointer is incorrect.
--- include/linux/ftrace_event.h | 5 + include/trace/bpf_trace.h | 41 ++++++++ include/trace/ftrace.h | 17 ++++ kernel/trace/Kconfig | 1 + kernel/trace/Makefile | 1 + kernel/trace/bpf_trace_callbacks.c | 193 ++++++++++++++++++++++++++++++++++++ kernel/trace/trace.c | 7 ++ kernel/trace/trace.h | 11 +- kernel/trace/trace_events.c | 9 +- kernel/trace/trace_events_filter.c | 61 +++++++++++- kernel/trace/trace_kprobe.c | 15 ++- 11 files changed, 356 insertions(+), 5 deletions(-) create mode 100644 include/trace/bpf_trace.h create mode 100644 kernel/trace/bpf_trace_callbacks.c
include/linux/ftrace_event.h +5 -0
@@ -204,6 +204,7 @@ enum {TRACE_EVENT_FL_IGNORE_ENABLE_BIT ,TRACE_EVENT_FL_WAS_ENABLED_BIT ,TRACE_EVENT_FL_USE_CALL_FILTER_BIT , + TRACE_EVENT_FL_BPF_BIT, };/*
@@ -224,6 +225,7 @@ enum {TRACE_EVENT_FL_IGNORE_ENABLE = (1 <<TRACE_EVENT_FL_IGNORE_ENABLE_BIT ),TRACE_EVENT_FL_WAS_ENABLED = (1 <<TRACE_EVENT_FL_WAS_ENABLED_BIT ),TRACE_EVENT_FL_USE_CALL_FILTER = (1 <<TRACE_EVENT_FL_USE_CALL_FILTER_BIT ), + TRACE_EVENT_FL_BPF = (1 << TRACE_EVENT_FL_BPF_BIT), }; struct ftrace_event_call {
@@ -487,6 +489,9 @@ event_trigger_unlock_commit_regs(struct ftrace_event_file *file,event_triggers_post_call (file, tt); } +struct bpf_context ; +void filter_call_bpf (struct event_filter *filter,struct bpf_context *ctx); +enum {FILTER_OTHER =0 ,FILTER_STATIC_STRING ,
include/trace/bpf_trace.h new file +41 -0
41 lines not on this page · raw message
include/trace/ftrace.h +17 -0
@@ -17,6 +17,8 @@*/ #include <linux/ftrace_event.h> +#include <linux/kexec.h> +#include <trace/bpf_trace.h> /* * DECLARE_EVENT_CLASS can be used to add a generic function
@@ -556,6 +558,21 @@ ftrace_raw_event_##call(void *__data, proto) \if (ftrace_trigger_soft_disabled (ftrace_file)) \return ; \ \ +if (unlikely (ftrace_file->flags &FTRACE_EVENT_FL_FILTERED ) && \ +unlikely (ftrace_file->event_call->flags &TRACE_EVENT_FL_BPF )) { \ +struct bpf_context _ctx; \ +struct pt_regs _regs; \ +void (*_fn)(struct bpf_context *,proto , \ +long ,long ,long ,long ); \ +crash_setup_regs (&_regs,NULL ); \ + _fn = (void (*)(struct bpf_context *,proto ,long ,long ,\ +long ,long ))init_bpf_context; \ +_fn (&_ctx, args,0 ,0 ,0 ,0 ); \ + _ctx.regs = &_regs; \ +filter_call_bpf (ftrace_file->filter, &_ctx); \ +return ; \ + } \ + \local_save_flags (irq_flags); \ pc =preempt_count (); \ \
kernel/trace/Kconfig +1 -0
@@ -80,6 +80,7 @@ config FTRACE_NMI_ENTER config EVENT_TRACING select CONTEXT_SWITCH_TRACER + select BPF64 bool config CONTEXT_SWITCH_TRACER
kernel/trace/Makefile +1 -0
@@ -51,6 +51,7 @@ obj-$(CONFIG_EVENT_TRACING) += trace_event_perf.o endif obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o obj-$(CONFIG_EVENT_TRACING) += trace_events_trigger.o +obj-$(CONFIG_EVENT_TRACING) += bpf_trace_callbacks.o obj-$(CONFIG_KPROBE_EVENT) += trace_kprobe.o obj-$(CONFIG_TRACEPOINTS) += power-traces.o ifeq ($(CONFIG_PM_RUNTIME),y)
kernel/trace/bpf_trace_callbacks.c new file +193 -0
193 lines not on this page · raw message
kernel/trace/trace.c +7 -0
@@ -1791,6 +1791,13 @@ void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); } +void __trace_stack_regs (unsigned long flags,int skip,int pc, +struct pt_regs *regs) +{ +__ftrace_trace_stack (global_trace.trace_buffer.buffer, flags, skip, + pc, regs); +} +/** * trace_dump_stack - record a stack back trace in the trace buffer * @skip: Number of functions to skip (helper handlers)
kernel/trace/trace.h +10 -1
@@ -619,6 +619,8 @@ void ftrace_trace_userstack(struct ring_buffer *buffer, unsigned long flags,void __trace_stack (struct trace_array *tr,unsigned long flags,int skip,int pc); +void __trace_stack_regs (unsigned long flags,int skip,int pc, +struct pt_regs *regs);#else static inline void ftrace_trace_stack (struct ring_buffer *buffer,unsigned long flags,int skip,int pc)
@@ -640,6 +642,10 @@ static inline void __trace_stack(struct trace_array *tr, unsigned long flags, int skip, int pc) { } +static inline void __trace_stack_regs (unsigned long flags,int skip,int pc, +struct pt_regs *regs) +{ +}#endif /* CONFIG_STACKTRACE */ extern cycle_t ftrace_now (int cpu);
@@ -939,12 +945,15 @@ struct ftrace_event_field {int is_signed; }; +struct bpf_program ; +struct event_filter {int n_preds;/* Number assigned */ int a_preds;/* allocated */ struct filter_pred *preds;struct filter_pred *root;char *filter_string; +struct bpf_program *prog; };struct event_subsystem {
@@ -1017,7 +1026,7 @@ filter_parse_regex(char *buff, int len, char **search, int *not);extern void print_event_filter (struct ftrace_event_file *file,struct trace_seq *s);extern int apply_event_filter (struct ftrace_event_file *file, -char *filter_string); +char *filter_string,int filter_len);extern int apply_subsystem_event_filter (struct ftrace_subsystem_dir *dir,char *filter_string); extern void print_subsystem_event_filter(struct event_subsystem *system,
kernel/trace/trace_events.c +8 -1
@@ -1042,9 +1042,16 @@ event_filter_write(struct file *filp, const char __user *ubuf, size_t cnt,mutex_lock (&event_mutex); file =event_file_data (filp);if (file) - err =apply_event_filter (file, buf); + err =apply_event_filter (file, buf, cnt);mutex_unlock (&event_mutex); +if (file->event_call->flags &TRACE_EVENT_FL_BPF ) +/* +* allocate per-cpu printk buffers, since BPF program +* might be calling bpf_trace_printk +*/ +trace_printk_init_buffers (); +free_page ((unsigned long ) buf);if (err <0 )return err;
kernel/trace/trace_events_filter.c +59 -2
97 lines not on this page · raw message
kernel/trace/trace_kprobe.c +14 -1
@@ -19,7 +19,7 @@#include <linux/module.h> #include <linux/uaccess.h> - +#include <trace/bpf_trace.h> #include "trace_probe.h" #define KPROBE_EVENT_SYSTEM "kprobes"
@@ -936,6 +936,19 @@ __kprobe_trace_func(struct trace_kprobe *tk, struct pt_regs *regs,if (ftrace_trigger_soft_disabled (ftrace_file))return ; +if (unlikely (ftrace_file->flags &FTRACE_EVENT_FL_FILTERED ) && +unlikely (ftrace_file->event_call->flags &TRACE_EVENT_FL_BPF )) { +struct bpf_context ctx; + ctx.regs = regs; + ctx.arg1 =regs_get_argument_nth (regs,0 ); + ctx.arg2 =regs_get_argument_nth (regs,1 ); + ctx.arg3 =regs_get_argument_nth (regs,2 ); + ctx.arg4 =regs_get_argument_nth (regs,3 ); + ctx.arg5 =regs_get_argument_nth (regs,4 ); +filter_call_bpf (ftrace_file->filter, &ctx); +return ; + } +local_save_flags (irq_flags); pc =preempt_count ();
Signature · 1 line
1.7.9.5
RFC PATCH v2 6/7 LLVM BPF backend
In reply to Alexei Starovoitovstandalone BPF backend for LLVM 3.2, 3.3 and 3.4
See tools/bpf/llvm/README.txt
Written in LLVM codying style and LLVM license.
Most of the lib/Target/BPF/* is boilerplate code
which is required for any LLVM backend.
Backend enforces presence of 'license' section in the source C file.
Makefile* is simplified LLVM build system
--- tools/bpf/llvm/LICENSE.TXT | 70 ++ tools/bpf/llvm/Makefile.rules | 641 +++++++++++++++++++ tools/bpf/llvm/README.txt | 23 + tools/bpf/llvm/bld/.gitignore | 2 + tools/bpf/llvm/bld/Makefile | 27 + tools/bpf/llvm/bld/Makefile.common | 14 + tools/bpf/llvm/bld/Makefile.config | 124 ++++ .../llvm/bld/include/llvm/Config/AsmParsers.def | 8 + .../llvm/bld/include/llvm/Config/AsmPrinters.def | 9 + .../llvm/bld/include/llvm/Config/Disassemblers.def | 8 + tools/bpf/llvm/bld/include/llvm/Config/Targets.def | 9 + .../bpf/llvm/bld/include/llvm/Support/DataTypes.h | 96 +++ tools/bpf/llvm/bld/lib/Makefile | 11 + .../llvm/bld/lib/Target/BPF/InstPrinter/Makefile | 10 + .../llvm/bld/lib/Target/BPF/MCTargetDesc/Makefile | 11 + tools/bpf/llvm/bld/lib/Target/BPF/Makefile | 17 + .../llvm/bld/lib/Target/BPF/TargetInfo/Makefile | 10 + tools/bpf/llvm/bld/lib/Target/Makefile | 11 + tools/bpf/llvm/bld/tools/Makefile | 12 + tools/bpf/llvm/bld/tools/llc/Makefile | 15 + tools/bpf/llvm/lib/Target/BPF/BPF.h | 30 + tools/bpf/llvm/lib/Target/BPF/BPF.td | 29 + tools/bpf/llvm/lib/Target/BPF/BPFAsmPrinter.cpp | 100 +++ tools/bpf/llvm/lib/Target/BPF/BPFCFGFixup.cpp | 62 ++ tools/bpf/llvm/lib/Target/BPF/BPFCallingConv.td | 24 + tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.cpp | 36 ++ tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.h | 35 + tools/bpf/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp | 182 ++++++ tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.cpp | 676 ++++++++++++++++++++ tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.h | 105 +++ tools/bpf/llvm/lib/Target/BPF/BPFInstrFormats.td | 29 + tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.cpp | 162 +++++ tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.h | 53 ++ tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.td | 455 +++++++++++++ tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.cpp | 77 +++ tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.h | 40 ++ tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.cpp | 122 ++++ tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.h | 65 ++ tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.td | 39 ++ tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.cpp | 23 + tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.h | 33 + tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.cpp | 72 +++ tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.h | 69 ++ .../lib/Target/BPF/InstPrinter/BPFInstPrinter.cpp | 79 +++ .../lib/Target/BPF/InstPrinter/BPFInstPrinter.h | 34 + .../lib/Target/BPF/MCTargetDesc/BPFAsmBackend.cpp | 85 +++ .../llvm/lib/Target/BPF/MCTargetDesc/BPFBaseInfo.h | 33 + .../Target/BPF/MCTargetDesc/BPFELFObjectWriter.cpp | 119 ++++ .../lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h | 34 + .../Target/BPF/MCTargetDesc/BPFMCCodeEmitter.cpp | 120 ++++ .../lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.h | 67 ++ .../Target/BPF/MCTargetDesc/BPFMCTargetDesc.cpp | 115 ++++ .../lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.h | 56 ++ .../lib/Target/BPF/TargetInfo/BPFTargetInfo.cpp | 13 + tools/bpf/llvm/tools/llc/llc.cpp | 381 +++++++++++ 55 files changed, 4782 insertions(+) create mode 100644 tools/bpf/llvm/LICENSE.TXT create mode 100644 tools/bpf/llvm/Makefile.rules create mode 100644 tools/bpf/llvm/README.txt create mode 100644 tools/bpf/llvm/bld/.gitignore create mode 100644 tools/bpf/llvm/bld/Makefile create mode 100644 tools/bpf/llvm/bld/Makefile.common create mode 100644 tools/bpf/llvm/bld/Makefile.config create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/AsmParsers.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/AsmPrinters.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/Disassemblers.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Config/Targets.def create mode 100644 tools/bpf/llvm/bld/include/llvm/Support/DataTypes.h create mode 100644 tools/bpf/llvm/bld/lib/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/InstPrinter/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/MCTargetDesc/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/BPF/TargetInfo/Makefile create mode 100644 tools/bpf/llvm/bld/lib/Target/Makefile create mode 100644 tools/bpf/llvm/bld/tools/Makefile create mode 100644 tools/bpf/llvm/bld/tools/llc/Makefile create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPF.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPF.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFAsmPrinter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFCFGFixup.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFCallingConv.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrFormats.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.td create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/InstPrinter/BPFInstPrinter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/InstPrinter/BPFInstPrinter.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFAsmBackend.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFBaseInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFELFObjectWriter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.cpp create mode 100644 tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.h create mode 100644 tools/bpf/llvm/lib/Target/BPF/TargetInfo/BPFTargetInfo.cpp create mode 100644 tools/bpf/llvm/tools/llc/llc.cpp
tools/bpf/llvm/LICENSE.TXT new file +70 -0
@@ -0,0 +1,70 @@ +============================================================================== +LLVM Release License +============================================================================== +University of Illinois/NCSA +Open Source License + +Copyright (c) 2003-2012 University of Illinois at Urbana-Champaign. +All rights reserved. + +Developed by: + + LLVM Team + + University of Illinois at Urbana-Champaign + + http://llvm.org + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal with +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimers. + + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimers in the + documentation and/or other materials provided with the distribution. + + * Neither the names of the LLVM Team, University of Illinois at + Urbana-Champaign, nor the names of its contributors may be used to + endorse or promote products derived from this Software without specific + prior written permission. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE +SOFTWARE. + +============================================================================== +Copyrights and Licenses for Third Party Software Distributed with LLVM: +============================================================================== +The LLVM software contains code written by third parties. Such software will +have its own individual LICENSE.TXT file in the directory in which it appears. +This file will describe the copyrights, license, and restrictions which apply +to that code. + +The disclaimer of warranty in the University of Illinois Open Source License +applies to all code in the LLVM Distribution, and nothing in any of the +other licenses gives permission to use the names of the LLVM Team or the +University of Illinois to endorse or promote products derived from this +Software. + +The following pieces of software have additional or alternate copyrights, +licenses, and/or restrictions: + +Program Directory +------- --------- +Autoconf llvm/autoconf + llvm/projects/ModuleMaker/autoconf + llvm/projects/sample/autoconf +CellSPU backend llvm/lib/Target/CellSPU/README.txt +Google Test llvm/utils/unittest/googletest +OpenBSD regex llvm/lib/Support/{reg*, COPYRIGHT.regex} +pyyaml tests llvm/test/YAMLParser/{*.data, LICENSE.TXT}
tools/bpf/llvm/Makefile.rules new file +641 -0
641 lines not on this page · raw message
tools/bpf/llvm/README.txt new file +23 -0
@@ -0,0 +1,23 @@ +LLVM BPF backend: +lib/Target/BPF/*.cpp + +Links with LLVM 3.2, 3.3 and 3.4 + +prerequisites: +apt-get install clang llvm-3.[234]-dev + +To build: +$cd bld +$make +if 'llvm-config-3.2' is not found in PATH, build with: +$make -j4 LLVM_CONFIG=/path_to/llvm-config + +To run: +$clang -O2 -emit-llvm -c file.c -o -|./bld/Debug+Asserts/bin/llc -o file.bpf + +'clang' - is unmodified clang used to build x86 code +'llc' - llvm bit-code to BPF compiler +file.bpf - BPF binary image, see include/linux/bpf_jit.h + +$clang -O2 -emit-llvm -c file.c -o -|llc -filetype=asm -o file.s +will emit human readable BPF assembler instead.
tools/bpf/llvm/bld/.gitignore new file +2 -0
@@ -0,0 +1,2 @@ +*.inc +Debug+Asserts
tools/bpf/llvm/bld/Makefile new file +27 -0
@@ -0,0 +1,27 @@ +#===- ./Makefile -------------------------------------------*- Makefile -*--===# +# +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. + +ifndef LLVM_CONFIG +LLVM_CONFIG := llvm-config-3.2 +export LLVM_CONFIG +endif + +LEVEL := . + +DIRS := lib tools + +include $(LEVEL)/Makefile.config + +# Include the main makefile machinery. +include $(LLVM_SRC_ROOT)/Makefile.rules + +# NOTE: This needs to remain as the last target definition in this file so +# that it gets executed last. +all:: + $(Echo) '*****' Completed $(BuildMode) Build + +# declare all targets at this level to be serial: +.NOTPARALLEL: +
tools/bpf/llvm/bld/Makefile.common new file +14 -0
@@ -0,0 +1,14 @@ +#===-- Makefile.common - Common make rules for LLVM --------*- Makefile -*--===# +# +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. + +# Configuration file to set paths specific to local installation of LLVM +ifndef LLVM_OBJ_ROOT +include $(LEVEL)/Makefile.config +else +include $(LLVM_OBJ_ROOT)/Makefile.config +endif + +# Include all of the build rules used for making LLVM +include $(LLVM_SRC_ROOT)/Makefile.rules
tools/bpf/llvm/bld/Makefile.config new file +124 -0
124 lines not on this page · raw message
tools/bpf/llvm/bld/include/llvm/Config/AsmParsers.def new file +8 -0
@@ -0,0 +1,8 @@ +/*===- llvm/Config/AsmParsers.def - LLVM Assembly Parsers -------*- C++ -*-===*\ +|* This file is distributed under the University of Illinois Open Source *| +|* License. See LICENSE.TXT for details. *| +\*===----------------------------------------------------------------------===*/ +#ifndef LLVM_ASM_PARSER +# error Please define the macro LLVM_ASM_PARSER(TargetName) +#endif +#undef LLVM_ASM_PARSER
tools/bpf/llvm/bld/include/llvm/Config/AsmPrinters.def new file +9 -0
@@ -0,0 +1,9 @@ +/*===- llvm/Config/AsmPrinters.def - LLVM Assembly Printers -----*- C++ -*-===*\ +|* This file is distributed under the University of Illinois Open Source *| +|* License. See LICENSE.TXT for details. *| +\*===----------------------------------------------------------------------===*/ +#ifndef LLVM_ASM_PRINTER +# error Please define the macro LLVM_ASM_PRINTER(TargetName) +#endif +LLVM_ASM_PRINTER(BPF) +#undef LLVM_ASM_PRINTER
tools/bpf/llvm/bld/include/llvm/Config/Disassemblers.def new file +8 -0
@@ -0,0 +1,8 @@ +/*===- llvm/Config/Disassemblers.def - LLVM Assembly Parsers ----*- C++ -*-===*\ +|* This file is distributed under the University of Illinois Open Source *| +|* License. See LICENSE.TXT for details. *| +\*===----------------------------------------------------------------------===*/ +#ifndef LLVM_DISASSEMBLER +# error Please define the macro LLVM_DISASSEMBLER(TargetName) +#endif +#undef LLVM_DISASSEMBLER
tools/bpf/llvm/bld/include/llvm/Config/Targets.def new file +9 -0
@@ -0,0 +1,9 @@ +/*===- llvm/Config/Targets.def - LLVM Target Architectures ------*- C++ -*-===*\ +|* This file is distributed under the University of Illinois Open Source *| +|* License. See LICENSE.TXT for details. *| +\*===----------------------------------------------------------------------===*/ +#ifndef LLVM_TARGET +# error Please define the macro LLVM_TARGET(TargetName) +#endif +LLVM_TARGET(BPF) +#undef LLVM_TARGET
tools/bpf/llvm/bld/include/llvm/Support/DataTypes.h new file +96 -0
96 lines not on this page · raw message
tools/bpf/llvm/bld/lib/Makefile new file +11 -0
@@ -0,0 +1,11 @@ +##===- lib/Makefile ----------------------------------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. +LEVEL = .. + +include $(LEVEL)/Makefile.config + +PARALLEL_DIRS := Target + +include $(LEVEL)/Makefile.common +
tools/bpf/llvm/bld/lib/Target/BPF/InstPrinter/Makefile new file +10 -0
@@ -0,0 +1,10 @@ +##===- lib/Target/BPF/InstPrinter/Makefile ----------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. +LEVEL = ../../../.. +LIBRARYNAME = LLVMBPFAsmPrinter + +# Hack: we need to include 'main' BPF target directory to grab private headers +CPP.Flags += -I$(PROJ_OBJ_DIR)/.. -I$(PROJ_SRC_DIR)/.. + +include $(LEVEL)/Makefile.common
tools/bpf/llvm/bld/lib/Target/BPF/MCTargetDesc/Makefile new file +11 -0
@@ -0,0 +1,11 @@ +##===- lib/Target/BPF/TargetDesc/Makefile ----------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. + +LEVEL = ../../../.. +LIBRARYNAME = LLVMBPFDesc + +# Hack: we need to include 'main' target directory to grab private headers +CPP.Flags += -I$(PROJ_OBJ_DIR)/.. -I$(PROJ_SRC_DIR)/.. + +include $(LEVEL)/Makefile.common
tools/bpf/llvm/bld/lib/Target/BPF/Makefile new file +17 -0
@@ -0,0 +1,17 @@ +##===- lib/Target/BPF/Makefile ---------------------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. + +LEVEL = ../../.. +LIBRARYNAME = LLVMBPFCodeGen +TARGET = BPF + +# Make sure that tblgen is run, first thing. +BUILT_SOURCES = BPFGenRegisterInfo.inc BPFGenInstrInfo.inc \ + BPFGenAsmWriter.inc BPFGenAsmMatcher.inc BPFGenDAGISel.inc \ + BPFGenMCCodeEmitter.inc BPFGenSubtargetInfo.inc BPFGenCallingConv.inc + +DIRS = InstPrinter TargetInfo MCTargetDesc + +include $(LEVEL)/Makefile.common +
tools/bpf/llvm/bld/lib/Target/BPF/TargetInfo/Makefile new file +10 -0
@@ -0,0 +1,10 @@ +##===- lib/Target/BPF/TargetInfo/Makefile ----------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. +LEVEL = ../../../.. +LIBRARYNAME = LLVMBPFInfo + +# Hack: we need to include 'main' target directory to grab private headers +CPPFLAGS = -I$(PROJ_OBJ_DIR)/.. -I$(PROJ_SRC_DIR)/.. + +include $(LEVEL)/Makefile.common
tools/bpf/llvm/bld/lib/Target/Makefile new file +11 -0
@@ -0,0 +1,11 @@ +#===- lib/Target/Makefile ----------------------------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. + +LEVEL = ../.. + +include $(LEVEL)/Makefile.config + +PARALLEL_DIRS := $(TARGETS_TO_BUILD) + +include $(LLVM_SRC_ROOT)/Makefile.rules
tools/bpf/llvm/bld/tools/Makefile new file +12 -0
@@ -0,0 +1,12 @@ +##===- tools/Makefile --------------------------------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. + +LEVEL := .. + +include $(LEVEL)/Makefile.config + +DIRS := +PARALLEL_DIRS := llc + +include $(LEVEL)/Makefile.common
tools/bpf/llvm/bld/tools/llc/Makefile new file +15 -0
@@ -0,0 +1,15 @@ +#===- tools/llc/Makefile -----------------------------------*- Makefile -*-===## +# This file is distributed under the University of Illinois Open Source +# License. See LICENSE.TXT for details. + +LEVEL := ../.. +TOOLNAME := llc +ifneq (,$(filter $(shell $(LLVM_CONFIG) --version),3.3 3.4)) +LINK_COMPONENTS := asmparser asmprinter codegen bitreader core mc selectiondag support target irreader +else +LINK_COMPONENTS := asmparser asmprinter codegen bitreader core mc selectiondag support target +endif +USEDLIBS := LLVMBPFCodeGen.a LLVMBPFDesc.a LLVMBPFInfo.a LLVMBPFAsmPrinter.a + +include $(LEVEL)/Makefile.common +
tools/bpf/llvm/lib/Target/BPF/BPF.h new file +30 -0
@@ -0,0 +1,30 @@ +//===-- BPF.h - Top-level interface for BPF representation ----*- C++ -*-===// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. + +#ifndef TARGET_BPF_H +#define TARGET_BPF_H +#include "llvm/Config/config.h" +#undef LLVM_NATIVE_TARGET +#undef LLVM_NATIVE_ASMPRINTER +#undef LLVM_NATIVE_ASMPARSER +#undef LLVM_NATIVE_DISASSEMBLER +#include "MCTargetDesc/BPFBaseInfo.h" +#include "MCTargetDesc/BPFMCTargetDesc.h" +#include "llvm/Target/TargetMachine.h" + +namespace llvm { +class FunctionPass; +class TargetMachine; +class BPFTargetMachine; + +/// createBPFISelDag - This pass converts a legalized DAG into a +/// BPF-specific DAG, ready for instruction scheduling. +FunctionPass *createBPFISelDag (BPFTargetMachine &TM ); + +FunctionPass *createBPFCFGFixup (BPFTargetMachine &TM ); + +extern Target TheBPFTarget; +} + +#endif
tools/bpf/llvm/lib/Target/BPF/BPF.td new file +29 -0
@@ -0,0 +1,29 @@ +//===- BPF.td - Describe the BPF Target Machine --------*- tablegen -*-===// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. + +// Target-independent interfaces which we are implementing +include "llvm/Target/Target.td" + +// BPF Subtarget features. +include "BPFRegisterInfo.td" +include "BPFCallingConv.td" +include "BPFInstrInfo.td" + +def BPFInstrInfo : InstrInfo; + +class Proc<string Name, list<SubtargetFeature> Features> + : Processor<Name, NoItineraries, Features>; + +def : Proc<"generic", []>; + +def BPFInstPrinter : AsmWriter { + string AsmWriterClassName = "InstPrinter"; + bit isMCAsmWriter = 1; +} + +// Declare the target which we are implementing +def BPF : Target { + let InstructionSet = BPFInstrInfo; + let AssemblyWriters = [BPFInstPrinter]; +}
tools/bpf/llvm/lib/Target/BPF/BPFAsmPrinter.cpp new file +100 -0
100 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFCFGFixup.cpp new file +62 -0
62 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFCallingConv.td new file +24 -0
@@ -0,0 +1,24 @@ +//===- BPFCallingConv.td - Calling Conventions BPF -------*- tablegen -*-===// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +// This describes the calling conventions for the BPF architectures. + +// BPF 64-bit C return-value convention. +def RetCC_BPF64 : CallingConv<[ + CCIfType<[i64], CCAssignToReg<[R0]>> +]>; + +// BPF 64-bit C Calling convention. +def CC_BPF64 : CallingConv<[ + // Promote i8/i16/i32 args to i64 + CCIfType<[i8, i16, i32], CCPromoteToType<i64>>, + + // All arguments get passed in integer registers if there is space. + CCIfType<[i64], CCAssignToReg<[R1, R2, R3, R4, R5]>>, + + // Alternatively, they are assigned to the stack in 8-byte aligned units. + CCAssignToStack<8, 8> +]>; + +def CSR: CalleeSavedRegs<(add R6, R7, R8, R9, R10)>;
tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.cpp new file +36 -0
@@ -0,0 +1,36 @@ +//===-- BPFFrameLowering.cpp - BPF Frame Information --------------------===// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +// This file contains the BPF implementation of TargetFrameLowering class. + +#include "BPFFrameLowering.h" +#include "BPFInstrInfo.h" +#include "llvm/CodeGen/MachineFrameInfo.h" +#include "llvm/CodeGen/MachineFunction.h" +#include "llvm/CodeGen/MachineInstrBuilder.h" +#include "llvm/CodeGen/MachineRegisterInfo.h" + +using namespace llvm; + +bool BPFFrameLowering::hasFP(const MachineFunction &MF) const { + return true; +} + +void BPFFrameLowering::emitPrologue(MachineFunction &MF) const { +} + +void BPFFrameLowering::emitEpilogue(MachineFunction &MF, + MachineBasicBlock &MBB) const { +} + +void BPFFrameLowering:: +processFunctionBeforeCalleeSavedScan(MachineFunction &MF, + RegScavenger *RS) const { + MachineRegisterInfo& MRI = MF.getRegInfo(); + + MRI.setPhysRegUnused(BPF::R6); + MRI.setPhysRegUnused(BPF::R7); + MRI.setPhysRegUnused(BPF::R8); + MRI.setPhysRegUnused(BPF::R9); +}
tools/bpf/llvm/lib/Target/BPF/BPFFrameLowering.h new file +35 -0
@@ -0,0 +1,35 @@ +//===-- BPFFrameLowering.h - Define frame lowering for BPF ---*- C++ -*--===// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. + +#ifndef BPF_FRAMEINFO_H +#define BPF_FRAMEINFO_H + +#include "BPF.h" +#include "BPFSubtarget.h" +#include "llvm/Target/TargetFrameLowering.h" + +namespace llvm { +class BPFSubtarget; + +class BPFFrameLowering : public TargetFrameLowering { +public: + explicit BPFFrameLowering(const BPFSubtarget &sti) + : TargetFrameLowering(TargetFrameLowering::StackGrowsDown, 8, 0) { + } + + void emitPrologue(MachineFunction &MF) const; + void emitEpilogue(MachineFunction &MF, MachineBasicBlock &MBB) const; + + bool hasFP(const MachineFunction &MF) const; + virtual void processFunctionBeforeCalleeSavedScan(MachineFunction &MF, + RegScavenger *RS) const; + +// llvm 3.3 defines it here + void eliminateCallFramePseudoInstr(MachineFunction &MF, MachineBasicBlock &MBB, + MachineBasicBlock::iterator MI) const { +MBB .erase (MI ); + } +}; +} +#endif
tools/bpf/llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp new file +182 -0
182 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.cpp new file +676 -0
676 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFISelLowering.h new file +105 -0
105 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFInstrFormats.td new file +29 -0
@@ -0,0 +1,29 @@ +//===- BPFInstrFormats.td - BPF Instruction Formats ----*- tablegen -*-===// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. + +class InstBPF<dag outs, dag ins, string asmstr, list<dag> pattern> + : Instruction { + field bits<64> Inst; + field bits<64> SoftFail = 0; + let Size = 8; + + let Namespace = "BPF"; + let DecoderNamespace = "BPF"; + + bits<3> bpfClass; + let Inst{58-56} = bpfClass; + + dag OutOperandList = outs; + dag InOperandList = ins; + let AsmString = asmstr; + let Pattern = pattern; +} + +// Pseudo instructions +class Pseudo<dag outs, dag ins, string asmstr, list<dag> pattern> + : InstBPF<outs, ins, asmstr, pattern> { + let Inst{63-0} = 0; + let isPseudo = 1; +} +
tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.cpp new file +162 -0
162 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.h new file +53 -0
53 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFInstrInfo.td new file +455 -0
349 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.cpp new file +77 -0
77 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFMCInstLower.h new file +40 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.cpp new file +122 -0
122 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.h new file +65 -0
65 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFRegisterInfo.td new file +39 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.cpp new file +23 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/lib/Target/BPF/BPFSubtarget.h new file +33 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.cpp new file +72 -0
72 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/BPFTargetMachine.h new file +69 -0
69 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/InstPrinter/BPFInstPrinter.cpp new file +79 -0
79 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/InstPrinter/BPFInstPrinter.h new file +34 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFAsmBackend.cpp new file +85 -0
85 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFBaseInfo.h new file +33 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFELFObjectWriter.cpp new file +119 -0
119 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h new file +34 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.cpp new file +120 -0
120 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCCodeEmitter.h new file +67 -0
67 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.cpp new file +115 -0
115 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/MCTargetDesc/BPFMCTargetDesc.h new file +56 -0
56 lines not on this page · raw message
tools/bpf/llvm/lib/Target/BPF/TargetInfo/BPFTargetInfo.cpp new file +13 -0
Lines of this file are not shown. View the raw message.
tools/bpf/llvm/tools/llc/llc.cpp new file +381 -0
381 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
1.7.9.5
Part of the diff is not shown here. View the raw message.
RFC PATCH v2 7/7 tracing filter examples in BPF
In reply to Alexei Starovoitovfilter_check/ - userspace correctness checker of BPF filter examples/ - BPF filter examples in C
will be compiled by LLVM into .bpf
$cd examples
$make - compile .c into .bpf
$make check - check correctness of *.bpf $make try - to apply netif_rcv.bpf as a tracing filter
--- tools/bpf/examples/Makefile | 71 +++++++++++++++++ tools/bpf/examples/README.txt | 59 ++++++++++++++ tools/bpf/examples/dropmon.c | 40 ++++++++++ tools/bpf/examples/netif_rcv.c | 34 ++++++++ tools/bpf/filter_check/Makefile | 32 ++++++++ tools/bpf/filter_check/README.txt | 3 + tools/bpf/filter_check/trace_filter_check.c | 115 +++++++++++++++++++++++++++ 7 files changed, 354 insertions(+) create mode 100644 tools/bpf/examples/Makefile create mode 100644 tools/bpf/examples/README.txt create mode 100644 tools/bpf/examples/dropmon.c create mode 100644 tools/bpf/examples/netif_rcv.c create mode 100644 tools/bpf/filter_check/Makefile create mode 100644 tools/bpf/filter_check/README.txt create mode 100644 tools/bpf/filter_check/trace_filter_check.c
tools/bpf/examples/Makefile new file +71 -0
@@ -0,0 +1,71 @@ +KOBJ := $(PWD)/../../.. + +VERSION_FILE := $(KOBJ)/include/generated/uapi/linux/version.h + +ifeq (,$(wildcard $(VERSION_FILE))) + $(error Linux kernel source not configured - missing version.h) +endif + +BLD=$(PWD) +LLC=$(BLD)/../llvm/bld/Debug+Asserts/bin/llc +CHK=$(BLD)/../filter_check/trace_filter_check + +EXTRA_CFLAGS= + +ifeq ($(NESTED),1) +# to get NOSTDINC_FLAGS and LINUXINCLUDE from kernel build +# have to trick top Makefile +# pretend that we're building a module +KBUILD_EXTMOD=$(PWD) +# and include main kernel Makefile +include Makefile + +# cannot have other targets (like all, clean) here +# since they will conflict +%.bpf: %.c + clang $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \ + -D__KERNEL__ -Wno-unused-value -Wno-pointer-sign \ + -O2 -emit-llvm -c $< -o -| $(LLC) -o $@ + +else + +SRCS := $(notdir $(wildcard *.c)) +BPFS = $(patsubst %.c,$(BLD)/%.bpf,$(SRCS)) + +all: $(LLC) +# invoke make recursively with current Makefile, but +# for specific .bpf targets + $(MAKE) -C $(KOBJ) -f $(BLD)/Makefile NESTED=1 $(BPFS) + +$(LLC): + $(MAKE) -C ../llvm/bld -j4 + +$(CHK): + $(MAKE) -C ../filter_check + +check: $(CHK) + @$(foreach bpf,$(patsubst %.c,%.bpf,$(SRCS)),echo Checking $(bpf) ...;$(CHK) $(bpf);) + +try: + @echo --- BPF filter for static tracepoint net:netif_receive_skb --- + @echo | sudo tee /sys/kernel/debug/tracing/trace > /dev/null + @cat netif_rcv.bpf | sudo tee /sys/kernel/debug/tracing/events/net/netif_receive_skb/filter > /dev/null + @echo 1 | sudo tee /sys/kernel/debug/tracing/events/net/netif_receive_skb/enable > /dev/null + ping -c1 localhost | grep req + sudo cat /sys/kernel/debug/tracing/trace + @echo 0 | sudo tee /sys/kernel/debug/tracing/events/net/netif_receive_skb/enable > /dev/null + @echo 0 | sudo tee /sys/kernel/debug/tracing/events/net/netif_receive_skb/filter > /dev/null + @echo | sudo tee /sys/kernel/debug/tracing/trace + @echo --- BPF filter for dynamic kprobe __netif_receive_skb --- + @echo "p:my __netif_receive_skb" | sudo tee /sys/kernel/debug/tracing/kprobe_events > /dev/null + @cat netif_rcv.bpf | sudo tee /sys/kernel/debug/tracing/events/kprobes/my/filter > /dev/null + @echo 1 | sudo tee /sys/kernel/debug/tracing/events/kprobes/my/enable > /dev/null + ping -c1 localhost | grep req + sudo cat /sys/kernel/debug/tracing/trace + @echo 0 | sudo tee /sys/kernel/debug/tracing/events/kprobes/my/filter > /dev/null + @echo 0 | sudo tee /sys/kernel/debug/tracing/events/kprobes/my/enable > /dev/null + @echo | sudo tee /sys/kernel/debug/tracing/kprobe_events > /dev/null + +clean: + rm -f *.bpf +endif
tools/bpf/examples/README.txt new file +59 -0
59 lines not on this page · raw message
tools/bpf/examples/dropmon.c new file +40 -0
@@ -0,0 +1,40 @@ +/* +* drop monitor in BPF, faster version of +* tools/perf/scripts/python/net_dropmonitor.py +*/ +#include <linux/bpf.h> +#include <trace/bpf_trace.h> + +#define DESC (NAME ) __attribute__((section(NAME), used)) + +DESC ("e skb:kfree_skb" ) +/* attaches to /sys/kernel/debug/tracing/events/skb/kfree_skb */ +void dropmon (struct bpf_context *ctx) +{ +void *loc; +uint64_t *drop_cnt; + +/* +* skb:kfree_skb is defined as: +* TRACE_EVENT(kfree_skb, +* TP_PROTO(struct sk_buff *skb, void *location), +* so ctx->arg2 is 'location' +*/ + loc = (void *)ctx->arg2; + + drop_cnt =bpf_table_lookup (ctx,0 , &loc); +if (drop_cnt) { +__sync_fetch_and_add (drop_cnt,1 ); + }else { +uint64_t init =0 ; +bpf_table_update (ctx,0 , &loc, &init); + } +} + +struct bpf_table t[] DESC("bpftables") = { + {BPF_TABLE_HASH, sizeof(void *), sizeof(uint64_t), 4096, 0} +}; + +/* filter code license: */ +char l[] DESC("license") = "GPL v2"; +
tools/bpf/examples/netif_rcv.c new file +34 -0
@@ -0,0 +1,34 @@ +/* +* tracing filter example +* attaches to /sys/kernel/debug/tracing/events/net/netif_receive_skb +* prints events for loobpack device only +*/ +#include <linux/skbuff.h> +#include <linux/netdevice.h> +#include <linux/bpf.h> +#include <trace/bpf_trace.h> + +#define DESC (NAME ) __attribute__((section(NAME), used)) + +DESC("e net:netif_receive_skb") +void my_filter (struct bpf_context *ctx) +{ +char devname[4 ] ="lo" ; +struct net_device *dev; +struct sk_buff *skb =0 ; + +/* +* for tracepoints arg1 is the 1st arg of TP_ARGS() macro +* defined in include/trace/events/.h +* for kprobe events arg1 is the 1st arg of probed function +*/ + skb = (struct sk_buff *)ctx->arg1; + dev =bpf_load_pointer (&skb->dev); +if (bpf_memcmp (dev->name, devname,2 ) ==0 ) { +char fmt[] ="skb %p dev %p \n" ; +bpf_trace_printk (fmt,sizeof (fmt), (long )skb, (long )dev,0 ); + } +} + +/* filter code license: */ +char license[] DESC("license") = "GPL";
tools/bpf/filter_check/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +CC = gcc + +all: trace_filter_check + +srctree=../../.. +src-perf=../../perf +ARCH=x86 + +CFLAGS += -I$(src-perf)/util/include +CFLAGS += -I$(src-perf)/arch/$(ARCH)/include +CFLAGS += -I$(srctree)/arch/$(ARCH)/include/uapi +CFLAGS += -I$(srctree)/arch/$(ARCH)/include +CFLAGS += -I$(srctree)/include/uapi +CFLAGS += -I$(srctree)/include +CFLAGS += -O2 -w + +$(srctree)/kernel/bpf_jit/bpf_check.o: $(srctree)/kernel/bpf_jit/bpf_check.c + $(MAKE) -C $(srctree) kernel/bpf_jit/bpf_check.o +$(srctree)/kernel/bpf_jit/bpf_run.o: $(srctree)/kernel/bpf_jit/bpf_run.c + $(MAKE) -C $(srctree) kernel/bpf_jit/bpf_run.o +$(srctree)/kernel/trace/bpf_trace_callbacks.o: $(srctree)/kernel/trace/bpf_trace_callbacks.c + $(MAKE) -C $(srctree) kernel/trace/bpf_trace_callbacks.o + +trace_filter_check: LDLIBS = -Wl,--unresolved-symbols=ignore-all +trace_filter_check: trace_filter_check.o \ + $(srctree)/kernel/bpf_jit/bpf_check.o \ + $(srctree)/kernel/bpf_jit/bpf_run.o \ + $(srctree)/kernel/trace/bpf_trace_callbacks.o + +clean: + rm -rf *.o trace_filter_check +
tools/bpf/filter_check/README.txt new file +3 -0
@@ -0,0 +1,3 @@ +To pre-check correctness of the filter do: +$ trace_filter_check filter_ex1.bpf +(final filter check always happens in kernel)
tools/bpf/filter_check/trace_filter_check.c new file +115 -0
115 lines not on this page · raw message
Signature · 1 line
1.7.9.5
From: Alexei Starovoitov <[email protected]>
Date: Wed, 5 Feb 2014 16:10:00 -0800
Quoted 2 lines
this patch set addresses main sticking points of the previous discussion:
http://thread.gmane.org/gmane.linux.kernel/1605783
You really need to properly CC: netdev on this patch series.
Quoted 8 lines · David Miller
From: Alexei Starovoitov <[email protected]>
Date: Wed, 5 Feb 2014 16:10:00 -0800Quoted 2 lines
this patch set addresses main sticking points of the previous discussion:
http://thread.gmane.org/gmane.linux.kernel/1605783You really need to properly CC: netdev on this patch series.
Sure. Happy to extend the audience.
Will repost with netdev included.