bpf: add bpf_get_stack helper
Subsystems (9)
v5 cover bpf: add bpf_get_stack helper
Currently, stackmap and bpf_get_stackid helper are provided for bpf program to get the stack trace. This approach has a limitation though. If two stack traces have the same hash, only one will get stored in the stackmap table regardless of whether BPF_F_REUSE_STACKID is specified or not, so some stack traces may be missing from user perspective.
This patch implements a new helper, bpf_get_stack, will send stack traces directly to bpf program. The bpf program is able to see all stack traces, and then can do in-kernel processing or send stack traces to user space through shared map or bpf_perf_event_output.
Patches #1 and #2 implemented the core kernel support. Patch #3 removes two never-hit branches in verifier. Patches #4 and #5 are two verifier improves to make bpf programming easier. Patch #6 synced the new helper to tools headers. Patch #7 moved perf_event polling code and ksym lookup code from samples/bpf to tools/testing/selftests/bpf. Patch #8 added a verifier test in tools/bpf for new verifier change. Patches #9 and #10 added tests for raw tracepoint prog and tracepoint prog respectively.
Changelogs:
v4 -> v5:
. relied on dst_reg->var_off to refine umin_val/umax_val
in verifier handling BPF_ARSH value range tracking, suggested by Edward.
v3 -> v4:
. fixed a bug when meta ptr is set to NULL in check_func_arg. . introduced tnum_arshift and added detailed comments for
the underlying implementation
. avoided using VLA in tools/bpf test_progs.
v2 -> v3:
. used meta to track helper memory size argument
. implemented range checking for ARSH in verifier
. moved perf event polling and ksym related functions
from samples/bpf to tools/bpf
. added test to compare build id's between bpf_get_stackid
and bpf_get_stack
v1 -> v2:
. fixed compilation error when CONFIG_PERF_EVENTS is not enabled
Yonghong Song (10):
bpf: change prototype for stack_map_get_build_id_offset
bpf: add bpf_get_stack helper
bpf/verifier: refine retval R0 state for bpf_get_stack helper
bpf: remove never-hit branches in verifier adjust_scalar_min_max_vals bpf/verifier: improve register value range tracking with ARSH tools/bpf: add bpf_get_stack helper to tools headers samples/bpf: move common-purpose trace functions to selftests tools/bpf: add a verifier test case for bpf_get_stack helper and ARSH tools/bpf: add a test for bpf_get_stack with raw tracepoint prog tools/bpf: add a test for bpf_get_stack with tracepoint prog
include/linux/bpf.h | 1 + include/linux/filter.h | 3 +- include/linux/tnum.h | 4 +- include/uapi/linux/bpf.h | 19 +- kernel/bpf/core.c | 5 + kernel/bpf/stackmap.c | 80 ++++++++- kernel/bpf/syscall.c | 10 ++ kernel/bpf/tnum.c | 10 ++ kernel/bpf/verifier.c | 80 ++++++++- kernel/trace/bpf_trace.c | 50 +++++- samples/bpf/Makefile | 11 +- samples/bpf/bpf_load.c | 63 ------- samples/bpf/bpf_load.h | 7 - samples/bpf/offwaketime_user.c | 1 + samples/bpf/sampleip_user.c | 1 + samples/bpf/spintest_user.c | 1 + samples/bpf/trace_event_user.c | 1 + samples/bpf/trace_output_user.c | 125 ++------------ tools/include/uapi/linux/bpf.h | 19 +- tools/testing/selftests/bpf/Makefile | 3 +- tools/testing/selftests/bpf/bpf_helpers.h | 3 +- tools/testing/selftests/bpf/test_get_stack_rawtp.c | 102 +++++++++++ tools/testing/selftests/bpf/test_progs.c | 192 ++++++++++++++++++++- .../selftests/bpf/test_stacktrace_build_id.c | 20 ++- tools/testing/selftests/bpf/test_stacktrace_map.c | 20 ++- tools/testing/selftests/bpf/test_verifier.c | 45 +++++ tools/testing/selftests/bpf/trace_helpers.c | 186 ++++++++++++++++++++ tools/testing/selftests/bpf/trace_helpers.h | 24 +++ 28 files changed, 867 insertions(+), 219 deletions(-) create mode 100644 tools/testing/selftests/bpf/test_get_stack_rawtp.c create mode 100644 tools/testing/selftests/bpf/trace_helpers.c create mode 100644 tools/testing/selftests/bpf/trace_helpers.h
Signature · 1 line
2.9.5
PATCH v5 1/10 bpf: change prototype for stack_map_get_build_id_offset
This patch didn't incur functionality change. The function prototype got changed so that the same function can be reused later.
--- kernel/bpf/stackmap.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-)
kernel/bpf/stackmap.c +5 -8
@@ -262,16 +262,11 @@ static int stack_map_get_build_id(struct vm_area_struct *vma,return ret; } -static void stack_map_get_build_id_offset (struct bpf_map *map, -struct stack_map_bucket *bucket, +static void stack_map_get_build_id_offset (struct bpf_stack_build_id *id_offs,u64 *ips,u32 trace_nr,bool user) {int i;struct vm_area_struct *vma; -struct bpf_stack_build_id *id_offs; - - bucket->nr = trace_nr; - id_offs = (struct bpf_stack_build_id *)bucket->data;/* * We cannot do up_read() in nmi context, so build_id lookup is
@@ -361,8 +356,10 @@ BPF_CALL_3(bpf_get_stackid, struct pt_regs *, regs, struct bpf_map *, map,pcpu_freelist_pop (&smap->freelist);if (unlikely (!new_bucket))return -ENOMEM ; -stack_map_get_build_id_offset (map, new_bucket, ips, - trace_nr, user); + new_bucket->nr = trace_nr; +stack_map_get_build_id_offset ( + (struct bpf_stack_build_id *)new_bucket->data, + ips, trace_nr, user); trace_len = trace_nr *sizeof (struct bpf_stack_build_id );if (hash_matches && bucket->nr == trace_nr &&memcmp (bucket->data, new_bucket->data, trace_len) ==0 ) {
Signature · 1 line
2.9.5
PATCH v5 2/10 bpf: add bpf_get_stack helper
In reply to Yonghong SongCurrently, stackmap and bpf_get_stackid helper are provided for bpf program to get the stack trace. This approach has a limitation though. If two stack traces have the same hash, only one will get stored in the stackmap table,
so some stack traces are missing from user perspective.
This patch implements a new helper, bpf_get_stack, will send stack traces directly to bpf program. The bpf program is able to see all stack traces, and then can do in-kernel processing or send stack traces to user space through shared map or bpf_perf_event_output.
--- include/linux/bpf.h | 1 + include/linux/filter.h | 3 ++- include/uapi/linux/bpf.h | 19 ++++++++++++-- kernel/bpf/core.c | 5 ++++ kernel/bpf/stackmap.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 10 ++++++++ kernel/bpf/verifier.c | 3 +++ kernel/trace/bpf_trace.c | 50 +++++++++++++++++++++++++++++++++++- 8 files changed, 154 insertions(+), 4 deletions(-)
include/linux/bpf.h +1 -0
@@ -690,6 +690,7 @@ extern const struct bpf_func_proto bpf_get_current_comm_proto;extern const struct bpf_func_proto bpf_skb_vlan_push_proto;extern const struct bpf_func_proto bpf_skb_vlan_pop_proto;extern const struct bpf_func_proto bpf_get_stackid_proto; +extern const struct bpf_func_proto bpf_get_stack_proto;extern const struct bpf_func_proto bpf_sock_map_update_proto;/* Shared helpers among cBPF and eBPF. */
include/linux/filter.h +2 -1
@@ -468,7 +468,8 @@ struct bpf_prog { dst_needed:1, /* Do we need dst entry? */ blinded:1, /* Was blinded */ is_func:1, /* program is a bpf function */ - kprobe_override:1; /* Do we override a kprobe? */ + kprobe_override:1, /* Do we override a kprobe? */ + need_callchain_buf:1; /* Needs callchain buffer? */enum bpf_prog_type type;/* Type of BPF program */ enum bpf_attach_type expected_attach_type;/* For some prog types */ u32 len;/* Number of filter blocks */
include/uapi/linux/bpf.h +17 -2
@@ -529,6 +529,17 @@ union bpf_attr {* other bits - reserved * Return: >= 0 stackid on success or negative error * +* int bpf_get_stack(ctx, buf, size, flags) +* walk user or kernel stack and store the ips in buf +* @ctx: struct pt_regs* +* @buf: user buffer to fill stack +* @size: the buf size +* @flags: bits 0-7 - numer of stack frames to skip +* bit 8 - collect user stack instead of kernel +* bit 11 - get build-id as well if user stack +* other bits - reserved +* Return: >= 0 size copied on success or negative error +* * s64 bpf_csum_diff(from, from_size, to, to_size, seed) * calculate csum diff * @from: raw from buffer
@@ -841,7 +852,8 @@ union bpf_attr {FN (msg_cork_bytes), \FN (msg_pull_data), \FN (bind), \ -FN (xdp_adjust_tail), +FN (xdp_adjust_tail), \ +FN (get_stack),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
@@ -875,11 +887,14 @@ enum bpf_func_id {/* BPF_FUNC_skb_set_tunnel_key and BPF_FUNC_skb_get_tunnel_key flags. */ #define BPF_F_TUNINFO_IPV6 (1ULL << 0) -/* BPF_FUNC_get_stackid flags. */ +/* flags for both BPF_FUNC_get_stackid and BPF_FUNC_get_stack. */ #define BPF_F_SKIP_FIELD_MASK 0xffULL#define BPF_F_USER_STACK (1ULL << 8) +/* flags used by BPF_FUNC_get_stackid only. */ #define BPF_F_FAST_STACK_CMP (1ULL << 9)#define BPF_F_REUSE_STACKID (1ULL << 10) +/* flags used by BPF_FUNC_get_stack only. */ +#define BPF_F_USER_BUILD_ID (1ULL << 11)/* BPF_FUNC_skb_set_tunnel_key flags. */ #define BPF_F_ZERO_CSUM_TX (1ULL << 1)
kernel/bpf/core.c +5 -0
@@ -31,6 +31,7 @@#include <linux/rbtree_latch.h> #include <linux/kallsyms.h> #include <linux/rcupdate.h> +#include <linux/perf_event.h> #include <asm/unaligned.h>
@@ -1709,6 +1710,10 @@ static void bpf_prog_free_deferred(struct work_struct *work) aux = container_of(work, struct bpf_prog_aux, work);if (bpf_prog_is_dev_bound (aux))bpf_prog_offload_destroy (aux->prog); +#ifdef CONFIG_PERF_EVENTS +if (aux->prog->need_callchain_buf) +put_callchain_buffers (); +#endif for (i =0 ; i < aux->func_cnt; i++)bpf_jit_free (aux->func[i]);if (aux->func_cnt) {
kernel/bpf/stackmap.c +67 -0
73 lines not on this page · raw message
kernel/bpf/syscall.c +10 -0
@@ -1360,6 +1360,16 @@ static int bpf_prog_load(union bpf_attr *attr)if (err) goto free_used_maps; +if (prog->need_callchain_buf) { +#ifdef CONFIG_PERF_EVENTS + err =get_callchain_buffers (sysctl_perf_event_max_stack); +#else + err = -ENOTSUPP ; +#endif +if (err) + goto free_used_maps; + } + err =bpf_prog_new_fd (prog);if (err <0 ) {/* failed to allocate fd.
kernel/bpf/verifier.c +3 -0
@@ -2460,6 +2460,9 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insnif (err)return err; +if (func_id == BPF_FUNC_get_stack) + env->prog->need_callchain_buf = true; +if (changes_data)clear_all_pkt_pointers (env);return 0 ;
kernel/trace/bpf_trace.c +49 -1
98 lines not on this page · raw message
Signature · 1 line
2.9.5
PATCH v5 3/10 bpf/verifier: refine retval R0 state for bpf_get_stack helper
In reply to Yonghong SongThe special property of return values for helpers bpf_get_stack and bpf_probe_read_str are captured in verifier.
Both helpers return a negative error code or
a length, which is equal to or smaller than the buffer size argument. This additional information in the verifier can avoid the condition such as "retval > bufsize" in the bpf program. For example, for the code blow,
usize = bpf_get_stack(ctx, raw_data, max_len, BPF_F_USER_STACK);
if (usize < 0 || usize > max_len)
return 0;
The verifier may have the following errors:
52: (85) call bpf_get_stack#65
R0=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R1_w=ctx(id=0,off=0,imm=0)
R2_w=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R3_w=inv800 R4_w=inv256
R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0)
R9_w=inv800 R10=fp0,call_-1
53: (bf) r8 = r0
54: (bf) r1 = r8
55: (67) r1 <<= 32
56: (bf) r2 = r1
57: (77) r2 >>= 32
58: (25) if r2 > 0x31f goto pc+33
R0=inv(id=0) R1=inv(id=0,smax_value=9223372032559808512,
umax_value=18446744069414584320,
var_off=(0x0; 0xffffffff00000000))
R2=inv(id=0,umax_value=799,var_off=(0x0; 0x3ff))
R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0)
R8=inv(id=0) R9=inv800 R10=fp0,call_-1
59: (1f) r9 -= r8
60: (c7) r1 s>>= 32
61: (bf) r2 = r7
62: (0f) r2 += r1
math between map_value pointer and register with unbounded
min value is not allowedThe failure is due to llvm compiler optimization where register "r2", which is a copy of "r1", is tested for condition while later on "r1" is used for map_ptr operation. The verifier is not able to track such inst sequence effectively.
Without the "usize > max_len" condition, there is no llvm optimization and the below generated code passed verifier:
52: (85) call bpf_get_stack#65
R0=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R1_w=ctx(id=0,off=0,imm=0)
R2_w=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R3_w=inv800 R4_w=inv256
R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0)
R9_w=inv800 R10=fp0,call_-1
53: (b7) r1 = 0
54: (bf) r8 = r0
55: (67) r8 <<= 32
56: (c7) r8 s>>= 32
57: (6d) if r1 s> r8 goto pc+24
R0=inv(id=0,umax_value=800) R1=inv0 R6=ctx(id=0,off=0,imm=0)
R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0)
R8=inv(id=0,umax_value=800,var_off=(0x0; 0x3ff)) R9=inv800
R10=fp0,call_-1
58: (bf) r2 = r7
59: (0f) r2 += r8
60: (1f) r9 -= r8
61: (bf) r1 = r6--- kernel/bpf/verifier.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
kernel/bpf/verifier.c +25 -0
@@ -164,6 +164,8 @@ struct bpf_call_arg_meta {bool pkt_access;int regno;int access_size; +s64 msize_smax_value; +u64 msize_umax_value; };static DEFINE_MUTEX (bpf_verifier_lock);
@@ -1994,6 +1996,12 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno, }else if (arg_type_is_mem_size (arg_type)) {bool zero_size_allowed = (arg_type ==ARG_CONST_SIZE_OR_ZERO ); +/* remember the mem_size which may be used later +* to refine return values. +*/ + meta->msize_smax_value = reg->smax_value; + meta->msize_umax_value = reg->umax_value; +/* The register is SCALAR_VALUE; the access check * happens using its boundaries. */
@@ -2333,6 +2341,21 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)return 0 ; } +static void do_refine_retval_range (struct bpf_reg_state *regs,int ret_type, +int func_id, +struct bpf_call_arg_meta *meta) +{ +struct bpf_reg_state *ret_reg = ®s[BPF_REG_0 ]; + +if (ret_type !=RET_INTEGER || + (func_id != BPF_FUNC_get_stack && + func_id != BPF_FUNC_probe_read_str)) +return ; + + ret_reg->smax_value = meta->msize_smax_value; + ret_reg->umax_value = meta->msize_umax_value; +} +static int check_helper_call (struct bpf_verifier_env *env,int func_id,int insn_idx) {const struct bpf_func_proto *fn =NULL ;
@@ -2456,6 +2479,8 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insnreturn -EINVAL ; } +do_refine_retval_range (regs, fn->ret_type, func_id, &meta); + err =check_map_func_compatibility (env, meta.map_ptr, func_id);if (err)return err;
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v5 03/10] bpf/verifier: refine retval R0 state for bpf_get_stack helper
Quoted 98 lines · Yonghong Song
The special property of return values for helpers
bpf_get_stackandbpf_probe_read_strare captured in verifier.
Both helpers return a negative error code or
a length, which is equal to or smaller than the buffer size argument. This additional information in the verifier can avoid the condition such as "retval > bufsize" in the bpf program. For example, for the code blow,usize = bpf_get_stack(ctx, raw_data, max_len, BPF_F_USER_STACK); if (usize < 0 || usize > max_len) return 0; The verifier may have the following errors: 52: (85) call bpf_get_stack#65 R0=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R1_w=ctx(id=0,off=0,imm=0) R2_w=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R3_w=inv800 R4_w=inv256 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R9_w=inv800 R10=fp0,call_-1 53: (bf) r8 = r0 54: (bf) r1 = r8 55: (67) r1 <<= 32 56: (bf) r2 = r1 57: (77) r2 >>= 32 58: (25) if r2 > 0x31f goto pc+33 R0=inv(id=0) R1=inv(id=0,smax_value=9223372032559808512, umax_value=18446744069414584320, var_off=(0x0; 0xffffffff00000000)) R2=inv(id=0,umax_value=799,var_off=(0x0; 0x3ff)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R8=inv(id=0) R9=inv800 R10=fp0,call_-1 59: (1f) r9 -= r8 60: (c7) r1 s>>= 32 61: (bf) r2 = r7 62: (0f) r2 += r1 math between map_value pointer and register with unbounded min value is not allowedThe failure is due to llvm compiler optimization where register "r2", which is a copy of "r1", is tested for condition while later on "r1" is used for
map_ptroperation. The verifier is not able to track such inst sequence effectively.Without the "usize >
max_len" condition, there is no llvm optimization and the below generated code passed verifier:52: (85) call bpf_get_stack#65 R0=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R1_w=ctx(id=0,off=0,imm=0) R2_w=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R3_w=inv800 R4_w=inv256 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R9_w=inv800 R10=fp0,call_-1 53: (b7) r1 = 0 54: (bf) r8 = r0 55: (67) r8 <<= 32 56: (c7) r8 s>>= 32 57: (6d) if r1 s> r8 goto pc+24 R0=inv(id=0,umax_value=800) R1=inv0 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=1600,imm=0) R8=inv(id=0,umax_value=800,var_off=(0x0; 0x3ff)) R9=inv800 R10=fp0,call_-1 58: (bf) r2 = r7 59: (0f) r2 += r8 60: (1f) r9 -= r8 61: (bf) r1 = r6Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/verifier.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)kernel/bpf/verifier.c +11
-0@@ -164,6 +164,8 @@ struct bpf_call_arg_meta {bool pkt_access;int regno;int access_size; +s64 msize_smax_value; +u64 msize_umax_value; };static DEFINE_MUTEX (bpf_verifier_lock);@@ -1994,6 +1996,12 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno, }else if (arg_type_is_mem_size (arg_type)) {bool zero_size_allowed = (arg_type ==ARG_CONST_SIZE_OR_ZERO ); +/* remember the mem_size which may be used later +* to refine return values. +*/ + meta->msize_smax_value = reg->smax_value; + meta->msize_umax_value = reg->umax_value; +/* The register is SCALAR_VALUE; the access check * happens using its boundaries. */ @@ -2333,6 +2341,21 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)return 0 ; } +static void do_refine_retval_range (struct bpf_reg_state *regs,int ret_type, +int func_id, +struct bpf_call_arg_meta *meta)
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 2356
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+{ +struct bpf_reg_state *ret_reg = ®s[BPF_REG_0 ]; + +if (ret_type !=RET_INTEGER || + (func_id != BPF_FUNC_get_stack && + func_id != BPF_FUNC_probe_read_str)) +return ; + + ret_reg->smax_value = meta->msize_smax_value; + ret_reg->umax_value = meta->msize_umax_value;
should we call:
__reg_deduce_bounds(ret_reg);
__reg_bound_offset(ret_reg);here?
It doesn't seem that it will be called later and ret_reg->var_off state will be more conservative than necessary.
PATCH v5 4/10 bpf: remove never-hit branches in verifier adjust_scalar_min_max_vals
In reply to Yonghong SongIn verifier function adjust_scalar_min_max_vals,
when src_known is false and the opcode is BPF_LSH/BPF_RSH, early return will happen in the function. So remove the branch in handling BPF_LSH/BPF_RSH when src_known is false.
--- kernel/bpf/verifier.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)
kernel/bpf/verifier.c +2 -9
@@ -2932,10 +2932,7 @@ static int adjust_scalar_min_max_vals(struct bpf_verifier_env *env, dst_reg->umin_value <<= umin_val; dst_reg->umax_value <<= umax_val; } -if (src_known) - dst_reg->var_off =tnum_lshift (dst_reg->var_off, umin_val); -else - dst_reg->var_off =tnum_lshift (tnum_unknown, umin_val); + dst_reg->var_off =tnum_lshift (dst_reg->var_off, umin_val);/* We may learn something more from the var_off */ __update_reg_bounds (dst_reg);break ;
@@ -2963,11 +2960,7 @@ static int adjust_scalar_min_max_vals(struct bpf_verifier_env *env,*/ dst_reg->smin_value =S64_MIN ; dst_reg->smax_value =S64_MAX ; -if (src_known) - dst_reg->var_off =tnum_rshift (dst_reg->var_off, - umin_val); -else - dst_reg->var_off =tnum_rshift (tnum_unknown, umin_val); + dst_reg->var_off =tnum_rshift (dst_reg->var_off, umin_val); dst_reg->umin_value >>= umax_val; dst_reg->umax_value >>= umin_val;/* We may learn something more from the var_off */
Signature · 1 line
2.9.5
PATCH v5 5/10 bpf/verifier: improve register value range tracking with ARSH
In reply to Yonghong SongWhen helpers like bpf_get_stack returns an int value and later on used for arithmetic computation, the LSH and ARSH operations are often required to get proper sign extension into 64-bit. For example, without this patch:
54: R0=inv(id=0,umax_value=800)
54: (bf) r8 = r0
55: R0=inv(id=0,umax_value=800) R8_w=inv(id=0,umax_value=800)
55: (67) r8 <<= 32
56: R8_w=inv(id=0,umax_value=3435973836800,var_off=(0x0; 0x3ff00000000))
56: (c7) r8 s>>= 32
57: R8=inv(id=0)
With this patch:
54: R0=inv(id=0,umax_value=800)
54: (bf) r8 = r0
55: R0=inv(id=0,umax_value=800) R8_w=inv(id=0,umax_value=800)
55: (67) r8 <<= 32
56: R8_w=inv(id=0,umax_value=3435973836800,var_off=(0x0; 0x3ff00000000))
56: (c7) r8 s>>= 32
57: R8=inv(id=0, umax_value=800,var_off=(0x0; 0x3ff))With better range of "R8", later on when "R8" is added to other register, e.g., a map pointer or scalar-value register, the better register range can be derived and verifier failure may be avoided.
In our later example,
......
usize = bpf_get_stack(ctx, raw_data, max_len, BPF_F_USER_STACK);
if (usize < 0)
return 0;
ksize = bpf_get_stack(ctx, raw_data + usize, max_len - usize, 0);
......Without improving ARSH value range tracking, the register representing "max_len - usize" will have smin_value equal to S64_MIN and will be rejected by verifier.
--- include/linux/tnum.h | 4 +++- kernel/bpf/tnum.c | 10 ++++++++++ kernel/bpf/verifier.c | 41 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 1 deletion(-)
include/linux/tnum.h +3 -1
@@ -23,8 +23,10 @@ struct tnum tnum_range(u64 min, u64 max);/* Arithmetic and logical ops */ /* Shift a tnum left (by a fixed shift) */ struct tnum tnum_lshift (struct tnum a,u8 shift); -/* Shift a tnum right (by a fixed shift) */ +/* Shift (rsh) a tnum right (by a fixed shift) */ struct tnum tnum_rshift (struct tnum a,u8 shift); +/* Shift (arsh) a tnum right (by a fixed min_shift) */ +struct tnum tnum_arshift (struct tnum a,u8 min_shift);/* Add two tnums, return @a + @b */ struct tnum tnum_add (struct tnum a,struct tnum b);/* Subtract two tnums, return @a - @b */
kernel/bpf/tnum.c +10 -0
@@ -43,6 +43,16 @@ struct tnum tnum_rshift(struct tnum a, u8 shift)return TNUM (a.value >> shift, a.mask >> shift); } +struct tnum tnum_arshift (struct tnum a,u8 min_shift) +{ +/* if a.value is negative, arithmetic shifting by minimum shift +* will have larger negative offset compared to more shifting. +* If a.value is nonnegative, arithmetic shifting by minimum shift +* will have larger positive offset compare to more shifting. +*/ +return TNUM ((s64 )a.value >> min_shift, (s64 )a.mask >> min_shift); +} +struct tnum tnum_add (struct tnum a,struct tnum b) {u64 sm, sv, sigma, chi, mu;
kernel/bpf/verifier.c +41 -0
47 lines not on this page · raw message
Signature · 1 line
2.9.5
PATCH v5 6/10 tools/bpf: add bpf_get_stack helper to tools headers
In reply to Yonghong Song--- tools/include/uapi/linux/bpf.h | 19 +++++++++++++++++-- tools/testing/selftests/bpf/bpf_helpers.h | 3 ++- 2 files changed, 19 insertions(+), 3 deletions(-)
tools/include/uapi/linux/bpf.h +17 -2
@@ -529,6 +529,17 @@ union bpf_attr {* other bits - reserved * Return: >= 0 stackid on success or negative error * +* int bpf_get_stack(ctx, buf, size, flags) +* walk user or kernel stack and store the ips in buf +* @ctx: struct pt_regs* +* @buf: user buffer to fill stack +* @size: the buf size +* @flags: bits 0-7 - numer of stack frames to skip +* bit 8 - collect user stack instead of kernel +* bit 11 - get build-id as well if user stack +* other bits - reserved +* Return: >= 0 size copied on success or negative error +* * s64 bpf_csum_diff(from, from_size, to, to_size, seed) * calculate csum diff * @from: raw from buffer
@@ -841,7 +852,8 @@ union bpf_attr {FN (msg_cork_bytes), \FN (msg_pull_data), \FN (bind), \ -FN (xdp_adjust_tail), +FN (xdp_adjust_tail), \ +FN (get_stack),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
@@ -875,11 +887,14 @@ enum bpf_func_id {/* BPF_FUNC_skb_set_tunnel_key and BPF_FUNC_skb_get_tunnel_key flags. */ #define BPF_F_TUNINFO_IPV6 (1ULL << 0) -/* BPF_FUNC_get_stackid flags. */ +/* flags for both BPF_FUNC_get_stackid and BPF_FUNC_get_stack. */ #define BPF_F_SKIP_FIELD_MASK 0xffULL#define BPF_F_USER_STACK (1ULL << 8) +/* flags used by BPF_FUNC_get_stackid only. */ #define BPF_F_FAST_STACK_CMP (1ULL << 9)#define BPF_F_REUSE_STACKID (1ULL << 10) +/* flags used by BPF_FUNC_get_stack only. */ +#define BPF_F_USER_BUILD_ID (1ULL << 11)/* BPF_FUNC_skb_set_tunnel_key flags. */ #define BPF_F_ZERO_CSUM_TX (1ULL << 1)
tools/testing/selftests/bpf/bpf_helpers.h +2 -1
@@ -98,7 +98,8 @@ static int (*bpf_bind)(void *ctx, void *addr, int addr_len) = (void *) BPF_FUNC_bind;static int (*bpf_xdp_adjust_tail)(void *ctx,int offset) = (void *) BPF_FUNC_xdp_adjust_tail; - +static int (*bpf_get_stack)(void *ctx,void *buf,int size,int flags) = + (void *) BPF_FUNC_get_stack;/* llvm builtin functions that eBPF C program may use to * emit BPF_LD_ABS and BPF_LD_IND instructions
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v5 06/10] tools/bpf: add bpf_get_stack helper to tools headers
Quoted 2 lines · Yonghong Song
Signed-off-by Yonghong Song <[email protected]>---
please add few words to commit log. Don't leave it empty even for trivial patches.
PATCH v5 7/10 samples/bpf: move common-purpose trace functions to selftests
In reply to Yonghong SongThere is no functionality change in this patch. The common-purpose trace functions, including perf_event polling and ksym lookup, are moved from trace_output_user.c and bpf_load.c to selftests/bpf/trace_helpers.c so that these function can be reused later in selftests.
--- samples/bpf/Makefile | 11 +- samples/bpf/bpf_load.c | 63 ---------- samples/bpf/bpf_load.h | 7 -- samples/bpf/offwaketime_user.c | 1 + samples/bpf/sampleip_user.c | 1 + samples/bpf/spintest_user.c | 1 + samples/bpf/trace_event_user.c | 1 + samples/bpf/trace_output_user.c | 125 +++---------------- tools/testing/selftests/bpf/trace_helpers.c | 186 ++++++++++++++++++++++++++++ tools/testing/selftests/bpf/trace_helpers.h | 24 ++++ 10 files changed, 238 insertions(+), 182 deletions(-) create mode 100644 tools/testing/selftests/bpf/trace_helpers.c create mode 100644 tools/testing/selftests/bpf/trace_helpers.h
samples/bpf/Makefile +6 -5
@@ -49,6 +49,7 @@ hostprogs-y += xdp_adjust_tail # Libbpf dependencies LIBBPF := ../../tools/lib/bpf/bpf.o ../../tools/lib/bpf/nlattr.o CGROUP_HELPERS := ../../tools/testing/selftests/bpf/cgroup_helpers.o +TRACE_HELPERS := ../../tools/testing/selftests/bpf/trace_helpers.o test_lru_dist-objs := test_lru_dist.o $(LIBBPF) sock_example-objs := sock_example.o $(LIBBPF)
@@ -65,10 +66,10 @@ tracex6-objs := bpf_load.o $(LIBBPF) tracex6_user.o tracex7-objs := bpf_load.o $(LIBBPF) tracex7_user.o load_sock_ops-objs := bpf_load.o $(LIBBPF) load_sock_ops.o test_probe_write_user-objs := bpf_load.o $(LIBBPF) test_probe_write_user_user.o -trace_output-objs := bpf_load.o $(LIBBPF) trace_output_user.o +trace_output-objs := bpf_load.o $(LIBBPF) trace_output_user.o $(TRACE_HELPERS) lathist-objs := bpf_load.o $(LIBBPF) lathist_user.o -offwaketime-objs := bpf_load.o $(LIBBPF) offwaketime_user.o -spintest-objs := bpf_load.o $(LIBBPF) spintest_user.o +offwaketime-objs := bpf_load.o $(LIBBPF) offwaketime_user.o $(TRACE_HELPERS) +spintest-objs := bpf_load.o $(LIBBPF) spintest_user.o $(TRACE_HELPERS) map_perf_test-objs := bpf_load.o $(LIBBPF) map_perf_test_user.o test_overhead-objs := bpf_load.o $(LIBBPF) test_overhead_user.o test_cgrp2_array_pin-objs := $(LIBBPF) test_cgrp2_array_pin.o
@@ -82,8 +83,8 @@ xdp2-objs := bpf_load.o $(LIBBPF) xdp1_user.o xdp_router_ipv4-objs := bpf_load.o $(LIBBPF) xdp_router_ipv4_user.o test_current_task_under_cgroup-objs := bpf_load.o $(LIBBPF) $(CGROUP_HELPERS) \ test_current_task_under_cgroup_user.o -trace_event-objs := bpf_load.o $(LIBBPF) trace_event_user.o -sampleip-objs := bpf_load.o $(LIBBPF) sampleip_user.o +trace_event-objs := bpf_load.o $(LIBBPF) trace_event_user.o $(TRACE_HELPERS) +sampleip-objs := bpf_load.o $(LIBBPF) sampleip_user.o $(TRACE_HELPERS) tc_l2_redirect-objs := bpf_load.o $(LIBBPF) tc_l2_redirect_user.o lwt_len_hist-objs := bpf_load.o $(LIBBPF) lwt_len_hist_user.o xdp_tx_iptunnel-objs := bpf_load.o $(LIBBPF) xdp_tx_iptunnel_user.o
samples/bpf/bpf_load.c +0 -63
66 lines not on this page · raw message
samples/bpf/bpf_load.h +0 -7
@@ -54,12 +54,5 @@ int load_bpf_file(char *path);int load_bpf_file_fixup_map (const char *path,fixup_map_cb fixup_map);void read_trace_pipe (void ); -struct ksym { -long addr; -char *name; -}; - -int load_kallsyms (void ); -struct ksym *ksym_search (long key);int bpf_set_link_xdp_fd (int ifindex,int fd,__u32 flags);#endif
samples/bpf/offwaketime_user.c +1 -0
@@ -17,6 +17,7 @@#include <sys/resource.h> #include "libbpf.h" #include "bpf_load.h" +#include "trace_helpers.h" #define PRINT_RAW_ADDR 0
samples/bpf/sampleip_user.c +1 -0
@@ -22,6 +22,7 @@#include "libbpf.h" #include "bpf_load.h" #include "perf-sys.h" +#include "trace_helpers.h" #define DEFAULT_FREQ 99#define DEFAULT_SECS 5
samples/bpf/spintest_user.c +1 -0
@@ -7,6 +7,7 @@#include <sys/resource.h> #include "libbpf.h" #include "bpf_load.h" +#include "trace_helpers.h" int main (int ac,char **argv) {
samples/bpf/trace_event_user.c +1 -0
@@ -21,6 +21,7 @@#include "libbpf.h" #include "bpf_load.h" #include "perf-sys.h" +#include "trace_helpers.h" #define SAMPLE_FREQ 50
samples/bpf/trace_output_user.c +18 -107
167 lines not on this page · raw message
tools/testing/selftests/bpf/trace_helpers.c new file +186 -0
186 lines not on this page · raw message
tools/testing/selftests/bpf/trace_helpers.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __TRACE_HELPER_H +#define __TRACE_HELPER_H + +struct ksym { +long addr; +char *name; +}; + +int load_kallsyms (void ); +struct ksym *ksym_search (long key); + +typedef void (*perf_event_exec_fn )(void ); +typedef int (*perf_event_print_fn )(void *data,int size); + +/* return code for perf_event_print_fn */ +#define PERF_EVENT_DONE 0 +#define PERF_EVENT_ERROR 1 +#define PERF_EVENT_CONT 2 + +/* return PERF_EVENT_DONE or PERF_EVENT_ERROR */ +int perf_event_poller (int fd,perf_event_exec_fn exec_fn, +perf_event_print_fn output_fn); +#endif
Signature · 1 line
2.9.5
PATCH v5 8/10 tools/bpf: add a verifier test case for bpf_get_stack helper and ARSH
In reply to Yonghong SongThe test_verifier already has a few ARSH test cases. This patch adds a new test case which takes advantage of newly improved verifier behavior for bpf_get_stack and ARSH.
--- tools/testing/selftests/bpf/test_verifier.c | 45 +++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
tools/testing/selftests/bpf/test_verifier.c +45 -0
@@ -11423,6 +11423,51 @@ static struct bpf_test tests[] = { .errstr = "BPF_XADD stores into R2 packet", .prog_type = BPF_PROG_TYPE_XDP, }, + { + "bpf_get_stack return R0 within range", + .insns = { +BPF_MOV64_REG (BPF_REG_6 ,BPF_REG_1 ), +BPF_ST_MEM (BPF_DW ,BPF_REG_10 ,-8 ,0 ), +BPF_MOV64_REG (BPF_REG_2 ,BPF_REG_10 ), +BPF_ALU64_IMM (BPF_ADD ,BPF_REG_2 ,-8 ), +BPF_LD_MAP_FD (BPF_REG_1 ,0 ), +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_map_lookup_elem), +BPF_JMP_IMM (BPF_JEQ ,BPF_REG_0 ,0 ,28 ), +BPF_MOV64_REG (BPF_REG_7 ,BPF_REG_0 ), +BPF_MOV64_IMM (BPF_REG_9 ,sizeof (struct test_val )), +BPF_MOV64_REG (BPF_REG_1 ,BPF_REG_6 ), +BPF_MOV64_REG (BPF_REG_2 ,BPF_REG_7 ), +BPF_MOV64_IMM (BPF_REG_3 ,sizeof (struct test_val )), +BPF_MOV64_IMM (BPF_REG_4 ,256 ), +BPF_EMIT_CALL (BPF_FUNC_get_stack), +BPF_MOV64_IMM (BPF_REG_1 ,0 ), +BPF_MOV64_REG (BPF_REG_8 ,BPF_REG_0 ), +BPF_ALU64_IMM (BPF_LSH ,BPF_REG_8 ,32 ), +BPF_ALU64_IMM (BPF_ARSH ,BPF_REG_8 ,32 ), +BPF_JMP_REG (BPF_JSLT ,BPF_REG_1 ,BPF_REG_8 ,16 ), +BPF_ALU64_REG (BPF_SUB ,BPF_REG_9 ,BPF_REG_8 ), +BPF_MOV64_REG (BPF_REG_2 ,BPF_REG_7 ), +BPF_ALU64_REG (BPF_ADD ,BPF_REG_2 ,BPF_REG_8 ), +BPF_MOV64_REG (BPF_REG_1 ,BPF_REG_9 ), +BPF_ALU64_IMM (BPF_LSH ,BPF_REG_1 ,32 ), +BPF_ALU64_IMM (BPF_ARSH ,BPF_REG_1 ,32 ), +BPF_MOV64_REG (BPF_REG_3 ,BPF_REG_2 ), +BPF_ALU64_REG (BPF_ADD ,BPF_REG_3 ,BPF_REG_1 ), +BPF_MOV64_REG (BPF_REG_1 ,BPF_REG_7 ), +BPF_MOV64_IMM (BPF_REG_5 ,sizeof (struct test_val )), +BPF_ALU64_REG (BPF_ADD ,BPF_REG_1 ,BPF_REG_5 ), +BPF_JMP_REG (BPF_JGE ,BPF_REG_3 ,BPF_REG_1 ,4 ), +BPF_MOV64_REG (BPF_REG_1 ,BPF_REG_6 ), +BPF_MOV64_REG (BPF_REG_3 ,BPF_REG_9 ), +BPF_MOV64_IMM (BPF_REG_4 ,0 ), +BPF_EMIT_CALL (BPF_FUNC_get_stack), + BPF_EXIT_INSN(), + }, + .fixup_map2 = { 4 }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_TRACEPOINT, + }, };static int probe_filter_length (const struct bpf_insn *fp)
Signature · 1 line
2.9.5
PATCH v5 9/10 tools/bpf: add a test for bpf_get_stack with raw tracepoint prog
In reply to Yonghong SongThe test attached a raw_tracepoint program to sched/sched_switch. It tested to get stack for user space, kernel space and user space with build_id request. It also tested to get user and kernel stack into the same buffer with back-to-back bpf_get_stack helper calls.
Whenever the kernel stack is available, the user space application will check to ensure that the kernel function for raw_tracepoint ___bpf_prog_run is part of the stack.
--- tools/testing/selftests/bpf/Makefile | 3 +- tools/testing/selftests/bpf/test_get_stack_rawtp.c | 102 +++++++++++++++++ tools/testing/selftests/bpf/test_progs.c | 122 +++++++++++++++++++++ 3 files changed, 226 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/test_get_stack_rawtp.c
tools/testing/selftests/bpf/Makefile +2 -1
@@ -32,7 +32,7 @@ TEST_GEN_FILES = test_pkt_access.o test_xdp.o test_l4lb.o test_tcp_estats.o test test_l4lb_noinline.o test_xdp_noinline.o test_stacktrace_map.o \ sample_map_ret0.o test_tcpbpf_kern.o test_stacktrace_build_id.o \ sockmap_tcp_msg_prog.o connect4_prog.o connect6_prog.o test_adjust_tail.o \ - test_btf_haskv.o test_btf_nokv.o + test_btf_haskv.o test_btf_nokv.o test_get_stack_rawtp.o # Order correspond to 'make run_tests' order TEST_PROGS := test_kmod.sh \
@@ -56,6 +56,7 @@ $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/libbpf.a $(OUTPUT)/test_dev_cgroup: cgroup_helpers.c $(OUTPUT)/test_sock: cgroup_helpers.c $(OUTPUT)/test_sock_addr: cgroup_helpers.c +$(OUTPUT)/test_progs: trace_helpers.c .PHONY: force
tools/testing/selftests/bpf/test_get_stack_rawtp.c new file +102 -0
102 lines not on this page · raw message
tools/testing/selftests/bpf/test_progs.c +122 -0
140 lines not on this page · raw message
Signature · 1 line
2.9.5
PATCH v5 10/10 tools/bpf: add a test for bpf_get_stack with tracepoint prog
In reply to Yonghong SongThe test_stacktrace_map and test_stacktrace_build_id are enhanced to call bpf_get_stack in the helper to get the stack trace as well. The stack traces from bpf_get_stack and bpf_get_stackid are compared to ensure that for the same stack as represented as the same hash, their ip addresses or build id's must be the same.
--- tools/testing/selftests/bpf/test_progs.c | 70 ++++++++++++++++++++-- .../selftests/bpf/test_stacktrace_build_id.c | 20 ++++++- tools/testing/selftests/bpf/test_stacktrace_map.c | 20 ++++++- 3 files changed, 99 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/test_progs.c +64 -6
@@ -897,11 +897,47 @@ static int compare_map_keys(int map1_fd, int map2_fd)return 0 ; } +static int compare_stack_ips (int smap_fd,int amap_fd,int stack_trace_len) +{ +__u32 key, next_key, *cur_key_p, *next_key_p; +char *val_buf1, *val_buf2; +int i, err =0 ; + + val_buf1 =malloc (stack_trace_len); + val_buf2 =malloc (stack_trace_len); + cur_key_p =NULL ; + next_key_p = &key; +while (bpf_map_get_next_key (smap_fd, cur_key_p, next_key_p) ==0 ) { + err =bpf_map_lookup_elem (smap_fd, next_key_p, val_buf1); +if (err) + goto out; + err =bpf_map_lookup_elem (amap_fd, next_key_p, val_buf2); +if (err) + goto out; +for (i =0 ; i < stack_trace_len; i++) { +if (val_buf1[i] != val_buf2[i]) { + err =-1 ; + goto out; + } + } + key = *next_key_p; + cur_key_p = &key; + next_key_p = &next_key; + } +if (errno !=ENOENT ) + err =-1 ; + +out: +free (val_buf1); +free (val_buf2); +return err; +} +static void test_stacktrace_map () { -int control_map_fd, stackid_hmap_fd, stackmap_fd; +int control_map_fd, stackid_hmap_fd, stackmap_fd, stack_amap_fd;const char *file ="./test_stacktrace_map.o" ; -int bytes, efd, err, pmu_fd, prog_fd; +int bytes, efd, err, pmu_fd, prog_fd, stack_trace_len;struct perf_event_attr attr = {};__u32 key, val, duration =0 ;struct bpf_object *obj;
@@ -957,6 +993,10 @@ static void test_stacktrace_map()if (stackmap_fd <0 ) goto disable_pmu; + stack_amap_fd =bpf_find_map (__func__, obj,"stack_amap" ); +if (stack_amap_fd <0 ) + goto disable_pmu; +/* give some time for bpf program run */ sleep (1 );
@@ -978,6 +1018,12 @@ static void test_stacktrace_map() "err %d errno %d\n", err, errno)) goto disable_pmu_noerr; + stack_trace_len =PERF_MAX_STACK_DEPTH *sizeof (__u64); + err =compare_stack_ips (stackmap_fd, stack_amap_fd, stack_trace_len); +if (CHECK (err,"compare_stack_ips stackmap vs. stack_amap" , +"err %d errno %d\n" , err, errno)) + goto disable_pmu_noerr; + goto disable_pmu_noerr; disable_pmu: error_cnt++;
@@ -1071,9 +1117,9 @@ static int extract_build_id(char *build_id, size_t size)static void test_stacktrace_build_id (void ) { -int control_map_fd, stackid_hmap_fd, stackmap_fd; +int control_map_fd, stackid_hmap_fd, stackmap_fd, stack_amap_fd;const char *file ="./test_stacktrace_build_id.o" ; -int bytes, efd, err, pmu_fd, prog_fd; +int bytes, efd, err, pmu_fd, prog_fd, stack_trace_len;struct perf_event_attr attr = {};__u32 key, previous_key, val, duration =0 ;struct bpf_object *obj;
@@ -1138,6 +1184,11 @@ static void test_stacktrace_build_id(void) err, errno)) goto disable_pmu; + stack_amap_fd =bpf_find_map (__func__, obj,"stack_amap" ); +if (CHECK (stack_amap_fd <0 ,"bpf_find_map stack_amap" , +"err %d errno %d\n" , err, errno)) + goto disable_pmu; +assert (system ("dd if=/dev/urandom of=/dev/zero count=4 2> /dev/null" ) ==0 );assert (system ("./urandom_read if=/dev/urandom of=/dev/zero count=4 2> /dev/null" ) ==0 );
@@ -1189,8 +1240,15 @@ static void test_stacktrace_build_id(void) previous_key = key; }while (bpf_map_get_next_key (stackmap_fd, &previous_key, &key) ==0 ); -CHECK (build_id_matches <1 ,"build id match" , -"Didn't find expected build ID from the map" ); +if (CHECK (build_id_matches <1 ,"build id match" , +"Didn't find expected build ID from the map" )) + goto disable_pmu; + + stack_trace_len =PERF_MAX_STACK_DEPTH + *sizeof (struct bpf_stack_build_id ); + err =compare_stack_ips (stackmap_fd, stack_amap_fd, stack_trace_len); +CHECK (err,"compare_stack_ips stackmap vs. stack_amap" , +"err %d errno %d\n" , err, errno); disable_pmu:ioctl (pmu_fd,PERF_EVENT_IOC_DISABLE );
tools/testing/selftests/bpf/test_stacktrace_build_id.c +18 -2
@@ -19,7 +19,7 @@ struct bpf_map_def SEC("maps") stackid_hmap = { .type = BPF_MAP_TYPE_HASH, .key_size = sizeof(__u32), .value_size = sizeof(__u32), - .max_entries = 10000, + .max_entries = 16384, }; struct bpf_map_def SEC("maps") stackmap = {
@@ -31,6 +31,14 @@ struct bpf_map_def SEC("maps") stackmap = { .map_flags = BPF_F_STACK_BUILD_ID, }; +struct bpf_map_def SEC("maps") stack_amap = { + .type = BPF_MAP_TYPE_ARRAY, + .key_size = sizeof(__u32), + .value_size =sizeof (struct bpf_stack_build_id ) + * PERF_MAX_STACK_DEPTH, + .max_entries = 128, +}; +/* taken from /sys/kernel/debug/tracing/events/random/urandom_read/format */ struct random_urandom_args {unsigned long long pad;
@@ -42,7 +50,10 @@ struct random_urandom_args { SEC("tracepoint/random/urandom_read")int oncpu (struct random_urandom_args *args) { +__u32 max_len =sizeof (struct bpf_stack_build_id ) + *PERF_MAX_STACK_DEPTH ;__u32 key =0 , val =0 , *value_p; +void *stack_p; value_p =bpf_map_lookup_elem (&control_map, &key);if (value_p && *value_p)
@@ -50,8 +61,13 @@ int oncpu(struct random_urandom_args *args)/* The size of stackmap and stackid_hmap should be the same */ key =bpf_get_stackid (args, &stackmap,BPF_F_USER_STACK ); -if ((int )key >=0 ) +if ((int )key >=0 ) {bpf_map_update_elem (&stackid_hmap, &key, &val,0 ); + stack_p =bpf_map_lookup_elem (&stack_amap, &key); +if (stack_p) +bpf_get_stack (args, stack_p, max_len, +BPF_F_USER_STACK |BPF_F_USER_BUILD_ID ); + }return 0 ; }
tools/testing/selftests/bpf/test_stacktrace_map.c +17 -3
@@ -19,14 +19,21 @@ struct bpf_map_def SEC("maps") stackid_hmap = { .type = BPF_MAP_TYPE_HASH, .key_size = sizeof(__u32), .value_size = sizeof(__u32), - .max_entries = 10000, + .max_entries = 16384, }; struct bpf_map_def SEC("maps") stackmap = { .type = BPF_MAP_TYPE_STACK_TRACE, .key_size = sizeof(__u32), .value_size = sizeof(__u64) * PERF_MAX_STACK_DEPTH, - .max_entries = 10000, + .max_entries = 16384, +}; + +struct bpf_map_def SEC("maps") stack_amap = { + .type = BPF_MAP_TYPE_ARRAY, + .key_size = sizeof(__u32), + .value_size = sizeof(__u64) * PERF_MAX_STACK_DEPTH, + .max_entries = 16384, };/* taken from /sys/kernel/debug/tracing/events/sched/sched_switch/format */
@@ -44,7 +51,10 @@ struct sched_switch_args { SEC("tracepoint/sched/sched_switch")int oncpu (struct sched_switch_args *ctx) { +__u32 max_len =PERF_MAX_STACK_DEPTH *sizeof (__u64);__u32 key =0 , val =0 , *value_p; +void *stack_p; + value_p =bpf_map_lookup_elem (&control_map, &key);if (value_p && *value_p)
@@ -52,8 +62,12 @@ int oncpu(struct sched_switch_args *ctx)/* The size of stackmap and stackid_hmap should be the same */ key =bpf_get_stackid (ctx, &stackmap,0 ); -if ((int )key >=0 ) +if ((int )key >=0 ) {bpf_map_update_elem (&stackid_hmap, &key, &val,0 ); + stack_p =bpf_map_lookup_elem (&stack_amap, &key); +if (stack_p) +bpf_get_stack (ctx, stack_p, max_len,0 ); + }return 0 ; }
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v5 10/10] tools/bpf: add a test for bpf_get_stack with tracepoint prog
Quoted 8 lines · Yonghong Song
The
test_stacktrace_mapandtest_stacktrace_build_idare enhanced to callbpf_get_stackin the helper to get the stack trace as well. The stack traces frombpf_get_stackandbpf_get_stackidare compared to ensure that for the same stack as represented as the same hash, their ip addresses or build id's must be the same.Signed-off-by Yonghong Song <[email protected]>
...
tools/testing/selftests/bpf/test_stacktrace_map.c struct sched_switch_args { · line 57
Quoted 9 lines
/* taken from
/sys/kernel/debug/tracing/events/sched/sched_switch/format*/tools/testing/selftests/bpf/test_stacktrace_map.c +3
-0@@ -44,7 +51,10 @@ struct sched_switch_args { SEC("tracepoint/sched/sched_switch")int oncpu (struct sched_switch_args *ctx) { +__u32 max_len =PERF_MAX_STACK_DEPTH *sizeof (__u64);__u32 key =0 , val =0 , *value_p; +void *stack_p; +
nit: unnecessary extra empty line.
otherwise