Misc fixes and preliminaries for iterators
Subsystems (4)
Andrii Nakryiko
maintainercover Misc fixes and preliminaries for iterators
A bunch of small fixes and improvements done as preliminaries for upcoming open-coded iterators. Sending them out a bit earlier to avoid dragging along and rebasing a bunch of smallish changes.
Andrii Nakryiko (8):
bpf: improve stack slot state printing
bpf: improve regsafe() checks for PTR_TO_{MEM,BUF,TP_BUFFER} selftests/bpf: enhance align selftest's expected log matching
bpf: honor env->test_state_freq flag in is_state_visited() selftests/bpf: adjust log_fixup's buffer size for proper truncation
bpf: clean up visit_insn()'s instruction processing
bpf: fix visit_insn()'s detection of BPF_FUNC_timer_set_callback
helper
bpf: ensure that r0 is marked scratched after any function call
kernel/bpf/verifier.c | 111 +++++++++++------- .../testing/selftests/bpf/prog_tests/align.c | 18 ++- .../selftests/bpf/prog_tests/log_fixup.c | 2 +- 3 files changed, 83 insertions(+), 48 deletions(-)
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 1/8 bpf: improve stack slot state printing
Improve stack slot state printing to provide more useful and relevant information, especially for dynptrs. While previously we'd see something like:
8: (85) call bpf_ringbuf_reserve_dynptr#198 ; R0_w=scalar() fp-8_w=dddddddd fp-16_w=dddddddd refs=2
Now we'll see way more useful:
8: (85) call bpf_ringbuf_reserve_dynptr#198 ; R0_w=scalar() fp-16_w=dynptr_ringbuf(ref_id=2) refs=2
I experimented with printing the range of slots taken by dynptr, something like:
fp-16..8_w=dynptr_ringbuf(ref_id=2)
But it felt very awkward and pretty useless. So we print the lowest address (most negative offset) only.
The general structure of this code is now also set up for easier extension and will accommodate ITER slots naturally.
--- kernel/bpf/verifier.c | 75 ++++++++++++++++++++++++++++--------------- 1 file changed, 49 insertions(+), 26 deletions(-)
kernel/bpf/verifier.c +49 -26
@@ -705,6 +705,25 @@ static const char *kernel_type_name(const struct btf* btf, u32 id)return btf_name_by_offset (btf,btf_type_by_id (btf, id)->name_off); } +static const char *dynptr_type_str (enum bpf_dynptr_type type) +{ +switch (type) { +case BPF_DYNPTR_TYPE_LOCAL : +return "local" ; +case BPF_DYNPTR_TYPE_RINGBUF : +return "ringbuf" ; +case BPF_DYNPTR_TYPE_SKB : +return "skb" ; +case BPF_DYNPTR_TYPE_XDP : +return "xdp" ; +case BPF_DYNPTR_TYPE_INVALID : +return "<invalid>" ; +default : +WARN_ONCE (1 ,"unknown dynptr type %d\n" , type); +return "<unknown>" ; + } +} +static void mark_reg_scratched (struct bpf_verifier_env *env,u32 regno) { env->scratched_regs |=1U << regno;
@@ -1176,26 +1195,49 @@ static void print_verifier_state(struct bpf_verifier_env *env,for (j =0 ; j <BPF_REG_SIZE ; j++) {if (state->stack[i].slot_type[j] !=STACK_INVALID ) valid = true; - types_buf[j] = slot_type_char[ - state->stack[i].slot_type[j]]; + types_buf[j] = slot_type_char[state->stack[i].slot_type[j]]; } types_buf[BPF_REG_SIZE ] =0 ;if (!valid)continue ;if (!print_all && !stack_slot_scratched (env, i))continue ; -verbose (env," fp%d" , (-i -1 ) *BPF_REG_SIZE ); -print_liveness (env, state->stack[i].spilled_ptr.live); -if (is_spilled_reg (&state->stack[i])) { +switch (state->stack[i].slot_type[BPF_REG_SIZE -1 ]) { +case STACK_SPILL : reg = &state->stack[i].spilled_ptr; t = reg->type; + +verbose (env," fp%d" , (-i -1 ) *BPF_REG_SIZE ); +print_liveness (env, reg->live);verbose (env,"=%s" , t ==SCALAR_VALUE ?"" :reg_type_str (env, t));if (t ==SCALAR_VALUE && reg->precise)verbose (env,"P" );if (t ==SCALAR_VALUE &&tnum_is_const (reg->var_off))verbose (env,"%lld" , reg->var_off.value + reg->off); - }else { +break ; +case STACK_DYNPTR : + i +=BPF_DYNPTR_NR_SLOTS -1 ; + reg = &state->stack[i].spilled_ptr; + +verbose (env," fp%d" , (-i -1 ) *BPF_REG_SIZE ); +print_liveness (env, reg->live); +verbose (env,"=dynptr_%s" ,dynptr_type_str (reg->dynptr.type)); +if (reg->ref_obj_id) +verbose (env,"(ref_id=%d)" , reg->ref_obj_id); +break ; +case STACK_MISC : +case STACK_ZERO : +default : + reg = &state->stack[i].spilled_ptr; + +for (j =0 ; j <BPF_REG_SIZE ; j++) + types_buf[j] = slot_type_char[state->stack[i].slot_type[j]]; + types_buf[BPF_REG_SIZE ] =0 ; + +verbose (env," fp%d" , (-i -1 ) *BPF_REG_SIZE ); +print_liveness (env, reg->live);verbose (env,"=%s" , types_buf); +break ; } }if (state->acquired_refs && state->refs[0 ].id) {
@@ -6312,28 +6354,9 @@ static int process_dynptr_func(struct bpf_verifier_env *env, int regno, int insn/* Fold modifiers (in this case, MEM_RDONLY) when checking expected type */ if (!is_dynptr_type_expected (env, reg, arg_type & ~MEM_RDONLY )) { -const char *err_extra ="" ; - -switch (arg_type &DYNPTR_TYPE_FLAG_MASK ) { -case DYNPTR_TYPE_LOCAL : - err_extra ="local" ; -break ; -case DYNPTR_TYPE_RINGBUF : - err_extra ="ringbuf" ; -break ; -case DYNPTR_TYPE_SKB : - err_extra ="skb " ; -break ; -case DYNPTR_TYPE_XDP : - err_extra ="xdp " ; -break ; -default : - err_extra ="<unknown>" ; -break ; - }verbose (env,"Expected a dynptr of type %s as arg #%d\n" , - err_extra, regno); +dynptr_type_str (arg_to_dynptr_type (arg_type)), regno);return -EINVAL ; }
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 2/8 bpf: improve regsafe() checks for PTR_TO_{MEM,BUF,TP_BUFFER}
In reply to Andrii NakryikoTeach regsafe() logic to handle PTR_TO_MEM, PTR_TO_BUF, and PTR_TO_TP_BUFFER similarly to PTR_TO_MAP_{KEY,VALUE}. That is, instead of exact match for var_off and range, use tnum_in() and range_within() checks, allowing more general verified state to subsume more specific current state. This allows to match wider range of valid and safe states, speeding up verification and detecting wider range of equivalent states for upcoming open-coded iteration looping logic.
--- kernel/bpf/verifier.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +5 -1
@@ -14114,13 +14114,17 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold,tnum_in (rold->var_off, rcur->var_off);case PTR_TO_MAP_KEY :case PTR_TO_MAP_VALUE : +case PTR_TO_MEM : +case PTR_TO_BUF : +case PTR_TO_TP_BUFFER :/* If the new min/max/var_off satisfy the old ones and * everything else matches, we are OK. */ return memcmp (rold, rcur, offsetof(struct bpf_reg_state , var_off)) ==0 &&range_within (rold, rcur) &&tnum_in (rold->var_off, rcur->var_off) && -check_ids (rold->id, rcur->id, idmap); +check_ids (rold->id, rcur->id, idmap) && +check_ids (rold->ref_obj_id, rcur->ref_obj_id, idmap);case PTR_TO_PACKET_META :case PTR_TO_PACKET :/* We must have at least as much range as the old ptr
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 3/8 selftests/bpf: enhance align selftest's expected log matching
In reply to Andrii NakryikoAllow to search for expected register state in all the verifier log output that's related to specified instruction number.
See added comment for an example of possible situation that is happening due to a simple enhancement done in the next patch, which fixes handling of env->test_state_freq flag in state checkpointing logic.
--- tools/testing/selftests/bpf/prog_tests/align.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
tools/testing/selftests/bpf/prog_tests/align.c +12 -6
@@ -660,16 +660,22 @@ static int do_test_single(struct bpf_align_test *test)* func#0 @0 * 0: R1=ctx(off=0,imm=0) R10=fp0 * 0: (b7) r3 = 2 ; R3_w=2 +* +* Sometimes it's actually two lines below, e.g. when +* searching for "6: R3_w=scalar(umax=255,var_off=(0x0; 0xff))": +* from 4 to 6: R0_w=pkt(off=8,r=8,imm=0) R1=ctx(off=0,imm=0) R2_w=pkt(off=0,r=8,imm=0) R3_w=pkt_end(off=0,imm=0) R10=fp0 +* 6: R0_w=pkt(off=8,r=8,imm=0) R1=ctx(off=0,imm=0) R2_w=pkt(off=0,r=8,imm=0) R3_w=pkt_end(off=0,imm=0) R10=fp0 +* 6: (71) r3 = *(u8 *)(r2 +0) ; R2_w=pkt(off=0,r=8,imm=0) R3_w=scalar(umax=255,var_off=(0x0; 0xff)) */ -if (!strstr (line_ptr, m.match)) { +while (!strstr (line_ptr, m.match)) { cur_line =-1 ; line_ptr =strtok (NULL ,"\n" ); -sscanf (line_ptr,"%u: " , &cur_line); +sscanf (line_ptr ?:"" ,"%u: " , &cur_line); +if (!line_ptr || cur_line != m.line) +break ; } -if (cur_line != m.line || !line_ptr || - !strstr (line_ptr, m.match)) { -printf ("Failed to find match %u: %s\n" , - m.line, m.match); +if (cur_line != m.line || !line_ptr || !strstr (line_ptr, m.match)) { +printf ("Failed to find match %u: %s\n" , m.line, m.match); ret =1 ;printf ("%s" , bpf_vlog);break ;
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 4/8 bpf: honor env->test_state_freq flag in is_state_visited()
In reply to Andrii Nakryikoenv->test_state_freq flag can be set by user by passing BPF_F_TEST_STATE_FREQ program flag. This is used in a bunch of selftests to have predictable state checkpoints at every jump and so on.
Currently, bounded loop handling heuristic ignores this flag if number of processed jumps and/or number of processed instructions is below some thresholds, which throws off that reliable state checkpointing.
Honor this flag in all circumstances by disabling heuristic if env->test_state_freq is set.
--- kernel/bpf/verifier.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +2 -1
@@ -14556,7 +14556,8 @@ static int is_state_visited(struct bpf_verifier_env *env, int insn_idx)* This threshold shouldn't be too high either, since states * at the end of the loop are likely to be useful in pruning. */ -if (env->jmps_processed - env->prev_jmps_processed <20 && +if (!env->test_state_freq && + env->jmps_processed - env->prev_jmps_processed <20 && env->insn_processed - env->prev_insn_processed <100 ) add_new_state = false; goto miss;
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 5/8 selftests/bpf: adjust log_fixup's buffer size for proper truncation
In reply to Andrii NakryikoAdjust log_fixup's expected buffer length to fix the test. It's pretty finicky in its length expectation, but it doesn't break often. So just adjust the length to work on current kernel and with follow up iterator changes as well.
--- tools/testing/selftests/bpf/prog_tests/log_fixup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/log_fixup.c +1 -1
@@ -141,7 +141,7 @@ void test_log_fixup(void)if (test__start_subtest ("bad_core_relo_trunc_partial" ))bad_core_relo (300 ,TRUNC_PARTIAL /* truncate original log a bit */ );if (test__start_subtest ("bad_core_relo_trunc_full" )) -bad_core_relo (250 ,TRUNC_FULL /* truncate also libbpf's message patch */ ); +bad_core_relo (210 ,TRUNC_FULL /* truncate also libbpf's message patch */ );if (test__start_subtest ("bad_core_relo_subprog" ))bad_core_relo_subprog ();if (test__start_subtest ("missing_map" ))
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 6/8 bpf: clean up visit_insn()'s instruction processing
In reply to Andrii NakryikoInstead of referencing processed instruction repeatedly as insns[t] throughout entire visit_insn() function, take a local insn pointer and work with it in a cleaner way.
It makes enhancing this function further a bit easier as well.
--- kernel/bpf/verifier.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-)
kernel/bpf/verifier.c +12 -13
@@ -13389,44 +13389,43 @@ static int visit_func_call_insn(int t, struct bpf_insn *insns,*/ static int visit_insn (int t,struct bpf_verifier_env *env) { -struct bpf_insn *insns = env->prog->insnsi; +struct bpf_insn *insns = env->prog->insnsi, *insn = &insns[t];int ret; -if (bpf_pseudo_func (insns + t)) +if (bpf_pseudo_func (insn))return visit_func_call_insn (t, insns, env, true);/* All non-branch instructions have a single fall-through edge. */ -if (BPF_CLASS (insns[t].code) !=BPF_JMP && -BPF_CLASS (insns[t].code) !=BPF_JMP32 ) +if (BPF_CLASS (insn->code) !=BPF_JMP && +BPF_CLASS (insn->code) !=BPF_JMP32 )return push_insn (t, t +1 ,FALLTHROUGH , env, false); -switch (BPF_OP (insns[t].code)) { +switch (BPF_OP (insn->code)) {case BPF_EXIT :return DONE_EXPLORING ;case BPF_CALL : -if (insns[t].imm == BPF_FUNC_timer_set_callback) +if (insn->imm == BPF_FUNC_timer_set_callback)/* Mark this call insn as a prune point to trigger * is_state_visited() check before call itself is * processed by __check_func_call(). Otherwise new * async state will be pushed for further exploration. */ mark_prune_point (env, t); -return visit_func_call_insn (t, insns, env, - insns[t].src_reg ==BPF_PSEUDO_CALL ); +return visit_func_call_insn (t, insns, env, insn->src_reg ==BPF_PSEUDO_CALL );case BPF_JA : -if (BPF_SRC (insns[t].code) !=BPF_K ) +if (BPF_SRC (insn->code) !=BPF_K )return -EINVAL ;/* unconditional jump with single edge */ - ret =push_insn (t, t + insns[t].off +1 ,FALLTHROUGH , env, + ret =push_insn (t, t + insn->off +1 ,FALLTHROUGH , env, true);if (ret)return ret; -mark_prune_point (env, t + insns[t].off +1 ); -mark_jmp_point (env, t + insns[t].off +1 ); +mark_prune_point (env, t + insn->off +1 ); +mark_jmp_point (env, t + insn->off +1 );return ret;
@@ -13438,7 +13437,7 @@ static int visit_insn(int t, struct bpf_verifier_env *env)if (ret)return ret; -return push_insn (t, t + insns[t].off +1 ,BRANCH , env, true); +return push_insn (t, t + insn->off +1 ,BRANCH , env, true); } }
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 7/8 bpf: fix visit_insn()'s detection of BPF_FUNC_timer_set_callback helper
In reply to Andrii NakryikoIt's not correct to assume that any BPF_CALL instruction is a helper call. Fix visit_insn()'s detection of bpf_timer_set_callback() helper by also checking insn->code == 0. For kfuncs insn->code would be set to BPF_PSEUDO_KFUNC_CALL, and for subprog calls it will be BPF_PSEUDO_CALL.
--- kernel/bpf/verifier.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/verifier.c +1 -1
@@ -13405,7 +13405,7 @@ static int visit_insn(int t, struct bpf_verifier_env *env)return DONE_EXPLORING ;case BPF_CALL : -if (insn->imm == BPF_FUNC_timer_set_callback) +if (insn->src_reg ==0 && insn->imm == BPF_FUNC_timer_set_callback)/* Mark this call insn as a prune point to trigger * is_state_visited() check before call itself is * processed by __check_func_call(). Otherwise new
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 8/8 bpf: ensure that r0 is marked scratched after any function call
In reply to Andrii Nakryikor0 is important (unless called function is void-returning, but that's taken care of by print_verifier_state() anyways) in verifier logs. Currently for helpers we seem to print it in verifier log, but for kfuncs we don't.
Instead of figuring out where in the maze of code we accidentally set r0 as scratched for helpers and why we don't do that for kfuncs, just enforce that after any function call r0 is marked as scratched.
Also, perhaps, we should reconsider "scratched" terminology, as it's mightily confusing. "Touched" would seem more appropriate. But I left that for follow ups for now.
--- kernel/bpf/verifier.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/bpf/verifier.c +2 -0
@@ -15001,6 +15001,8 @@ static int do_check(struct bpf_verifier_env *env) err =check_helper_call (env, insn, &env->insn_idx);if (err)return err; + +mark_reg_scratched (env,BPF_REG_0 ); }else if (opcode ==BPF_JA ) { if (BPF_SRC(insn->code) != BPF_K || insn->imm != 0 ||
Signature · 1 line
2.30.2