bpf: Track aligned st store as imprecise spilled registers
Subsystems (4)
Yonghong Song
reviewerPATCH 1/1 bpf: Track aligned st store as imprecise spilled registers
With patch set [1], precision backtracing supports register spill/fill to/from the stack. The patch [2] allows initial imprecise register spill with content 0. This is a common case for cpuv3 and lower for initializing the stack variables with pattern
r1 = 0
*(u64 *)(r10 - 8) = r1
and the [2] has demonstrated good verification improvement.
For cpuv4, the initialization could be
*(u64 *)(r10 - 8) = 0
The current verifier marks the r10-8 contents with STACK_ZERO. Similar to [2], let us permit the above insn to behave like imprecise register spill which can reduce number of verified states.
I checked cpuv3 and cpuv4 with and without this patch. There is no change for cpuv3 since '*(u64 *)(r10 - 8) = 0' is only generated with cpuv4.
For cpuv4:
$ ../veristat -C old.cpuv4.csv new.cpuv4.csv -e file,prog,insns,states -s '|insns_diff|' File Program Insns (A) Insns (B) Insns (DIFF) States (A) States (B) States (DIFF) ----------------------------------------------------- ---------------------------------------------------- --------- --------- --------------- ---------- ---------- ------------- pyperf600_bpf_loop.bpf.linked3.o on_event 6066 4889 -1177 (-19.40%) 403 321 -82 (-20.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_tc 12412 11719 -693 (-5.58%) 345 330 -15 (-4.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_xdp 12478 11794 -684 (-5.48%) 346 331 -15 (-4.34%) test_cls_redirect.bpf.linked3.o cls_redirect 35483 35387 -96 (-0.27%) 2179 2177 -2 (-0.09%) local_storage_bench.bpf.linked3.o get_local 228 168 -60 (-26.32%) 17 14 -3 (-17.65%) test_l4lb_noinline.bpf.linked3.o balancer_ingress 4494 4522 +28 (+0.62%) 217 219 +2 (+0.92%) test_l4lb_noinline_dynptr.bpf.linked3.o balancer_ingress 1432 1455 +23 (+1.61%) 92 94 +2 (+2.17%) verifier_iterating_callbacks.bpf.linked3.o widening 52 41 -11 (-21.15%) 4 3 -1 (-25.00%) test_xdp_noinline.bpf.linked3.o balancer_ingress_v6 3462 3458 -4 (-0.12%) 216 216 +0 (+0.00%) ...
test_l4lb_noinline and test_l4lb_noinline_dynptr has minor regression, but pyperf600_bpf_loop and local_storage_bench gets pretty good improvement.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
--- kernel/bpf/verifier.c | 2 +- tools/testing/selftests/bpf/progs/verifier_spill_fill.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +1 -1
@@ -4491,7 +4491,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,if (fls64 (reg->umax_value) >BITS_PER_BYTE * size) state->stack[spi].spilled_ptr.id =0 ; }else if (!reg && !(off %BPF_REG_SIZE ) &&is_bpf_st_mem (insn) && - insn->imm !=0 && env->bpf_capable) { + env->bpf_capable) {struct bpf_reg_state fake_reg = {};__mark_reg_known (&fake_reg, insn->imm);
tools/testing/selftests/bpf/progs/verifier_spill_fill.c +2 -2
@@ -496,13 +496,13 @@ SEC("raw_tp") __log_level(2) __success /* make sure fp-8 is all STACK_ZERO */ -__msg("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=00000000") +__msg("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=0")/* but fp-16 is spilled IMPRECISE zero const reg */ __msg ("4: (7b) *(u64 *)(r10 -16) = r0 ; R0_w=0 R10=fp0 fp-16_w=0" )/* validate that assigning R2 from STACK_ZERO doesn't mark register * precise immediately; if necessary, it will be marked precise later */ -__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=00000000" ) +__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=0" )/* similarly, when R2 is assigned from spilled register, it is initially * imprecise, but will be marked precise later once it is used in precise context */
Signature · 1 line
2.34.1
Quoted 18 lines · Yonghong Song
With patch set [1], precision backtracing supports register spill/fill to/from the stack. The patch [2] allows initial imprecise register spill with content 0. This is a common case for cpuv3 and lower for initializing the stack variables with pattern
r1 = 0
*(u64 *)(r10 - 8) = r1
and the [2] has demonstrated good verification improvement.For cpuv4, the initialization could be
*(u64 *)(r10 - 8) = 0
The current verifier marks the r10-8 contents withSTACK_ZERO. Similar to [2], let us permit the above insn to behave like imprecise register spill which can reduce number of verified states.I checked cpuv3 and cpuv4 with and without this patch. There is no change for cpuv3 since '*(u64 *)(r10 - 8) = 0' is only generated with cpuv4.
[...]
Quoted 10 lines
test_l4lb_noinlineandtest_l4lb_noinline_dynptrhas minor regression, butpyperf600_bpf_loopandlocal_storage_benchgets pretty good improvement.[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/Cc Kuniyuki Iwashima <[email protected]>Cc Martin KaFai Lau <[email protected]>Signed-off-by Yonghong Song <[email protected]>
Quoted 53 lines · Yonghong Song
With patch set [1], precision backtracing supports register spill/fill to/from the stack. The patch [2] allows initial imprecise register spill with content 0. This is a common case for cpuv3 and lower for initializing the stack variables with pattern
r1 = 0
*(u64 *)(r10 - 8) = r1
and the [2] has demonstrated good verification improvement.For cpuv4, the initialization could be
*(u64 *)(r10 - 8) = 0
The current verifier marks the r10-8 contents withSTACK_ZERO. Similar to [2], let us permit the above insn to behave like imprecise register spill which can reduce number of verified states.I checked cpuv3 and cpuv4 with and without this patch. There is no change for cpuv3 since '*(u64 *)(r10 - 8) = 0' is only generated with cpuv4.
For cpuv4:
$ ../veristat -C old.cpuv4.csv new.cpuv4.csv -e file,prog,insns,states -s '|insns_diff|' File Program Insns (A) Insns (B) Insns (DIFF) States (A) States (B) States (DIFF) ----------------------------------------------------- ---------------------------------------------------- --------- --------- --------------- ---------- ---------- ------------- pyperf600_bpf_loop.bpf.linked3.o on_event 6066 4889 -1177 (-19.40%) 403 321 -82 (-20.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_tc 12412 11719 -693 (-5.58%) 345 330 -15 (-4.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_xdp 12478 11794 -684 (-5.48%) 346 331 -15 (-4.34%) test_cls_redirect.bpf.linked3.o cls_redirect 35483 35387 -96 (-0.27%) 2179 2177 -2 (-0.09%) local_storage_bench.bpf.linked3.o get_local 228 168 -60 (-26.32%) 17 14 -3 (-17.65%) test_l4lb_noinline.bpf.linked3.o balancer_ingress 4494 4522 +28 (+0.62%) 217 219 +2 (+0.92%) test_l4lb_noinline_dynptr.bpf.linked3.o balancer_ingress 1432 1455 +23 (+1.61%) 92 94 +2 (+2.17%) verifier_iterating_callbacks.bpf.linked3.o widening 52 41 -11 (-21.15%) 4 3 -1 (-25.00%) test_xdp_noinline.bpf.linked3.o balancer_ingress_v6 3462 3458 -4 (-0.12%) 216 216 +0 (+0.00%) ...
test_l4lb_noinlineandtest_l4lb_noinline_dynptrhas minor regression, butpyperf600_bpf_loopandlocal_storage_benchgets pretty good improvement.[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/Cc Kuniyuki Iwashima <[email protected]>Cc Martin KaFai Lau <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/verifier.c | 2 +- tools/testing/selftests/bpf/progs/verifier_spill_fill.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)kernel/bpf/verifier.c +0
-0
kernel/bpf/verifier.c static int check_stack_write_fixed_off(struct bpf_verifier_env *env, · line 4494
Quoted 5 lines
kernel/bpf/verifier.c +1
-1@@ -4491,7 +4491,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,if (fls64 (reg->umax_value) >BITS_PER_BYTE * size) state->stack[spi].spilled_ptr.id =0 ; }else if (!reg && !(off %BPF_REG_SIZE ) &&is_bpf_st_mem (insn) && - insn->imm !=0 && env->bpf_capable) { + env->bpf_capable) {
the change makes sense, there is nothing special about insn->imm == 0 case, so LGTM
tools/testing/selftests/bpf/progs/verifier_spill_fill.c SEC("raw_tp") · line 498
Quoted 11 lines
kernel/bpf/verifier.c +0
-0struct bpf_reg_state fake_reg = {};__mark_reg_known (&fake_reg, insn->imm);tools/testing/selftests/bpf/progs/verifier_spill_fill.c +0
-0@@ -496,13 +496,13 @@ SEC("raw_tp")__log_level (2 ) __success/* make sure fp-8 is all STACK_ZERO */
but we should update STACK_ZERO comments in this test
and also, STACK_ZERO situation is still possible, right? E.g., when we spill register at -4 offset, not -8. So I'd either extend or add another test to make sure we still validate that STACK_ZERO slots return precise zero. Can you add something like this?
Quoted 15 lines
tools/testing/selftests/bpf/progs/verifier_spill_fill.c +2
-3-__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=00000000" ) +__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=0" ) /* but fp-16 is spilled IMPRECISE zero const reg */__msg ("4: (7b) *(u64 *)(r10 -16) = r0 ; R0_w=0 R10=fp0 fp-16_w=0" )/* validate that assigning R2 from STACK_ZERO doesn't mark register * precise immediately; if necessary, it will be marked precise later */ -__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=00000000" ) +__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=0" )/* similarly, when R2 is assigned from spilled register, it is initially * imprecise, but will be marked precise later once it is used in precise context */ --2.34.1
Quoted 84 lines · Andrii Nakryiko
Quoted 53 lines · Yonghong Song
With patch set [1], precision backtracing supports register spill/fill to/from the stack. The patch [2] allows initial imprecise register spill with content 0. This is a common case for cpuv3 and lower for initializing the stack variables with pattern
r1 = 0
*(u64 *)(r10 - 8) = r1
and the [2] has demonstrated good verification improvement.For cpuv4, the initialization could be
*(u64 *)(r10 - 8) = 0
The current verifier marks the r10-8 contents withSTACK_ZERO. Similar to [2], let us permit the above insn to behave like imprecise register spill which can reduce number of verified states.I checked cpuv3 and cpuv4 with and without this patch. There is no change for cpuv3 since '*(u64 *)(r10 - 8) = 0' is only generated with cpuv4.
For cpuv4:
$ ../veristat -C old.cpuv4.csv new.cpuv4.csv -e file,prog,insns,states -s '|insns_diff|' File Program Insns (A) Insns (B) Insns (DIFF) States (A) States (B) States (DIFF) ----------------------------------------------------- ---------------------------------------------------- --------- --------- --------------- ---------- ---------- ------------- pyperf600_bpf_loop.bpf.linked3.o on_event 6066 4889 -1177 (-19.40%) 403 321 -82 (-20.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_tc 12412 11719 -693 (-5.58%) 345 330 -15 (-4.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_xdp 12478 11794 -684 (-5.48%) 346 331 -15 (-4.34%) test_cls_redirect.bpf.linked3.o cls_redirect 35483 35387 -96 (-0.27%) 2179 2177 -2 (-0.09%) local_storage_bench.bpf.linked3.o get_local 228 168 -60 (-26.32%) 17 14 -3 (-17.65%) test_l4lb_noinline.bpf.linked3.o balancer_ingress 4494 4522 +28 (+0.62%) 217 219 +2 (+0.92%) test_l4lb_noinline_dynptr.bpf.linked3.o balancer_ingress 1432 1455 +23 (+1.61%) 92 94 +2 (+2.17%) verifier_iterating_callbacks.bpf.linked3.o widening 52 41 -11 (-21.15%) 4 3 -1 (-25.00%) test_xdp_noinline.bpf.linked3.o balancer_ingress_v6 3462 3458 -4 (-0.12%) 216 216 +0 (+0.00%) ...
test_l4lb_noinlineandtest_l4lb_noinline_dynptrhas minor regression, butpyperf600_bpf_loopandlocal_storage_benchgets pretty good improvement.[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/Cc Kuniyuki Iwashima <[email protected]>Cc Martin KaFai Lau <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/verifier.c | 2 +- tools/testing/selftests/bpf/progs/verifier_spill_fill.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)kernel/bpf/verifier.c +0
-0kernel/bpf/verifier.c static int check_stack_write_fixed_off(struct bpf_verifier_env *env, · line 4494
Quoted 5 lines
kernel/bpf/verifier.c +1
-1@@ -4491,7 +4491,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,if (fls64 (reg->umax_value) >BITS_PER_BYTE * size) state->stack[spi].spilled_ptr.id =0 ; }else if (!reg && !(off %BPF_REG_SIZE ) &&is_bpf_st_mem (insn) && - insn->imm !=0 && env->bpf_capable) { + env->bpf_capable) {the change makes sense, there is nothing special about
insn->imm== 0 case, so LGTMtools/testing/selftests/bpf/progs/verifier_spill_fill.c SEC("raw_tp") · line 498
Quoted 11 lines
kernel/bpf/verifier.c +0
-0struct bpf_reg_state fake_reg = {};__mark_reg_known (&fake_reg, insn->imm);tools/testing/selftests/bpf/progs/verifier_spill_fill.c +0
-0@@ -496,13 +496,13 @@ SEC("raw_tp")__log_level (2 ) __success/* make sure fp-8 is all STACK_ZERO */ but we should update
STACK_ZEROcomments in this testand also,
STACK_ZEROsituation is still possible, right? E.g., when we spill register at -4 offset, not -8. So I'd either extend or add another test to make sure we still validate thatSTACK_ZEROslots return precise zero. Can you add something like this?
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_spill_fill.c SEC("raw_tp") · line 508
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_spill_fill.c +2
-2-__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=00000000" ) +__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=0" ) /* but fp-16 is spilled IMPRECISE zero const reg */__msg ("4: (7b) *(u64 *)(r10 -16) = r0 ; R0_w=0 R10=fp0 fp-16_w=0" )/* validate that assigning R2 from STACK_ZERO doesn't mark register * precise immediately; if necessary, it will be marked precise later */ -__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=00000000" ) +__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=0" )/* similarly, when R2 is assigned from spilled register, it is initially * imprecise, but will be marked precise later once it is used in precise context */
And seems like test_verifier test is failing now ([0]):
#114/p BPF_ST_MEM stack imm zero, variable offset FAIL
Failed to load prog 'Invalid argument'!
At program exit the register R0 has smin=0 smax=255 should have been in [0, 1] verification time 19 usec
stack depth 32
processed 11 insns (limit 1000000) max_states_per_insn 0total_states 0 peak_states 0 mark_read 0
[0] https://github.com/kernel-patches/bpf/actions/runs/7389645653/job/20103046755
Quoted 3 lines
Quoted 3 lines
--
2.34.1
Yonghong Song
reviewerQuoted 105 lines · Andrii Nakryiko
Quoted 80 lines · Andrii Nakryiko
Quoted 52 lines · Yonghong Song
With patch set [1], precision backtracing supports register spill/fill to/from the stack. The patch [2] allows initial imprecise register spill with content 0. This is a common case for cpuv3 and lower for initializing the stack variables with pattern
r1 = 0
*(u64 *)(r10 - 8) = r1
and the [2] has demonstrated good verification improvement.For cpuv4, the initialization could be
*(u64 *)(r10 - 8) = 0
The current verifier marks the r10-8 contents withSTACK_ZERO. Similar to [2], let us permit the above insn to behave like imprecise register spill which can reduce number of verified states.I checked cpuv3 and cpuv4 with and without this patch. There is no change for cpuv3 since '*(u64 *)(r10 - 8) = 0' is only generated with cpuv4.
For cpuv4:
$ ../veristat -C old.cpuv4.csv new.cpuv4.csv -e file,prog,insns,states -s '|insns_diff|' File Program Insns (A) Insns (B) Insns (DIFF) States (A) States (B) States (DIFF) ----------------------------------------------------- ---------------------------------------------------- --------- --------- --------------- ---------- ---------- ------------- pyperf600_bpf_loop.bpf.linked3.o on_event 6066 4889 -1177 (-19.40%) 403 321 -82 (-20.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_tc 12412 11719 -693 (-5.58%) 345 330 -15 (-4.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_xdp 12478 11794 -684 (-5.48%) 346 331 -15 (-4.34%) test_cls_redirect.bpf.linked3.o cls_redirect 35483 35387 -96 (-0.27%) 2179 2177 -2 (-0.09%) local_storage_bench.bpf.linked3.o get_local 228 168 -60 (-26.32%) 17 14 -3 (-17.65%) test_l4lb_noinline.bpf.linked3.o balancer_ingress 4494 4522 +28 (+0.62%) 217 219 +2 (+0.92%) test_l4lb_noinline_dynptr.bpf.linked3.o balancer_ingress 1432 1455 +23 (+1.61%) 92 94 +2 (+2.17%) verifier_iterating_callbacks.bpf.linked3.o widening 52 41 -11 (-21.15%) 4 3 -1 (-25.00%) test_xdp_noinline.bpf.linked3.o balancer_ingress_v6 3462 3458 -4 (-0.12%) 216 216 +0 (+0.00%) ...
test_l4lb_noinlineandtest_l4lb_noinline_dynptrhas minor regression, butpyperf600_bpf_loopandlocal_storage_benchgets pretty good improvement.[1] https://lore.kernel.org/all/[email protected]/ [2] https://lore.kernel.org/all/[email protected]/Cc Kuniyuki Iwashima <[email protected]>Cc Martin KaFai Lau <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/verifier.c | 2 +- tools/testing/selftests/bpf/progs/verifier_spill_fill.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)kernel/bpf/verifier.c +0
-0kernel/bpf/verifier.c static int check_stack_write_fixed_off(struct bpf_verifier_env *env, · line 4494
Quoted 5 lines
kernel/bpf/verifier.c +1
-1@@ -4491,7 +4491,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,if (fls64 (reg->umax_value) >BITS_PER_BYTE * size) state->stack[spi].spilled_ptr.id =0 ; }else if (!reg && !(off %BPF_REG_SIZE ) &&is_bpf_st_mem (insn) && - insn->imm !=0 && env->bpf_capable) { + env->bpf_capable) {the change makes sense, there is nothing special about
insn->imm== 0 case, so LGTMtools/testing/selftests/bpf/progs/verifier_spill_fill.c SEC("raw_tp") · line 498
Quoted 11 lines
kernel/bpf/verifier.c +0
-0struct bpf_reg_state fake_reg = {};__mark_reg_known (&fake_reg, insn->imm);tools/testing/selftests/bpf/progs/verifier_spill_fill.c +0
-0@@ -496,13 +496,13 @@ SEC("raw_tp")__log_level (2 ) __success/* make sure fp-8 is all STACK_ZERO */ but we should update
STACK_ZEROcomments in this testand also,
STACK_ZEROsituation is still possible, right? E.g., when we spill register at -4 offset, not -8. So I'd either extend or add another test to make sure we still validate thatSTACK_ZEROslots return precise zero. Can you add something like this?Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_spill_fill.c SEC("raw_tp") · line 508
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_spill_fill.c +2
-2-__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=00000000" ) +__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=0" ) /* but fp-16 is spilled IMPRECISE zero const reg */__msg ("4: (7b) *(u64 *)(r10 -16) = r0 ; R0_w=0 R10=fp0 fp-16_w=0" )/* validate that assigning R2 from STACK_ZERO doesn't mark register * precise immediately; if necessary, it will be marked precise later */ -__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=00000000" ) +__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=0" )/* similarly, when R2 is assigned from spilled register, it is initially * imprecise, but will be marked precise later once it is used in precise context */ And seems like
test_verifiertest is failing now ([0]):#114/p
BPF_ST_MEMstack imm zero, variable offset FAIL
Failed to load prog 'Invalid argument'!
At program exit the register R0 has smin=0 smax=255 should have been in [0, 1] verification time 19 usec
stack depth 32
processed 11 insns (limit 1000000)max_states_per_insn0total_states0peak_states0mark_read0
Ack. The CI also sent an email to me about this. Will investigate.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
--
2.34.1
Yonghong Song
reviewerQuoted 63 lines · Andrii Nakryiko
Quoted 52 lines · Yonghong Song
With patch set [1], precision backtracing supports register spill/fill to/from the stack. The patch [2] allows initial imprecise register spill with content 0. This is a common case for cpuv3 and lower for initializing the stack variables with pattern
r1 = 0
*(u64 *)(r10 - 8) = r1
and the [2] has demonstrated good verification improvement.For cpuv4, the initialization could be
*(u64 *)(r10 - 8) = 0
The current verifier marks the r10-8 contents withSTACK_ZERO. Similar to [2], let us permit the above insn to behave like imprecise register spill which can reduce number of verified states.I checked cpuv3 and cpuv4 with and without this patch. There is no change for cpuv3 since '*(u64 *)(r10 - 8) = 0' is only generated with cpuv4.
For cpuv4:
$ ../veristat -C old.cpuv4.csv new.cpuv4.csv -e file,prog,insns,states -s '|insns_diff|' File Program Insns (A) Insns (B) Insns (DIFF) States (A) States (B) States (DIFF) ----------------------------------------------------- ---------------------------------------------------- --------- --------- --------------- ---------- ---------- ------------- pyperf600_bpf_loop.bpf.linked3.o on_event 6066 4889 -1177 (-19.40%) 403 321 -82 (-20.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_tc 12412 11719 -693 (-5.58%) 345 330 -15 (-4.35%) xdp_synproxy_kern.bpf.linked3.o syncookie_xdp 12478 11794 -684 (-5.48%) 346 331 -15 (-4.34%) test_cls_redirect.bpf.linked3.o cls_redirect 35483 35387 -96 (-0.27%) 2179 2177 -2 (-0.09%) local_storage_bench.bpf.linked3.o get_local 228 168 -60 (-26.32%) 17 14 -3 (-17.65%) test_l4lb_noinline.bpf.linked3.o balancer_ingress 4494 4522 +28 (+0.62%) 217 219 +2 (+0.92%) test_l4lb_noinline_dynptr.bpf.linked3.o balancer_ingress 1432 1455 +23 (+1.61%) 92 94 +2 (+2.17%) verifier_iterating_callbacks.bpf.linked3.o widening 52 41 -11 (-21.15%) 4 3 -1 (-25.00%) test_xdp_noinline.bpf.linked3.o balancer_ingress_v6 3462 3458 -4 (-0.12%) 216 216 +0 (+0.00%) ...
test_l4lb_noinlineandtest_l4lb_noinline_dynptrhas minor regression, butpyperf600_bpf_loopandlocal_storage_benchgets pretty good improvement.[1] https://lore.kernel.org/all/[email protected]/ [2] https://lore.kernel.org/all/[email protected]/Cc Kuniyuki Iwashima <[email protected]>Cc Martin KaFai Lau <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/verifier.c | 2 +- tools/testing/selftests/bpf/progs/verifier_spill_fill.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)kernel/bpf/verifier.c +0
-0kernel/bpf/verifier.c static int check_stack_write_fixed_off(struct bpf_verifier_env *env, · line 4494
Quoted 5 lines
kernel/bpf/verifier.c +1
-1@@ -4491,7 +4491,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,if (fls64 (reg->umax_value) >BITS_PER_BYTE * size) state->stack[spi].spilled_ptr.id =0 ; }else if (!reg && !(off %BPF_REG_SIZE ) &&is_bpf_st_mem (insn) && - insn->imm !=0 && env->bpf_capable) { + env->bpf_capable) {the change makes sense, there is nothing special about
insn->imm== 0 case, so LGTMQuoted 2 lines
kernel/bpf/verifier.c +0
-0struct bpf_reg_state fake_reg = {};
Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_spill_fill.c SEC("raw_tp") · line 498
Quoted 9 lines
kernel/bpf/verifier.c +0
-0__mark_reg_known (&fake_reg, insn->imm);tools/testing/selftests/bpf/progs/verifier_spill_fill.c +0
-0@@ -496,13 +496,13 @@ SEC("raw_tp")__log_level (2 ) __success/* make sure fp-8 is all STACK_ZERO */ but we should update
STACK_ZEROcomments in this test
Missed this. Will update comments.
Quoted 5 lines
and also,
STACK_ZEROsituation is still possible, right? E.g., when we spill register at -4 offset, not -8. So I'd either extend or add another test to make sure we still validate thatSTACK_ZEROslots return precise zero. Can you add something like this?
Yes, if offset not 8 byte aligned, e.g., -4, it will be STACK_ZERO. Will add another test case to capture this.
Quoted 17 lines
Quoted 17 lines
tools/testing/selftests/bpf/progs/verifier_spill_fill.c +2
-3-__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=00000000" ) +__msg ("2: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=0" ) /* but fp-16 is spilled IMPRECISE zero const reg */__msg ("4: (7b) *(u64 *)(r10 -16) = r0 ; R0_w=0 R10=fp0 fp-16_w=0" )/* validate that assigning R2 from STACK_ZERO doesn't mark register * precise immediately; if necessary, it will be marked precise later */ -__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=00000000" ) +__msg ("6: (71) r2 = *(u8 *)(r10 -1) ; R2_w=0 R10=fp0 fp-8_w=0" )/* similarly, when R2 is assigned from spilled register, it is initially * imprecise, but will be marked precise later once it is used in precise context */ --2.34.1