bpf: linked scalar precision fixes
Subsystems (4)
v3 cover bpf: linked scalar precision fixes
Hi,
This v3 targets 6.6.y and changes the backport strategy based on review feedback on v2.
The original observed failure was found in Rust/Aya-generated eBPF around helper calls. Rust match lowering can keep a helper return value and a scalar filled through a by-reference helper argument in the same enum-style control flow. That makes it easy for the verifier-visible scalar values to become linked by scalar id.
The relevant verifier-log bytecode from the original reproducer is below. The later instructions only store r7 into a map so user space can observe which branch the verifier kept.
15: (85) call bpf_get_func_ret#184 ; R0_w=scalar() fp-8_w=mmmmmmmm 16: (79) r7 = *(u64 *)(r10 -8) ; R7_w=scalar() R10=fp0 17: (15) if r0 == 0x0 goto pc+1 ; R0_w=scalar() 18: (bf) r7 = r0 ; R0=scalar(id=1) R7=scalar(id=1) 19: (55) if r0 != 0x0 goto pc+6 ; R0=0 20: (67) r7 <<= 32 ; R7_w=0 21: (77) r7 >>= 32 ; R7_w=0 22: (b7) r1 = 1 ; R1_w=1
23: (55) if r7 != 0xf goto pc+1
The important verifier state shape is:
1. The program checks "if r0 == 0". The jump target is the success path, and the fallthrough path is the failure path.
2. On the failure path, "r7 = r0" gives r0 and r7 the same scalar id. The real success path skips that assignment, so r7 is independent there.
3. At the later "if r0 != 0" check, an affected verifier can explore an impossible continuation where r0 is zero and r7 is narrowed through the shared scalar id as well.
4. That impossible continuation reaches the return-value comparison with the wrong r7 value. When the real success path is analyzed later, state pruning can consider it safe against the earlier cached verifier state and skip the real continuation.
The root cause is verifier scalar state tracking, not helper-specific behavior. A helper return value in r0 and another scalar can become linked by scalar id on one branch. The real success path can skip that assignment, so the two verifier states are not equivalent.
The relevant pruning point is that regsafe()/states_equal() accepted the real success-path state against an earlier cached state where r0 was an imprecise scalar and r7 constraints were loose enough to cover the current r7. In the impossible path, r0 and r7 are linked by scalar id after instruction 18. In the real success path, instruction 18 is skipped and that scalar-id relationship does not exist. These states should therefore not be treated as equivalent for pruning.
The upstream linked-scalar precision series fixes that root cause by recording, in jmp_history, which linked registers were synchronized at each conditional jump and by using that per-instruction history during precision backtracking. This covers both the original r0 == 0 / r0 != 0 shape and the r0 == 1 / r0 != 1 shape used by the separate runtime selftest.
A Rust/Aya-specific runtime reproducer/selftest discussed in the v2 thread has been submitted separately to bpf-next for review:
https://lore.kernel.org/bpf/[email protected]/
That reproducer keeps the same helper-return/control-flow shape but shifts the success value to 1 before branching. This avoids depending on the not-equal-zero refinement and exercises linked scalar precision during state pruning directly. It uses bpf_skb_load_bytes() in the normal tc test-run path and does not require fexit attach or bpf_get_func_ret(). It is not included in this stable series because per review feedback it should go through bpf-next first before being considered for stable.
Targeted results for that separate helper-status runtime reproducer are:
v6.6.142 + reproducer: FAIL
v6.6.142 + v2 d028/9e backport path + reproducer: FAIL
v6.6.142 + this linked-scalars series + reproducer: PASS
bpf-next + reproducer: PASS
Changes since v2:
- update the subject from the v2 not-equal title to reflect the
linked-scalar precision backport used in this version;
- replace the d028f87517d6/9e314f5d8682 backport path with the full
upstream linked-scalar precision-tracking series suggested during
review;
- drop the custom Rust/Aya selftest from the stable series and point
to the separate bpf-next review instead; - adapt the linked_regs_broken_link_2 selftest log expectations for 6.6.y, where the verifier does not derive the same non-constant JMP_X scalar-vs-scalar range used by the upstream log check;
- keep 6.6.y as the first stable target and document that older LTS trees need separate adaptations.
v2:
https://lore.kernel.org/r/[email protected]/
RFC v1:
https://lore.kernel.org/r/[email protected]/
Backport details:
This series is based on v6.6.142 / stable/linux-6.6.y commit 924b4a879cbb ("Linux 6.6.142"). I would like it applied to 6.6.y first. The same issue is reproducible on 6.1.y, 5.15.y, and 5.10.y, but those trees need separate older-layout adaptations.
Instead of backporting the d028f87517d6 not-equal refinement plus the 9e314f5d8682 range-combining prerequisite, this series backports the full upstream linked-scalar precision-tracking series:
4bf79f9be434
bpf: Track equal scalars history on per-instruction level
842edb5507a1
bpf: Remove mark_precise_scalar_ids()
bebc17b1c03b
selftests/bpf: Tests for per-insn sync_linked_regs() precision tracking
cfbf25481d6d
selftests/bpf: Update comments find_equal_scalars->sync_linked_regs
Upstream series:
https://lore.kernel.org/r/[email protected]/
Patches 1 and 2 are the verifier changes from that upstream series. The main 6.6.y-specific verifier adaptation is in patch 1: 6.6.y does not yet have the newer BPF_ADD_CONST scalar-id representation, so sync_linked_regs() is adapted to the older scalar-id layout. Patch 2 then follows on top of that adapted layout.
Patches 3 and 4 bring the upstream verifier selftests and comment updates. Patch 3 has one 6.6.y-specific log adaptation:linked_regs_broken_link_2 keeps the "div by zero" reject check, but drops the upstream mark_precise log expectations because 6.6.y does not derive the scalar-vs-scalar range for that non-constant JMP_X comparison. Patch 4 only updates the two pre-existing comments that are present in 6.6.y.
Relevant QEMU selftest results on 6.6.y with this backport:
verifier_scalar_ids passed all 18 subtests, including the newly backported linked-scalar precision tests and the related check_ids_in_regsafe tests.
Thanks to Shung-Hsi Yu for reviewing v2 and suggesting the upstream linked-scalar precision series as the preferred backport direction.
Eduard Zingerman (4):
bpf: Track equal scalars history on per-instruction level
bpf: Remove mark_precise_scalar_ids()
selftests/bpf: Tests for per-insn sync_linked_regs() precision
tracking
selftests/bpf: Update comments find_equal_scalars->sync_linked_regs
include/linux/bpf_verifier.h | 4 + kernel/bpf/verifier.c | 367 +++++++++++------- .../selftests/bpf/progs/verifier_scalar_ids.c | 253 ++++++++---- .../selftests/bpf/progs/verifier_spill_fill.c | 4 +- .../bpf/progs/verifier_subprog_precision.c | 2 +- .../testing/selftests/bpf/verifier/precise.c | 2 +- 6 files changed, 417 insertions(+), 215 deletions(-)
base-commit: 924b4a879cbb75aef37c160b955b92f6894b11a4
Signature · 1 line
2.43.0
PATCH v3 1/4 bpf: Track equal scalars history on per-instruction level
From: Eduard Zingerman <[email protected]>
[ Upstream commit 4bf79f9be434e000c8e12fe83b2f4402480f1460 ]
Use bpf_verifier_state->jmp_history to track which registers were updated by find_equal_scalars() (renamed to collect_linked_regs()) when conditional jump was verified. Use recorded information in backtrack_insn() to propagate precision.
E.g. for the following program:
while verifying instructions
1: r1 = r0 | 2: if r1 < 8 goto ... | push r0,r1 as linked registers in jmp_history 3: if r0 > 16 goto ... | push r0,r1 as linked registers in jmp_history 4: r2 = r10 | 5: r2 += r0 v mark_chain_precision(r0)
while doing mark_chain_precision(r0)
5: r2 += r0 | mark r0 precise 4: r2 = r10 | 3: if r0 > 16 goto ... | mark r0,r1 as precise 2: if r1 < 8 goto ... | mark r0,r1 as precise 1: r1 = r0 v
Technically, do this as follows:
- Use 10 bits to identify each register that gains range because of sync_linked_regs():
- 3 bits for frame number;
- 6 bits for register or stack slot number;
- 1 bit to indicate if register is spilled.
- Use u64 as a vector of 6 such records + 4 bits for vector length.
- Augment struct bpf_jmp_history_entry with a field 'linked_regs' representing such vector.
- When doing check_cond_jmp_op() remember up to 6 registers that gain range because of sync_linked_regs() in such a vector.
- Don't propagate range information and reset IDs for registers that don't fit in 6-value vector.
- Push a pair {instruction index, linked registers vector} to bpf_verifier_state->jmp_history.
- When doing backtrack_insn() check if any of recorded linked registers is currently marked precise, if so mark all linked registers as precise.
This also requires fixes for two test_verifier tests:
- precise: test 1
- precise: test 2
Both tests contain the following instruction sequence:
19: (bf) r2 = r9 ; R2=scalar(id=3) R9=scalar(id=3)
20: (a5) if r2 < 0x8 goto pc+1 ; R2=scalar(id=3,umin=8)
21: (95) exit
22: (07) r2 += 1 ; R2_w=scalar(id=3+1,...) 23: (bf) r1 = r10 ; R1_w=fp0 R10=fp0 24: (07) r1 += -8 ; R1_w=fp-8 25: (b7) r3 = 0 ; R3_w=0
26: (85) call bpf_probe_read_kernel#113
The call to bpf_probe_read_kernel() at (26) forces r2 to be precise. Previously, this forced all registers with same id to become precise immediately when mark_chain_precision() is called.
After this change, the precision is propagated to registers sharing same id only when 'if' instruction is backtracked.
Hence verification log for both tests is changed:
regs=r2,r9 -> regs=r2 for instructions 25..20.
[ zhenzhong: backport to 6.6.y verifier layout and adapt
sync_linked_regs() to the pre-BPF_ADD_CONST scalar-id code. ]
--- include/linux/bpf_verifier.h | 4 + kernel/bpf/verifier.c | 256 ++++++++++++++++-- .../bpf/progs/verifier_subprog_precision.c | 2 +- 3 files changed, 239 insertions(+), 23 deletions(-)
include/linux/bpf_verifier.h +4 -0
@@ -345,6 +345,10 @@ struct bpf_jmp_history_entry { u32 prev_idx : 22;/* special flags, e.g., whether insn is doing register stack spill/load */ u32 flags : 10; +/* additional registers that need precision tracking when this +* jump is backtracked, vector of six 10-bit records +*/ +u64 linked_regs; };/* Maximum number of register states that can exist at once */
kernel/bpf/verifier.c +234 -22
403 lines not on this page · raw message
tools/testing/selftests/bpf/progs/verifier_subprog_precision.c +1 -1
@@ -141,7 +141,7 @@ __msg("mark_precise: frame0: last_idx 14 first_idx 9") __msg("mark_precise: frame0: regs=r6 stack= before 13: (bf) r1 = r7") __msg("mark_precise: frame0: regs=r6 stack= before 12: (27) r6 *= 4") __msg("mark_precise: frame0: regs=r6 stack= before 11: (25) if r6 > 0x3 goto pc+4") -__msg("mark_precise: frame0: regs=r6 stack= before 10: (bf) r6 = r0") +__msg("mark_precise: frame0: regs=r0,r6 stack= before 10: (bf) r6 = r0")__msg ("mark_precise: frame0: regs=r0 stack= before 9: (85) call bpf_loop" ) /* State entering callback body popped from states stack */__msg ("from 9 to 17: frame1:" )
Signature · 1 line
2.43.0
PATCH v3 2/4 bpf: Remove mark_precise_scalar_ids()
In reply to Zhenzhong WuFrom: Eduard Zingerman <[email protected]>
[ Upstream commit 842edb5507a1038e009d27e69d13b94b6f085763 ]
Function mark_precise_scalar_ids() is superseded by bt_sync_linked_regs() and equal scalars tracking in jump history. mark_precise_scalar_ids() propagates precision over registers sharing same ID on parent/child state boundaries, while jump history records allow bt_sync_linked_regs() to propagate same information with instruction level granularity, which is strictly more precise.
This commit removes mark_precise_scalar_ids() and updates test cases in progs/verifier_scalar_ids to reflect new verifier behavior.
The tests are updated in the following manner:
- mark_precise_scalar_ids() propagated precision regardless of presence of conditional jumps, while new jump history based logic only kicks in when conditional jumps are present.
Hence test cases are augmented with conditional jumps to still trigger precision propagation.
- As equal scalars tracking no longer relies on parent/child state boundaries some test cases are no longer interesting, such test cases are removed, namely:
- precision_same_state and precision_cross_state are superseded by
linked_regs_bpf_k;
- precision_same_state_broken_link and equal_scalars_broken_link are superseded by linked_regs_broken_link.
[ zhenzhong: backport to 6.6.y after adapting the first linked-regs
history commit to the older scalar-id verifier layout. ]
--- kernel/bpf/verifier.c | 115 ------------ .../selftests/bpf/progs/verifier_scalar_ids.c | 171 ++++++------------ .../testing/selftests/bpf/verifier/precise.c | 2 +- 3 files changed, 56 insertions(+), 232 deletions(-)
kernel/bpf/verifier.c +0 -115
@@ -4265,96 +4265,6 @@ static void mark_all_scalars_imprecise(struct bpf_verifier_env *env, struct bpf_ } } -static bool idset_contains (struct bpf_idset *s,u32 id) -{ -u32 i; - -for (i =0 ; i < s->count; ++i) -if (s->ids[i] == id) -return true; - -return false; -} - -static int idset_push (struct bpf_idset *s,u32 id) -{ -if (WARN_ON_ONCE (s->count >=ARRAY_SIZE (s->ids))) -return -EFAULT ; - s->ids[s->count++] = id; -return 0 ; -} - -static void idset_reset (struct bpf_idset *s) -{ - s->count =0 ; -} - -/* Collect a set of IDs for all registers currently marked as precise in env->bt. -* Mark all registers with these IDs as precise. -*/ -static int mark_precise_scalar_ids (struct bpf_verifier_env *env,struct bpf_verifier_state *st) -{ -struct bpf_idset *precise_ids = &env->idset_scratch; -struct backtrack_state *bt = &env->bt; -struct bpf_func_state *func; -struct bpf_reg_state *reg; -DECLARE_BITMAP (mask,64 ); -int i, fr; - -idset_reset (precise_ids); - -for (fr = bt->frame; fr >=0 ; fr--) { - func = st->frame[fr]; - -bitmap_from_u64 (mask,bt_frame_reg_mask (bt, fr)); -for_each_set_bit (i, mask,32 ) { - reg = &func->regs[i]; -if (!reg->id || reg->type !=SCALAR_VALUE ) -continue ; -if (idset_push (precise_ids, reg->id)) -return -EFAULT ; - } - -bitmap_from_u64 (mask,bt_frame_stack_mask (bt, fr)); -for_each_set_bit (i, mask,64 ) { -if (i >= func->allocated_stack /BPF_REG_SIZE ) -break ; -if (!is_spilled_scalar_reg (&func->stack[i])) -continue ; - reg = &func->stack[i].spilled_ptr; -if (!reg->id) -continue ; -if (idset_push (precise_ids, reg->id)) -return -EFAULT ; - } - } - -for (fr =0 ; fr <= st->curframe; ++fr) { - func = st->frame[fr]; - -for (i =BPF_REG_0 ; i <BPF_REG_10 ; ++i) { - reg = &func->regs[i]; -if (!reg->id) -continue ; -if (!idset_contains (precise_ids, reg->id)) -continue ; -bt_set_frame_reg (bt, fr, i); - } -for (i =0 ; i < func->allocated_stack /BPF_REG_SIZE ; ++i) { -if (!is_spilled_scalar_reg (&func->stack[i])) -continue ; - reg = &func->stack[i].spilled_ptr; -if (!reg->id) -continue ; -if (!idset_contains (precise_ids, reg->id)) -continue ; -bt_set_frame_slot (bt, fr, i); - } - } - -return 0 ; -} -/* * __mark_chain_precision() backtracks BPF program instruction sequence and * chain of verifier states making sure that register *regno* (if regno >= 0)
@@ -4487,31 +4397,6 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int regno) bt->frame, last_idx, first_idx, subseq_idx); } -/* If some register with scalar ID is marked as precise, -* make sure that all registers sharing this ID are also precise. -* This is needed to estimate effect of sync_linked_regs(). -* Do this at the last instruction of each state, -* bpf_reg_state::id fields are valid for these instructions. -* -* Allows to track precision in situation like below: -* -* r2 = unknown value -* ... -* --- state #0 --- -* ... -* r1 = r2 // r1 and r2 now share the same ID -* ... -* --- state #1 {r1.id = A, r2.id = A} --- -* ... -* if (r2 > 10) goto exit; // sync_linked_regs() assigns range to r1 -* ... -* --- state #2 {r1.id = A, r2.id = A} --- -* r3 = r10 -* r3 += r1 // need to mark both r1 and r2 -*/ -if (mark_precise_scalar_ids (env, st)) -return -EFAULT ; -if (last_idx <0 ) {/* we are at the entry into subprog, which * is expected for global funcs, but only if
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +55 -116
263 lines not on this page · raw message
tools/testing/selftests/bpf/verifier/precise.c +1 -1
@@ -106,7 +106,7 @@ mark_precise: frame0: regs=r2 stack= before 22\ mark_precise: frame0: parent state regs=r2 stack=:\ mark_precise: frame0: last_idx 20 first_idx 20\ - mark_precise: frame0: regs=r2,r9 stack= before 20\ + mark_precise: frame0: regs=r2 stack= before 20\ mark_precise: frame0: parent state regs=r2,r9 stack=:\ mark_precise: frame0: last_idx 19 first_idx 17\ mark_precise: frame0: regs=r2,r9 stack= before 19\
Signature · 1 line
2.43.0
PATCH v3 3/4 selftests/bpf: Tests for per-insn sync_linked_regs() precision tracking
In reply to Zhenzhong WuFrom: Eduard Zingerman <[email protected]>
[ Upstream commit bebc17b1c03b224a0b4aec6a171815e39f8ba9bc ]
Add a few test cases to verify precision tracking for scalars gaining range because of sync_linked_regs():
- check what happens when more than 6 registers might gain range in sync_linked_regs();
- check if precision is propagated correctly when operand of conditional jump gained range in sync_linked_regs() and one of linked registers is marked precise;
- check if precision is propagated correctly when operand of conditional jump gained range in sync_linked_regs() and a other-linked operand of the conditional jump is marked precise;
- add a minimized reproducer for precision tracking bug reported in [0];
- Check that mark_chain_precision() for one of the conditional jump operands does not trigger equal scalars precision propagation.
[0] https://lore.kernel.org/bpf/CAEf4BzZ0xidVCqB47XnkXcNhkPWF6_nTV7yt+_Lf0kcFEut2Mg@mail.gmail.com/
[ zhenzhong: keep the linked_regs_broken_link_2 reject check, but
drop the mark_precise log expectations because 6.6.y does not derive the scalar-vs-scalar range for that non-constant JMP_X comparison. ]
--- .../selftests/bpf/progs/verifier_scalar_ids.c | 162 ++++++++++++++++++ 1 file changed, 162 insertions(+)
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +162 -0
@@ -47,6 +47,72 @@ __naked void linked_regs_bpf_k(void) : __clobber_all); } +/* Registers r{0,1,2} share same ID when 'if r1 > ...' insn is processed, +* check that verifier marks r{1,2} as precise while backtracking +* 'if r1 > ...' with r0 already marked. +*/ +SEC("socket") +__success __log_level(2) +__flag(BPF_F_TEST_STATE_FREQ) +__msg("frame0: regs=r0 stack= before 5: (2d) if r1 > r3 goto pc+0") +__msg("frame0: parent state regs=r0,r1,r2,r3 stack=:") +__msg("frame0: regs=r0,r1,r2,r3 stack= before 4: (b7) r3 = 7") +__naked void linked_regs_bpf_x_src (void ) +{ + asmvolatile ( +/* r0 = random number up to 0xff */ +"call %[bpf_ktime_get_ns];" +"r0 &= 0xff;" +/* tie r0.id == r1.id == r2.id */ +"r1 = r0;" +"r2 = r0;" +"r3 = 7;" +"if r1 > r3 goto +0;" +/* force r0 to be precise, this eventually marks r1 and r2 as +* precise as well because of shared IDs +*/ +"r4 = r10;" +"r4 += r0;" +"r0 = 0;" +"exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} + +/* Registers r{0,1,2} share same ID when 'if r1 > r3' insn is processed, +* check that verifier marks r{0,1,2} as precise while backtracking +* 'if r1 > r3' with r3 already marked. +*/ +SEC("socket") +__success __log_level(2) +__flag(BPF_F_TEST_STATE_FREQ) +__msg("frame0: regs=r3 stack= before 5: (2d) if r1 > r3 goto pc+0") +__msg("frame0: parent state regs=r0,r1,r2,r3 stack=:") +__msg("frame0: regs=r0,r1,r2,r3 stack= before 4: (b7) r3 = 7") +__naked void linked_regs_bpf_x_dst (void ) +{ + asmvolatile ( +/* r0 = random number up to 0xff */ +"call %[bpf_ktime_get_ns];" +"r0 &= 0xff;" +/* tie r0.id == r1.id == r2.id */ +"r1 = r0;" +"r2 = r0;" +"r3 = 7;" +"if r1 > r3 goto +0;" +/* force r0 to be precise, this eventually marks r1 and r2 as +* precise as well because of shared IDs +*/ +"r4 = r10;" +"r4 += r3;" +"r0 = 0;" +"exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} +/* Same as linked_regs_bpf_k, but break one of the * links, note that r1 is absent from regs=... in __msg below. */
@@ -280,6 +346,102 @@ __naked void precision_two_ids(void) : __clobber_all); } +SEC("socket") +__success __log_level(2) +__flag(BPF_F_TEST_STATE_FREQ) +/* check thar r0 and r6 have different IDs after 'if', + * collect_linked_regs() can't tie more than 6 registers for a single insn. + */ +__msg("8: (25) if r0 > 0x7 goto pc+0 ; R0=scalar(id=1") +__msg ("9: (bf) r6 = r6 ; R6_w=scalar(id=2" ) +/* check that r{0-5} are marked precise after 'if' */ +__msg("frame0: regs=r0 stack= before 8: (25) if r0 > 0x7 goto pc+0") +__msg("frame0: parent state regs=r0,r1,r2,r3,r4,r5 stack=:") +__naked void linked_regs_too_many_regs (void ) +{ + asmvolatile ( +/* r0 = random number up to 0xff */ +"call %[bpf_ktime_get_ns];" +"r0 &= 0xff;" +/* tie r{0-6} IDs */ +"r1 = r0;" +"r2 = r0;" +"r3 = r0;" +"r4 = r0;" +"r5 = r0;" +"r6 = r0;" +/* propagate range for r{0-6} */ +"if r0 > 7 goto +0;" +/* make r6 appear in the log */ +"r6 = r6;" +/* force r0 to be precise, +* this would cause r{0-4} to be precise because of shared IDs +*/ +"r7 = r10;" +"r7 += r0;" +"r0 = 0;" +"exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} + +SEC("socket") +__failure __log_level(2) +__flag(BPF_F_TEST_STATE_FREQ) +__msg("div by zero") +__naked void linked_regs_broken_link_2 (void ) +{ + asmvolatile ( +"call %[bpf_get_prandom_u32];" +"r7 = r0;" +"r8 = r0;" +"call %[bpf_get_prandom_u32];" +"if r0 > 1 goto +0;" +/* r7.id == r8.id, +* thus r7 precision implies r8 precision, +* which implies r0 precision because of the conditional below. +*/ +"if r8 >= r0 goto 1f;" +/* break id relation between r7 and r8 */ +"r8 += r8;" +/* make r7 precise */ +"if r7 == 0 goto 1f;" +"r0 /= 0;" +"1:" +"r0 = 42;" +"exit;" + : + : __imm(bpf_get_prandom_u32) + : __clobber_all); +} + +/* Check that mark_chain_precision() for one of the conditional jump +* operands does not trigger equal scalars precision propagation. +*/ +SEC("socket") +__success __log_level(2) +__msg("3: (25) if r1 > 0x100 goto pc+0") +__msg("frame0: regs=r1 stack= before 2: (bf) r1 = r0") +__naked void cjmp_no_linked_regs_trigger (void ) +{ + asmvolatile ( +/* r0 = random number up to 0xff */ +"call %[bpf_ktime_get_ns];" +"r0 &= 0xff;" +/* tie r0.id == r1.id */ +"r1 = r0;" +/* the jump below would be predicted, thus r1 would be marked precise, +* this should not imply precision mark for r0 +*/ +"if r1 > 256 goto +0;" +"r0 = 0;" +"exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +} +/* Verify that check_ids() is used by regsafe() for scalars. * * r9 = ... some pointer with range X ...
--
2.43.0
PATCH v3 4/4 selftests/bpf: Update comments find_equal_scalars->sync_linked_regs
In reply to Zhenzhong WuFrom: Eduard Zingerman <[email protected]>
[ Upstream commit cfbf25481d6dec0089c99c9d33a2ea634fe8f008 ]
find_equal_scalars() is renamed to sync_linked_regs(), this commit updates existing references in the selftests comments.
--- tools/testing/selftests/bpf/progs/verifier_spill_fill.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/verifier_spill_fill.c +2 -2
@@ -392,7 +392,7 @@ __naked void spill_32bit_of_64bit_fail(void) *(u32 *)(r10 -8 ) = r1; \/* 32-bit fill r2 from stack. */ \ r2 = *(u32 *)(r10 -8 ); \ -/* Compare r2 with another register to trigger find_equal_scalars.\ +/* Compare r2 with another register to trigger sync_linked_regs.\ * Having one random bit is important here, otherwise the verifier cuts\ * the corners. If the ID was mistakenly preserved on spill, this would\ * cause the verifier to think that r1 is also equal to zero in one of\
@@ -431,7 +431,7 @@ __naked void spill_16bit_of_32bit_fail(void) *(u16 *)(r10 -8 ) = r1; \/* 16-bit fill r2 from stack. */ \ r2 = *(u16 *)(r10 -8 ); \ -/* Compare r2 with another register to trigger find_equal_scalars.\ +/* Compare r2 with another register to trigger sync_linked_regs.\ * Having one random bit is important here, otherwise the verifier cuts\ * the corners. If the ID was mistakenly preserved on spill, this would\ * cause the verifier to think that r1 is also equal to zero in one of\
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH stable 6.6.y v3 1/4] bpf: Track equal scalars history on per-instruction level
In reply to Zhenzhong WuThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Flawed tracking of linked registers in collect_linked_regs() wastes array capacity, leading to…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Flawed tracking of linked registers in collect_linked_regs() wastes array capacity, leading to silent clearing of valid scalar IDs and precision loss. --
commit 1fe1c2de8417cc37930a0c9654b28c2004c45842
Author: Eduard Zingerman <[email protected]>
Subject: bpf: Track equal scalars history on per-instruction level
This commit tracks registers updated by collect_linked_regs() during conditional jumps using jmp_history, and propagates precision via backtrack_insn().
Quoted 16 lines
kernel/bpf/verifier.c +5
-3@@ -14699,19 +14699,78 @@ static bool try_match_pkt_pointers(const struct bpf_insn *insn,return true; } -static void find_equal_scalars (struct bpf_verifier_state *vstate, -struct bpf_reg_state *known_reg) +static void __collect_linked_regs (struct linked_regs *reg_set,struct bpf_reg_state *reg, +u32 id,u32 frameno,u32 spi_or_reg,bool is_reg) { -struct bpf_func_state *state; +struct linked_reg *e; + +if (reg->type !=SCALAR_VALUE || reg->id != id)
kernel/bpf/verifier.c static bool try_match_pkt_pointers(const struct bpf_insn *insn, · line 14717
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+return ; + + e =linked_regs_push (reg_set); +if (e) { + e->frameno = frameno; + e->is_reg = is_reg; + e->regno = spi_or_reg; + }else { + reg->id =0 ; + }
[Severity: High]
When the array capacity is exhausted (which could happen prematurely due to the duplication and cleanup issues mentioned below), this silently clears reg->id = 0.
Could this precision loss cause the verifier to unnecessarily reject valid BPF programs that rely on equal scalar precision propagation?
Quoted 9 lines
kernel/bpf/verifier.c +9
-0+} + +/* For all R being scalar registers or spilled scalar registers +* in verifier state, save R in linked_regs if R->id == id. +* If there are too many Rs sharing same id, reset id for leftover Rs. +*/ +static void collect_linked_regs (struct bpf_verifier_state *vstate,u32 id, +struct linked_regs *linked_regs) +{
[ ... ]
Quoted 7 lines
kernel/bpf/verifier.c +7
-0+for (i = vstate->curframe; i >=0 ; i--) { + func = vstate->frame[i]; +for (j =0 ; j <BPF_REG_FP ; j++) { + reg = &func->regs[j]; +__collect_linked_regs (linked_regs, reg, id, i, j, true); + } +for (j =0 ; j < func->allocated_stack /BPF_REG_SIZE ; j++) {
kernel/bpf/verifier.c static bool try_match_pkt_pointers(const struct bpf_insn *insn, · line 14747
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+if (!is_spilled_reg (&func->stack[j])) +continue ; + reg = &func->stack[j].spilled_ptr; +__collect_linked_regs (linked_regs, reg, id, i, j, false); + } + } + +if (linked_regs->cnt ==1 ) + linked_regs->cnt =0 ; +}
[Severity: High]
Does this correctly discard isolated registers if the array is already populated?
If linked_regs already contains elements from a previous collect_linked_regs() call for src_reg, adding a single isolated dst_reg will increase the total count beyond 1. This bypasses the linked_regs->cnt == 1 cleanup logic, leaving the isolated register in the array and wasting capacity.
[ ... ]
kernel/bpf/verifier.c static int check_cond_jmp_op(struct bpf_verifier_env *env, · line 14910
Quoted 13 lines
kernel/bpf/verifier.c +9
-0@@ -14899,6 +14899,21 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,return 0 ; } +/* Push scalar registers sharing same ID to jump history, +* do this before creating 'other_branch', so that both +* 'this_branch' and 'other_branch' share this history +* if parent state is created. +*/ +if (BPF_SRC (insn->code) ==BPF_X && src_reg->type ==SCALAR_VALUE && src_reg->id) +collect_linked_regs (this_branch, src_reg->id, &linked_regs); +if (dst_reg->type ==SCALAR_VALUE && dst_reg->id) +collect_linked_regs (this_branch, dst_reg->id, &linked_regs);
[Severity: High]
If src_reg->id and dst_reg->id are identical, does this sequence append all associated registers into linked_regs twice? Since LINKED_REGS_MAX is only 6, this duplication might quickly exhaust the array capacity.
Quoted 5 lines
kernel/bpf/verifier.c +5
-0+if (linked_regs.cnt >0 ) { + err =push_jmp_history (env, this_branch,0 ,linked_regs_pack (&linked_regs)); +if (err) +return err; + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[email protected]
AI reviewRe: [PATCH stable 6.6.y v3 3/4] selftests/bpf: Tests for per-insn sync_linked_regs() precision tracking
In reply to Zhenzhong WuThank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Low] Newly added multi-line comments violate the strict BPF subsystem comment style by placing text on…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Low] Newly added multi-line comments violate the strict BPF subsystem comment style by placing text on the same line as the opening /*.
- [Low] A comment in linked_regs_bpf_x_dst incorrectly claims to force r0 to be precise, but the corresponding code actually targets r3.
- [Low] A comment in linked_regs_too_many_regs incorrectly states that r{0-4} will be precise, missing r5.
--
commit 85ad29f2c85dcd0b0ee951c5fc1bc2711d79a7a5
Author: Eduard Zingerman <[email protected]>
selftests/bpf: Tests for per-insn sync_linked_regs() precision tracking
This commit adds test cases to verify precision tracking for scalars gaining range because of sync_linked_regs(). It includes checks for when more than 6 registers might gain range, precision propagation, and adds a reproducer for a precision tracking bug.
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c __naked void linked_regs_bpf_k(void) · line 53
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +4
-0@@ -47,6 +47,72 @@ __naked void linked_regs_bpf_k(void) : __clobber_all); } +/* Registers r{0,1,2} share same ID when 'if r1 > ...' insn is processed, +* check that verifier marks r{1,2} as precise while backtracking +* 'if r1 > ...' with r0 already marked. +*/
[Severity: Low]
This isn't a bug, but the BPF subsystem multi-line comment style requires the opening /* to be on its own line, rather than starting text on the first line.
Quoted 2 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +2
-0+SEC("socket") +__success __log_level(2)
[ ... ]
Quoted 4 lines
+/* Registers r{0,1,2} share same ID when 'if r1 > r3' insn is processed,
+ * check that verifier marks r{0,1,2} as precise while backtracking
+ * 'if r1 > r3' with r3 already marked.
+ */
[Severity: Low]
This isn't a bug, but here's another instance where the multi-line comment starts text on the opening line instead of placing the /* on its own line.
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +12
-0+SEC("socket") +__success __log_level(2) +__flag(BPF_F_TEST_STATE_FREQ) +__msg("frame0: regs=r3 stack= before 5: (2d) if r1 > r3 goto pc+0") +__msg("frame0: parent state regs=r0,r1,r2,r3 stack=:") +__msg("frame0: regs=r0,r1,r2,r3 stack= before 4: (b7) r3 = 7") +__naked void linked_regs_bpf_x_dst (void ) +{ + asmvolatile ( +/* r0 = random number up to 0xff */ +"call %[bpf_ktime_get_ns];" + "r0 &= 0xff;"
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c __naked void linked_regs_bpf_k(void) · line 108
Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +10
-0+/* tie r0.id == r1.id == r2.id */ +"r1 = r0;" +"r2 = r0;" +"r3 = 7;" +"if r1 > r3 goto +0;" +/* force r0 to be precise, this eventually marks r1 and r2 as +* precise as well because of shared IDs +*/ +"r4 = r10;" +"r4 += r3;"
[Severity: Low]
This isn't a bug, but should the comment here say "force r3 to be precise" instead of "force r0"? The code actually targets r3, so this appears to be a copy-paste error from linked_regs_bpf_x_src.
Quoted 6 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +6
-0+ "r0 = 0;" + "exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +}
[ ... ]
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c __naked void precision_two_ids(void) · line 354
Quoted 6 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +6
-0+SEC("socket") +__success __log_level(2) +__flag (BPF_F_TEST_STATE_FREQ ) +/* check thar r0 and r6 have different IDs after 'if', +* collect_linked_regs() can't tie more than 6 registers for a single insn. +*/
[Severity: Low]
This isn't a bug, but just a quick note that this multi-line comment also doesn't follow the preferred BPF style of having the opening /* on its own line.
Quoted 15 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +15
-0+__msg("8: (25) if r0 > 0x7 goto pc+0 ; R0=scalar(id=1") +__msg ("9: (bf) r6 = r6 ; R6_w=scalar(id=2" ) +/* check that r{0-5} are marked precise after 'if' */ +__msg("frame0: regs=r0 stack= before 8: (25) if r0 > 0x7 goto pc+0") +__msg("frame0: parent state regs=r0,r1,r2,r3,r4,r5 stack=:") +__naked void linked_regs_too_many_regs (void ) +{ + asmvolatile ( +/* r0 = random number up to 0xff */ +"call %[bpf_ktime_get_ns];" +"r0 &= 0xff;" +/* tie r{0-6} IDs */ +"r1 = r0;" +"r2 = r0;" + "r3 = r0;"
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c __naked void precision_two_ids(void) · line 379
Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +10
-0+"r4 = r0;" +"r5 = r0;" +"r6 = r0;" +/* propagate range for r{0-6} */ +"if r0 > 7 goto +0;" +/* make r6 appear in the log */ +"r6 = r6;" +/* force r0 to be precise, +* this would cause r{0-4} to be precise because of shared IDs +*/
[Severity: Low]
This isn't a bug, but could the comment mention r{0-5} will be precise instead of r{0-4}? The __msg assertion properly verifies regs=r0,r1,r2,r3,r4,r5, so the text might be accidentally omitting r5.
Quoted 8 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +8
-0+ "r7 = r10;" + "r7 += r0;" + "r0 = 0;" + "exit;" + : + : __imm(bpf_ktime_get_ns) + : __clobber_all); +}
[ ... ]
Quoted 3 lines
+/* Check that
mark_chain_precision()for one of the conditional jump
+ * operands does not trigger equal scalars precision propagation.
+ */
[Severity: Low]
This isn't a bug, but this multi-line comment starts text on the opening line, similar to the ones above.
Quoted 5 lines
tools/testing/selftests/bpf/progs/verifier_scalar_ids.c +5
-0+SEC("socket") +__success __log_level(2) +__msg("3: (25) if r1 > 0x100 goto pc+0") +__msg("frame0: regs=r1 stack= before 2: (bf) r1 = r0") +__naked void cjmp_no_linked_regs_trigger (void )
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH stable 6.6.y v3 2/4] bpf: Remove mark_precise_scalar_ids()
In reply to Zhenzhong WuThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Low] Leftover unused struct bpf_idset and union member idset_scratch. - [Low] Multi-line comments do not…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Low] Leftover unused struct bpf_idset and union member idset_scratch.
- [Low] Multi-line comments do not follow the BPF subsystem modern comment style. --
Note: Due to recitation policy restrictions, this report does not quote the original patch code inline. Instead, it provides a free-form summary of the findings with references to the modified files and functions.
commit b92dc3ff591ac22c49e14f36339cfd789c618695
Author: Eduard Zingerman <[email protected]>
bpf: Remove mark_precise_scalar_ids()
This commit removes the mark_precise_scalar_ids() function, as it is superseded by jump history tracking and bt_sync_linked_regs(). Test cases in progs/verifier_scalar_ids are updated to reflect the new verifier behavior requiring conditional jumps.
File: include/linux/bpf_verifier.h
[Severity: Low]
Since the patch completely removes mark_precise_scalar_ids(), does this leave struct bpf_idset and the idset_scratch member in struct bpf_verifier_env as unused dead code?
It appears mark_precise_scalar_ids() was the only consumer of these definitions. Removing them from include/linux/bpf_verifier.h might help keep the verifier environment structure clean.
File: tools/testing/selftests/bpf/progs/verifier_scalar_ids.c
[Severity: Low]
This isn't a bug, but do the modified and newly added multi-line comments in the file, including inside linked_regs_broken_link(), deviate from the preferred BPF subsystem comment style?
The BPF guidelines request that multi-line comments have the opening indicator on its own line, but several comments in this file keep the text on the same line as the opening indicator.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Quoted 2 lines · Zhenzhong Wu
This v3 targets 6.6.y and changes the backport strategy based on review feedback on v2.
Queued all four for 6.6.y, thanks.
--
Thanks,
Sasha
Quoted 4 lines · Zhenzhong Wu
Hi,
This v3 targets 6.6.y and changes the backport strategy based on review feedback on v2.
[...]
Quoted 5 lines
Relevant QEMU selftest results on 6.6.y with this backport:
verifier_scalar_idspassed all 18 subtests, including the newly backported linked-scalar precision tests and the relatedcheck_ids_in_regsafetests.
The first patch in this backport series is actually breaking the "precise: test 1" selftest from test_verifier. You can see the full error at [1]. I haven't yet checked if it's the test or the backport that needs to be adjusted.
1: https://github.com/shunghsiyu/libbpf/actions/runs/27575831217/job/81523786835
Quoted 23 lines
Thanks to Shung-Hsi Yu for reviewing v2 and suggesting the upstream linked-scalar precision series as the preferred backport direction.
Eduard Zingerman (4):
bpf: Track equal scalars history on per-instruction level
bpf: Removemark_precise_scalar_ids()
selftests/bpf: Tests for per-insnsync_linked_regs()precision
tracking
selftests/bpf: Update commentsfind_equal_scalars->sync_linked_regsinclude/linux/bpf_verifier.h | 4 + kernel/bpf/verifier.c | 367 +++++++++++------- .../selftests/bpf/progs/verifier_scalar_ids.c | 253 ++++++++---- .../selftests/bpf/progs/verifier_spill_fill.c | 4 +- .../bpf/progs/verifier_subprog_precision.c | 2 +- .../testing/selftests/bpf/verifier/precise.c | 2 +- 6 files changed, 417 insertions(+), 215 deletions(-)base-commit:
924b4a879cbb75aef37c160b955b92f6894b11a4Signature · 1 line
2.43.0
Quoted 10 lines · Paul Chaignon
Quoted 4 lines · Zhenzhong Wu
Hi,
This v3 targets 6.6.y and changes the backport strategy based on review feedback on v2.
[...]
Quoted 2 lines
Relevant QEMU selftest results on 6.6.y with this backport:
Quoted 8 lines
Quoted 3 lines
verifier_scalar_idspassed all 18 subtests, including the newly backported linked-scalar precision tests and the relatedcheck_ids_in_regsafetests.The first patch in this backport series is actually breaking the "precise: test 1" selftest from
test_verifier. You can see the full error at [1]. I haven't yet checked if it's the test or the backport that needs to be adjusted.
I had a quick look, and believe it was that test that needs to be adjusted to include r9 into the precise register set.
So unless Sasha have other preference, I suggest Zhenzhong send a v4, with changes to tools/testing/selftests/bpf/verifier/precise.c (including "r9" the the expected verifier output) merged into "bpf:
Track equal scalars history on per-instruction level".
---
The program under test is:
00: BPF_MOV64_IMM(BPF_REG_0, 1),
01: BPF_LD_MAP_FD(BPF_REG_6, 0),
03: BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
04: BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
05: BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
06: BPF_ST_MEM(BPF_DW, BPF_REG_FP, -8, 0),
07: BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
08: BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
09: BPF_EXIT_INSN(),
10: BPF_MOV64_REG(BPF_REG_9, BPF_REG_0),
11: BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
12: BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
13: BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
14: BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
15: BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
16: BPF_EXIT_INSN(),
17: BPF_MOV64_REG(BPF_REG_8, BPF_REG_0),
18: BPF_ALU64_REG(BPF_SUB, BPF_REG_9, BPF_REG_8), /* map_value_ptr -= map_value_ptr */ 19: BPF_MOV64_REG(BPF_REG_2, BPF_REG_9),
20: BPF_JMP_IMM(BPF_JLT, BPF_REG_2, 8, 1),
21: BPF_EXIT_INSN(),
22: BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1), /* R2=scalar(umin=1, umax=8) */ 23: BPF_MOV64_REG(BPF_REG_1, BPF_REG_FP),
24: BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8),
25: BPF_MOV64_IMM(BPF_REG_3, 0),
26: BPF_EMIT_CALL(BPF_FUNC_probe_read_kernel),
27: BPF_EXIT_INSN(),
The test was expecting the following line in the verifier log that was shown during the backtracking start at instruction 26 (call bpf_probe_read_kernel#113)
mark_precise: frame0: regs=r2 stack= before 20: (a5) if r2 < 0x8 goto pc+1
mark_precise: frame0: parent state regs=r2 stack=: ...
mark_precise: frame0: last_idx 19 first_idx 10 ...
But after applying the patchset, we now got an additional register r9 in the precise set:
mark_precise: frame0: regs=r2 stack= before 20: (a5) if r2 < 0x8 goto pc+1
mark_precise: frame0: parent state regs=r2,r9 stack=: ....
mark_precise: frame0: last_idx 19 first_idx 10 ...
The additional r9 in the precise set seems actually correct, this is because r2 and r9 share the same scalar ID at instruction 20 (before the link got broken in instruction 21), and hence at that point, both register should be marked as precise.
---
In upstream the test already has the expected verifier log to include r9, and hence no failure, but it simply comes from the fact that r2 and r9 maintain a link even after instruction 22 (r2 += 1).
commit 98d7ca374ba4b39e7535613d40e159f09ca14da2
Author: Alexei Starovoitov <[email protected]>
Date: Wed Jun 12 18:38:13 2024 -0700
bpf: Track delta between "linked" registers.
...
--- a/tools/testing/selftests/bpf/verifier/precise.c +++ b/tools/testing/selftests/bpf/verifier/precise.c @@ -39,12 +39,12 @@
.result = VERBOSE_ACCEPT,
.errstr =
"mark_precise: frame0: last_idx 26 first_idx 20\
- mark_precise: frame0: regs=r2 stack= before 25\
- mark_precise: frame0: regs=r2 stack= before 24\
- mark_precise: frame0: regs=r2 stack= before 23\
- mark_precise: frame0: regs=r2 stack= before 22\
- mark_precise: frame0: regs=r2 stack= before 20\
- mark_precise: frame0: parent state regs=r2 stack=:\
+ mark_precise: frame0: regs=r2,r9 stack= before 25\
+ mark_precise: frame0: regs=r2,r9 stack= before 24\
+ mark_precise: frame0: regs=r2,r9 stack= before 23\
+ mark_precise: frame0: regs=r2,r9 stack= before 22\
+ mark_precise: frame0: regs=r2,r9 stack= before 20\
+ mark_precise: frame0: parent state regs=r2,r9 stack=:\
mark_precise: frame0: last_idx 19 first_idx 10\
mark_precise: frame0: regs=r2,r9 stack= before 19\
mark_precise: frame0: regs=r9 stack= before 18\
...
---
Full test log below
#492/p precise: test 1 FAIL
Unexpected verifier log!
EXP: mark_precise: frame0: parent state regs=r2 stack=:
RES:
func#0 @0
0: R1=ctx(off=0,imm=0) R10=fp0
0: (b7) r0 = 1 ; R0_w=1 1: (18) r6 = 0xffff9eb644619000 ; R6_w=map_ptr(off=0,ks=4,vs=48,imm=0) 3: (bf) r1 = r6 ; R1_w=map_ptr(off=0,ks=4,vs=48,imm=0) R6_w=map_ptr(off=0,ks=4,vs=48,imm=0) 4: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 5: (07) r2 += -8 ; R2_w=fp-8 6: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=00000000 7: (85) call bpf_map_lookup_elem#1 ; R0_w=map_value_or_null(id=1,off=0,ks=4,vs=48,imm=0) 8: (55) if r0 != 0x0 goto pc+1 ; R0_w=0
9: (95) exit
from 8 to 10: R0=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm 10: R0=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm
10: (bf) r9 = r0 ; R0=map_value(off=0,ks=4,vs=48,imm=0) R9_w=map_value(off=0,ks=4,vs=48,imm=0) 11: (bf) r1 = r6 ; R1_w=map_ptr(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) 12: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 13: (07) r2 += -8 ; R2_w=fp-8 14: (85) call bpf_map_lookup_elem#1 ; R0_w=map_value_or_null(id=2,off=0,ks=4,vs=48,imm=0) 15: (55) if r0 != 0x0 goto pc+1 ; R0_w=0
16: (95) exit
from 15 to 17: R0_w=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R9_w=map_value(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm 17: R0_w=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R9_w=map_value(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm
17: (bf) r8 = r0 ; R0_w=map_value(off=0,ks=4,vs=48,imm=0) R8_w=map_value(off=0,ks=4,vs=48,imm=0) 18: (1f) r9 -= r8 ; R8_w=map_value(off=0,ks=4,vs=48,imm=0) R9_w=scalar() 19: (bf) r2 = r9 ; R2=scalar(id=3) R9=scalar(id=3) 20: (a5) if r2 < 0x8 goto pc+1 ; R2=scalar(id=3,umin=8)
21: (95) exit
from 20 to 22: R0=map_value(off=0,ks=4,vs=48,imm=0) R2=scalar(id=3,umax=7,var_off=(0x0; 0x7)) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R8=map_value(off=0,ks=4,vs=48,imm=0) R9=scalar(id=3,umax=7,var_off=(0x0; 0x7)) R10=fp0 fp-8=0000mmmm 22: R0=map_value(off=0,ks=4,vs=48,imm=0) R2=scalar(id=3,umax=7,var_off=(0x0; 0x7)) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R8=map_value(off=0,ks=4,vs=48,imm=0) R9=scalar(id=3,umax=7,var_off=(0x0; 0x7)) R10=fp0 fp-8=0000mmmm
22: (07) r2 += 1 ; R2_w=scalar(umin=1,umax=8,var_off=(0x0; 0xf)) 23: (bf) r1 = r10 ; R1_w=fp0 R10=fp0 24: (07) r1 += -8 ; R1_w=fp-8 25: (b7) r3 = 0 ; R3_w=0
26: (85) call bpf_probe_read_kernel#113
mark_precise: frame0: last_idx 26 first_idx 20 subseq_idx -1
mark_precise: frame0: regs=r2 stack= before 25: (b7) r3 = 0
mark_precise: frame0: regs=r2 stack= before 24: (07) r1 += -8
mark_precise: frame0: regs=r2 stack= before 23: (bf) r1 = r10
mark_precise: frame0: regs=r2 stack= before 22: (07) r2 += 1
mark_precise: frame0: regs=r2 stack= before 20: (a5) if r2 < 0x8 goto pc+1
mark_precise: frame0: parent state regs=r2,r9 stack=: R0_rw=map_value(off=0,ks=4,vs=48,imm=0) R2_rw=Pscalar(id=3) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R8_w=map_value(off=0,ks=4,vs=48,imm=0) R9_w=Pscalar(id=3) R10=fp0 fp-8_r=0000mmmm
mark_precise: frame0: last_idx 19 first_idx 10 subseq_idx 20
mark_precise: frame0: regs=r2,r9 stack= before 19: (bf) r2 = r9
mark_precise: frame0: regs=r9 stack= before 18: (1f) r9 -= r8
mark_precise: frame0: regs=r8,r9 stack= before 17: (bf) r8 = r0
mark_precise: frame0: regs=r0,r9 stack= before 15: (55) if r0 != 0x0 goto pc+1
mark_precise: frame0: regs=r0,r9 stack= before 14: (85) call bpf_map_lookup_elem#1
mark_precise: frame0: regs=r9 stack= before 13: (07) r2 += -8
mark_precise: frame0: regs=r9 stack= before 12: (bf) r2 = r10
mark_precise: frame0: regs=r9 stack= before 11: (bf) r1 = r6
mark_precise: frame0: regs=r9 stack= before 10: (bf) r9 = r0
mark_precise: frame0: parent state regs= stack=: R0_rw=map_value(off=0,ks=4,vs=48,imm=0) R6_rw=map_ptr(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8_rw=0000mmmm
27: R0_w=scalar()
27: (95) exit
processed 27 insns (limit 1000000) max_states_per_insn 0 total_states 2 peak_states 2 mark_read 1
[...]
Re: [PATCH stable 6.6.y v3 1/4] bpf: Track equal scalars history on per-instruction level
In reply to Zhenzhong WuOn Mon, Jun 15, 2026 at 12:58:38AM +0800, Zhenzhong Wu wrote:
[...]
Quoted 19 lines
kernel/bpf/verifier.c +15
-2+/* For all R being scalar registers or spilled scalar registers +* in verifier state, save R in linked_regs if R->id == id. +* If there are too many Rs sharing same id, reset id for leftover Rs. +*/ +static void collect_linked_regs (struct bpf_verifier_state *vstate,u32 id, +struct linked_regs *linked_regs) +{ +struct bpf_func_state *func;struct bpf_reg_state *reg; +int i, j; - bpf_for_each_reg_in_vstate(vstate, state, reg, ({ - if (reg->type == SCALAR_VALUE && reg->id == known_reg->id) { +for (i = vstate->curframe; i >=0 ; i--) { + func = vstate->frame[i]; +for (j =0 ; j <BPF_REG_FP ; j++) { + reg = &func->regs[j]; +__collect_linked_regs (linked_regs, reg, id, i, j, true); + }
kernel/bpf/verifier.c static bool try_match_pkt_pointers(const struct bpf_insn *insn, · line 14746
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+for (j =0 ; j < func->allocated_stack /BPF_REG_SIZE ; j++) { +if (!is_spilled_reg (&func->stack[j])) +continue ; + reg = &func->stack[j].spilled_ptr; +__collect_linked_regs (linked_regs, reg, id, i, j, false); + } + } + +if (linked_regs->cnt ==1 ) + linked_regs->cnt =0 ;
This part seems new, not found on the original commit, and also not in bpf-next. Can you add some more explaining (in the notes before your signed-off-by) regarding why this is needed?
Quoted 1 line
+}
[...]
Quoted 4 lines
kernel/bpf/verifier.c +0
-0@@ -14704,6 +14899,21 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,return 0 ; }
kernel/bpf/verifier.c static int check_cond_jmp_op(struct bpf_verifier_env *env, · line 14911
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+/* Push scalar registers sharing same ID to jump history, +* do this before creating 'other_branch', so that both +* 'this_branch' and 'other_branch' share this history +* if parent state is created. +*/ +if (BPF_SRC (insn->code) ==BPF_X && src_reg->type ==SCALAR_VALUE && src_reg->id) +collect_linked_regs (this_branch, src_reg->id, &linked_regs); +if (dst_reg->type ==SCALAR_VALUE && dst_reg->id) +collect_linked_regs (this_branch, dst_reg->id, &linked_regs); +if (linked_regs.cnt >0 ) {
Same here, the original commit and bpf-next has the '> 1' conditional, where as your has '> 0'. Can you also added some explanation on this part?
Quoted 5 lines
kernel/bpf/verifier.c +5
-0+ err =push_jmp_history (env, this_branch,0 ,linked_regs_pack (&linked_regs)); +if (err) +return err; + } +
...
Sorry for the late reply.
For v3, I only ran the targeted test_progs coverage and missed the legacy test_verifier coverage, so I did not catch the expectation mismatch in precise.c. Thanks for the detailed analysis. I have rechecked this and will send v4 shortly with the precise.c fix included.
Quoted 189 lines · Shung-Hsi Yu
Quoted 10 lines · Paul Chaignon
Quoted 4 lines · Zhenzhong Wu
Hi,
This v3 targets 6.6.y and changes the backport strategy based on review feedback on v2.
[...]
Quoted 2 lines
Relevant QEMU selftest results on 6.6.y with this backport:
Quoted 8 lines
Quoted 3 lines
verifier_scalar_idspassed all 18 subtests, including the newly backported linked-scalar precision tests and the relatedcheck_ids_in_regsafetests.The first patch in this backport series is actually breaking the "precise: test 1" selftest from
test_verifier. You can see the full error at [1]. I haven't yet checked if it's the test or the backport that needs to be adjusted.I had a quick look, and believe it was that test that needs to be adjusted to include r9 into the precise register set.
So unless Sasha have other preference, I suggest Zhenzhong send a v4, with changes to
tools/testing/selftests/bpf/verifier/precise.c(including "r9" the the expected verifier output) merged into "bpf:
Track equal scalars history on per-instruction level".---
The program under test is:
00:
BPF_MOV64_IMM(BPF_REG_0, 1),
01:BPF_LD_MAP_FD(BPF_REG_6, 0),
03:BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
04:BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
05:BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
06:BPF_ST_MEM(BPF_DW, BPF_REG_FP, -8, 0),
07:BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
08:BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
09:BPF_EXIT_INSN(),10:
BPF_MOV64_REG(BPF_REG_9, BPF_REG_0),11:
BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
12:BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
13:BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
14:BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
15:BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
16:BPF_EXIT_INSN(),17:
BPF_MOV64_REG(BPF_REG_8, BPF_REG_0),18:
BPF_ALU64_REG(BPF_SUB, BPF_REG_9, BPF_REG_8), /*map_value_ptr-=map_value_ptr*/ 19:BPF_MOV64_REG(BPF_REG_2, BPF_REG_9),
20:BPF_JMP_IMM(BPF_JLT, BPF_REG_2, 8, 1),
21:BPF_EXIT_INSN(),22:
BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1), /* R2=scalar(umin=1, umax=8) */ 23:BPF_MOV64_REG(BPF_REG_1, BPF_REG_FP),
24:BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8),
25:BPF_MOV64_IMM(BPF_REG_3, 0),
26:BPF_EMIT_CALL(BPF_FUNC_probe_read_kernel),
27:BPF_EXIT_INSN(),The test was expecting the following line in the verifier log that was shown during the backtracking start at instruction 26 (call bpf_probe_read_kernel#113)
mark_precise: frame0: regs=r2 stack= before 20: (a5) if r2 <0x8goto pc+1
mark_precise: frame0: parent state regs=r2 stack=: ...
mark_precise: frame0:last_idx19first_idx10 ...But after applying the patchset, we now got an additional register r9 in the precise set:
mark_precise: frame0: regs=r2 stack= before 20: (a5) if r2 <0x8goto pc+1
mark_precise: frame0: parent state regs=r2,r9 stack=: ....
mark_precise: frame0:last_idx19first_idx10 ...The additional r9 in the precise set seems actually correct, this is because r2 and r9 share the same scalar ID at instruction 20 (before the link got broken in instruction 21), and hence at that point, both register should be marked as precise.
---
In upstream the test already has the expected verifier log to include r9, and hence no failure, but it simply comes from the fact that r2 and r9 maintain a link even after instruction 22 (r2 += 1).
commit
98d7ca374ba4b39e7535613d40e159f09ca14da2
Author: Alexei Starovoitov <[email protected]>
Date: Wed Jun 12 18:38:13 2024 -0700bpf: Track delta between "linked" registers.
...
---a/tools/testing/selftests/bpf/verifier/precise.c+++b/tools/testing/selftests/bpf/verifier/precise.c@@ -39,12 +39,12 @@
.result =VERBOSE_ACCEPT,
.errstr =
"mark_precise: frame0:last_idx26first_idx20\- mark_precise: frame0: regs=r2 stack= before 25\ - mark_precise: frame0: regs=r2 stack= before 24\ - mark_precise: frame0: regs=r2 stack= before 23\ - mark_precise: frame0: regs=r2 stack= before 22\ - mark_precise: frame0: regs=r2 stack= before 20\ - mark_precise: frame0: parent state regs=r2 stack=:\ + mark_precise: frame0: regs=r2,r9 stack= before 25\ + mark_precise: frame0: regs=r2,r9 stack= before 24\ + mark_precise: frame0: regs=r2,r9 stack= before 23\ + mark_precise: frame0: regs=r2,r9 stack= before 22\ + mark_precise: frame0: regs=r2,r9 stack= before 20\ + mark_precise: frame0: parent state regs=r2,r9 stack=:\ mark_precise: frame0: last_idx 19 first_idx 10\ mark_precise: frame0: regs=r2,r9 stack= before 19\ mark_precise: frame0: regs=r9 stack= before 18\...
---
Full test log below
#492/p precise: test 1 FAIL
Unexpected verifier log!
EXP:mark_precise: frame0: parent state regs=r2 stack=:
RES:
func#0 @0
0: R1=ctx(off=0,imm=0) R10=fp00: (b7) r0 = 1 ; R0_w=1 1: (18) r6 = 0xffff9eb644619000 ; R6_w=map_ptr(off=0,ks=4,vs=48,imm=0) 3: (bf) r1 = r6 ; R1_w=map_ptr(off=0,ks=4,vs=48,imm=0) R6_w=map_ptr(off=0,ks=4,vs=48,imm=0) 4: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 5: (07) r2 += -8 ; R2_w=fp-8 6: (7a) *(u64 *)(r10 -8) = 0 ; R10=fp0 fp-8_w=00000000 7: (85) call bpf_map_lookup_elem#1 ; R0_w=map_value_or_null(id=1,off=0,ks=4,vs=48,imm=0) 8: (55) if r0 != 0x0 goto pc+1 ; R0_w=09: (95) exit
from 8 to 10: R0=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm 10: R0=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm
10: (bf) r9 = r0 ; R0=map_value(off=0,ks=4,vs=48,imm=0) R9_w=map_value(off=0,ks=4,vs=48,imm=0) 11: (bf) r1 = r6 ; R1_w=map_ptr(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) 12: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 13: (07) r2 += -8 ; R2_w=fp-8 14: (85) call bpf_map_lookup_elem#1 ; R0_w=map_value_or_null(id=2,off=0,ks=4,vs=48,imm=0) 15: (55) if r0 != 0x0 goto pc+1 ; R0_w=016: (95) exit
from 15 to 17: R0_w=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R9_w=map_value(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm 17: R0_w=map_value(off=0,ks=4,vs=48,imm=0) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R9_w=map_value(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8=0000mmmm
17: (bf) r8 = r0 ; R0_w=map_value(off=0,ks=4,vs=48,imm=0) R8_w=map_value(off=0,ks=4,vs=48,imm=0) 18: (1f) r9 -= r8 ; R8_w=map_value(off=0,ks=4,vs=48,imm=0) R9_w=scalar() 19: (bf) r2 = r9 ; R2=scalar(id=3) R9=scalar(id=3) 20: (a5) if r2 < 0x8 goto pc+1 ; R2=scalar(id=3,umin=8)21: (95) exit
from 20 to 22: R0=map_value(off=0,ks=4,vs=48,imm=0) R2=scalar(id=3,umax=7,var_off=(0x0;
0x7)) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R8=map_value(off=0,ks=4,vs=48,imm=0) R9=scalar(id=3,umax=7,var_off=(0x0;0x7)) R10=fp0 fp-8=0000mmmm 22: R0=map_value(off=0,ks=4,vs=48,imm=0) R2=scalar(id=3,umax=7,var_off=(0x0;0x7)) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R8=map_value(off=0,ks=4,vs=48,imm=0) R9=scalar(id=3,umax=7,var_off=(0x0;0x7)) R10=fp0 fp-8=0000mmmm22: (07) r2 += 1 ; R2_w=scalar(umin=1,umax=8,var_off=(0x0; 0xf)) 23: (bf) r1 = r10 ; R1_w=fp0 R10=fp0 24: (07) r1 += -8 ; R1_w=fp-8 25: (b7) r3 = 0 ; R3_w=026: (85) call bpf_probe_read_kernel#113
mark_precise: frame0:last_idx26first_idx20subseq_idx-1
mark_precise: frame0: regs=r2 stack= before 25: (b7) r3 = 0
mark_precise: frame0: regs=r2 stack= before 24: (07) r1 += -8
mark_precise: frame0: regs=r2 stack= before 23: (bf) r1 = r10
mark_precise: frame0: regs=r2 stack= before 22: (07) r2 += 1
mark_precise: frame0: regs=r2 stack= before 20: (a5) if r2 <0x8goto pc+1
mark_precise: frame0: parent state regs=r2,r9 stack=: R0_rw=map_value(off=0,ks=4,vs=48,imm=0) R2_rw=Pscalar(id=3) R6=map_ptr(off=0,ks=4,vs=48,imm=0) R8_w=map_value(off=0,ks=4,vs=48,imm=0) R9_w=Pscalar(id=3) R10=fp0 fp-8_r=0000mmmm
mark_precise: frame0:last_idx19first_idx10subseq_idx20
mark_precise: frame0: regs=r2,r9 stack= before 19: (bf) r2 = r9
mark_precise: frame0: regs=r9 stack= before 18: (1f) r9 -= r8
mark_precise: frame0: regs=r8,r9 stack= before 17: (bf) r8 = r0
mark_precise: frame0: regs=r0,r9 stack= before 15: (55) if r0 !=0x0goto pc+1
mark_precise: frame0: regs=r0,r9 stack= before 14: (85) call bpf_map_lookup_elem#1
mark_precise: frame0: regs=r9 stack= before 13: (07) r2 += -8
mark_precise: frame0: regs=r9 stack= before 12: (bf) r2 = r10
mark_precise: frame0: regs=r9 stack= before 11: (bf) r1 = r6
mark_precise: frame0: regs=r9 stack= before 10: (bf) r9 = r0
mark_precise: frame0: parent state regs= stack=: R0_rw=map_value(off=0,ks=4,vs=48,imm=0) R6_rw=map_ptr(off=0,ks=4,vs=48,imm=0) R10=fp0 fp-8_rw=0000mmmm
27: R0_w=scalar()
27: (95) exit
processed 27 insns (limit 1000000)max_states_per_insn0total_states2peak_states2mark_read1[...]
Re: [PATCH stable 6.6.y v3 1/4] bpf: Track equal scalars history on per-instruction level
In reply to Shung-Hsi YuOn second thought, I'll drop the cnt == 1 reset (and the matching cnt > 0) and follow upstream's cnt > 1 in v4. It gives no benefit over upstream, it only adds the corner cases the bot flagged. And The other two points the bot raised (clearing id on overflow, collecting src/dst twice) are upstream behaviour, so I will keep them as-is.Thanks for the pointer.
Quoted 64 lines · Shung-Hsi Yu
On Mon, Jun 15, 2026 at 12:58:38AM +0800, Zhenzhong Wu wrote:
[...]Quoted 19 lines
kernel/bpf/verifier.c +15
-2+/* For all R being scalar registers or spilled scalar registers +* in verifier state, save R in linked_regs if R->id == id. +* If there are too many Rs sharing same id, reset id for leftover Rs. +*/ +static void collect_linked_regs (struct bpf_verifier_state *vstate,u32 id, +struct linked_regs *linked_regs) +{ +struct bpf_func_state *func;struct bpf_reg_state *reg; +int i, j; - bpf_for_each_reg_in_vstate(vstate, state, reg, ({ - if (reg->type == SCALAR_VALUE && reg->id == known_reg->id) { +for (i = vstate->curframe; i >=0 ; i--) { + func = vstate->frame[i]; +for (j =0 ; j <BPF_REG_FP ; j++) { + reg = &func->regs[j]; +__collect_linked_regs (linked_regs, reg, id, i, j, true); + }kernel/bpf/verifier.c static bool try_match_pkt_pointers(const struct bpf_insn *insn, · line 14746
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+for (j =0 ; j < func->allocated_stack /BPF_REG_SIZE ; j++) { +if (!is_spilled_reg (&func->stack[j])) +continue ; + reg = &func->stack[j].spilled_ptr; +__collect_linked_regs (linked_regs, reg, id, i, j, false); + } + } + +if (linked_regs->cnt ==1 ) + linked_regs->cnt =0 ;This part seems new, not found on the original commit, and also not in bpf-next. Can you add some more explaining (in the notes before your signed-off-by) regarding why this is needed?
Quoted 1 line
+}
[...]
Quoted 4 lines
kernel/bpf/verifier.c +0
-0@@ -14704,6 +14899,21 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,return 0 ; }kernel/bpf/verifier.c static int check_cond_jmp_op(struct bpf_verifier_env *env, · line 14911
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+/* Push scalar registers sharing same ID to jump history, +* do this before creating 'other_branch', so that both +* 'this_branch' and 'other_branch' share this history +* if parent state is created. +*/ +if (BPF_SRC (insn->code) ==BPF_X && src_reg->type ==SCALAR_VALUE && src_reg->id) +collect_linked_regs (this_branch, src_reg->id, &linked_regs); +if (dst_reg->type ==SCALAR_VALUE && dst_reg->id) +collect_linked_regs (this_branch, dst_reg->id, &linked_regs); +if (linked_regs.cnt >0 ) {Same here, the original commit and bpf-next has the '> 1' conditional, where as your has '> 0'. Can you also added some explanation on this part?
Quoted 5 lines
kernel/bpf/verifier.c +5
-0+ err =push_jmp_history (env, this_branch,0 ,linked_regs_pack (&linked_regs)); +if (err) +return err; + } +...