bpf: Fix uninit read for non-fetch atomics on partially spilled slots
Subsystems (4)
PATCH 1/2 bpf: Fix uninit read for non-fetch atomics on partially spilled slots
check_stack_read_fixed_off() skips partial spill checks for non-fetch atomics; the following prog can be loaded:
0: (b7) r1 = 1 ; R1=1 1: (63) *(u32 *)(r10 -8) = r1 ; R1=1 R10=fp0 fp-8=????1 2: (db) lock *(u64 *)(r10 -8) += r1 ; R1=1 R10=fp0 fp-8=mmmmmmmm 3: (79) r0 = *(u64 *)(r10 -8) ; R0=scalar() R10=fp0 fp-8=mmmmmmmm 4: (77) r0 >>= 32 ; R0=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))
5: (95) exit
When test run:
retval=4294967295
Note fp-8 is ????1 at #1, yet it becomes fp-8=mmmmmmmm after the non-fetching atomic add at #2; hence the high 32 bits are leaked.
Fix by applying the partial load check; after the patch, the prog is rejected:
Verification failed: Memory Safety: Uninitialized stack read
Reason:
This rejected read uses 8 bytes at stack offset -8, but byte 4 in that range is uninitialized on this path. Programs loaded with CAP_PERFMON can be allowed to read uninitialized stack bytes, but this program is being rejected without that allowance.
At:
...
0 | (b7) r1 = 1
1 | (63) *(u32 *)(r10 -8) = r1
>>> 2 | (db) lock *(u64 *)(r10 -8) += r1
3 | (79) r0 = *(u64 *)(r10 -8)
4 | (77) r0 >>= 32This affects CAP_BPF only.
--- kernel/bpf/verifier.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +6 -3
@@ -4038,11 +4038,11 @@ static int check_stack_read_fixed_off(struct bpf_verifier_env *env,return -EACCES ; } -if (dst_regno <0 ) -return 0 ; -if (size <= spill_size &&bpf_stack_narrow_access_ok (off, size, spill_size)) { +if (dst_regno <0 ) +return 0 ; +if (env->bpf_capable && size ==4 && spill_size ==4 &&get_reg_width (reg) <=32 )/* Ensure stack slot has an ID to build a relation
@@ -4084,6 +4084,9 @@ static int check_stack_read_fixed_off(struct bpf_verifier_env *env,return -EACCES ; } +if (dst_regno <0 ) +return 0 ; +if (spill_cnt == size &&tnum_is_const (reg->var_off) && reg->var_off.value ==0 ) {__mark_reg_const_zero (env, &state->regs[dst_regno]);
Signature · 1 line
2.34.1
PATCH 2/2 selftests/bpf: Test non-fetch atomic on a narrow stack spill
Spill a 32-bit scalar into an 8-byte stack slot, leaving the other half of the slot uninitialized, and apply a non-fetch 64-bit atomic add to the slot. With CAP_BPF but without CAP_PERFMON the verifier rejects it.
--- .../selftests/bpf/progs/verifier_spill_fill.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
tools/testing/selftests/bpf/progs/verifier_spill_fill.c +17 -0
@@ -1344,6 +1344,23 @@ __naked void old_imprecise_scalar32_vs_cur_stack_misc(void) : __clobber_all); } +SEC("socket") +__description("stack_noperfmon: reject non-fetch atomic on narrow spill") +__success +__caps_unpriv(CAP_BPF) +__failure_unpriv __msg_unpriv("invalid read from stack off -8+4 size 8") +__naked void stack_noperfmon_reject_atomic_on_narrow_spill (void ) +{ + asmvolatile ( +"r1 = 1;" +"*(u32 *)(r10 - 8) = r1;" +/* A non-fetch atomic reads all 8 bytes of the slot. */ +"lock *(u64 *)(r10 - 8) += r1;" +"r0 = 0;" +"exit;" + ::: __clobber_all); +} + SEC("raw_tp") __success__naked void var_off_write_over_scalar_spill (void )
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerQuoted 34 lines · Hao Sun
check_stack_read_fixed_off()skips partial spill checks for non-fetch atomics; the following prog can be loaded:0: (b7) r1 = 1 ; R1=1 1: (63) *(u32 *)(r10 -8) = r1 ; R1=1 R10=fp0 fp-8=????1 2: (db) lock *(u64 *)(r10 -8) += r1 ; R1=1 R10=fp0 fp-8=mmmmmmmm 3: (79) r0 = *(u64 *)(r10 -8) ; R0=scalar() R10=fp0 fp-8=mmmmmmmm 4: (77) r0 >>= 32 ; R0=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))5: (95) exit
When test run:
retval=4294967295Note fp-8 is ????1 at #1, yet it becomes fp-8=mmmmmmmm after the non-fetching atomic add at #2; hence the high 32 bits are leaked.
Fix by applying the partial load check; after the patch, the prog is rejected:
Verification failed: Memory Safety: Uninitialized stack read
Reason:
This rejected read uses 8 bytes at stack offset -8, but byte 4 in that range is uninitialized on this path. Programs loaded withCAP_PERFMONcan be allowed to read uninitialized stack bytes, but this program is being rejected without that allowance.At:
...0 | (b7) r1 = 1 1 | (63) *(u32 *)(r10 -8) = r1 >>> 2 | (db) lock *(u64 *)(r10 -8) += r1 3 | (79) r0 = *(u64 *)(r10 -8) 4 | (77) r0 >>= 32
Quoted 6 lines
This affects
CAP_BPFonly.Fixes 354e8f1970f8 ("bpf: Support <8-byte scalar spill and refill")Signed-off-by Hao Sun <[email protected]>---
The same root cause is also reachable through check_stack_range_initialized(). It takes the spilled register path for every byte of a slot holding a spilled scalar, without looking at the byte's own slot_type, so a helper or kfunc memory argument spanning a narrow spill still reads the uninitialized half without CAP_PERFMON:
r1 = 1; *(u32 *)(r10 - 8) = r1; r1 = map_ringbuf ll; r2 = r10; r2 += -8; r3 = 8; r4 = 0; call bpf_ringbuf_output;
loads with CAP_BPF alone and copies four uninitialized bytes of kernel stack to user space. The fix is to only take that path when *stype == STACK_SPILL, so the rest of the slot goes through the usual MISC/ZERO/INVALID checks like any other slot. Could you add that to v2 with a matching test (same Fixes: tag), since it's the same bug on the helper path?
Locally I tried something like this and it addressed the problem.
kernel/bpf/verifier.c +6 -1
@@ -7481,7 +7481,12 @@ static int check_stack_range_initialized( goto mark; } -if (bpf_is_spilled_reg (ss) && +/* +* Only the bytes marked STACK_SPILL hold the spilled register. +* The rest of a narrowly spilled slot keeps its previous type +* and must be initialized on its own. +*/ +if (*stype ==STACK_SPILL && (ss->spilled_ptr.type ==SCALAR_VALUE || env->allow_ptr_leaks)) {if (clobber) {
Please also target bpf-next in next revision.
pw-bot: cr
Kumar Kartikeya Dwivedi
maintainerQuoted 84 lines · Kumar Kartikeya Dwivedi
Quoted 34 lines · Hao Sun
check_stack_read_fixed_off()skips partial spill checks for non-fetch atomics; the following prog can be loaded:0: (b7) r1 = 1 ; R1=1 1: (63) *(u32 *)(r10 -8) = r1 ; R1=1 R10=fp0 fp-8=????1 2: (db) lock *(u64 *)(r10 -8) += r1 ; R1=1 R10=fp0 fp-8=mmmmmmmm 3: (79) r0 = *(u64 *)(r10 -8) ; R0=scalar() R10=fp0 fp-8=mmmmmmmm 4: (77) r0 >>= 32 ; R0=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))5: (95) exit
When test run:
retval=4294967295Note fp-8 is ????1 at #1, yet it becomes fp-8=mmmmmmmm after the non-fetching atomic add at #2; hence the high 32 bits are leaked.
Fix by applying the partial load check; after the patch, the prog is rejected:
Verification failed: Memory Safety: Uninitialized stack read
Reason:
This rejected read uses 8 bytes at stack offset -8, but byte 4 in that range is uninitialized on this path. Programs loaded withCAP_PERFMONcan be allowed to read uninitialized stack bytes, but this program is being rejected without that allowance.At:
...0 | (b7) r1 = 1 1 | (63) *(u32 *)(r10 -8) = r1 >>> 2 | (db) lock *(u64 *)(r10 -8) += r1 3 | (79) r0 = *(u64 *)(r10 -8) 4 | (77) r0 >>= 32Quoted 6 lines
This affects
CAP_BPFonly.Fixes 354e8f1970f8 ("bpf: Support <8-byte scalar spill and refill")Signed-off-by Hao Sun <[email protected]>---
The same root cause is also reachable through
check_stack_range_initialized(). It takes the spilled register path for every byte of a slot holding a spilled scalar, without looking at the byte's ownslot_type, so a helper or kfunc memory argument spanning a narrow spill still reads the uninitialized half withoutCAP_PERFMON:r1 = 1; *(u32 *)(r10 - 8) = r1; r1 = map_ringbuf ll; r2 = r10; r2 += -8; r3 = 8; r4 = 0; call bpf_ringbuf_output;loads with
CAP_BPFalone and copies four uninitialized bytes of kernel stack to user space. The fix is to only take that path when *stype ==STACK_SPILL, so the rest of the slot goes through the usual MISC/ZERO/INVALID checks like any other slot. Could you add that to v2 with a matching test (same Fixes: tag), since it's the same bug on the helper path?Locally I tried something like this and it addressed the problem.
kernel/bpf/verifier.c +6
-1@@ -7481,7 +7481,12 @@ static int check_stack_range_initialized( goto mark; } -if (bpf_is_spilled_reg (ss) && +/* +* Only the bytes marked STACK_SPILL hold the spilled register. +* The rest of a narrowly spilled slot keeps its previous type +* and must be initialized on its own. +*/ +if (*stype ==STACK_SPILL && (ss->spilled_ptr.type ==SCALAR_VALUE || env->allow_ptr_leaks)) {if (clobber) {
Quoted 2 lines
Please also target bpf-next in next revision.
Actually, I just applied these two, so you can just follow up with extra fix + test.
Quoted 1 line
pw-bot: cr