bpf: Check load-acquire src ptr type before the load
Subsystems (4)
Daniel Borkmann
maintainerPATCH 1/2 bpf: Check load-acquire src ptr type before the load
check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok(). For a load-acquire that fetches into its own source register (dst_reg == src_reg), check_load_mem() overwrites src_reg's type with the type of the loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the source pointer and fails to reject the disallowed types (ctx, pkt, flow_keys, sock).
Since bpf_convert_ctx_accesses() does not rewrite atomic loads, the raw access to the underlying kernel object is left in place. The destination type is taken from the ctx access itself, so a load-acquire of the sk field of struct __sk_buff for example leaves the register typed as PTR_TO_SOCK_COMMON_OR_NULL, which type_is_sk_pointer() does not match either, while it actually holds unconverted struct sk_buff bytes. Once the NULL check has passed this is a type confusion, not just a leak of kernel data.
Validate src_reg with check_reg_arg() and check the source pointer type with atomic_ptr_type_ok() before the load again, mirroring check_atomic_rmw(). Out-of-range register numbers are already rejected earlier by check_and_resolve_insns() (commit 503d21ef8eac ("bpf: Do register range validation early")), and the only exemption there, is_stack_arg_ldx(), requires BPF_LDX | BPF_MEM | BPF_DW and thus never matches a BPF_ATOMIC insn. atomic_ptr_type_ok() can therefore not dereference register state out of bounds, that is, the out-of-bounds read addressed by the Fixes commit below does not reappear (as proven also via selftest).
--- kernel/bpf/verifier.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +2 -2
@@ -6616,7 +6616,7 @@ static int check_atomic_load(struct bpf_verifier_env *env, {int err; - err =check_load_mem (env, insn, true, false, false,"atomic_load" ); + err =check_reg_arg (env, insn->src_reg,SRC_OP );if (err)return err;
@@ -6627,7 +6627,7 @@ static int check_atomic_load(struct bpf_verifier_env *env,return -EACCES ; } -return 0 ; +return check_load_mem (env, insn, true, false, false,"atomic_load" ); } static int check_atomic_store(struct bpf_verifier_env *env,
Signature · 1 line
2.43.0
Daniel Borkmann
maintainerPATCH 2/2 selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx
Add a verifier test that a load-acquire fetching into its own source register (dst_reg == src_reg) from a ctx pointer is rejected.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_load_acquire [...]
#614/1 verifier_load_acquire/load-acquire, 8-bit:OK #614/2 verifier_load_acquire/load-acquire, 8-bit @unpriv:OK #614/3 verifier_load_acquire/load-acquire, 16-bit:OK #614/4 verifier_load_acquire/load-acquire, 16-bit @unpriv:OK #614/5 verifier_load_acquire/load-acquire, 32-bit:OK #614/6 verifier_load_acquire/load-acquire, 32-bit @unpriv:OK #614/7 verifier_load_acquire/load-acquire, 64-bit:OK #614/8 verifier_load_acquire/load-acquire, 64-bit @unpriv:OK #614/9 verifier_load_acquire/load-acquire with uninitialized src_reg:OK #614/10 verifier_load_acquire/load-acquire with uninitialized src_reg @unpriv:OK #614/11 verifier_load_acquire/load-acquire with non-pointer src_reg:OK #614/12 verifier_load_acquire/load-acquire with non-pointer src_reg @unpriv:OK #614/13 verifier_load_acquire/misaligned load-acquire:OK #614/14 verifier_load_acquire/misaligned load-acquire @unpriv:OK #614/15 verifier_load_acquire/load-acquire from ctx pointer:OK #614/16 verifier_load_acquire/load-acquire from ctx pointer @unpriv:OK #614/17 verifier_load_acquire/load-acquire from ctx pointer, same dst and src register:OK #614/18 verifier_load_acquire/load-acquire from ctx pointer, same dst and src register @unpriv:OK #614/19 verifier_load_acquire/load-acquire with invalid register R15:OK #614/20 verifier_load_acquire/load-acquire with invalid register R15 @unpriv:OK #614/21 verifier_load_acquire/load-acquire from pkt pointer:OK #614/22 verifier_load_acquire/load-acquire from flow_keys pointer:OK #614/23 verifier_load_acquire/load-acquire from sock pointer:OK #614 verifier_load_acquire:OK
Summary: 1/23 PASSED, 0 SKIPPED, 0 FAILED
--- .../selftests/bpf/progs/verifier_load_acquire.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
tools/testing/selftests/bpf/progs/verifier_load_acquire.c +16 -0
@@ -148,6 +148,22 @@ __naked void load_acquire_from_ctx_pointer(void) : __clobber_all); } +SEC("socket") +__description("load-acquire from ctx pointer, same dst and src register") +__failure __failure_unpriv __msg("BPF_ATOMIC loads from R6 ctx is not allowed") +__naked void load_acquire_ctx_same_dst_src (void ) +{ + asmvolatile ( +"r6 = r1;" +".8byte %[load_acquire_insn];" // w6 = load_acquire((u32 *)(r6 + 0)); +"r0 = 0;" +"exit;" + : + : __imm_insn(load_acquire_insn, + BPF_ATOMIC_OP(BPF_W, BPF_LOAD_ACQ, BPF_REG_6, BPF_REG_6, 0)) + : __clobber_all); +} + SEC("xdp") __description("load-acquire from pkt pointer") __failure __msg("BPF_ATOMIC loads from R2 pkt is not allowed")
Signature · 1 line
2.43.0
Re: [PATCH bpf-next 2/2] selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx
Quoted 31 lines · Daniel Borkmann
Add a verifier test that a load-acquire fetching into its own source register (
dst_reg==src_reg) from a ctx pointer is rejected.# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t
verifier_load_acquire[...]
#614/1 verifier_load_acquire/load-acquire, 8-bit:OK
#614/2 verifier_load_acquire/load-acquire, 8-bit @unpriv:OK
#614/3 verifier_load_acquire/load-acquire, 16-bit:OK
#614/4 verifier_load_acquire/load-acquire, 16-bit @unpriv:OK
#614/5 verifier_load_acquire/load-acquire, 32-bit:OK
#614/6 verifier_load_acquire/load-acquire, 32-bit @unpriv:OK
#614/7 verifier_load_acquire/load-acquire, 64-bit:OK
#614/8 verifier_load_acquire/load-acquire, 64-bit @unpriv:OK
#614/9 verifier_load_acquire/load-acquire with uninitialized src_reg:OK
#614/10 verifier_load_acquire/load-acquire with uninitializedsrc_reg@unpriv:OK #614/11 verifier_load_acquire/load-acquire with non-pointer src_reg:OK
#614/12 verifier_load_acquire/load-acquire with non-pointersrc_reg@unpriv:OK #614/13 verifier_load_acquire/misaligned load-acquire:OK
#614/14 verifier_load_acquire/misaligned load-acquire @unpriv:OK
#614/15 verifier_load_acquire/load-acquire from ctx pointer:OK
#614/16 verifier_load_acquire/load-acquire from ctx pointer @unpriv:OK
#614/17 verifier_load_acquire/load-acquire from ctx pointer, same dst and src register:OK #614/18 verifier_load_acquire/load-acquire from ctx pointer, same dst and src register @unpriv:OK #614/19 verifier_load_acquire/load-acquire with invalid register R15:OK
#614/20 verifier_load_acquire/load-acquire with invalid register R15 @unpriv:OK #614/21 verifier_load_acquire/load-acquire from pkt pointer:OK
#614/22 verifier_load_acquire/load-acquire fromflow_keyspointer:OK
#614/23 verifier_load_acquire/load-acquire from sock pointer:OK
#614 verifier_load_acquire:OK
Summary: 1/23 PASSED, 0 SKIPPED, 0 FAILED
Quoted 2 lines
Signed-off-by Daniel Borkmann <[email protected]>---
...
Quoted 27 lines · Daniel Borkmann
check_atomic_load()callscheck_load_mem()beforeatomic_ptr_type_ok(). For a load-acquire that fetches into its own source register (dst_reg==src_reg),check_load_mem()overwrites src_reg's type with the type of the loaded value, so the subsequentatomic_ptr_type_ok()no longer sees the source pointer and fails to reject the disallowed types (ctx, pkt,flow_keys, sock).Since
bpf_convert_ctx_accesses()does not rewrite atomic loads, the raw access to the underlying kernel object is left in place. The destination type is taken from the ctx access itself, so a load-acquire of the sk field of struct__sk_bufffor example leaves the register typed asPTR_TO_SOCK_COMMON_OR_NULL, whichtype_is_sk_pointer()does not match either, while it actually holds unconverted structsk_buffbytes. Once the NULL check has passed this is a type confusion, not just a leak of kernel data.Validate
src_regwithcheck_reg_arg()and check the source pointer type withatomic_ptr_type_ok()before the load again, mirroringcheck_atomic_rmw(). Out-of-range register numbers are already rejected earlier bycheck_and_resolve_insns()(commit503d21ef8eac("bpf: Do register range validation early")), and the only exemption there,is_stack_arg_ldx(), requiresBPF_LDX|BPF_MEM|BPF_DWand thus never matches aBPF_ATOMICinsn.atomic_ptr_type_ok()can therefore not dereference register state out of bounds, that is, the out-of-bounds read addressed by the Fixes commit below does not reappear (as proven also via selftest).
Quoted 4 lines
Fixes c03bb2fa327e ("bpf: Fix out-of-bounds read in check_atomic_load/store()")Reported-by STAR Labs SG <[email protected]>Signed-off-by Daniel Borkmann <[email protected]>---
...
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Quoted 8 lines · On Tue, 4 Aug 2026 22:19:16 +0200 you wrote:
check_atomic_load()callscheck_load_mem()beforeatomic_ptr_type_ok(). For a load-acquire that fetches into its own source register (dst_reg==src_reg),check_load_mem()overwrites src_reg's type with the type of the loaded value, so the subsequentatomic_ptr_type_ok()no longer sees the source pointer and fails to reject the disallowed types (ctx, pkt,flow_keys, sock).[...]
Here is the summary with links:
- [bpf-next,1/2] bpf: Check load-acquire src ptr type before the load
https://git.kernel.org/bpf/bpf-next/c/b87803391baa - [bpf-next,2/2] selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx https://git.kernel.org/bpf/bpf-next/c/363b15d8551e
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html