Skip to contentSkip to conversation

bpf: Check load-acquire src ptr type before the load

PATCH Applied Daniel Borkmann 3 participants 5 messages Aug 4 – Aug 5

Subsystems (4)
lore
Share & tools
lore

Daniel Borkmann

author maintainer
#0 lore raw

PATCH 1/2 bpf: Check load-acquire src ptr type before the load

check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok(). For a load-acquire that fetches into its own source register (dst_reg == src_reg), check_load_mem() overwrites src_reg's type with the type of the loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the source pointer and fails to reject the disallowed types (ctx, pkt, flow_keys, sock).

Since bpf_convert_ctx_accesses() does not rewrite atomic loads, the raw access to the underlying kernel object is left in place. The destination type is taken from the ctx access itself, so a load-acquire of the sk field of struct __sk_buff for example leaves the register typed as PTR_TO_SOCK_COMMON_OR_NULL, which type_is_sk_pointer() does not match either, while it actually holds unconverted struct sk_buff bytes. Once the NULL check has passed this is a type confusion, not just a leak of kernel data.

Validate src_reg with check_reg_arg() and check the source pointer type with atomic_ptr_type_ok() before the load again, mirroring check_atomic_rmw(). Out-of-range register numbers are already rejected earlier by check_and_resolve_insns() (commit 503d21ef8eac ("bpf: Do register range validation early")), and the only exemption there, is_stack_arg_ldx(), requires BPF_LDX | BPF_MEM | BPF_DW and thus never matches a BPF_ATOMIC insn. atomic_ptr_type_ok() can therefore not dereference register state out of bounds, that is, the out-of-bounds read addressed by the Fixes commit below does not reappear (as proven also via selftest).

Fixes c03bb2fa327e ("bpf: Fix out-of-bounds read in check_atomic_load/store()")
---
 kernel/bpf/verifier.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +2 -2
index b274004fccfd..9513e18836c2 100644
@@ -6616,7 +6616,7 @@ static int check_atomic_load(struct bpf_verifier_env *env,
 {
 	int err;
 
-	err = check_load_mem(env, insn, true, false, false, "atomic_load");
+	err = check_reg_arg(env, insn->src_reg, SRC_OP);
 	if (err)
 		return err;
 
@@ -6627,7 +6627,7 @@ static int check_atomic_load(struct bpf_verifier_env *env,
 		return -EACCES;
 	}
 
-	return 0;
+	return check_load_mem(env, insn, true, false, false, "atomic_load");
 }
 
 static int check_atomic_store(struct bpf_verifier_env *env,
Signature · 1 line
2.43.0
author maintainer
#1 lore raw

PATCH 2/2 selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx

Add a verifier test that a load-acquire fetching into its own source register (dst_reg == src_reg) from a ctx pointer is rejected.

# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_load_acquire [...]

  #614/1   verifier_load_acquire/load-acquire, 8-bit:OK
  #614/2   verifier_load_acquire/load-acquire, 8-bit @unpriv:OK
  #614/3   verifier_load_acquire/load-acquire, 16-bit:OK
  #614/4   verifier_load_acquire/load-acquire, 16-bit @unpriv:OK
  #614/5   verifier_load_acquire/load-acquire, 32-bit:OK
  #614/6   verifier_load_acquire/load-acquire, 32-bit @unpriv:OK
  #614/7   verifier_load_acquire/load-acquire, 64-bit:OK
  #614/8   verifier_load_acquire/load-acquire, 64-bit @unpriv:OK
  #614/9   verifier_load_acquire/load-acquire with uninitialized src_reg:OK
  #614/10  verifier_load_acquire/load-acquire with uninitialized src_reg @unpriv:OK
  #614/11  verifier_load_acquire/load-acquire with non-pointer src_reg:OK
  #614/12  verifier_load_acquire/load-acquire with non-pointer src_reg @unpriv:OK
  #614/13  verifier_load_acquire/misaligned load-acquire:OK
  #614/14  verifier_load_acquire/misaligned load-acquire @unpriv:OK
  #614/15  verifier_load_acquire/load-acquire from ctx pointer:OK
  #614/16  verifier_load_acquire/load-acquire from ctx pointer @unpriv:OK
  #614/17  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register:OK
  #614/18  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register @unpriv:OK
  #614/19  verifier_load_acquire/load-acquire with invalid register R15:OK
  #614/20  verifier_load_acquire/load-acquire with invalid register R15 @unpriv:OK
  #614/21  verifier_load_acquire/load-acquire from pkt pointer:OK
  #614/22  verifier_load_acquire/load-acquire from flow_keys pointer:OK
  #614/23  verifier_load_acquire/load-acquire from sock pointer:OK
  #614     verifier_load_acquire:OK

Summary: 1/23 PASSED, 0 SKIPPED, 0 FAILED

---
 .../selftests/bpf/progs/verifier_load_acquire.c  | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
tools/testing/selftests/bpf/progs/verifier_load_acquire.c +16 -0
index 74f4f19c10b8..ae1dab1b0cbb 100644
@@ -148,6 +148,22 @@ __naked void load_acquire_from_ctx_pointer(void)
 	: __clobber_all);
 }
 
+SEC("socket")
+__description("load-acquire from ctx pointer, same dst and src register")
+__failure __failure_unpriv __msg("BPF_ATOMIC loads from R6 ctx is not allowed")
+__naked void load_acquire_ctx_same_dst_src(void)
+{
+	asm volatile (
+	"r6 = r1;"
+	".8byte %[load_acquire_insn];"	// w6 = load_acquire((u32 *)(r6 + 0));
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm_insn(load_acquire_insn,
+		     BPF_ATOMIC_OP(BPF_W, BPF_LOAD_ACQ, BPF_REG_6, BPF_REG_6, 0))
+	: __clobber_all);
+}
+
 SEC("xdp")
 __description("load-acquire from pkt pointer")
 __failure __msg("BPF_ATOMIC loads from R2 pkt is not allowed")
Signature · 1 line
2.43.0
maintainer Acked-by on v1 2/2
#2 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx

Quoted 31 lines · Daniel Borkmann

Add a verifier test that a load-acquire fetching into its own source register (dst_reg == src_reg) from a ctx pointer is rejected.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_load_acquire [...]
  #614/1   verifier_load_acquire/load-acquire, 8-bit:OK
  #614/2   verifier_load_acquire/load-acquire, 8-bit @unpriv:OK
  #614/3   verifier_load_acquire/load-acquire, 16-bit:OK
  #614/4   verifier_load_acquire/load-acquire, 16-bit @unpriv:OK
  #614/5   verifier_load_acquire/load-acquire, 32-bit:OK
  #614/6   verifier_load_acquire/load-acquire, 32-bit @unpriv:OK
  #614/7   verifier_load_acquire/load-acquire, 64-bit:OK
  #614/8   verifier_load_acquire/load-acquire, 64-bit @unpriv:OK
  #614/9   verifier_load_acquire/load-acquire with uninitialized src_reg:OK
  #614/10  verifier_load_acquire/load-acquire with uninitialized src_reg @unpriv:OK #614/11  verifier_load_acquire/load-acquire with non-pointer src_reg:OK
  #614/12  verifier_load_acquire/load-acquire with non-pointer src_reg @unpriv:OK #614/13  verifier_load_acquire/misaligned load-acquire:OK
  #614/14  verifier_load_acquire/misaligned load-acquire @unpriv:OK
  #614/15  verifier_load_acquire/load-acquire from ctx pointer:OK
  #614/16  verifier_load_acquire/load-acquire from ctx pointer @unpriv:OK
  #614/17  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register:OK #614/18  verifier_load_acquire/load-acquire from ctx pointer, same dst and src register @unpriv:OK #614/19  verifier_load_acquire/load-acquire with invalid register R15:OK
  #614/20  verifier_load_acquire/load-acquire with invalid register R15 @unpriv:OK #614/21  verifier_load_acquire/load-acquire from pkt pointer:OK
  #614/22  verifier_load_acquire/load-acquire from flow_keys pointer:OK
  #614/23  verifier_load_acquire/load-acquire from sock pointer:OK
  #614     verifier_load_acquire:OK
  Summary: 1/23 PASSED, 0 SKIPPED, 0 FAILED

Quoted 2 lines

---

...

maintainer Acked-by on v1 1/2
#3 lore raw
In reply to Daniel Borkmann
Quoted 27 lines · Daniel Borkmann

check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok(). For a load-acquire that fetches into its own source register (dst_reg == src_reg), check_load_mem() overwrites src_reg's type with the type of the loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the source pointer and fails to reject the disallowed types (ctx, pkt, flow_keys, sock).

Since bpf_convert_ctx_accesses() does not rewrite atomic loads, the raw access to the underlying kernel object is left in place. The destination type is taken from the ctx access itself, so a load-acquire of the sk field of struct __sk_buff for example leaves the register typed as PTR_TO_SOCK_COMMON_OR_NULL, which type_is_sk_pointer() does not match either, while it actually holds unconverted struct sk_buff bytes. Once the NULL check has passed this is a type confusion, not just a leak of kernel data.

Validate src_reg with check_reg_arg() and check the source pointer type with atomic_ptr_type_ok() before the load again, mirroring check_atomic_rmw(). Out-of-range register numbers are already rejected earlier by check_and_resolve_insns() (commit 503d21ef8eac ("bpf: Do register range validation early")), and the only exemption there, is_stack_arg_ldx(), requires BPF_LDX | BPF_MEM | BPF_DW and thus never matches a BPF_ATOMIC insn. atomic_ptr_type_ok() can therefore not dereference register state out of bounds, that is, the out-of-bounds read addressed by the Fixes commit below does not reappear (as proven also via selftest).

Quoted 4 lines
Fixes c03bb2fa327e ("bpf: Fix out-of-bounds read in check_atomic_load/store()")

---

...

#4 lore raw
In reply to Daniel Borkmann
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Quoted 8 lines · On Tue, 4 Aug 2026 22:19:16 +0200 you wrote:

check_atomic_load() calls check_load_mem() before atomic_ptr_type_ok(). For a load-acquire that fetches into its own source register (dst_reg == src_reg), check_load_mem() overwrites src_reg's type with the type of the loaded value, so the subsequent atomic_ptr_type_ok() no longer sees the source pointer and fails to reject the disallowed types (ctx, pkt, flow_keys, sock).

[...]

Here is the summary with links:
- [bpf-next,1/2] bpf: Check load-acquire src ptr type before the load

    https://git.kernel.org/bpf/bpf-next/c/b87803391baa
  - [bpf-next,2/2] selftests/bpf: Add load-acquire test for dst_reg == src_reg from ctx
    https://git.kernel.org/bpf/bpf-next/c/363b15d8551e

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html