Skip to contentSkip to conversation

bpf, x86: fix per-CPU address resolution into an extended register

PATCH Superseded Vineet Gupta 3 participants 9 messages Aug 14

Subsystems (6)
Stable branch x86 Kselftest BPF +2 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Vineet Gupta

author
#0 lore raw

v2 cover bpf, x86: fix per-CPU address resolution into an extended register

The JIT resolves a per-CPU address with

add <dst>, gs:[this_cpu_off]

but builds the REX prefix with add_1mod(), which sets REX.B. The destination is encoded in ModRM.reg, which REX.R extends, and the memory operand is disp32 with no base, so REX.B does nothing and the high register bit is dropped. Every extended destination therefore resolves into whichever register shares the low three bits:

R5 -> RAX R7 -> RBP R8 -> RSI R9 -> RDI

The address is left unadjusted and an unrelated register is clobbered. Patch 1 switches to add_2mod() so the bit goes through REX.R.

Clang reloads the address into R1 before each per-CPU access, so the destination is never an extended register and the bug has been dormant since v6.10. GCC keeps several per-CPU addresses live at once, which is how it turned up: test_progs-bpf_gcc panics the kernel in global_percpu_data/init, with the address of a .percpu variable in R5.

Patch 2 adds a test per register. A functional test only catches this if the address happens to land in an extended register, so the test matches the JITed add instead, one __naked program per register.

Changes in v2:
- Add the selftest, patch 2/2 (Eduard Zingerman). It uses __jited() rather than __xlated(): the xlated stream is identical for every register, and the wrong prefix is only visible in the native encoding.
- No functional change to patch 1.

v1: https://lore.kernel.org/bpf/[email protected]/T/#u

Vineet Gupta (2):
bpf, x86: Fix per-CPU address resolution into an extended register selftests/bpf: Check per-CPU address resolution per register

 arch/x86/net/bpf_jit_comp.c                   |   2 +-
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_percpu_addr.c          | 114 ++++++++++++++++++
 3 files changed, 117 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_percpu_addr.c
Signature · 1 line
2.53.0-Meta
author
#1 lore raw

PATCH v2 1/2 bpf, x86: Fix per-CPU address resolution into an extended register

The destination of the per-CPU address MOV is encoded in ModRM.reg, which is extended by REX.R, but the REX prefix is built with add_1mod(), which sets REX.B. REX.B extends ModRM.rm and SIB.base, and this instruction addresses memory as disp32 with no base, so the bit has no effect at all and the high register bit is simply lost.

Every is_ereg() destination therefore resolves to the wrong register, picking whichever one shares the low three bits:

R5 -> RAX R7 -> RBP R8 -> RSI R9 -> RDI

With BPF_REG_5, whose reg2hex is 0, the emitted

65 49 03 04 25 <off> add %gs:<off>,%rax

adds the per-CPU offset to RAX rather than R8. The destination keeps the unadjusted address and RAX is clobbered, so the program goes on to dereference a pointer that was never made per-CPU:

  BUG: unable to handle page fault for address: 0000607e386a8894
  RIP: bpf_prog_707837aafd2aa9ae_update_percpu_data+0x93/0xc9
  Call Trace:
   __bpf_prog_test_run_raw_tp+0x2dc/0x7d0
   __flush_smp_call_function_queue+0x1e9/0xc80
  Kernel panic - not syncing: Fatal exception in interrupt

R5 is the mildest of the four, aliasing a scratch register and faulting at the store. R7 aliases RBP and would corrupt the frame pointer, R8 and R9 alias the argument registers.

Use add_2mod() so the register goes through REX.R, matching how add_2reg() places it in ModRM.reg and how emit_priv_frame_ptr() hardcodes 0x4c for the same instruction with R9. Encodings for the non-extended registers are unchanged.

Problem showed up when trying to resurrect BPF_GCC CI (selftests built with BPF_GCC).

This has gone unnoticed because clang reloads the address into R1 before each per-CPU access, so the destination is never an extended register. GCC keeps several per-CPU addresses live at once, and test_progs-bpf_gcc panics the kernel in global_percpu_data/init, where the address of a .percpu variable ends up in R5.

Fixes 7bdbf7446305 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Cc [email protected]
Signed-off-by Vineet Gupta <[email protected]>
---
 arch/x86/net/bpf_jit_comp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/net/bpf_jit_comp.c +1 -1
index d920772af7d5..1a9fb530adc3 100644
@@ -1935,7 +1935,7 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 				EMIT_mov(dst_reg, src_reg);
 #ifdef CONFIG_SMP
 				/* add <dst>, gs:[<off>] */
-				EMIT2(0x65, add_1mod(0x48, dst_reg));
+				EMIT2(0x65, add_2mod(0x48, 0, dst_reg));
 				EMIT3(0x03, add_2reg(0x04, 0, dst_reg), 0x25);
 				EMIT((u32)(unsigned long)&this_cpu_off, 4);
 #endif
Signature · 1 line
2.53.0-Meta
maintainer Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH bpf v2 1/2] bpf, x86: Fix per-CPU address resolution into an extended register

Quoted 44 lines · Vineet Gupta

The destination of the per-CPU address MOV is encoded in ModRM.reg, which is extended by REX.R, but the REX prefix is built with add_1mod(), which sets REX.B. REX.B extends ModRM.rm and SIB.base, and this instruction addresses memory as disp32 with no base, so the bit has no effect at all and the high register bit is simply lost.

Every is_ereg() destination therefore resolves to the wrong register, picking whichever one shares the low three bits:

R5 -> RAX R7 -> RBP R8 -> RSI R9 -> RDI

With BPF_REG_5, whose reg2hex is 0, the emitted

65 49 03 04 25 <off> add %gs:<off>,%rax

adds the per-CPU offset to RAX rather than R8. The destination keeps the unadjusted address and RAX is clobbered, so the program goes on to dereference a pointer that was never made per-CPU:

  BUG: unable to handle page fault for address: 0000607e386a8894
  RIP: bpf_prog_707837aafd2aa9ae_update_percpu_data+0x93/0xc9
  Call Trace:
   __bpf_prog_test_run_raw_tp+0x2dc/0x7d0
   __flush_smp_call_function_queue+0x1e9/0xc80
  Kernel panic - not syncing: Fatal exception in interrupt

R5 is the mildest of the four, aliasing a scratch register and faulting at the store. R7 aliases RBP and would corrupt the frame pointer, R8 and R9 alias the argument registers.

Use add_2mod() so the register goes through REX.R, matching how add_2reg() places it in ModRM.reg and how emit_priv_frame_ptr() hardcodes 0x4c for the same instruction with R9. Encodings for the non-extended registers are unchanged.

Problem showed up when trying to resurrect BPF_GCC CI (selftests built with BPF_GCC).

This has gone unnoticed because clang reloads the address into R1 before each per-CPU access, so the destination is never an extended register. GCC keeps several per-CPU addresses live at once, and test_progs-bpf_gcc panics the kernel in global_percpu_data/init, where the address of a .percpu variable ends up in R5.

Quoted 4 lines
Fixes 7bdbf7446305 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Cc [email protected]
Signed-off-by Vineet Gupta <[email protected]>

---

Reviewed-by Eduard Zingerman <[email protected]>

Please don't drop acks.

AI review
#3 lore raw

Re: [PATCH bpf v2 1/2] bpf, x86: Fix per-CPU address resolution into an extended register

In reply to Vineet Gupta
Should this include Eduard Zingerman's Reviewed-by tag? In the v2 review thread, Eduard provided "Reviewed-by: Eduard Zingerman <[email protected]>" and specifically noted "Please don't drop acks."
Quoted 46 lines

bpf, x86: Fix per-CPU address resolution into an extended register

The destination of the per-CPU address MOV is encoded in ModRM.reg, which is extended by REX.R, but the REX prefix is built with add_1mod(), which sets REX.B. REX.B extends ModRM.rm and SIB.base, and this instruction addresses memory as disp32 with no base, so the bit has no effect at all and the high register bit is simply lost.

Every is_ereg() destination therefore resolves to the wrong register, picking whichever one shares the low three bits:

R5 -> RAX R7 -> RBP R8 -> RSI R9 -> RDI

With BPF_REG_5, whose reg2hex is 0, the emitted

65 49 03 04 25 <off> add %gs:<off>,%rax

adds the per-CPU offset to RAX rather than R8. The destination keeps the unadjusted address and RAX is clobbered, so the program goes on to dereference a pointer that was never made per-CPU:

  BUG: unable to handle page fault for address: 0000607e386a8894
  RIP: bpf_prog_707837aafd2aa9ae_update_percpu_data+0x93/0xc9
  Call Trace:
   __bpf_prog_test_run_raw_tp+0x2dc/0x7d0
   __flush_smp_call_function_queue+0x1e9/0xc80
  Kernel panic - not syncing: Fatal exception in interrupt

R5 is the mildest of the four, aliasing a scratch register and faulting at the store. R7 aliases RBP and would corrupt the frame pointer, R8 and R9 alias the argument registers.

Use add_2mod() so the register goes through REX.R, matching how add_2reg() places it in ModRM.reg and how emit_priv_frame_ptr() hardcodes 0x4c for the same instruction with R9. Encodings for the non-extended registers are unchanged.

Problem showed up when trying to resurrect BPF_GCC CI (selftests built with BPF_GCC).

This has gone unnoticed because clang reloads the address into R1 before each per-CPU access, so the destination is never an extended register. GCC keeps several per-CPU addresses live at once, and test_progs-bpf_gcc panics the kernel in global_percpu_data/init, where the address of a .percpu variable ends up in R5.

Quoted 3 lines
Fixes 7bdbf7446305 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Cc [email protected]
Signed-off-by Vineet Gupta <[email protected]>

Should this include Eduard Zingerman's Reviewed-by tag? In the v2 review thread, Eduard provided "Reviewed-by: Eduard Zingerman <[email protected]>" and specifically noted "Please don't drop acks."

https://lore.kernel.org/bpf/[email protected]/

arch/x86/net/bpf_jit_comp.c static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int * · line 1893

Quoted 13 lines
arch/x86/net/bpf_jit_comp.c +1 -1
index de7515ea1bea..276d076d2993 100644
@@ -1887,7 +1887,7 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 				EMIT_mov(dst_reg, src_reg);
 #ifdef CONFIG_SMP
 				/* add <dst>, gs:[<off>] */
-				EMIT2(0x65, add_1mod(0x48, dst_reg));
+				EMIT2(0x65, add_2mod(0x48, 0, dst_reg));
 				EMIT3(0x03, add_2reg(0x04, 0, dst_reg), 0x25);
 				EMIT((u32)(unsigned long)&this_cpu_off, 4);
 #endif

In the v1 review thread, Eduard requested an inline assembly test with __xlated() matcher to verify the JIT generates correct code for per-CPU address resolution. A test was apparently added in v2 as patch 2/2 (tools/testing/selftests/bpf/progs/verifier_percpu_addr.c), but that test is not present in this commit.

Given that this bug affects all four extended registers (R5, R7, R8, R9) and can cause kernel panics or frame pointer corruption, should the test be included here to verify the fix works correctly?

https://lore.kernel.org/bpf/[email protected]/

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31839526403

author
#4 lore raw

Re: [PATCH bpf v2 1/2] bpf, x86: Fix per-CPU address resolution into an extended register

Quoted 46 lines · On 8/14/26 2:20 PM, [email protected] wrote:
Quoted 46 lines

bpf, x86: Fix per-CPU address resolution into an extended register

The destination of the per-CPU address MOV is encoded in ModRM.reg, which is extended by REX.R, but the REX prefix is built with add_1mod(), which sets REX.B. REX.B extends ModRM.rm and SIB.base, and this instruction addresses memory as disp32 with no base, so the bit has no effect at all and the high register bit is simply lost.

Every is_ereg() destination therefore resolves to the wrong register, picking whichever one shares the low three bits:

   R5 -> RAX    R7 -> RBP    R8 -> RSI    R9 -> RDI

With BPF_REG_5, whose reg2hex is 0, the emitted

   65 49 03 04 25 <off>	add %gs:<off>,%rax

adds the per-CPU offset to RAX rather than R8. The destination keeps the unadjusted address and RAX is clobbered, so the program goes on to dereference a pointer that was never made per-CPU:

   BUG: unable to handle page fault for address: 0000607e386a8894
   RIP: bpf_prog_707837aafd2aa9ae_update_percpu_data+0x93/0xc9
   Call Trace:
    __bpf_prog_test_run_raw_tp+0x2dc/0x7d0
    __flush_smp_call_function_queue+0x1e9/0xc80
   Kernel panic - not syncing: Fatal exception in interrupt

R5 is the mildest of the four, aliasing a scratch register and faulting at the store. R7 aliases RBP and would corrupt the frame pointer, R8 and R9 alias the argument registers.

Use add_2mod() so the register goes through REX.R, matching how add_2reg() places it in ModRM.reg and how emit_priv_frame_ptr() hardcodes 0x4c for the same instruction with R9. Encodings for the non-extended registers are unchanged.

Problem showed up when trying to resurrect BPF_GCC CI (selftests built with BPF_GCC).

This has gone unnoticed because clang reloads the address into R1 before each per-CPU access, so the destination is never an extended register. GCC keeps several per-CPU addresses live at once, and test_progs-bpf_gcc panics the kernel in global_percpu_data/init, where the address of a .percpu variable ends up in R5.

Quoted 8 lines
Quoted 3 lines
Fixes 7bdbf7446305 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Cc [email protected]
Signed-off-by Vineet Gupta <[email protected]>

Should this include Eduard Zingerman's Reviewed-by tag? In the v2 review thread, Eduard provided "Reviewed-by: Eduard Zingerman <[email protected]>" and specifically noted "Please don't drop acks."

https://lore.kernel.org/bpf/[email protected]/

Yes I missed it. Will carry it in v3.

Quoted 19 lines

arch/x86/net/bpf_jit_comp.c static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int * · line 1893

Quoted 13 lines
arch/x86/net/bpf_jit_comp.c +1 -1
index de7515ea1bea..276d076d2993 100644
@@ -1887,7 +1887,7 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
  				EMIT_mov(dst_reg, src_reg);
  #ifdef CONFIG_SMP
  				/* add <dst>, gs:[<off>] */
-				EMIT2(0x65, add_1mod(0x48, dst_reg));
+				EMIT2(0x65, add_2mod(0x48, 0, dst_reg));
  				EMIT3(0x03, add_2reg(0x04, 0, dst_reg), 0x25);
  				EMIT((u32)(unsigned long)&this_cpu_off, 4);
  #endif

In the v1 review thread, Eduard requested an inline assembly test with __xlated() matcher to verify the JIT generates correct code for per-CPU address resolution. A test was apparently added in v2 as patch 2/2 (tools/testing/selftests/bpf/progs/verifier_percpu_addr.c), but that test is not present in this commit.

Quoted 5 lines

Given that this bug affects all four extended registers (R5, R7, R8, R9) and can cause kernel panics or frame pointer corruption, should the test be included here to verify the fix works correctly?

https://lore.kernel.org/bpf/[email protected]/

Per BPF convention tests are generally separate patch and it is in 2/2 of this series.

Thx,
-Vineet

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31839526403

author
#5 lore raw

PATCH v2 2/2 selftests/bpf: Check per-CPU address resolution per register

In reply to Vineet Gupta

An ld_imm64 of a per-CPU map value is followed by a mov_percpu_addr that reuses the same register, so which register the address lands in decides how the JIT encodes the add. Getting the REX prefix wrong there is invisible to a functional test on x86 unless the address happens to land in an extended register, which is why this went unnoticed.

Add one __naked program per extended register, R5, R7, R8 and R9, each loading a .percpu variable into that register, and match the JITed add against the register it must resolve into. R1 is covered too, so that a fix which sets REX.R unconditionally does not pass either.

Signed-off-by Vineet Gupta <[email protected]>
---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_percpu_addr.c          | 114 ++++++++++++++++++
 2 files changed, 116 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_percpu_addr.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 8113fea7ba86..64ac49ad67e6 100644
@@ -79,6 +79,7 @@
 #include "verifier_netfilter_retcode.skel.h"
 #include "verifier_bpf_fastcall.skel.h"
 #include "verifier_or_jmp32_k.skel.h"
+#include "verifier_percpu_addr.skel.h"
 #include "verifier_precision.skel.h"
 #include "verifier_prevent_map_lookup.skel.h"
 #include "verifier_private_stack.skel.h"

@@ -240,6 +241,7 @@ void test_verifier_netfilter_ctx(void)        { RUN(verifier_netfilter_ctx); }
 void test_verifier_netfilter_retcode(void)    { RUN(verifier_netfilter_retcode); }
 void test_verifier_bpf_fastcall(void)         { RUN(verifier_bpf_fastcall); }
 void test_verifier_or_jmp32_k(void)           { RUN(verifier_or_jmp32_k); }
+void test_verifier_percpu_addr(void)          { RUN(verifier_percpu_addr); }
 void test_verifier_precision(void)            { RUN(verifier_precision); }
 void test_verifier_prevent_map_lookup(void)   { RUN(verifier_prevent_map_lookup); }
 void test_verifier_private_stack(void)        { RUN(verifier_private_stack); }
tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +114 -0

114 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
maintainer
#6 lore raw

Re: [PATCH bpf v2 2/2] selftests/bpf: Check per-CPU address resolution per register

On Fri, 2026-08-14 at 13:32 -0700, Vineet Gupta wrote:

...

Quoted 15 lines
tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +15 -0
+/*
+ * An ld_imm64 of a per-CPU map value is followed by a mov_percpu_addr that
+ * reuses the same register, so the register the address lands in decides how
+ * the JIT encodes the add. On x86 R5, R7, R8 and R9 are the extended
+ * registers, whose high bit needs REX.R because the destination sits in
+ * ModRM.reg. Check one program per extended register, since getting the
+ * prefix wrong resolves the address into whichever register shares the low
+ * three bits instead.
+ */
+
+SEC("raw_tp")
+__description("per-CPU address into r5")
+__success
+__arch_x86_64
+__jited("	addq	%gs:{{.*}}, %r8")

tools/testing/selftests/bpf/progs/verifier_percpu_addr.c line 35

Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +10 -0
+__naked void percpu_addr_into_r5(void)
+{
+	asm volatile ("					\
+	r5 = %[percpu_data] ll;				\
+	r0 = *(u32 *)(r5 + 0);				\
+	exit;						\
+"	:
+	: __imm_addr(percpu_data)
+	: __clobber_all);
+}

Thank you for adding the tests.
Maybe save some space by just packing it as a single test?

__jited(" addq %gs:{{.*}}, %...")
__jited(" addq %gs:{{.*}}, %...")
__jited(" addq %gs:{{.*}}, %...")
__jited(" addq %gs:{{.*}}, %...")
__jited(" addq %gs:{{.*}}, %...")

	r0 = %[percpu_data] ll;				\
	r1 = %[percpu_data] ll;				\
	r2 = %[percpu_data] ll;				\
	r3 = %[percpu_data] ll;				\
	r4 = %[percpu_data] ll;				\
	r5 = %[percpu_data] ll;				\
	r6 = %[percpu_data] ll;				\
	r7 = %[percpu_data] ll;				\
	r8 = %[percpu_data] ll;				\
	r9 = %[percpu_data] ll;				\

More compact and no need to describe in length why only R{5,7,8,9} are selected.

...

AI review
#7 lore raw

Re: [PATCH bpf v2 2/2] selftests/bpf: Check per-CPU address resolution per register

In reply to Vineet Gupta
This isn't a bug, but would it be tidier to move the percpu_data declaration inside the __TARGET_ARCH_x86 block?
Quoted 31 lines
tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +25 -0
new file mode 100644
index 000000000000..962faea8ef90
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+int percpu_data SEC(".percpu");
+
+#if defined(__TARGET_ARCH_x86)
+
+/*
+ * An ld_imm64 of a per-CPU map value is followed by a mov_percpu_addr that
+ * reuses the same register, so the register the address lands in decides how
+ * the JIT encodes the add. On x86 R5, R7, R8 and R9 are the extended
+ * registers, whose high bit needs REX.R because the destination sits in
+ * ModRM.reg. Check one program per extended register, since getting the
+ * prefix wrong resolves the address into whichever register shares the low
+ * three bits instead.
+ */
+
+SEC("raw_tp")
+__description("per-CPU address into r5")
+__success
+__arch_x86_64
+__jited("	addq	%gs:{{.*}}, %r8")

tools/testing/selftests/bpf/progs/verifier_percpu_addr.c line 35

Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +10 -0
+__naked void percpu_addr_into_r5(void)
+{
+	asm volatile ("					\
+	r5 = %[percpu_data] ll;				\
+	r0 = *(u32 *)(r5 + 0);				\
+	exit;						\
+"	:
+	: __imm_addr(percpu_data)
+	: __clobber_all);
+}

This isn't a bug, but would it be tidier to move the percpu_data declaration inside the __TARGET_ARCH_x86 block?

Currently percpu_data is declared above the arch guard, but its only readers (percpu_addr_into_r1/r5/r7/r8/r9) are inside the guard:

+int percpu_data SEC(".percpu");
+
+#if defined(__TARGET_ARCH_x86)
+
+/* ... test functions using percpu_data ... */

On every other target the object still carries a .percpu section, so libbpf still creates an unused percpu data map for it.

Comparable files in the same directory keep objects used only by guarded programs inside the guard. verifier_private_stack.c declares its 'struct elem' and 'array SEC(".maps")' after the '#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)' guard, and verifier_ldsx.c declares its 'arena SEC(".maps")' map inside its arch guard.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31839526403

author
#8 lore raw

Re: [PATCH bpf v2 2/2] selftests/bpf: Check per-CPU address resolution per register

Quoted 53 lines · On 8/14/26 2:20 PM, [email protected] wrote:
Quoted 31 lines
tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +25 -0
new file mode 100644
index 000000000000..962faea8ef90
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+int percpu_data SEC(".percpu");
+
+#if defined(__TARGET_ARCH_x86)
+
+/*
+ * An ld_imm64 of a per-CPU map value is followed by a mov_percpu_addr that
+ * reuses the same register, so the register the address lands in decides how
+ * the JIT encodes the add. On x86 R5, R7, R8 and R9 are the extended
+ * registers, whose high bit needs REX.R because the destination sits in
+ * ModRM.reg. Check one program per extended register, since getting the
+ * prefix wrong resolves the address into whichever register shares the low
+ * three bits instead.
+ */
+
+SEC("raw_tp")
+__description("per-CPU address into r5")
+__success
+__arch_x86_64
+__jited("	addq	%gs:{{.*}}, %r8")

tools/testing/selftests/bpf/progs/verifier_percpu_addr.c line 35

Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +10 -0
+__naked void percpu_addr_into_r5(void)
+{
+	asm volatile ("					\
+	r5 = %[percpu_data] ll;				\
+	r0 = *(u32 *)(r5 + 0);				\
+	exit;						\
+"	:
+	: __imm_addr(percpu_data)
+	: __clobber_all);
+}

This isn't a bug, but would it be tidier to move the percpu_data declaration inside the __TARGET_ARCH_x86 block?

Currently percpu_data is declared above the arch guard, but its only readers (percpu_addr_into_r1/r5/r7/r8/r9) are inside the guard:

tools/testing/selftests/bpf/progs/verifier_percpu_addr.c new file +5 -0
+int percpu_data SEC(".percpu");
+
+#if defined(__TARGET_ARCH_x86)
+
+/* ... test functions using percpu_data ... */
Quoted 2 lines

On every other target the object still carries a .percpu section, so libbpf still creates an unused percpu data map for it.

That's a good point, will address this in v3

Thx,
-Vineet

Quoted 13 lines

Comparable files in the same directory keep objects used only by guarded programs inside the guard. verifier_private_stack.c declares its 'struct elem' and 'array SEC(".maps")' after the '#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)' guard, and verifier_ldsx.c declares its 'arena SEC(".maps")' map inside its arch guard.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31839526403