bpf, riscv: Add support for signed arena loads
Subsystems (5)
v3 cover bpf, riscv: Add support for signed arena loads
Hi,
Signed loads from arena memory are currently unsupported on riscv64:bpf_jit_supports_insn() rejects BPF_MEMSX loads when in_arena is set, so the verifier fails such programs with "sign extending loads from arena are not supported yet". The x86 and arm64 JITs gained support for them in v6.18 (a91ae3c89311, eab2a71f3a6a). Since compilers are free to generate signed loads into arena memory (e.g. GCC-14 was reported to do so), otherwise valid BPF programs fail to load on riscv64.
This series adds BPF_PROBE_MEM32SX support to the RV64 JIT and enables the corresponding selftests on riscv64:
1 implements signed arena loads in the RV64 JIT. The verifier
already converts MEMSX loads from PTR_TO_ARENA to BPF_PROBE_MEM32SX once bpf_jit_supports_insn() allows them, so the JIT reuses the existing arena handling: the arena base (RV_REG_ARENA) is added to the source register and the load is emitted with sign extension (lb/lh/lw). BPF_PROBE_MEM32SX is also added to the add_exception_handler() mode gate so faulting loads register an exception table entry that clears the destination register and resumes execution.
2 enables the arena LDSX tests on riscv64: JIT disassembly
assertions are added to arena_ldsx_disasm, and arena_ldsx_exception/s8/s16/s32 are now run on riscv64.
The series was verified on riscv64 with QEMU (-M virt): all five arena_ldsx tests pass, including the exception path (load from unallocated arena memory returns 0) and the sign-extension values (s8/s16/s32 tests return -1 as expected). The arena_atomics test suite also loads and runs cleanly.
Changes in v3:
- Restore the pseudo-code form of the PROBE_MEM32SX case comment.
Changes in v2:
- Fix an extable entry overflow breaking the arena_atomics load.
- Collect Reviewed-by on patch 2.
Note: the __jited assertions in patch 2 were derived from the JIT register allocation (R0->a5, R1->a0, R8->s3, R9->s4, arena base in s7) and the emit_ldx() code paths; happy to adjust them if a disassembler output detail differs.
Thanks,
Pei
Chen Pei (2):
bpf, riscv: Add support for signed arena loads selftests/bpf: Enable arena LDSX tests for riscv64
arch/riscv/net/bpf_jit_comp64.c | 16 ++++++++++------ .../testing/selftests/bpf/progs/verifier_ldsx.c | 17 +++++++++++++++++ 2 files changed, 27 insertions(+), 6 deletions(-)
Signature · 1 line
2.50.1
PATCH v3 1/2 bpf, riscv: Add support for signed arena loads
Signed loads from arena memory are currently rejected on riscv64, as bpf_jit_supports_insn() refuses BPF_MEMSX loads when in_arena is set, while x86 and arm64 gained support for them in v6.18. Compilers such as GCC-14 are free to generate signed loads into arena memory, which breaks loading of otherwise valid BPF programs on riscv64.
Implement BPF_PROBE_MEM32SX support in the RV64 JIT by reusing the existing arena handling: the arena base (RV_REG_ARENA) is added to the source register and the load is emitted with sign extension (lb/lh/lw). Add BPF_PROBE_MEM32SX to the add_exception_handler() mode gate so that faulting loads get an exception table entry which clears the destination register and resumes execution. Since BPF_PROBE_MEM32SX shares its mode value (0xc0) with BPF_ATOMIC, the gate accepts it only for LDX class instructions so that plain atomic instructions do not register exception table entries.
Verified by running the arena LDSX selftests (arena_ldsx_disasm, arena_ldsx_exception, arena_ldsx_s8/s16/s32) and the full arena_atomics test suite on riscv64 QEMU, all passing.
---
Changes in v3:
- Restore the pseudo-code form of the PROBE_MEM32SX case comment.
Changes in v2:
- Fix extable entry overflow breaking arena_atomics load
arch/riscv/net/bpf_jit_comp64.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +10 -6
@@ -777,6 +777,8 @@ static int add_exception_handler(const struct bpf_insn *insn, int dst_reg,if (BPF_MODE (insn->code) !=BPF_PROBE_MEM &&BPF_MODE (insn->code) !=BPF_PROBE_MEMSX &&BPF_MODE (insn->code) !=BPF_PROBE_MEM32 && + !(BPF_MODE (insn->code) ==BPF_PROBE_MEM32SX && +BPF_CLASS (insn->code) ==BPF_LDX ) &&BPF_MODE (insn->code) !=BPF_PROBE_ATOMIC )return 0 ;
@@ -1902,13 +1904,19 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,case BPF_LDX |BPF_PROBE_MEM32 |BPF_H :case BPF_LDX |BPF_PROBE_MEM32 |BPF_W :case BPF_LDX |BPF_PROBE_MEM32 |BPF_DW : +/* LDX | PROBE_MEM32SX: dst = *(signed size *)(src + RV_REG_ARENA + off) */ +case BPF_LDX |BPF_PROBE_MEM32SX |BPF_B : +case BPF_LDX |BPF_PROBE_MEM32SX |BPF_H : +case BPF_LDX |BPF_PROBE_MEM32SX |BPF_W : {bool sign_ext; sign_ext =BPF_MODE (insn->code) ==BPF_MEMSX || -BPF_MODE (insn->code) ==BPF_PROBE_MEMSX ; +BPF_MODE (insn->code) ==BPF_PROBE_MEMSX || +BPF_MODE (insn->code) ==BPF_PROBE_MEM32SX ; -if (BPF_MODE (insn->code) ==BPF_PROBE_MEM32 ) { +if (BPF_MODE (insn->code) ==BPF_PROBE_MEM32 || +BPF_MODE (insn->code) ==BPF_PROBE_MEM32SX ) {emit_add (RV_REG_T2 , rs,RV_REG_ARENA , ctx); rs =RV_REG_T2 ; }
@@ -2126,10 +2134,6 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)if (insn->imm ==BPF_CMPXCHG )return rv_ext_enabled (ZACAS );break ; -case BPF_LDX |BPF_MEMSX |BPF_B : -case BPF_LDX |BPF_MEMSX |BPF_H : -case BPF_LDX |BPF_MEMSX |BPF_W : -return false; } }
Signature · 1 line
2.50.1
Quoted 68 lines · Chen Pei
Signed loads from arena memory are currently rejected on riscv64, as
bpf_jit_supports_insn()refusesBPF_MEMSXloads whenin_arenais set, while x86 and arm64 gained support for them in v6.18. Compilers such as GCC-14 are free to generate signed loads into arena memory, which breaks loading of otherwise valid BPF programs on riscv64.Implement
BPF_PROBE_MEM32SXsupport in the RV64 JIT by reusing the existing arena handling: the arena base (RV_REG_ARENA) is added to the source register and the load is emitted with sign extension (lb/lh/lw). AddBPF_PROBE_MEM32SXto theadd_exception_handler()mode gate so that faulting loads get an exception table entry which clears the destination register and resumes execution. SinceBPF_PROBE_MEM32SXshares its mode value (0xc0) withBPF_ATOMIC, the gate accepts it only for LDX class instructions so that plain atomic instructions do not register exception table entries.Verified by running the arena LDSX selftests (
arena_ldsx_disasm,arena_ldsx_exception, arena_ldsx_s8/s16/s32) and the fullarena_atomicstest suite on riscv64 QEMU, all passing.Signed-off-by Chen Pei <[email protected]>---
Changes in v3:
- Restore the pseudo-code form of thePROBE_MEM32SXcase comment.Changes in v2:
- Fix extable entry overflow breakingarena_atomicsloadarch/riscv/net/bpf_jit_comp64.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)arch/riscv/net/bpf_jit_comp64.c +10
-2@@ -777,6 +777,8 @@ static int add_exception_handler(const struct bpf_insn *insn, int dst_reg,if (BPF_MODE (insn->code) !=BPF_PROBE_MEM &&BPF_MODE (insn->code) !=BPF_PROBE_MEMSX &&BPF_MODE (insn->code) !=BPF_PROBE_MEM32 && + !(BPF_MODE (insn->code) ==BPF_PROBE_MEM32SX && +BPF_CLASS (insn->code) ==BPF_LDX ) &&BPF_MODE (insn->code) !=BPF_PROBE_ATOMIC )return 0 ;@@ -1902,13 +1904,19 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,case BPF_LDX |BPF_PROBE_MEM32 |BPF_H :case BPF_LDX |BPF_PROBE_MEM32 |BPF_W :case BPF_LDX |BPF_PROBE_MEM32 |BPF_DW : +/* LDX | PROBE_MEM32SX: dst = *(signed size *)(src + RV_REG_ARENA + off) */ +case BPF_LDX |BPF_PROBE_MEM32SX |BPF_B : +case BPF_LDX |BPF_PROBE_MEM32SX |BPF_H : +case BPF_LDX |BPF_PROBE_MEM32SX |BPF_W : {bool sign_ext; sign_ext =BPF_MODE (insn->code) ==BPF_MEMSX || -BPF_MODE (insn->code) ==BPF_PROBE_MEMSX ; +BPF_MODE (insn->code) ==BPF_PROBE_MEMSX || +BPF_MODE (insn->code) ==BPF_PROBE_MEM32SX ; -if (BPF_MODE (insn->code) ==BPF_PROBE_MEM32 ) { +if (BPF_MODE (insn->code) ==BPF_PROBE_MEM32 || +BPF_MODE (insn->code) ==BPF_PROBE_MEM32SX ) {emit_add (RV_REG_T2 , rs,RV_REG_ARENA , ctx); rs =RV_REG_T2 ; }
arch/riscv/net/bpf_jit_comp64.c bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena) · line 2139
Quoted 10 lines
arch/riscv/net/bpf_jit_comp64.c +0
-4@@ -2126,10 +2134,6 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena)if (insn->imm ==BPF_CMPXCHG )return rv_ext_enabled (ZACAS );break ; -case BPF_LDX |BPF_MEMSX |BPF_B : -case BPF_LDX |BPF_MEMSX |BPF_H : -case BPF_LDX |BPF_MEMSX |BPF_W : -return false; } }
PATCH v3 2/2 selftests/bpf: Enable arena LDSX tests for riscv64
In reply to Chen PeiNow that the riscv64 JIT supports signed arena loads (BPF_PROBE_MEM32SX), enable the arena LDSX tests on riscv64:
add JIT disassembly assertions for arena_ldsx_disasm (arena base in s7, add into t2, sign-extending lw/lh/lb loads) and run arena_ldsx_exception and arena_ldsx_s8/s16/s32 on riscv64.
--- .../testing/selftests/bpf/progs/verifier_ldsx.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
tools/testing/selftests/bpf/progs/verifier_ldsx.c +17 -0
@@ -286,6 +286,19 @@ __jited("add x11, x0, x28") __jited("ldrsh x22, [x11, #0x18]") __jited("add x11, x0, x28") __jited("ldrsb x22, [x11, #0x20]") +__arch_riscv64 +__jited("add t2, a5, s7") +__jited("lw s3, 0x10(t2)") +__jited("add t2, a5, s7") +__jited("lh s3, 0x18(t2)") +__jited("add t2, a5, s7") +__jited("lb s3, 0x20(t2)") +__jited("add t2, a0, s7") +__jited("lw s4, 0x10(t2)") +__jited("add t2, a0, s7") +__jited("lh s4, 0x18(t2)") +__jited("add t2, a0, s7") +__jited("lb s4, 0x20(t2)")__naked void arena_ldsx_disasm (void *ctx) { asm volatile (
@@ -317,6 +330,7 @@ __description("Arena LDSX Exception") __success __retval(0) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_exception (void *ctx) { asm volatile (
@@ -338,6 +352,7 @@ __description("Arena LDSX, S8") __success __retval(-1) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_s8 (void *ctx) { asm volatile (
@@ -369,6 +384,7 @@ __description("Arena LDSX, S16") __success __retval(-1) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_s16 (void *ctx) { asm volatile (
@@ -400,6 +416,7 @@ __description("Arena LDSX, S32") __success __retval(-1) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_s32 (void *ctx) { asm volatile (
Signature · 1 line
2.50.1
Re: [PATCH bpf-next v3 2/2] selftests/bpf: Enable arena LDSX tests for riscv64
Quoted 61 lines · Chen Pei
Now that the riscv64 JIT supports signed arena loads (
BPF_PROBE_MEM32SX), enable the arena LDSX tests on riscv64:
add JIT disassembly assertions forarena_ldsx_disasm(arena base in s7, add into t2, sign-extending lw/lh/lb loads) and runarena_ldsx_exceptionand arena_ldsx_s8/s16/s32 on riscv64.Reviewed-by Pu Lehui <[email protected]>Signed-off-by Chen Pei <[email protected]>--- .../testing/selftests/bpf/progs/verifier_ldsx.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)tools/testing/selftests/bpf/progs/verifier_ldsx.c +16
-0@@ -286,6 +286,19 @@ __jited("add x11, x0, x28") __jited("ldrsh x22, [x11, #0x18]") __jited("add x11, x0, x28") __jited("ldrsb x22, [x11, #0x20]") +__arch_riscv64 +__jited("add t2, a5, s7") +__jited("lw s3, 0x10(t2)") +__jited("add t2, a5, s7") +__jited("lh s3, 0x18(t2)") +__jited("add t2, a5, s7") +__jited("lb s3, 0x20(t2)") +__jited("add t2, a0, s7") +__jited("lw s4, 0x10(t2)") +__jited("add t2, a0, s7") +__jited("lh s4, 0x18(t2)") +__jited("add t2, a0, s7") +__jited("lb s4, 0x20(t2)")__naked void arena_ldsx_disasm (void *ctx) { asm volatile (@@ -317,6 +330,7 @@ __description("Arena LDSX Exception") __success __retval(0) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_exception (void *ctx) { asm volatile (@@ -338,6 +352,7 @@ __description("Arena LDSX, S8") __success __retval(-1) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_s8 (void *ctx) { asm volatile (@@ -369,6 +384,7 @@ __description("Arena LDSX, S16") __success __retval(-1) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_s16 (void *ctx) { asm volatile (
tools/testing/selftests/bpf/progs/verifier_ldsx.c __description("Arena LDSX, S32") · line 422
Quoted 7 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c +1
-0@@ -400,6 +416,7 @@ __description("Arena LDSX, S32") __success __retval(-1) __arch_x86_64 __arch_arm64 +__arch_riscv64__naked voidarena_ldsx_s32 (void *ctx) { asm volatile (
Quoted 7 lines · Chen Pei
Chen Pei (2):
bpf, riscv: Add support for signed arena loads selftests/bpf: Enable arena LDSX tests for riscv64arch/riscv/net/bpf_jit_comp64.c | 16 ++++++++++------ .../testing/selftests/bpf/progs/verifier_ldsx.c | 17 +++++++++++++++++ 2 files changed, 27 insertions(+), 6 deletions(-)
Back from summer vacation! Apologies for the delay!
For the series:
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Quoted 11 lines · On Wed, 19 Aug 2026 20:09:22 +0800 you wrote:
Hi,
Signed loads from arena memory are currently unsupported on riscv64:
bpf_jit_supports_insn()rejectsBPF_MEMSXloads whenin_arenais set, so the verifier fails such programs with "sign extending loads from arena are not supported yet". The x86 and arm64 JITs gained support for them in v6.18 (a91ae3c89311,eab2a71f3a6a). Since compilers are free to generate signed loads into arena memory (e.g. GCC-14 was reported to do so), otherwise valid BPF programs fail to load on riscv64.
Quoted 1 line
[...]
Here is the summary with links:
- [bpf-next,v3,1/2] bpf, riscv: Add support for signed arena loads
https://git.kernel.org/bpf/bpf-next/c/b4757714cc63 - [bpf-next,v3,2/2] selftests/bpf: Enable arena LDSX tests for riscv64 https://git.kernel.org/bpf/bpf-next/c/8f2087a1b43b
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html