bpf, riscv: Add support for indirect jumps
Subsystems (6)
v2 cover bpf, riscv: Add support for indirect jumps
The indirect jump instruction (BPF_JMP | BPF_JA | BPF_X, "gotox") and the BPF_MAP_TYPE_INSN_ARRAY jump tables it consumes are core features: the verifier accepts them on every architecture, and the x86-64, arm64 and powerpc JITs implement them. On riscv64 a program using gotox passes verification and then fails to load, because the JIT does not know the opcode and CONFIG_BPF_JIT_ALWAYS_ON leaves no interpreter to fall back onto.
This series adds the riscv64 support and turns the existing selftests on for that architecture.
Patch 1 emits "jalr zero, rd, 0" for gotox and publishes the xlated to jitted offsets through bpf_prog_update_insn_ptrs(), which is what fills in the jump table addresses. Patch 2 widens the arch guard in verifier_gotox.c to riscv64, the same way arm64 and powerpc were enabled.
Changes since v1:
- Rebased onto bpf-next.
- Patch 1: reword the code comment to say "shift ctx->offset[] by one".
- Patch 2: collapse the multi-line #endif marker to a single line.
- Added Reviewed-by/Acked-by from Björn Töpel.
Testing
=======
Environment: QEMU virt rv64, with CONFIG_BPF_JIT=y, CONFIG_BPF_JIT_ALWAYS_ON=y, CONFIG_DEBUG_INFO_BTF=y; selftests cross-built with clang 22.
- test_progs -t verifier_gotox: 27/27 subtests pass on bpf-next, 13 of them executed through BPF_PROG_TEST_RUN.
- test_progs-cpuv4 -t bpf_gotox: 14/14 subtests pass, none skipped. The cpuv4 flavor is needed here because bpf_gotox gates its subtests on __BPF_FEATURE_GOTOX, which clang only defines for -mcpu=v4.
Chen Pei (2):
bpf, riscv: Add support for indirect jumps
selftests/bpf: Enable gotox tests for riscv64
arch/riscv/net/bpf_jit_comp64.c | 5 +++++ arch/riscv/net/bpf_jit_core.c | 16 ++++++++++++++-- .../testing/selftests/bpf/progs/verifier_gotox.c | 6 ++++-- 3 files changed, 23 insertions(+), 4 deletions(-)
Signature · 1 line
2.50.1
PATCH v2 1/2 bpf, riscv: Add support for indirect jumps
Implement JIT support for the indirect jump instruction (BPF_JMP | BPF_JA | BPF_X), a.k.a. gotox, which lets a BPF program jump through a BPF_MAP_TYPE_INSN_ARRAY jump table.
Emit "jalr zero, rd, 0" and hand the xlated to jitted offsets to bpf_prog_update_insn_ptrs(), which is what fills in the jump table entries; without that call the load fails with -EFAULT in bpf_insn_array_ready(). ctx->offset[] holds the offset of the insn *following* insn i, as bpf_prog_fill_jited_linfo() expects, so it is shifted by one and offset[0] comes from the prologue length. build_body() now records both halves of a multi-insn record, so no slot keeps a fabricated offset.
Only the RV64 JIT is covered; RV32 keeps failing to load as before.
---
Changes since v1:
- Reword the code comment to say "shift ctx->offset[] by one".
arch/riscv/net/bpf_jit_comp64.c | 5 +++++ arch/riscv/net/bpf_jit_core.c | 16 ++++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +5 -0
@@ -1691,6 +1691,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,emit_zextw (rd, rd, ctx);break ; +/* JUMP reg */ +case BPF_JMP |BPF_JA |BPF_X : +emit_jalr (RV_REG_ZERO , rd,0 , ctx); +break ; +/* JUMP off */ case BPF_JMP |BPF_JA :case BPF_JMP32 |BPF_JA :
arch/riscv/net/bpf_jit_core.c +14 -2
@@ -26,10 +26,13 @@ static int build_body(struct rv_jit_context *ctx, bool extra_pass, int *offset)int ret; ret =bpf_jit_emit_insn (insn, ctx, extra_pass); -if (ret >0 ) - i++;/* skip the next instruction */ if (offset) offset[i] = ctx->ninsns; +if (ret >0 ) { + i++;/* skip the next instruction */ +if (offset) + offset[i] = ctx->ninsns; + }if (ret <0 )return ret; }
@@ -176,6 +179,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prfor (i =0 ; i < prog->len; i++) ctx->offset[i] =ninsns_rvoff (ctx->offset[i]);bpf_prog_fill_jited_linfo (prog, ctx->offset); + +/* +* bpf_prog_update_insn_ptrs() wants the start of each insn, so +* shift ctx->offset[] by one and get insn 0 from the prologue. +*/ +for (i = prog->len -1 ; i >0 ; i--) + ctx->offset[i] = ctx->offset[i -1 ]; + ctx->offset[0 ] =ninsns_rvoff (ctx->prologue_len); +bpf_prog_update_insn_ptrs (prog, ctx->offset, jit_data->ro_image); out_offset:kvfree (ctx->offset);kfree (jit_data);
Signature · 7 lines
2.50.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 69 lines · Chen Pei
Implement JIT support for the indirect jump instruction (
BPF_JMP|BPF_JA|BPF_X), a.k.a. gotox, which lets a BPF program jump through aBPF_MAP_TYPE_INSN_ARRAYjump table.Emit "jalr zero, rd, 0" and hand the xlated to jitted offsets to
bpf_prog_update_insn_ptrs(), which is what fills in the jump table entries; without that call the load fails with-EFAULTinbpf_insn_array_ready(). ctx->offset[] holds the offset of the insn *following* insn i, asbpf_prog_fill_jited_linfo()expects, so it is shifted by one and offset[0] comes from the prologue length.build_body()now records both halves of a multi-insn record, so no slot keeps a fabricated offset.Only the RV64 JIT is covered; RV32 keeps failing to load as before.
Signed-off-by Chen Pei <[email protected]>Reviewed-by Björn Töpel <[email protected]>Acked-by Björn Töpel <[email protected]>---
Changes since v1:
- Reword the code comment to say "shift ctx->offset[] by one".arch/riscv/net/bpf_jit_comp64.c | 5 +++++ arch/riscv/net/bpf_jit_core.c | 16 ++++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-)arch/riscv/net/bpf_jit_comp64.c +5
-0@@ -1691,6 +1691,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,emit_zextw (rd, rd, ctx);break ; +/* JUMP reg */ +case BPF_JMP |BPF_JA |BPF_X : +emit_jalr (RV_REG_ZERO , rd,0 , ctx); +break ; +/* JUMP off */ case BPF_JMP |BPF_JA :case BPF_JMP32 |BPF_JA :arch/riscv/net/bpf_jit_core.c +7
-2@@ -26,10 +26,13 @@ static int build_body(struct rv_jit_context *ctx, bool extra_pass, int *offset)int ret; ret =bpf_jit_emit_insn (insn, ctx, extra_pass); -if (ret >0 ) - i++;/* skip the next instruction */ if (offset) offset[i] = ctx->ninsns; +if (ret >0 ) { + i++;/* skip the next instruction */ +if (offset) + offset[i] = ctx->ninsns; + }if (ret <0 )return ret; }@@ -176,6 +179,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prfor (i =0 ; i < prog->len; i++) ctx->offset[i] =ninsns_rvoff (ctx->offset[i]);bpf_prog_fill_jited_linfo (prog, ctx->offset); + +/*
arch/riscv/net/bpf_jit_core.c struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr · line 193
Quoted 10 lines
arch/riscv/net/bpf_jit_core.c +7
-0+* bpf_prog_update_insn_ptrs() wants the start of each insn, so +* shift ctx->offset[] by one and get insn 0 from the prologue. +*/ +for (i = prog->len -1 ; i >0 ; i--) + ctx->offset[i] = ctx->offset[i -1 ]; + ctx->offset[0 ] =ninsns_rvoff (ctx->prologue_len); +bpf_prog_update_insn_ptrs (prog, ctx->offset, jit_data->ro_image); out_offset:kvfree (ctx->offset);kfree (jit_data);
PATCH v2 2/2 selftests/bpf: Enable gotox tests for riscv64
In reply to Chen PeiThe riscv64 JIT now supports the gotox instruction and jump tables, so run the tests in verifier_gotox.c on riscv64 too, mirroring what was done for arm64 and powerpc.
The guard is 64-bit only because the RV32 JIT does not implement gotox.
---
Changes since v1:
- Collapse the multi-line #endif marker to a single line.
tools/testing/selftests/bpf/progs/verifier_gotox.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/verifier_gotox.c +4 -2
@@ -6,7 +6,9 @@#include "bpf_misc.h" #include "../../../include/linux/filter.h" -#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_powerpc) +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \ + defined(__TARGET_ARCH_powerpc) || \ + (defined(__TARGET_ARCH_riscv) && __riscv_xlen ==64 )#define DEFINE_SIMPLE_JUMP_TABLE_PROG (NAME ,SRC_REG ,OFF ,IMM ,OUTCOME ) \ \
@@ -580,6 +582,6 @@ nop_%=: \ : __clobber_all); } -#endif /* __TARGET_ARCH_x86 || __TARGET_ARCH_arm64 || __TARGET_ARCH_powerpc*/ +#endif /* gotox: x86, arm64, powerpc, riscv64 */ char _license[] SEC("license") = "GPL";
Signature · 1 line
2.50.1
[email protected]
AI reviewRe: [PATCH bpf-next v2 2/2] selftests/bpf: Enable gotox tests for riscv64
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] bpf: missing architecture guards for riscv64 gotox tests --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] bpf: missing architecture guards for riscv64 gotox tests
--
commit 1d33b2e23d145afdb7fdcec7bed2452e5b940956
Author: Chen Pei <[email protected]>
selftests/bpf: Enable gotox tests for riscv64
This commit adds riscv64 to the architecture whitelist in verifier_gotox.c because the JIT now supports the gotox instruction and jump tables.
tools/testing/selftests/bpf/progs/verifier_gotox.c line 11
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +3
-1@@ -6,7 +6,9 @@#include "bpf_misc.h" #include "../../../include/linux/filter.h" -#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_powerpc) +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \ + defined(__TARGET_ARCH_powerpc) || \ + (defined(__TARGET_ARCH_riscv) && __riscv_xlen ==64 )
[Severity: Low]
Are there other files that need their architecture guards updated for riscv64 gotox tests to prevent a regression in test coverage?
For example, tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c has:
#if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__) static int map_create(__u32 map_type, __u32 max_entries)
{
...This skips the bpf_insn_array test suite on riscv64, which heavily exercises the gotox jump table logic.
Similarly, tools/testing/selftests/bpf/progs/compute_live_registers.c has:
#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)
SEC("socket")__log_level(2)
__msg("2: .1........ (07) r1 += 8")
__msg("3: .1........ (79) r2 = *(u64 *)(r1 +0)")
__msg("4: ..2....... (b7) r3 = 1")
__msg("5: ..23...... (b7) r4 = 2")
__msg("6: ..234..... (0d) gotox r2")
...
This prevents the gotox liveness tracking tests from being compiled and run on riscv64.
Should these architecture guards also be updated to ensure comprehensive test coverage for the newly introduced JIT feature on riscv64?
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Pu Lehui
maintainerRe: [PATCH bpf-next v2 2/2] selftests/bpf: Enable gotox tests for riscv64
Quoted 52 lines · On 2026/9/28 10:31, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] bpf: missing architecture guards for riscv64 gotox tests
--commit
1d33b2e23d145afdb7fdcec7bed2452e5b940956
Author: Chen Pei <[email protected]>selftests/bpf: Enable gotox tests for riscv64
This commit adds riscv64 to the architecture whitelist in
verifier_gotox.cbecause the JIT now supports the gotox instruction and jump tables.tools/testing/selftests/bpf/progs/verifier_gotox.c line 11
Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +3
-1@@ -6,7 +6,9 @@#include "bpf_misc.h" #include "../../../include/linux/filter.h" -#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_powerpc) +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \ + defined(__TARGET_ARCH_powerpc) || \ + (defined(__TARGET_ARCH_riscv) && __riscv_xlen ==64 )[Severity: Low]
Are there other files that need their architecture guards updated for riscv64 gotox tests to prevent a regression in test coverage?For example,
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.chas:#if
defined(__x86_64__)||defined(__powerpc__)||defined(__aarch64__)static intmap_create(__u32 map_type, __u32 max_entries){ ...This skips the
bpf_insn_arraytest suite on riscv64, which heavily exercises the gotox jump table logic.Similarly,
tools/testing/selftests/bpf/progs/compute_live_registers.chas:#if
defined(__TARGET_ARCH_x86)||defined(__TARGET_ARCH_arm64)
SEC("socket")__log_level(2)
__msg("2: .1........ (07) r1 += 8")
__msg("3: .1........ (79) r2 = *(u64 *)(r1 +0)")
__msg("4: ..2....... (b7) r3 = 1")
__msg("5: ..23...... (b7) r4 = 2")
__msg("6: ..234..... (0d) gotox r2")
...
Quoted 8 lines
This prevents the gotox liveness tracking tests from being compiled and run on riscv64.
Should these architecture guards also be updated to ensure comprehensive test coverage for the newly introduced JIT feature on riscv64?
[ ... ]
yah, should go with bpf_insn_array and compute_live_registers testcases
Re: [PATCH bpf-next v2 2/2] selftests/bpf: Enable gotox tests for riscv64
Hi Lehui,
Thanks for the review.
Quoted 1 line
yah, should go with
bpf_insn_arrayandcompute_live_registerstestcases
Agreed. I've enabled both testcases for riscv64 and they pass in my local testing. I'll include them in v3.
Best regards,
Pei