Skip to contentSkip to conversation

bpf, riscv: Add support for indirect jumps

PATCH Superseded Chen Pei 3 participants 7 messages Sep 28

Subsystems (6)
Kselftest BPF RISC-V BPF selftests +2 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Chen Pei

author
#0 lore raw

v2 cover bpf, riscv: Add support for indirect jumps

The indirect jump instruction (BPF_JMP | BPF_JA | BPF_X, "gotox") and the BPF_MAP_TYPE_INSN_ARRAY jump tables it consumes are core features: the verifier accepts them on every architecture, and the x86-64, arm64 and powerpc JITs implement them. On riscv64 a program using gotox passes verification and then fails to load, because the JIT does not know the opcode and CONFIG_BPF_JIT_ALWAYS_ON leaves no interpreter to fall back onto.

This series adds the riscv64 support and turns the existing selftests on for that architecture.

Patch 1 emits "jalr zero, rd, 0" for gotox and publishes the xlated to jitted offsets through bpf_prog_update_insn_ptrs(), which is what fills in the jump table addresses. Patch 2 widens the arch guard in verifier_gotox.c to riscv64, the same way arm64 and powerpc were enabled.

Changes since v1:
- Rebased onto bpf-next.
- Patch 1: reword the code comment to say "shift ctx->offset[] by one".
- Patch 2: collapse the multi-line #endif marker to a single line.
- Added Reviewed-by/Acked-by from Björn Töpel.

Testing
=======

Environment: QEMU virt rv64, with CONFIG_BPF_JIT=y, CONFIG_BPF_JIT_ALWAYS_ON=y, CONFIG_DEBUG_INFO_BTF=y; selftests cross-built with clang 22.

- test_progs -t verifier_gotox: 27/27 subtests pass on bpf-next, 13 of them executed through BPF_PROG_TEST_RUN.
- test_progs-cpuv4 -t bpf_gotox: 14/14 subtests pass, none skipped. The cpuv4 flavor is needed here because bpf_gotox gates its subtests on __BPF_FEATURE_GOTOX, which clang only defines for -mcpu=v4.

Chen Pei (2):
bpf, riscv: Add support for indirect jumps
selftests/bpf: Enable gotox tests for riscv64

 arch/riscv/net/bpf_jit_comp64.c                  |  5 +++++
 arch/riscv/net/bpf_jit_core.c                    | 16 ++++++++++++++--
 .../testing/selftests/bpf/progs/verifier_gotox.c |  6 ++++--
 3 files changed, 23 insertions(+), 4 deletions(-)
Signature · 1 line
2.50.1
author
#1 lore raw

PATCH v2 1/2 bpf, riscv: Add support for indirect jumps

Implement JIT support for the indirect jump instruction (BPF_JMP | BPF_JA | BPF_X), a.k.a. gotox, which lets a BPF program jump through a BPF_MAP_TYPE_INSN_ARRAY jump table.

Emit "jalr zero, rd, 0" and hand the xlated to jitted offsets to bpf_prog_update_insn_ptrs(), which is what fills in the jump table entries; without that call the load fails with -EFAULT in bpf_insn_array_ready(). ctx->offset[] holds the offset of the insn *following* insn i, as bpf_prog_fill_jited_linfo() expects, so it is shifted by one and offset[0] comes from the prologue length. build_body() now records both halves of a multi-insn record, so no slot keeps a fabricated offset.

Only the RV64 JIT is covered; RV32 keeps failing to load as before.

---

Changes since v1:
- Reword the code comment to say "shift ctx->offset[] by one".

 arch/riscv/net/bpf_jit_comp64.c |  5 +++++
 arch/riscv/net/bpf_jit_core.c   | 16 ++++++++++++++--
 2 files changed, 19 insertions(+), 2 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +5 -0
index ed0a6f871dea..9de3749fb268 100644
@@ -1691,6 +1691,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
 			emit_zextw(rd, rd, ctx);
 		break;
 
+	/* JUMP reg */
+	case BPF_JMP | BPF_JA | BPF_X:
+		emit_jalr(RV_REG_ZERO, rd, 0, ctx);
+		break;
+
 	/* JUMP off */
 	case BPF_JMP | BPF_JA:
 	case BPF_JMP32 | BPF_JA:
arch/riscv/net/bpf_jit_core.c +14 -2
index 470a6ace5662..5265dd5bd39e 100644
@@ -26,10 +26,13 @@ static int build_body(struct rv_jit_context *ctx, bool extra_pass, int *offset)
 		int ret;
 
 		ret = bpf_jit_emit_insn(insn, ctx, extra_pass);
-		if (ret > 0)
-			i++; /* skip the next instruction */
 		if (offset)
 			offset[i] = ctx->ninsns;
+		if (ret > 0) {
+			i++; /* skip the next instruction */
+			if (offset)
+				offset[i] = ctx->ninsns;
+		}
 		if (ret < 0)
 			return ret;
 	}
@@ -176,6 +179,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
 		for (i = 0; i < prog->len; i++)
 			ctx->offset[i] = ninsns_rvoff(ctx->offset[i]);
 		bpf_prog_fill_jited_linfo(prog, ctx->offset);
+
+		/*
+		 * bpf_prog_update_insn_ptrs() wants the start of each insn, so
+		 * shift ctx->offset[] by one and get insn 0 from the prologue.
+		 */
+		for (i = prog->len - 1; i > 0; i--)
+			ctx->offset[i] = ctx->offset[i - 1];
+		ctx->offset[0] = ninsns_rvoff(ctx->prologue_len);
+		bpf_prog_update_insn_ptrs(prog, ctx->offset, jit_data->ro_image);
 out_offset:
 		kvfree(ctx->offset);
 		kfree(jit_data);
Signature · 7 lines
2.50.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
maintainer Reviewed-by on v2 1/2
#2 lore raw
Quoted 69 lines · Chen Pei

Implement JIT support for the indirect jump instruction (BPF_JMP | BPF_JA | BPF_X), a.k.a. gotox, which lets a BPF program jump through a BPF_MAP_TYPE_INSN_ARRAY jump table.

Emit "jalr zero, rd, 0" and hand the xlated to jitted offsets to bpf_prog_update_insn_ptrs(), which is what fills in the jump table entries; without that call the load fails with -EFAULT in bpf_insn_array_ready(). ctx->offset[] holds the offset of the insn *following* insn i, as bpf_prog_fill_jited_linfo() expects, so it is shifted by one and offset[0] comes from the prologue length. build_body() now records both halves of a multi-insn record, so no slot keeps a fabricated offset.

Only the RV64 JIT is covered; RV32 keeps failing to load as before.

---

Changes since v1:
- Reword the code comment to say "shift ctx->offset[] by one".

  arch/riscv/net/bpf_jit_comp64.c |  5 +++++
  arch/riscv/net/bpf_jit_core.c   | 16 ++++++++++++++--
  2 files changed, 19 insertions(+), 2 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +5 -0
index ed0a6f871dea..9de3749fb268 100644
@@ -1691,6 +1691,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
  			emit_zextw(rd, rd, ctx);
  		break;
 
+	/* JUMP reg */
+	case BPF_JMP | BPF_JA | BPF_X:
+		emit_jalr(RV_REG_ZERO, rd, 0, ctx);
+		break;
+
  	/* JUMP off */
  	case BPF_JMP | BPF_JA:
  	case BPF_JMP32 | BPF_JA:
arch/riscv/net/bpf_jit_core.c +7 -2
index 470a6ace5662..5265dd5bd39e 100644
@@ -26,10 +26,13 @@ static int build_body(struct rv_jit_context *ctx, bool extra_pass, int *offset)
  		int ret;
 
  		ret = bpf_jit_emit_insn(insn, ctx, extra_pass);
-		if (ret > 0)
-			i++; /* skip the next instruction */
  		if (offset)
  			offset[i] = ctx->ninsns;
+		if (ret > 0) {
+			i++; /* skip the next instruction */
+			if (offset)
+				offset[i] = ctx->ninsns;
+		}
  		if (ret < 0)
  			return ret;
  	}
@@ -176,6 +179,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
  		for (i = 0; i < prog->len; i++)
  			ctx->offset[i] = ninsns_rvoff(ctx->offset[i]);
  		bpf_prog_fill_jited_linfo(prog, ctx->offset);
+
+		/*

arch/riscv/net/bpf_jit_core.c struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr · line 193

Quoted 10 lines
arch/riscv/net/bpf_jit_core.c +7 -0
+		 * bpf_prog_update_insn_ptrs() wants the start of each insn, so
+		 * shift ctx->offset[] by one and get insn 0 from the prologue.
+		 */
+		for (i = prog->len - 1; i > 0; i--)
+			ctx->offset[i] = ctx->offset[i - 1];
+		ctx->offset[0] = ninsns_rvoff(ctx->prologue_len);
+		bpf_prog_update_insn_ptrs(prog, ctx->offset, jit_data->ro_image);
  out_offset:
  		kvfree(ctx->offset);
  		kfree(jit_data);
author
#3 lore raw

PATCH v2 2/2 selftests/bpf: Enable gotox tests for riscv64

In reply to Chen Pei

The riscv64 JIT now supports the gotox instruction and jump tables, so run the tests in verifier_gotox.c on riscv64 too, mirroring what was done for arm64 and powerpc.

The guard is 64-bit only because the RV32 JIT does not implement gotox.

---

Changes since v1:
- Collapse the multi-line #endif marker to a single line.

 tools/testing/selftests/bpf/progs/verifier_gotox.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/verifier_gotox.c +4 -2
index f5a9878c7b8d..96a9d914930e 100644
@@ -6,7 +6,9 @@
 #include "bpf_misc.h"
 #include "../../../include/linux/filter.h"
 
-#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_powerpc)
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \
+	defined(__TARGET_ARCH_powerpc) || \
+	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64)
 
 #define DEFINE_SIMPLE_JUMP_TABLE_PROG(NAME, SRC_REG, OFF, IMM, OUTCOME)	\
 									\
@@ -580,6 +582,6 @@ nop_%=:								\
 	: __clobber_all);
 }
 
-#endif /* __TARGET_ARCH_x86 || __TARGET_ARCH_arm64 || __TARGET_ARCH_powerpc*/
+#endif /* gotox: x86, arm64, powerpc, riscv64 */
 
 char _license[] SEC("license") = "GPL";
Signature · 1 line
2.50.1
#4 lore raw

Re: [PATCH bpf-next v2 2/2] selftests/bpf: Enable gotox tests for riscv64

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] bpf: missing architecture guards for riscv64 gotox tests --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] bpf: missing architecture guards for riscv64 gotox tests
--

commit 1d33b2e23d145afdb7fdcec7bed2452e5b940956
Author: Chen Pei <[email protected]>

selftests/bpf: Enable gotox tests for riscv64

This commit adds riscv64 to the architecture whitelist in verifier_gotox.c because the JIT now supports the gotox instruction and jump tables.

tools/testing/selftests/bpf/progs/verifier_gotox.c line 11

Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +3 -1
index f5a9878c7b8d4..96a9d914930ee 100644
@@ -6,7 +6,9 @@
 #include "bpf_misc.h"
 #include "../../../include/linux/filter.h"
 
-#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_powerpc)
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \
+	defined(__TARGET_ARCH_powerpc) || \
+	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64)

[Severity: Low]
Are there other files that need their architecture guards updated for riscv64 gotox tests to prevent a regression in test coverage?

For example, tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c has:

#if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__) static int map_create(__u32 map_type, __u32 max_entries)

{
    ...

This skips the bpf_insn_array test suite on riscv64, which heavily exercises the gotox jump table logic.

Similarly, tools/testing/selftests/bpf/progs/compute_live_registers.c has:

#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)

SEC("socket")
__log_level(2)
__msg("2: .1........ (07) r1 += 8")
__msg("3: .1........ (79) r2 = *(u64 *)(r1 +0)")
__msg("4: ..2....... (b7) r3 = 1")
__msg("5: ..23...... (b7) r4 = 2")
__msg("6: ..234..... (0d) gotox r2")
...

This prevents the gotox liveness tracking tests from being compiled and run on riscv64.

Should these architecture guards also be updated to ensure comprehensive test coverage for the newly introduced JIT feature on riscv64?

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
maintainer
#5 lore raw

Re: [PATCH bpf-next v2 2/2] selftests/bpf: Enable gotox tests for riscv64

Quoted 52 lines · On 2026/9/28 10:31, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] bpf: missing architecture guards for riscv64 gotox tests
--

commit 1d33b2e23d145afdb7fdcec7bed2452e5b940956
Author: Chen Pei <[email protected]>

selftests/bpf: Enable gotox tests for riscv64

This commit adds riscv64 to the architecture whitelist in verifier_gotox.c because the JIT now supports the gotox instruction and jump tables.

tools/testing/selftests/bpf/progs/verifier_gotox.c line 11

Quoted 12 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +3 -1
index f5a9878c7b8d4..96a9d914930ee 100644
@@ -6,7 +6,9 @@
  #include "bpf_misc.h"
  #include "../../../include/linux/filter.h"
 
-#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_powerpc)
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \
+	defined(__TARGET_ARCH_powerpc) || \
+	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64)

[Severity: Low]
Are there other files that need their architecture guards updated for riscv64 gotox tests to prevent a regression in test coverage?

For example, tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c has:

#if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__) static int map_create(__u32 map_type, __u32 max_entries)

{
     ...

This skips the bpf_insn_array test suite on riscv64, which heavily exercises the gotox jump table logic.

Similarly, tools/testing/selftests/bpf/progs/compute_live_registers.c has:

#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)

SEC("socket")
__log_level(2)
__msg("2: .1........ (07) r1 += 8")
__msg("3: .1........ (79) r2 = *(u64 *)(r1 +0)")
__msg("4: ..2....... (b7) r3 = 1")
__msg("5: ..23...... (b7) r4 = 2")
__msg("6: ..234..... (0d) gotox r2")
...

Quoted 8 lines

This prevents the gotox liveness tracking tests from being compiled and run on riscv64.

Should these architecture guards also be updated to ensure comprehensive test coverage for the newly introduced JIT feature on riscv64?

[ ... ]

yah, should go with bpf_insn_array and compute_live_registers testcases

author
#6 lore raw

Re: [PATCH bpf-next v2 2/2] selftests/bpf: Enable gotox tests for riscv64

Hi Lehui,

Thanks for the review.

Quoted 1 line

yah, should go with bpf_insn_array and compute_live_registers testcases

Agreed. I've enabled both testcases for riscv64 and they pass in my local testing. I'll include them in v3.

Best regards,
Pei