bpf: arm64: Indirect jumps
Subsystems (5)
Puranjay Mohan
maintainerv2 cover bpf: arm64: Indirect jumps
Changes in v1->v2:
v1: https://lore.kernel.org/all/[email protected]/
- Dropped patch 3 that was ignoring relocations for .jumptables. LLVM has been fixed to not emit relocations for .jumptables, so this patch is not needed.
- Added Reviewed-by: Anton Protopopov <[email protected]>
This set adds the support of indirect jumps to the arm64 JIT. It involves calling bpf_prog_update_insn_ptrs() to support instructions array map. The second piece is supporting BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0 instruction that is trivial to implement on arm64.
The final patch enables selftests on arm64:
[root@localhost bpf]# ./test_progs-cpuv4 -a "*gotox*"
#20/1 bpf_gotox/one-switch:OK #20/2 bpf_gotox/one-switch-non-zero-sec-offset:OK #20/3 bpf_gotox/two-switches:OK #20/4 bpf_gotox/big-jump-table:OK #20/5 bpf_gotox/static-global:OK #20/6 bpf_gotox/nonstatic-global:OK #20/7 bpf_gotox/other-sec:OK #20/8 bpf_gotox/static-global-other-sec:OK #20/9 bpf_gotox/nonstatic-global-other-sec:OK #20/10 bpf_gotox/one-jump-two-maps:OK #20/11 bpf_gotox/one-map-two-jumps:OK #20 bpf_gotox:OK #537/1 verifier_gotox/jump_table_ok:OK #537/2 verifier_gotox/jump_table_reserved_field_src_reg:OK #537/3 verifier_gotox/jump_table_reserved_field_non_zero_off:OK #537/4 verifier_gotox/jump_table_reserved_field_non_zero_imm:OK #537/5 verifier_gotox/jump_table_no_jump_table:OK #537/6 verifier_gotox/jump_table_incorrect_dst_reg_type:OK #537/7 verifier_gotox/jump_table_invalid_read_size_u32:OK #537/8 verifier_gotox/jump_table_invalid_read_size_u16:OK #537/9 verifier_gotox/jump_table_invalid_read_size_u8:OK #537/10 verifier_gotox/jump_table_misaligned_access:OK #537/11 verifier_gotox/jump_table_invalid_mem_acceess_pos:OK #537/12 verifier_gotox/jump_table_invalid_mem_acceess_neg:OK #537/13 verifier_gotox/jump_table_add_sub_ok:OK #537/14 verifier_gotox/jump_table_no_writes:OK #537/15 verifier_gotox/jump_table_use_reg_r0:OK #537/16 verifier_gotox/jump_table_use_reg_r1:OK #537/17 verifier_gotox/jump_table_use_reg_r2:OK #537/18 verifier_gotox/jump_table_use_reg_r3:OK #537/19 verifier_gotox/jump_table_use_reg_r4:OK #537/20 verifier_gotox/jump_table_use_reg_r5:OK #537/21 verifier_gotox/jump_table_use_reg_r6:OK #537/22 verifier_gotox/jump_table_use_reg_r7:OK #537/23 verifier_gotox/jump_table_use_reg_r8:OK #537/24 verifier_gotox/jump_table_use_reg_r9:OK #537/25 verifier_gotox/jump_table_outside_subprog:OK #537/26 verifier_gotox/jump_table_contains_non_unique_values:OK #537 verifier_gotox:OK
Summary: 2/37 PASSED, 0 SKIPPED, 0 FAILED
Puranjay Mohan (3):
bpf: arm64: Add support for instructions array
bpf: arm64: Add support for indirect jumps
selftests: bpf: Enable gotox tests from arm64
arch/arm64/net/bpf_jit_comp.c | 11 +++++++++++ tools/testing/selftests/bpf/progs/verifier_gotox.c | 4 ++-- 2 files changed, 13 insertions(+), 2 deletions(-)
Signature · 1 line
2.47.1
Puranjay Mohan
maintainerPATCH v2 1/3 bpf: arm64: Add support for instructions array
Add support for the instructions array map type in the arm64 JIT by calling bpf_prog_update_insn_ptrs() with the offsets that map xlated_offset to the jited_offset in the final image. arm64 JIT already has this offset array which was being used for bpf_prog_fill_jited_linfo() and can be used directly for bpf_prog_update_insn_ptrs.
--- arch/arm64/net/bpf_jit_comp.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/net/bpf_jit_comp.c +7 -0
@@ -2231,6 +2231,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)for (i =0 ; i <= prog->len; i++) ctx.offset[i] *=AARCH64_INSN_SIZE ;bpf_prog_fill_jited_linfo (prog, ctx.offset +1 ); +/* +* The bpf_prog_update_insn_ptrs function expects offsets to +* point to the first byte of the jitted instruction (unlike +* the bpf_prog_fill_jited_linfo above, which, for historical +* reasons, expects to point to the next instruction) +*/ +bpf_prog_update_insn_ptrs (prog, ctx.offset, ctx.ro_image); out_off:if (!ro_header && priv_stack_ptr) {free_percpu (priv_stack_ptr);
Signature · 1 line
2.47.1
Re: [PATCH bpf-next v2 1/3] bpf: arm64: Add support for instructions array
Quoted 21 lines · Puranjay Mohan
Add support for the instructions array map type in the arm64 JIT by calling
bpf_prog_update_insn_ptrs()with the offsets that mapxlated_offsetto thejited_offsetin the final image. arm64 JIT already has this offset array which was being used forbpf_prog_fill_jited_linfo()and can be used directly forbpf_prog_update_insn_ptrs.Signed-off-by Puranjay Mohan <[email protected]>Reviewed-by Anton Protopopov <[email protected]>--- arch/arm64/net/bpf_jit_comp.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/net/bpf_jit_comp.c +0
-0@@ -2231,6 +2231,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)for (i =0 ; i <= prog->len; i++) ctx.offset[i] *=AARCH64_INSN_SIZE ;bpf_prog_fill_jited_linfo (prog, ctx.offset +1 );
arch/arm64/net/bpf_jit_comp.c struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) · line 2243
Quoted 10 lines
arch/arm64/net/bpf_jit_comp.c +7
-0+/* +* The bpf_prog_update_insn_ptrs function expects offsets to +* point to the first byte of the jitted instruction (unlike +* the bpf_prog_fill_jited_linfo above, which, for historical +* reasons, expects to point to the next instruction) +*/ +bpf_prog_update_insn_ptrs (prog, ctx.offset, ctx.ro_image); out_off:if (!ro_header && priv_stack_ptr) {free_percpu (priv_stack_ptr);
Puranjay Mohan
maintainerPATCH v2 2/3 bpf: arm64: Add support for indirect jumps
In reply to Puranjay MohanAdd support for a new instruction
BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0
which does an indirect jump to a location stored in Rx. The register Rx should have type PTR_TO_INSN. This new type assures that the Rx register contains a value (or a range of values) loaded from a correct jump table – map of type instruction array.
ARM64 JIT supports indirect jumps to all registers through the A64_BR() macro, use it to implement this new instruction.
--- arch/arm64/net/bpf_jit_comp.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/net/bpf_jit_comp.c +4 -0
@@ -1452,6 +1452,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,emit (A64_ASR (is64, dst, dst, imm), ctx);break ; +/* JUMP reg */ +case BPF_JMP |BPF_JA |BPF_X : +emit (A64_BR (dst), ctx); +break ;/* JUMP off */ case BPF_JMP |BPF_JA :case BPF_JMP32 |BPF_JA :
Signature · 1 line
2.47.1
Re: [PATCH bpf-next v2 2/3] bpf: arm64: Add support for indirect jumps
Quoted 23 lines · Puranjay Mohan
Add support for a new instruction
BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0which does an indirect jump to a location stored in Rx. The register Rx should have type
PTR_TO_INSN. This new type assures that the Rx register contains a value (or a range of values) loaded from a correct jump table – map of type instruction array.ARM64 JIT supports indirect jumps to all registers through the
A64_BR()macro, use it to implement this new instruction.Signed-off-by Puranjay Mohan <[email protected]>Reviewed-by Anton Protopopov <[email protected]>--- arch/arm64/net/bpf_jit_comp.c | 4 ++++ 1 file changed, 4 insertions(+)arch/arm64/net/bpf_jit_comp.c +0
-0
arch/arm64/net/bpf_jit_comp.c static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · line 1461
Quoted 10 lines
arch/arm64/net/bpf_jit_comp.c +4
-0@@ -1452,6 +1452,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,emit (A64_ASR (is64, dst, dst, imm), ctx);break ; +/* JUMP reg */ +case BPF_JMP |BPF_JA |BPF_X : +emit (A64_BR (dst), ctx); +break ;/* JUMP off */ case BPF_JMP |BPF_JA :case BPF_JMP32 |BPF_JA :
Puranjay Mohan
maintainerPATCH v2 3/3 selftests: bpf: Enable gotox tests from arm64
In reply to Puranjay Mohanarm64 JIT now supports gotox instruction and jumptables, so run tests in verifier_gotox.c for arm64.
--- tools/testing/selftests/bpf/progs/verifier_gotox.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/verifier_gotox.c +2 -2
@@ -6,7 +6,7 @@#include "bpf_misc.h" #include "../../../include/linux/filter.h" -#ifdef __TARGET_ARCH_x86 +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)#define DEFINE_SIMPLE_JUMP_TABLE_PROG (NAME ,SRC_REG ,OFF ,IMM ,OUTCOME ) \ \
@@ -384,6 +384,6 @@ jt0_%=: \ : __clobber_all); } -#endif /* __TARGET_ARCH_x86 */ +#endif /* __TARGET_ARCH_x86 || __TARGET_ARCH_arm64 */ char _license[] SEC("license") = "GPL";
Signature · 1 line
2.47.1
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Mon, 17 Nov 2025 13:07:28 +0000 you wrote:
Changes in
v1->v2:
v1: https://lore.kernel.org/all/[email protected]/
- Dropped patch 3 that was ignoring relocations for .jumptables. LLVM has been fixed to not emit relocations for .jumptables, so this patch is not needed.
- Added Reviewed-by: Anton Protopopov <[email protected]>[...]
Here is the summary with links:
- [bpf-next,v2,1/3] bpf: arm64: Add support for instructions array
https://git.kernel.org/bpf/bpf-next/c/84b1c40d5b4d - [bpf-next,v2,2/3] bpf: arm64: Add support for indirect jumps https://git.kernel.org/bpf/bpf-next/c/f4a66cf1cb14 - [bpf-next,v2,3/3] selftests: bpf: Enable gotox tests from arm64 https://git.kernel.org/bpf/bpf-next/c/d8774a36235e
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html