Skip to contentSkip to conversation

bpf: arm64: Indirect jumps

PATCH Superseded Puranjay Mohan 2 participants 10 messages Nov 17, 2025

Subsystems (6)
Kselftest BPF arm64 BPF selftests +2 more subsystems

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Puranjay Mohan

author maintainer
#0 lore raw

cover bpf: arm64: Indirect jumps

This set adds the support of indirect jumps to the arm64 JIT. It involves calling bpf_prog_update_insn_ptrs() to support instructions array map. The second piece is supporting BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0 instruction that is trivial to implement on arm64.

When running the selftests after doing the above changes, I found that on arm64 builds of llvm, a relocation section was being generated for .jumptables sections and it was making libbpf fail like:

libbpf: relocation against STT_SECTION in non-exec section is not supported!
Error: failed to link 'tools/testing/selftests/bpf/cpuv4/bpf_gotox.bpf.o': Invalid argument (22)

Which is due to:

Relocation section '.rel.jumptables' at offset 0x5b50 contains 263 entries:
Offset Info Type Symbol's Value Symbol's Name

0000000000000000  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall
0000000000000008  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall
0000000000000010  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall

This rel section is not generated by x86 builds of LLVM. The third patch of this set makes libbpf ignore relocation sections for .jumptables.

The final patch enables selftests on arm64:

[root@localhost bpf]# ./test_progs-cpuv4 -a "*gotox*"

 #20/1    bpf_gotox/one-switch:OK
 #20/2    bpf_gotox/one-switch-non-zero-sec-offset:OK
 #20/3    bpf_gotox/two-switches:OK
 #20/4    bpf_gotox/big-jump-table:OK
 #20/5    bpf_gotox/static-global:OK
 #20/6    bpf_gotox/nonstatic-global:OK
 #20/7    bpf_gotox/other-sec:OK
 #20/8    bpf_gotox/static-global-other-sec:OK
 #20/9    bpf_gotox/nonstatic-global-other-sec:OK
 #20/10   bpf_gotox/one-jump-two-maps:OK
 #20/11   bpf_gotox/one-map-two-jumps:OK
 #20      bpf_gotox:OK
 #537/1   verifier_gotox/jump_table_ok:OK
 #537/2   verifier_gotox/jump_table_reserved_field_src_reg:OK
 #537/3   verifier_gotox/jump_table_reserved_field_non_zero_off:OK
 #537/4   verifier_gotox/jump_table_reserved_field_non_zero_imm:OK
 #537/5   verifier_gotox/jump_table_no_jump_table:OK
 #537/6   verifier_gotox/jump_table_incorrect_dst_reg_type:OK
 #537/7   verifier_gotox/jump_table_invalid_read_size_u32:OK
 #537/8   verifier_gotox/jump_table_invalid_read_size_u16:OK
 #537/9   verifier_gotox/jump_table_invalid_read_size_u8:OK
 #537/10  verifier_gotox/jump_table_misaligned_access:OK
 #537/11  verifier_gotox/jump_table_invalid_mem_acceess_pos:OK
 #537/12  verifier_gotox/jump_table_invalid_mem_acceess_neg:OK
 #537/13  verifier_gotox/jump_table_add_sub_ok:OK
 #537/14  verifier_gotox/jump_table_no_writes:OK
 #537/15  verifier_gotox/jump_table_use_reg_r0:OK
 #537/16  verifier_gotox/jump_table_use_reg_r1:OK
 #537/17  verifier_gotox/jump_table_use_reg_r2:OK
 #537/18  verifier_gotox/jump_table_use_reg_r3:OK
 #537/19  verifier_gotox/jump_table_use_reg_r4:OK
 #537/20  verifier_gotox/jump_table_use_reg_r5:OK
 #537/21  verifier_gotox/jump_table_use_reg_r6:OK
 #537/22  verifier_gotox/jump_table_use_reg_r7:OK
 #537/23  verifier_gotox/jump_table_use_reg_r8:OK
 #537/24  verifier_gotox/jump_table_use_reg_r9:OK
 #537/25  verifier_gotox/jump_table_outside_subprog:OK
 #537/26  verifier_gotox/jump_table_contains_non_unique_values:OK
 #537     verifier_gotox:OK

Summary: 2/37 PASSED, 0 SKIPPED, 0 FAILED

Puranjay Mohan (4):
bpf: arm64: Add support for instructions array
bpf: arm64: Add support for indirect jumps
libbpf: Ignore relocations for .jumptables sections
selftests: bpf: Enable gotox tests from arm64

 arch/arm64/net/bpf_jit_comp.c                      | 11 +++++++++++
 tools/lib/bpf/linker.c                             |  4 ++++
 tools/testing/selftests/bpf/progs/verifier_gotox.c |  4 ++--
 3 files changed, 17 insertions(+), 2 deletions(-)
Signature · 1 line
2.47.3
author maintainer
#1 lore raw

PATCH 1/4 bpf: arm64: Add support for instructions array

Add support for the instructions array map type in the arm64 JIT by calling bpf_prog_update_insn_ptrs() with the offsets that map xlated_offset to the jited_offset in the final image. arm64 JIT already has this offset array which was being used for bpf_prog_fill_jited_linfo() and can be used directly for bpf_prog_update_insn_ptrs.

---
 arch/arm64/net/bpf_jit_comp.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/net/bpf_jit_comp.c +7 -0
index 0c9a50a1e73e..4a2afc0cefc4 100644
@@ -2231,6 +2231,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
 		for (i = 0; i <= prog->len; i++)
 			ctx.offset[i] *= AARCH64_INSN_SIZE;
 		bpf_prog_fill_jited_linfo(prog, ctx.offset + 1);
+		/*
+		 * The bpf_prog_update_insn_ptrs function expects offsets to
+		 * point to the first byte of the jitted instruction (unlike
+		 * the bpf_prog_fill_jited_linfo above, which, for historical
+		 * reasons, expects to point to the next instruction)
+		 */
+		bpf_prog_update_insn_ptrs(prog, ctx.offset, ctx.ro_image);
 out_off:
 		if (!ro_header && priv_stack_ptr) {
 			free_percpu(priv_stack_ptr);
Signature · 1 line
2.47.3
author maintainer
#3 lore raw

PATCH 2/4 bpf: arm64: Add support for indirect jumps

In reply to Puranjay Mohan

Add support for a new instruction

	BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0

which does an indirect jump to a location stored in Rx. The register Rx should have type PTR_TO_INSN. This new type assures that the Rx register contains a value (or a range of values) loaded from a correct jump table – map of type instruction array.

ARM64 JIT supports indirect jumps to all registers through the A64_BR() macro, use it to implement this new instruction.

---
 arch/arm64/net/bpf_jit_comp.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/arm64/net/bpf_jit_comp.c +4 -0
index 4a2afc0cefc4..4cfb549f2b43 100644
@@ -1452,6 +1452,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
 		emit(A64_ASR(is64, dst, dst, imm), ctx);
 		break;
 
+	/* JUMP reg */
+	case BPF_JMP | BPF_JA | BPF_X:
+		emit(A64_BR(dst), ctx);
+		break;
 	/* JUMP off */
 	case BPF_JMP | BPF_JA:
 	case BPF_JMP32 | BPF_JA:
Signature · 1 line
2.47.3
author maintainer
#5 lore raw

PATCH 3/4 libbpf: Ignore relocations for .jumptables sections

In reply to Puranjay Mohan

.jumptables sections generated by LLVM store byte offsets from the start of the section to the jump's target instruction. On arm64 builds of LLVM, it generates a rel section to add the section address to each entry in the jump table, therefore storing the absolute address of the jump target into the table. For BPF usage of .jumptables, we only care about the instruction offsets and calculate the final address much later after the JIT has finished, and called bpf_prog_update_insn_ptrs()

Without the change in this commit, compilation fails with:

libbpf: relocation against STT_SECTION in non-exec section is not supported! Error: failed to link 'tools/testing/selftests/bpf/cpuv4/bpf_gotox.bpf.o': Invalid argument (22)

This is due to the presence of a relocation section for .jumptables (non-exec) against syscall (STT_SECTION):

Relocation section '.rel.jumptables' at offset 0x5b50 contains 263 entries:
Offset Info Type Symbol's Value Symbol's Name

000000000000  0000000300000002 R_BPF_64_ABS64  0000000000000000 syscall
000000000008  0000000300000002 R_BPF_64_ABS64  0000000000000000 syscall

[...]

---
 tools/lib/bpf/linker.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/lib/bpf/linker.c +4 -0
index f4403e3cf994..f4295962983b 100644
@@ -634,6 +634,10 @@ static bool is_ignored_sec(struct src_sec *sec)
 		if (strcmp(name, BTF_ELF_SEC) == 0 ||
 		    strcmp(name, BTF_EXT_ELF_SEC) == 0)
 			return true;
+
+		/* .jumptables don't need relocations */
+		if (strcmp(name, JUMPTABLES_SEC) == 0)
+			return true;
 	}
 
 	return false;
Signature · 1 line
2.47.3
author maintainer
#6 lore raw

PATCH 4/4 selftests: bpf: Enable gotox tests from arm64

In reply to Puranjay Mohan

arm64 JIT now supports gotox instruction and jumptables, so run tests in verifier_gotox.c for arm64.

---
 tools/testing/selftests/bpf/progs/verifier_gotox.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/verifier_gotox.c +2 -2
index b6710f134a1d..536c9f3e2170 100644
@@ -6,7 +6,7 @@
 #include "bpf_misc.h"
 #include "../../../include/linux/filter.h"
 
-#ifdef __TARGET_ARCH_x86
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)
 
 #define DEFINE_SIMPLE_JUMP_TABLE_PROG(NAME, SRC_REG, OFF, IMM, OUTCOME)	\
 									\
@@ -384,6 +384,6 @@ jt0_%=:								\
 	: __clobber_all);
 }
 
-#endif /* __TARGET_ARCH_x86 */
+#endif /* __TARGET_ARCH_x86 || __TARGET_ARCH_arm64 */
 
 char _license[] SEC("license") = "GPL";
Signature · 1 line
2.47.3
#8 lore raw
In reply to Puranjay Mohan
Quoted 14 lines · Puranjay Mohan

This set adds the support of indirect jumps to the arm64 JIT. It involves calling bpf_prog_update_insn_ptrs() to support instructions array map. The second piece is supporting BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0 instruction that is trivial to implement on arm64.

When running the selftests after doing the above changes, I found that on arm64 builds of llvm, a relocation section was being generated for .jumptables sections and it was making libbpf fail like:

libbpf: relocation against STT_SECTION in non-exec section is not supported!
Error: failed to link 'tools/testing/selftests/bpf/cpuv4/bpf_gotox.bpf.o': Invalid argument (22)

Which is due to:

Quoted 8 lines

Relocation section '.rel.jumptables' at offset 0x5b50 contains 263 entries:
Offset Info Type Symbol's Value Symbol's Name

0000000000000000  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall
0000000000000008  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall
0000000000000010  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall

This rel section is not generated by x86 builds of LLVM. The third patch of this set makes libbpf ignore relocation sections for .jumptables.

I added Yonghong to this thread. He had fixed this problem in
https://github.com/llvm/llvm-project/pull/166301 changes doesn't seem to be x86-specific...

Quoted 43 lines

The final patch enables selftests on arm64:

[root@localhost bpf]# ./test_progs-cpuv4 -a "*gotox*"

 #20/1    bpf_gotox/one-switch:OK
 #20/2    bpf_gotox/one-switch-non-zero-sec-offset:OK
 #20/3    bpf_gotox/two-switches:OK
 #20/4    bpf_gotox/big-jump-table:OK
 #20/5    bpf_gotox/static-global:OK
 #20/6    bpf_gotox/nonstatic-global:OK
 #20/7    bpf_gotox/other-sec:OK
 #20/8    bpf_gotox/static-global-other-sec:OK
 #20/9    bpf_gotox/nonstatic-global-other-sec:OK
 #20/10   bpf_gotox/one-jump-two-maps:OK
 #20/11   bpf_gotox/one-map-two-jumps:OK
 #20      bpf_gotox:OK
 #537/1   verifier_gotox/jump_table_ok:OK
 #537/2   verifier_gotox/jump_table_reserved_field_src_reg:OK
 #537/3   verifier_gotox/jump_table_reserved_field_non_zero_off:OK
 #537/4   verifier_gotox/jump_table_reserved_field_non_zero_imm:OK
 #537/5   verifier_gotox/jump_table_no_jump_table:OK
 #537/6   verifier_gotox/jump_table_incorrect_dst_reg_type:OK
 #537/7   verifier_gotox/jump_table_invalid_read_size_u32:OK
 #537/8   verifier_gotox/jump_table_invalid_read_size_u16:OK
 #537/9   verifier_gotox/jump_table_invalid_read_size_u8:OK
 #537/10  verifier_gotox/jump_table_misaligned_access:OK
 #537/11  verifier_gotox/jump_table_invalid_mem_acceess_pos:OK
 #537/12  verifier_gotox/jump_table_invalid_mem_acceess_neg:OK
 #537/13  verifier_gotox/jump_table_add_sub_ok:OK
 #537/14  verifier_gotox/jump_table_no_writes:OK
 #537/15  verifier_gotox/jump_table_use_reg_r0:OK
 #537/16  verifier_gotox/jump_table_use_reg_r1:OK
 #537/17  verifier_gotox/jump_table_use_reg_r2:OK
 #537/18  verifier_gotox/jump_table_use_reg_r3:OK
 #537/19  verifier_gotox/jump_table_use_reg_r4:OK
 #537/20  verifier_gotox/jump_table_use_reg_r5:OK
 #537/21  verifier_gotox/jump_table_use_reg_r6:OK
 #537/22  verifier_gotox/jump_table_use_reg_r7:OK
 #537/23  verifier_gotox/jump_table_use_reg_r8:OK
 #537/24  verifier_gotox/jump_table_use_reg_r9:OK
 #537/25  verifier_gotox/jump_table_outside_subprog:OK
 #537/26  verifier_gotox/jump_table_contains_non_unique_values:OK
 #537     verifier_gotox:OK
Quoted 1 line

Summary: 2/37 PASSED, 0 SKIPPED, 0 FAILED

Cool!

Quoted 14 lines

Puranjay Mohan (4):
bpf: arm64: Add support for instructions array
bpf: arm64: Add support for indirect jumps
libbpf: Ignore relocations for .jumptables sections
selftests: bpf: Enable gotox tests from arm64

 arch/arm64/net/bpf_jit_comp.c                      | 11 +++++++++++
 tools/lib/bpf/linker.c                             |  4 ++++
 tools/testing/selftests/bpf/progs/verifier_gotox.c |  4 ++--
 3 files changed, 17 insertions(+), 2 deletions(-)
Signature · 1 line
2.47.3
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH bpf-next 1/4] bpf: arm64: Add support for instructions array

In reply to Puranjay Mohan
Quoted 33 lines · Puranjay Mohan

Add support for the instructions array map type in the arm64 JIT by calling bpf_prog_update_insn_ptrs() with the offsets that map xlated_offset to the jited_offset in the final image. arm64 JIT already has this offset array which was being used for bpf_prog_fill_jited_linfo() and can be used directly for bpf_prog_update_insn_ptrs.

---
 arch/arm64/net/bpf_jit_comp.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/net/bpf_jit_comp.c +7 -0
index 0c9a50a1e73e..4a2afc0cefc4 100644
@@ -2231,6 +2231,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
 		for (i = 0; i <= prog->len; i++)
 			ctx.offset[i] *= AARCH64_INSN_SIZE;
 		bpf_prog_fill_jited_linfo(prog, ctx.offset + 1);
+		/*
+		 * The bpf_prog_update_insn_ptrs function expects offsets to
+		 * point to the first byte of the jitted instruction (unlike
+		 * the bpf_prog_fill_jited_linfo above, which, for historical
+		 * reasons, expects to point to the next instruction)
+		 */
+		bpf_prog_update_insn_ptrs(prog, ctx.offset, ctx.ro_image);
 out_off:
 		if (!ro_header && priv_stack_ptr) {
 			free_percpu(priv_stack_ptr);
Signature · 1 line
2.47.3
Reviewed-by on v1 2/4
#4 lore raw

Re: [PATCH bpf-next 2/4] bpf: arm64: Add support for indirect jumps

In reply to Puranjay Mohan
Quoted 35 lines · Puranjay Mohan

Add support for a new instruction

	BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0

which does an indirect jump to a location stored in Rx. The register Rx should have type PTR_TO_INSN. This new type assures that the Rx register contains a value (or a range of values) loaded from a correct jump table – map of type instruction array.

ARM64 JIT supports indirect jumps to all registers through the A64_BR() macro, use it to implement this new instruction.

---
 arch/arm64/net/bpf_jit_comp.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/arm64/net/bpf_jit_comp.c +4 -0
index 4a2afc0cefc4..4cfb549f2b43 100644
@@ -1452,6 +1452,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
 		emit(A64_ASR(is64, dst, dst, imm), ctx);
 		break;
 
+	/* JUMP reg */
+	case BPF_JMP | BPF_JA | BPF_X:
+		emit(A64_BR(dst), ctx);
+		break;
 	/* JUMP off */
 	case BPF_JMP | BPF_JA:
 	case BPF_JMP32 | BPF_JA:
Signature · 1 line
2.47.3
Reviewed-by on v1 4/4
#7 lore raw

Re: [PATCH bpf-next 4/4] selftests: bpf: Enable gotox tests from arm64

In reply to Puranjay Mohan
Quoted 13 lines · Puranjay Mohan

arm64 JIT now supports gotox instruction and jumptables, so run tests in verifier_gotox.c for arm64.

---
 tools/testing/selftests/bpf/progs/verifier_gotox.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/verifier_gotox.c +0 -0
index b6710f134a1d..536c9f3e2170 100644

tools/testing/selftests/bpf/progs/verifier_gotox.c line 9

Quoted 5 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +1 -1
@@ -6,7 +6,7 @@
 #include "bpf_misc.h"
 #include "../../../include/linux/filter.h"
 
-#ifdef __TARGET_ARCH_x86
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)

Great to see this :)

Quoted 13 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +1 -1
 #define DEFINE_SIMPLE_JUMP_TABLE_PROG(NAME, SRC_REG, OFF, IMM, OUTCOME)	\
 									\
@@ -384,6 +384,6 @@ jt0_%=:								\
 	: __clobber_all);
 }
 
-#endif /* __TARGET_ARCH_x86 */
+#endif /* __TARGET_ARCH_x86 || __TARGET_ARCH_arm64 */
 
 char _license[] SEC("license") = "GPL";
Signature · 1 line
2.47.3
author maintainer
#9 lore raw
In reply to Anton Protopopov
Quoted 21 lines · Anton Protopopov
Quoted 20 lines · Puranjay Mohan

This set adds the support of indirect jumps to the arm64 JIT. It involves calling bpf_prog_update_insn_ptrs() to support instructions array map. The second piece is supporting BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0 instruction that is trivial to implement on arm64.

When running the selftests after doing the above changes, I found that on arm64 builds of llvm, a relocation section was being generated for .jumptables sections and it was making libbpf fail like:

libbpf: relocation against STT_SECTION in non-exec section is not supported!
Error: failed to link 'tools/testing/selftests/bpf/cpuv4/bpf_gotox.bpf.o': Invalid argument (22)

Which is due to:

Relocation section '.rel.jumptables' at offset 0x5b50 contains 263 entries:
Offset Info Type Symbol's Value Symbol's Name

0000000000000000  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall
0000000000000008  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall
0000000000000010  0000000300000002 R_BPF_64_ABS64         0000000000000000 syscall
Quoted 6 lines
Quoted 2 lines

This rel section is not generated by x86 builds of LLVM. The third patch of this set makes libbpf ignore relocation sections for .jumptables.

I added Yonghong to this thread. He had fixed this problem in
https://github.com/llvm/llvm-project/pull/166301 changes doesn't seem to be x86-specific...

My arm64 build didn't have this change, that is why it was failing. After pulling the latest changes and building again, it works without the libbpf patch.

So, I will send v2 and drop the libbpf patch.

Thanks,
Puranjay