bpf: Allow terminal gotox instructions
Subsystems (4)
PATCH v4 1/2 bpf: Allow terminal gotox instructions
check_subprogs() treats gotox as a direct jump and validates its reserved zero offset. When gotox is the final instruction, this produces a synthetic successor one instruction past the end of the subprogram and rejects an otherwise valid program.
Skip direct-offset validation for gotox and accept it as a non-fallthrough terminal instruction. Its actual targets remain validated from the instruction-array jump table during CFG construction.
--- kernel/bpf/verifier.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +4 -1
@@ -3088,6 +3088,8 @@ static int check_subprogs(struct bpf_verifier_env *env) subprog[cur_subprog].exit_idx = i; goto next; } +if (insn_is_gotox (&insn[i])) + goto next; off = i +bpf_jmp_offset (&insn[i]) +1 ;if (off < subprog_start || off >= subprog_end) {verbose (env,"jump out of range from insn %d to %d\n" , i, off);
@@ -3107,7 +3109,8 @@ static int check_subprogs(struct bpf_verifier_env *env)*/ if (code != (BPF_JMP |BPF_EXIT ) && code != (BPF_JMP32 |BPF_JA ) && - code != (BPF_JMP |BPF_JA )) { + code != (BPF_JMP |BPF_JA ) && + !insn_is_gotox (&insn[i])) {verbose (env,"last insn is not an exit or jmp\n" ); bpf_diag_program_structure( env, i, "subprogram can fall through",
base-commit: d761934c9483ecde93fe99d8705282f716dfee50
Signature · 1 line
2.43.0
PATCH v4 2/2 selftests/bpf: Test terminal gotox instructions
Add tests that place gotox at the end of the main program and a subprogram, with each jump-table target preceding the gotox instruction. This tests gotox as a valid non-fallthrough terminal instruction.
--- .../selftests/bpf/progs/verifier_gotox.c | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+)
tools/testing/selftests/bpf/progs/verifier_gotox.c +48 -0
@@ -47,6 +47,54 @@ DEFINE_SIMPLE_JUMP_TABLE_PROG(reserved_field_src_reg, BPF_REG_1, 0, 0, __fa DEFINE_SIMPLE_JUMP_TABLE_PROG(reserved_field_non_zero_off, BPF_REG_0, 1, 0, __failure __msg("BPF_JA|BPF_X uses reserved fields")) DEFINE_SIMPLE_JUMP_TABLE_PROG(reserved_field_non_zero_imm, BPF_REG_0, 0, 1, __failure __msg("BPF_JA|BPF_X uses reserved fields")) +#define DEFINE_TERMINAL_GOTOX_PROG (NAME ,BASE ) \ + __naked void NAME(void) \ + { \ + asm volatile (" \ + .pushsection .jumptables,\"\",@progbits; \ +jt0_%=: \ + .quad ret0_%= - " BASE "; \ + .size jt0_%=, 8; \ + .global jt0_%=; \ + .popsection; \ + \ + r0 = jt0_%= ll; \ + r0 = *(u64 *)(r0 + 0); \ + goto end_%=; \ +ret0_%=: \ + r0 = 0; \ + exit; \ +end_%=: \ + .8byte %[gotox_r0]; \ +" : \ + : __imm_insn(gotox_r0, BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, \ + BPF_REG_0, 0, 0, 0)) \ + : __clobber_all); \ + } + +SEC("socket") +__success __retval(0) +DEFINE_TERMINAL_GOTOX_PROG(jump_table_terminal_gotox, "socket") + +static __noinline __used +DEFINE_TERMINAL_GOTOX_PROG(terminal_gotox_subprog1, ".text") + +static __noinline__used intterminal_gotox_subprog2 (void) +{ +return 0 ; +} + +SEC("socket") +__success __retval(0) +__naked void jump_table_terminal_gotox_subprog(void) +{ + asmvolatile (" \ +call terminal_gotox_subprog1; \ +call terminal_gotox_subprog2; \ +exit; \ +" ::: __clobber_all); +} +/* * Gotox is forbidden when there is no jump table loaded * which points to the sub-function where the gotox is used
Signature · 1 line
2.43.0
Re: [PATCH bpf-next v4 2/2] selftests/bpf: Test terminal gotox instructions
Quoted 43 lines · Siddharth Chintamaneni
Add tests that place gotox at the end of the main program and a subprogram, with each jump-table target preceding the gotox instruction. This tests gotox as a valid non-fallthrough terminal instruction.
Signed-off-by Siddharth Chintamaneni <[email protected]>--- .../selftests/bpf/progs/verifier_gotox.c | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+)tools/testing/selftests/bpf/progs/verifier_gotox.c +26
-0@@ -47,6 +47,54 @@ DEFINE_SIMPLE_JUMP_TABLE_PROG(reserved_field_src_reg, BPF_REG_1, 0, 0, __fa DEFINE_SIMPLE_JUMP_TABLE_PROG(reserved_field_non_zero_off, BPF_REG_0, 1, 0, __failure __msg("BPF_JA|BPF_X uses reserved fields")) DEFINE_SIMPLE_JUMP_TABLE_PROG(reserved_field_non_zero_imm, BPF_REG_0, 0, 1, __failure __msg("BPF_JA|BPF_X uses reserved fields")) +#define DEFINE_TERMINAL_GOTOX_PROG (NAME ,BASE ) \ + __naked void NAME(void) \ + { \ + asm volatile (" \ + .pushsection .jumptables,\"\",@progbits; \ +jt0_%=: \ + .quad ret0_%= - " BASE "; \ + .size jt0_%=, 8; \ + .global jt0_%=; \ + .popsection; \ + \ + r0 = jt0_%= ll; \ + r0 = *(u64 *)(r0 + 0); \ + goto end_%=; \ +ret0_%=: \ + r0 = 0; \ + exit; \ +end_%=: \ + .8byte %[gotox_r0]; \ +" : \ + : __imm_insn(gotox_r0, BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, \ + BPF_REG_0, 0, 0, 0)) \ + : __clobber_all); \ + } + +SEC ("socket" )
tools/testing/selftests/bpf/progs/verifier_gotox.c DEFINE_SIMPLE_JUMP_TABLE_PROG(reserved_field_src_reg, BPF_REG_1, 0, 0, __fa · line 85
Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +10
-0+__success __retval(0) +DEFINE_TERMINAL_GOTOX_PROG(jump_table_terminal_gotox, "socket") + +static __noinline__used +DEFINE_TERMINAL_GOTOX_PROG(terminal_gotox_subprog1, ".text") + +static __noinline__used intterminal_gotox_subprog2 (void) +{ +return 0 ; +}
TBH, I was thinking about
DEFINE_TERMINAL_GOTOX_PROG(terminal_gotox_subprog1, ".text") DEFINE_TERMINAL_GOTOX_PROG(terminal_gotox_subprog2, ".text")
such that one shouldn't think in which order subprogs are added. But your variant should work.
Quoted 17 lines
tools/testing/selftests/bpf/progs/verifier_gotox.c +11
-0+SEC("socket") +__success __retval(0) +__naked void jump_table_terminal_gotox_subprog (void ) +{ + asmvolatile (" \ +call terminal_gotox_subprog1; \ +call terminal_gotox_subprog2; \ +exit; \ +" ::: __clobber_all); +} +/* * Gotox is forbidden when there is no jump table loaded * which points to the sub-function where the gotox is used Signature · 1 line
2.43.0
This series was applied to bpf/bpf.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf.git (master)
by Alexei Starovoitov <[email protected]>:
Quoted 10 lines · On Wed, 2 Sep 2026 17:14:13 +0000 you wrote:
check_subprogs()treats gotox as a direct jump and validates its reserved zero offset. When gotox is the final instruction, this produces a synthetic successor one instruction past the end of the subprogram and rejects an otherwise valid program.Skip direct-offset validation for gotox and accept it as a non-fallthrough terminal instruction. Its actual targets remain validated from the instruction-array jump table during CFG construction.
[...]
Here is the summary with links:
- [bpf-next,v4,1/2] bpf: Allow terminal gotox instructions
https://git.kernel.org/bpf/bpf/c/0d7823cd4cda - [bpf-next,v4,2/2] selftests/bpf: Test terminal gotox instructions https://git.kernel.org/bpf/bpf/c/ac781acaef49
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html