bpf: Support aggregate return values up to 16 bytes
Subsystems (12)
Yonghong Song
reviewerv4 cover bpf: Support aggregate return values up to 16 bytes
LLVM 23 can return an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair [1][2]. Before that the BPF backend could not return such values at all: a by-value aggregate return was rejected at compile time with "aggregate returns are not supported", and an __int128 return failed in the backend with "unable to allocate function return #1".
This series teaches the kernel the same convention, so that BPF programs and kfuncs can return these values. The first 8 bytes of the value come back in R0 and the second 8 bytes in R2. It applies to kfunc returns and to BPF-to-BPF subprogram returns, both global and static. The main program is unchanged: its return value is the program's exit code, so a return larger than 8 bytes is still rejected at BPF_EXIT.
Patches 1-2 are preparation: patch 1 factors out the per-register check used by the global return path, and patch 2 adds the shared helpers that answer "does this subprogram return a register pair", so that the patches which follow can be ordered independently. Patch 3 wires up the JIT side. Patches 4-5 teach precision backtracking and live register analysis about R2 as a second return register, ahead of the patch that starts modeling it. Patch 6 rejects callbacks returning more than 8 bytes, since neither bpf_callback_t nor bpf_exception_cb has a second return register. Patch 7 adds the verifier support proper. Patch 8 rejects a pair return once the subprogram's BTF has been marked unreliable, and patch 9 relaxes btf_distill_func_proto() and btf_validate_return_type(), which is what makes the whole thing reachable. Patches 10-12 add selftests and patch 13 documents the convention.
Constraints worth calling out:
- A by-value struct or union returned by a kfunc or by a global subprogram must be composed only of scalars. The verifier models the returned register bits as an unknown scalar, so a pointer member would be laundered into one and escape provenance and reference tracking. A static subprogram is verified inline and is not restricted this way.
- Returning the pair from a kfunc needs the JIT to place the second half into R2, which is architecture-specific work. Architectures opt in through bpf_jit_supports_kfunc_ret_reg_pair(); x86_64, arm64 and riscv64 do so here, and elsewhere bpf_add_kfunc_call() rejects such a kfunc with -EOPNOTSUPP. A register-pair return from a BPF subprogram needs no such capability.
- The interpreter propagates R0 alone out of a subprogram, so the JIT is forced wherever a caller can observe the pair.
- The compiler side requires LLVM 23 or newer. The selftests written in C record which compiler built them in a read-only flag and report a skip rather than a pass when built by anything older; the inline-asm tests do not depend on the compiler and run everywhere.
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876
Changelog:
v3 -> v4:
- v3: https://lore.kernel.org/bpf/[email protected]/
- Fix a few kernel comment format.
- Guard more kfunc's with x86_64/arm64 only to avoid s390x failure.
v2 -> v3:
- v2: https://lore.kernel.org/bpf/[email protected]/
- Add additional guard with __SIZEOF_INT128__ for 32bit kernel.
- Guard little endian to avoid arm64 big endian for selftests.
- Fix test failures for gcc15.
- Rebase to avoid conflict with latest master branch.
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- Split the R0:R2 helpers out of the verifier patch into their own preparation patch, and reordered the series so the core verifier patch comes after the infrastructure it depends on.
- New patch rejecting callbacks that return more than 8 bytes, both helper/kfunc callbacks and exception callbacks, with selftests.
- New patch rejecting a register-pair return once btf_check_subprog_call() has marked the subprogram's BTF unreliable, rather than silently mistracking R2.
- Folded "bpf: Force JIT for programs using the R0:R2 register pair" into the verifier patch.
- Dropped "bpf: Reject >8 byte return values on return-reading trampoline paths" and its selftests; that went in separately as commit c48796aa6c39.
- Described the register mapping as the first and second 8 bytes rather than the low and high 64 bits, which is only correct on little-endian, and reworded "16-byte" to "up to 16 bytes" where the range 9..16 was meant.
Yonghong Song (13):
bpf: Factor check_global_ret_scalar_reg() out of the global return
check
bpf: Add helpers to describe the R0:R2 return register pair
bpf: Wire up JIT support for 16-byte kfunc returns
bpf: Track R2 of register-pair returns in precision backtracking
bpf: Account R2 of register-pair returns in live register analysis
bpf: Reject callbacks returning more than 8 bytes
bpf: Add verifier support for 16-byte returns in R0:R2
bpf: Reject register-pair returns when the subprog BTF is unreliable
bpf: Enable aggregate return types up to 16 bytes selftests/bpf: Add C tests for 16-byte returns in R0:R2 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns selftests/bpf: Add tests for callbacks returning more than 8 bytes Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2
Documentation/bpf/kfuncs.rst | 62 +++ arch/arm64/net/bpf_jit_comp.c | 5 + arch/riscv/net/bpf_jit_comp64.c | 5 + arch/x86/net/bpf_jit_comp.c | 21 + include/linux/bpf_verifier.h | 16 + include/linux/filter.h | 1 + kernel/bpf/backtrack.c | 63 ++- kernel/bpf/btf.c | 44 +- kernel/bpf/core.c | 5 + kernel/bpf/liveness.c | 25 +- kernel/bpf/verifier.c | 271 +++++++++-- .../selftests/bpf/prog_tests/aggregate_ret.c | 206 +++++++++ .../selftests/bpf/prog_tests/exceptions.c | 2 + .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 15 + .../testing/selftests/bpf/prog_tests/timer.c | 2 + .../selftests/bpf/progs/aggregate_ret_func.c | 420 ++++++++++++++++++ .../bpf/progs/aggregate_ret_int128_c.c | 48 ++ .../selftests/bpf/progs/aggregate_ret_kfunc.c | 126 ++++++ .../bpf/progs/aggregate_ret_kfunc_c.c | 66 +++ .../selftests/bpf/progs/aggregate_ret_run.c | 178 ++++++++ .../bpf/progs/aggregate_ret_struct_c.c | 82 ++++ .../bpf/progs/aggregate_ret_target.c | 29 ++ .../bpf/progs/aggregate_ret_union_c.c | 58 +++ .../bpf/progs/btf__exceptions_ret_pair_fail.c | 10 + .../bpf/progs/btf__timer_ret_pair_fail.c | 10 + .../selftests/bpf/progs/exceptions_fail.c | 2 +- .../bpf/progs/exceptions_ret_pair_fail.c | 30 ++ .../selftests/bpf/progs/freplace_ret_pair.c | 20 + .../selftests/bpf/progs/timer_ret_pair_fail.c | 49 ++ .../selftests/bpf/progs/verifier_arena.c | 38 ++ .../selftests/bpf/test_kmods/bpf_testmod.c | 87 ++++ .../bpf/test_kmods/bpf_testmod_kfunc.h | 48 ++ 32 files changed, 1986 insertions(+), 58 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_run.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c create mode 100644 tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c create mode 100644 tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c create mode 100644 tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c create mode 100644 tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c
Signature · 1 line
2.53.0-Meta
Yonghong Song
reviewerPATCH v4 1/13 bpf: Factor check_global_ret_scalar_reg() out of the global return check
check_global_subprog_return_code() verifies that a global subprogram returns void, an arena pointer, or register R0 holding a scalar value. Later patches in this series add 16-byte aggregate return support, whose second half is returned in R2 and needs the same validation.
Factor the per-register check into check_global_ret_scalar_reg(env, regno) so that it can be reused for R2.
--- kernel/bpf/verifier.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-)
kernel/bpf/verifier.c +20 -12
@@ -16617,37 +16617,45 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charreturn 0 ; } -static int check_global_subprog_return_code (struct bpf_verifier_env *env) +static int check_global_ret_scalar_reg (struct bpf_verifier_env *env,u32 regno) { -struct bpf_reg_state *reg =reg_state (env,BPF_REG_0 ); -struct bpf_func_state *cur_frame =cur_func (env); +struct bpf_reg_state *reg;int err; -if (subprog_returns_void (env, cur_frame->subprogno)) -return 0 ; - - err =check_reg_arg (env,BPF_REG_0 ,SRC_OP ); + err =check_reg_arg (env, regno,SRC_OP );if (err)return err;/* Pointers to arena are safe to pass between subprograms. */ -if (is_arena_reg (env,BPF_REG_0 )) +if (is_arena_reg (env, regno))return 0 ; -if (is_pointer_value (env,BPF_REG_0 )) { -verbose (env,"R%d leaks addr as return value\n" ,BPF_REG_0 ); +if (is_pointer_value (env, regno)) { +verbose (env,"R%d leaks addr as return value\n" , regno);return -EACCES ; } + reg =reg_state (env, regno);if (reg->type !=SCALAR_VALUE ) { -verbose (env,"At subprogram exit the register R0 is not a scalar value (%s)\n" , -reg_type_str (env, reg->type)); +verbose (env,"At subprogram exit the register R%d is not a scalar value (%s)\n" , + regno,reg_type_str (env, reg->type));return -EINVAL ; }return 0 ; } +static int check_global_subprog_return_code (struct bpf_verifier_env *env) +{ +struct bpf_func_state *cur_frame =cur_func (env); +u32 subprog = cur_frame->subprogno; + +if (subprog_returns_void (env, subprog)) +return 0 ; + +return check_global_ret_scalar_reg (env,BPF_REG_0 ); +} +/* Bitmask with 1s for all caller saved registers */ #define ALL_CALLER_SAVED_REGS ((1u << CALLER_SAVED_REGS) - 1)
Signature · 1 line
2.53.0-Meta
Yonghong Song
reviewerPATCH v4 2/13 bpf: Add helpers to describe the R0:R2 return register pair
In reply to Yonghong SongLLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes: such a value comes back in the R0:R2 register pair, with R2 holding the upper half. See LLVM patches [1] and [2].
Later patches teach the JIT, precision backtracking, live register analysis and the verifier itself about that convention. All of them need to answer the same question: does this subprogram return its value in a register pair? Add the shared helpers up front so that those patches can be ordered independently of each other:
- subprog_ret_type() resolves a subprogram's BTF return type. It is factored out of subprog_returns_void(). The verifier_bug_if(!func) and !func_proto checks it replaces are redundant, since check_btf_func_early() already rejects a func_info record whose type_id is not a BTF_KIND_FUNC pointing at a BTF_KIND_FUNC_PROTO. A check on prog->aux->{btf,func_info} is added instead: unlike subprog_returns_void(), which is only used for global subprograms, later callers ask about static subprograms too, and those may belong to a program loaded without BTF.
- ret_regs_cnt() maps the size of a return value to the number of registers holding it.
- bpf_ret_reg_pair() answers the question above. Its users query it at every subprogram call and at every subprogram exit, that is once per verifier state rather than once per subprogram, so the answer is precomputed into bpf_subprog_info->ret_reg_pair by bpf_compute_subprog_ret_regs() and the helper itself is a flag test. It lives in bpf_verifier.h because kernel/bpf/backtrack.c and kernel/bpf/liveness.c need it as well.
bpf_compute_subprog_ret_regs() runs in bpf_check() right before bpf_compute_live_registers(), which is the first of those users: by then BTF func_info has been validated and the subprogram list is final.
No functional change, bpf_ret_reg_pair() has no callers yet.
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876
--- include/linux/bpf_verifier.h | 12 ++++++++ kernel/bpf/verifier.c | 55 ++++++++++++++++++++++++++++-------- 2 files changed, 56 insertions(+), 11 deletions(-)
include/linux/bpf_verifier.h +12 -0
@@ -811,6 +811,8 @@ struct bpf_subprog_info { bool is_async_cb: 1; bool is_exception_cb: 1; bool args_cached: 1; +/* true if the return value is passed in the R0:R2 register pair */ + bool ret_reg_pair: 1;/* true if bpf_fastcall stack region is used by functions that can't be inlined */ bool keep_fastcall_stack: 1; bool changes_pkt_data: 1;
@@ -1044,6 +1046,16 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *envreturn &env->subprog_info[subprog]; } +/* +* True if @subprog returns its value in the R0:R2 register pair. Cached by +* bpf_compute_subprog_ret_regs(), since this is queried on hot paths: at +* every subprogram call and at every subprogram exit. +*/ +static inline bool bpf_ret_reg_pair (struct bpf_verifier_env *env,int subprog) +{ +return subprog_info (env, subprog)->ret_reg_pair; +} +struct bpf_call_summary {u8 num_params;bool is_void;
kernel/bpf/verifier.c +44 -11
77 lines not on this page · raw message
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 02/13] bpf: Add helpers to describe the R0:R2 return register pair
Is check_btf_func_early() the correct function name here? Running `git grep check_btf_func_early` at this commit returns nothing, and the validation the changelog describes lives in…
Quoted 5 lines
-
subprog_ret_type()resolves a subprogram's BTF return type. It is factored out ofsubprog_returns_void(). Theverifier_bug_if(!func)and !func_proto checks it replaces are redundant, sincecheck_btf_func_early()already rejects afunc_inforecord whosetype_idis not aBTF_KIND_FUNCpointing at aBTF_KIND_FUNC_PROTO.
Is check_btf_func_early() the correct function name here? Running git grep check_btf_func_early at this commit returns nothing, and the validation the changelog describes lives in prepare_btf_func() (kernel/bpf/check_btf.c:31):
/* check type_id */
type = btf_type_by_id(btf, krecord[i].type_id);
if (!type || !btf_type_is_func(type)) { ... goto err_free; }
func_proto = btf_type_by_id(btf, type->type);
if (unlikely(!func_proto || !btf_type_is_func_proto(func_proto)))
goto err_free;The in-code comment added by the same hunk already uses the correct
name: "Both already validated by prepare_btf_func() at prog load."
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 02/13] bpf: Add helpers to describe the R0:R2 return register pair
Quoted 17 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 5 lines
-
subprog_ret_type()resolves a subprogram's BTF return type. It is factored out ofsubprog_returns_void(). Theverifier_bug_if(!func)and !func_proto checks it replaces are redundant, sincecheck_btf_func_early()already rejects afunc_inforecord whosetype_idis not aBTF_KIND_FUNCpointing at aBTF_KIND_FUNC_PROTO.Is
check_btf_func_early()the correct function name here? Runninggit grep check_btf_func_earlyat this commit returns nothing, and the validation the changelog describes lives inprepare_btf_func()(kernel/bpf/check_btf.c:31):/* check type_id */ type = btf_type_by_id(btf, krecord[i].type_id); if (!type || !btf_type_is_func(type)) { ... goto err_free; } func_proto = btf_type_by_id(btf, type->type); if (unlikely(!func_proto || !btf_type_is_func_proto(func_proto))) goto err_free;
Quoted 2 lines
The in-code comment added by the same hunk already uses the correct
name: "Both already validated byprepare_btf_func()at prog load."
Okay, replaced check_btf_func_early() with prepare_btf_func() in the commit message.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 02/13] bpf: Add helpers to describe the R0:R2 return register pair
In reply to Yonghong SongQuoted 36 lines · Yonghong Song
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes: such a value comes back in the R0:R2 register pair, with R2 holding the upper half. See LLVM patches [1] and [2].Later patches teach the JIT, precision backtracking, live register analysis and the verifier itself about that convention. All of them need to answer the same question: does this subprogram return its value in a register pair? Add the shared helpers up front so that those patches can be ordered independently of each other:
-
subprog_ret_type()resolves a subprogram's BTF return type. It is factored out ofsubprog_returns_void(). Theverifier_bug_if(!func)and !func_proto checks it replaces are redundant, sincecheck_btf_func_early()already rejects afunc_inforecord whosetype_idis not aBTF_KIND_FUNCpointing at aBTF_KIND_FUNC_PROTO. A check on prog->aux->{btf,func_info} is added instead: unlikesubprog_returns_void(), which is only used for global subprograms, later callers ask about static subprograms too, and those may belong to a program loaded without BTF.-
ret_regs_cnt()maps the size of a return value to the number of registers holding it.-
bpf_ret_reg_pair()answers the question above. Its users query it at every subprogram call and at every subprogram exit, that is once per verifier state rather than once per subprogram, so the answer is precomputed intobpf_subprog_info->ret_reg_pairbybpf_compute_subprog_ret_regs()and the helper itself is a flag test. It lives inbpf_verifier.hbecausekernel/bpf/backtrack.candkernel/bpf/liveness.cneed it as well.
bpf_compute_subprog_ret_regs()runs inbpf_check()right beforebpf_compute_live_registers(), which is the first of those users: by then BTFfunc_infohas been validated and the subprogram list is final.
Quoted 7 lines
No functional change,
bpf_ret_reg_pair()has no callers yet.[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876Signed-off-by Yonghong Song <[email protected]>---
I still think that bpf_compute_live_registers() can be used to compute this information w/o the need to resort to BTF.
...
Quoted 59 lines
kernel/bpf/verifier.c +0
-0@@ -382,27 +382,57 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) return aux && aux[subprog].linkage == BTF_FUNC_GLOBAL; }kernel/bpf/verifier.c +2
-1-static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +/* Return type of a subprogram, NULL if it cannot be resolved */ +static const struct btf_type *subprog_ret_type (struct bpf_verifier_env *env,int subprog){
kernel/bpf/verifier.c +1
-1-const struct btf_type *type, *func, *func_proto; +const struct btf_type *func, *func_proto;const struct btf *btf =
env->prog->aux->btf; u32btf_id;kernel/bpf/verifier.c +3
-0+if (!btf || !env->prog->aux->func_info) +return NULL ; +
btf_id= env->prog->aux->func_info[subprog].type_id;kernel/bpf/verifier.c +1
-0+/* Both already validated by prepare_btf_func() at prog load. */ func =
btf_type_by_id(btf, btf_id);kernel/bpf/verifier.c +0
-3-if (verifier_bug_if (!func, env,"btf_id %u not found" , btf_id)) -return false; -
func_proto=btf_type_by_id(btf, func->type);kernel/bpf/verifier.c +0
-2-if (!func_proto) -return false;kernel/bpf/verifier.c +6
-3- type =btf_type_skip_modifiers (btf, func_proto->type,NULL ); -if (!type) -return false; +return btf_type_skip_modifiers (btf, func_proto->type,NULL ); +} + +static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +{ +const struct btf_type *type =subprog_ret_type (env, subprog);kernel/bpf/verifier.c +17
-1-return btf_type_is_void (type); +return type &&btf_type_is_void (type); +} + +/* +* Number of registers holding a function return value: a value of up to 8 +* bytes is returned in R0, a value of more than 8 bytes and no more than 16 +* bytes (an __int128 or a struct/union of such size) is returned in the R0:R2 +* register pair, with R2 holding the upper half. +*/ +static u32 ret_regs_cnt (u32 size) +{ +return size >8 && size <=16 ?2 :1 ; +} + +/* +* Resolve the return convention of every subprogram once, so that +* bpf_ret_reg_pair() is a plain flag test on the hot paths that use it.
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 434
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+*/ +static void bpf_compute_subprog_ret_regs (struct bpf_verifier_env *env) +{ +const struct btf_type *type; +int subprog; + +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) { + type =subprog_ret_type (env, subprog); +if (type && (btf_type_is_struct (type) ||btf_type_is_scalar (type))) +subprog_info (env, subprog)->ret_reg_pair =ret_regs_cnt (type->size) >1 ;
Nit: there is a btf.c:btf_resolve_size() api function, it would be
better to use it instead of calculating the size ad-hoc.
Should the jit_required flag be set here instead of the main verification pass?
Quoted 4 lines
kernel/bpf/verifier.c +1
-0+ }}
static const char *subprog_name(const structbpf_verifier_env*env, int subprog)
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 02/13] bpf: Add helpers to describe the R0:R2 return register pair
Quoted 39 lines · Eduard Zingerman
Quoted 38 lines · Yonghong Song
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes: such a value comes back in the R0:R2 register pair, with R2 holding the upper half. See LLVM patches [1] and [2].Later patches teach the JIT, precision backtracking, live register analysis and the verifier itself about that convention. All of them need to answer the same question: does this subprogram return its value in a register pair? Add the shared helpers up front so that those patches can be ordered independently of each other:
-
subprog_ret_type()resolves a subprogram's BTF return type. It is factored out ofsubprog_returns_void(). Theverifier_bug_if(!func)and !func_proto checks it replaces are redundant, sincecheck_btf_func_early()already rejects afunc_inforecord whosetype_idis not aBTF_KIND_FUNCpointing at aBTF_KIND_FUNC_PROTO. A check on prog->aux->{btf,func_info} is added instead: unlikesubprog_returns_void(), which is only used for global subprograms, later callers ask about static subprograms too, and those may belong to a program loaded without BTF.-
ret_regs_cnt()maps the size of a return value to the number of registers holding it.-
bpf_ret_reg_pair()answers the question above. Its users query it at every subprogram call and at every subprogram exit, that is once per verifier state rather than once per subprogram, so the answer is precomputed intobpf_subprog_info->ret_reg_pairbybpf_compute_subprog_ret_regs()and the helper itself is a flag test. It lives inbpf_verifier.hbecausekernel/bpf/backtrack.candkernel/bpf/liveness.cneed it as well.
bpf_compute_subprog_ret_regs()runs inbpf_check()right beforebpf_compute_live_registers(), which is the first of those users: by then BTFfunc_infohas been validated and the subprogram list is final.No functional change,
bpf_ret_reg_pair()has no callers yet.
Quoted 7 lines
Quoted 5 lines
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876Signed-off-by Yonghong Song <[email protected]>---
I still think that
bpf_compute_live_registers()can be used to compute this information w/o the need to resort to BTF.
Maybe it is possible. I think current bpf_compute_subprog_ret_regs() is more clear. In llvm23, we have true signatures for bpf programs, so we should have precise return types for each subprogram.
Quoted 66 lines
...
Quoted 63 lines
kernel/bpf/verifier.c +34
-11@@ -382,27 +382,57 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) return aux && aux[subprog].linkage == BTF_FUNC_GLOBAL; } -static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +/* Return type of a subprogram, NULL if it cannot be resolved */ +static const struct btf_type *subprog_ret_type (struct bpf_verifier_env *env,int subprog) { -const struct btf_type *type, *func, *func_proto; +const struct btf_type *func, *func_proto; const struct btf *btf = env->prog->aux->btf; u32 btf_id; +if (!btf || !env->prog->aux->func_info) +return NULL ; + btf_id = env->prog->aux->func_info[subprog].type_id; +/* Both already validated by prepare_btf_func() at prog load. */ func = btf_type_by_id(btf, btf_id); -if (verifier_bug_if (!func, env,"btf_id %u not found" , btf_id)) -return false; - func_proto = btf_type_by_id(btf, func->type); -if (!func_proto) -return false; - type =btf_type_skip_modifiers (btf, func_proto->type,NULL ); -if (!type) -return false; +return btf_type_skip_modifiers (btf, func_proto->type,NULL ); +} + +static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +{ +const struct btf_type *type =subprog_ret_type (env, subprog); -return btf_type_is_void (type); +return type &&btf_type_is_void (type); +} + +/* +* Number of registers holding a function return value: a value of up to 8 +* bytes is returned in R0, a value of more than 8 bytes and no more than 16 +* bytes (an __int128 or a struct/union of such size) is returned in the R0:R2 +* register pair, with R2 holding the upper half. +*/ +static u32 ret_regs_cnt (u32 size) +{ +return size >8 && size <=16 ?2 :1 ; +} + +/* +* Resolve the return convention of every subprogram once, so that +* bpf_ret_reg_pair() is a plain flag test on the hot paths that use it. +*/ +static void bpf_compute_subprog_ret_regs (struct bpf_verifier_env *env) +{ +const struct btf_type *type;
Quoted 8 lines
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 434
Quoted 6 lines
kernel/bpf/verifier.c +6
-0+int subprog; + +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) { + type =subprog_ret_type (env, subprog); +if (type && (btf_type_is_struct (type) ||btf_type_is_scalar (type))) +subprog_info (env, subprog)->ret_reg_pair =ret_regs_cnt (type->size) >1 ;Nit: there is a btf.c:btf_resolve_size() api function, it would be
better to use it instead of calculating the size ad-hoc.
Will use btf_resolve_size() to get type size.
Quoted 3 lines
Should the
jit_requiredflag be set here instead of the main verification pass?
Indeed, this is much better.
Quoted 6 lines
Quoted 5 lines
kernel/bpf/verifier.c +1
-0+ } } static const char *subprog_name(const struct bpf_verifier_env *env, int subprog)...
Yonghong Song
reviewerPATCH v4 3/13 bpf: Wire up JIT support for 16-byte kfunc returns
In reply to Yonghong SongLLVM 23 returns an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch taught the verifier about that convention; wire up the JIT side so that the second half of the return value actually lands in R2.
A kfunc returning more than 8 bytes hands the second half of the result back in RDX, the native x86-64 ABI's second return register. BPF R0 maps to RAX so it needs no move, but BPF R2 maps to RSI, so emit a RDX->RSI move after a BPF_PSEUDO_KFUNC_CALL whose function model reports ret_size > 8.
Placing the second return half into R2 is possible on any JIT, but it needs architecture-specific JIT work. Rather than requiring every JIT to implement it at once, add a bpf_jit_supports_kfunc_ret_reg_pair() capability, defaulting to false in the generic core; an architecture opts in once its JIT handles the R0:R2 pair, and the remaining ones are left for future work. The verifier enforces it in bpf_add_kfunc_call(), rejecting a kfunc whose return is larger than 8 bytes with -EOPNOTSUPP when the JIT lacks the capability. Only x86, arm64 and riscv are supported so far.
On arm64 and riscv the native second return register is already BPF R2 (x1 in bpf2a64[] and a1 in regmap[] respectively), so the value is in the R0:R2 register pair on return with no extra move, unlike x86 (RDX->RSI). This has been tested on x86 and arm64. The riscv path is expected to work by the same register-mapping reasoning as arm64 but has not been tested.
bpf_add_kfunc_call() also rejects a kfunc that is marked KF_FASTCALL and returns more than 8 bytes. The bpf_fastcall contract implemented by mark_fastcall_pattern_for_call() assumes a call clobbers R0 plus the registers holding its arguments, so a return in the R0:R2 pair would clobber an R2 the caller expects the fastcall pattern to preserve. Such a kfunc is rejected with -EOPNOTSUPP as well.
--- arch/arm64/net/bpf_jit_comp.c | 5 +++++ arch/riscv/net/bpf_jit_comp64.c | 5 +++++ arch/x86/net/bpf_jit_comp.c | 21 +++++++++++++++++++++ include/linux/filter.h | 1 + kernel/bpf/core.c | 5 +++++ kernel/bpf/verifier.c | 13 +++++++++++++ 6 files changed, 50 insertions(+)
arch/arm64/net/bpf_jit_comp.c +5 -0
@@ -2330,6 +2330,11 @@ bool bpf_jit_supports_kfunc_call(void)return true; } +bool bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return true; +} +bool bpf_jit_supports_stack_args (void ) {return true;
arch/riscv/net/bpf_jit_comp64.c +5 -0
@@ -2111,6 +2111,11 @@ bool bpf_jit_supports_kfunc_call(void)return true; } +bool bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return true; +} +bool bpf_jit_supports_ptr_xchg (void ) {return true;
arch/x86/net/bpf_jit_comp.c +21 -0
@@ -2647,6 +2647,22 @@ st: insn_off = insn->off;return -EINVAL ;if (priv_frame_ptr)pop_r9 (&prog); +if (src_reg ==BPF_PSEUDO_KFUNC_CALL ) { +const struct btf_func_model *fm; + +/* +* A kfunc returning a >8 byte aggregate hands the +* second half back in RDX (the native ABI's second +* return reg), but BPF expects it in R0:R2. BPF R0 +* is RAX (no move needed), while BPF R2 is RSI, so +* copy RDX into RSI. +*/ + fm =bpf_jit_find_kfunc_model (bpf_prog, insn); +if (!fm) +return -EFAULT ; +if (fm->ret_size >8 ) +emit_mov_reg (&prog, true,BPF_REG_2 ,BPF_REG_3 ); + }break ; }
@@ -4137,6 +4153,11 @@ bool bpf_jit_supports_kfunc_call(void)return true; } +bool bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return true; +} +bool bpf_jit_supports_stack_args (void ) {return true;
include/linux/filter.h +1 -0
@@ -1213,6 +1213,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);bool bpf_jit_supports_subprog_tailcalls (void );bool bpf_jit_supports_percpu_insn (void );bool bpf_jit_supports_kfunc_call (void ); +bool bpf_jit_supports_kfunc_ret_reg_pair (void );bool bpf_jit_supports_stack_args (void );bool bpf_jit_supports_arena_args (void );bool bpf_jit_supports_far_kfunc_call (void );
kernel/bpf/core.c +5 -0
@@ -3303,6 +3303,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)return false; } +bool __weak bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return false; +} +bool __weak bpf_jit_supports_stack_args (void ) {return false;
kernel/bpf/verifier.c +13 -0
@@ -2837,6 +2837,19 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) err =btf_distill_func_proto (&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);if (err)return err; +if (func_model.ret_size >8 ) { +if (kfunc.flags && (*kfunc.flags &KF_FASTCALL )) { +verbose (env, +"kfunc %s with >8-byte return is not supported with KF_FASTCALL\n" , + kfunc.name); +return -EOPNOTSUPP ; + } +if (!bpf_jit_supports_kfunc_ret_reg_pair ()) { +verbose (env,"kfunc %s with >8-byte return is not supported by JIT\n" , + kfunc.name); +return -EOPNOTSUPP ; + } + }memset (&meta,0 ,sizeof (meta)); meta.btf = kfunc.btf;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 03/13] bpf: Wire up JIT support for 16-byte kfunc returns
This isn't a bug, but should this say the previous patch added the shared R0:R2 helpers, rather than that it taught the verifier the convention? Looking at 882cac43a2cef ("bpf: Add helpers to…
Quoted 4 lines
LLVM 23 returns an
__int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch taught the verifier about that convention; wire up the JIT side so that the second half of the return value actually lands in R2.
This isn't a bug, but should this say the previous patch added the shared R0:R2 helpers, rather than that it taught the verifier the convention? Looking at 882cac43a2cef ("bpf: Add helpers to describe the R0:R2 return register pair"), its changelog states "No functional change, bpf_ret_reg_pair() has no callers yet" and "Later patches teach the JIT, precision backtracking, live register analysis and the verifier itself about that convention." The verifier actually learns the convention in d280e9c973c59 ("bpf: Add verifier support for 16-byte returns in R0:R2"), two patches later.
Quoted 8 lines
arch/x86/net/bpf_jit_comp.c +0
-0@@ -2647,6 +2647,22 @@ st: insn_off = insn->off;return -EINVAL ;if (priv_frame_ptr)pop_r9 (&prog);
arch/x86/net/bpf_jit_comp.c st: insn_off = insn->off; · line 2659
Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +10
-0+if (src_reg ==BPF_PSEUDO_KFUNC_CALL ) { +const struct btf_func_model *fm; + +/* +* A kfunc returning a >8 byte aggregate hands the +* second half back in RDX (the native ABI's second +* return reg), but BPF expects it in R0:R2. BPF R0 +* is RAX (no move needed), while BPF R2 is RSI, so +* copy RDX into RSI. +*/
This isn't a bug, but since struct/union returns are still rejected by btf_distill_func_proto() at this point in the series (kernel/bpf/btf.c checks "if (ret < 0 || btf_type_is_struct(t))"), would "a kfunc returning more than 8 bytes" read more accurately here than ">8 byte aggregate"? Aggregates only become reachable later in b12c58bbc5e59 ("bpf: Enable aggregate return types up to 16 bytes"), so the only return type that can reach fm->ret_size > 8 at this commit is __int128.
arch/x86/net/bpf_jit_comp.c st: insn_off = insn->off; · line 2667
Quoted 8 lines
arch/x86/net/bpf_jit_comp.c +6
-0+ fm =bpf_jit_find_kfunc_model (bpf_prog, insn); +if (!fm) +return -EFAULT ; +if (fm->ret_size >8 ) +emit_mov_reg (&prog, true,BPF_REG_2 ,BPF_REG_3 ); + }break ; }
This isn't a bug, but emit_kfunc_arena_args() already does this same bpf_jit_find_kfunc_model() lookup earlier in do_jit() and bails out on NULL. Looking at emit_kfunc_arena_args() in arch/x86/net/bpf_jit_comp.c:
fm = bpf_jit_find_kfunc_model(prog, insn);
if (!fm)
return -EINVAL;Since bpf_jit_find_kfunc_model() is a pure bsearch over an immutable table, if the first lookup succeeded the second cannot fail. Could the two BPF_PSEUDO_KFUNC_CALL blocks share one lookup, so the second bsearch and the unreachable "if (!fm)" guard go away? The two sites also disagree on the errno for the identical condition (-EINVAL vs -EFAULT).
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 03/13] bpf: Wire up JIT support for 16-byte kfunc returns
Quoted 4 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 4 lines
LLVM 23 returns an
__int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch taught the verifier about that convention; wire up the JIT side so that the second half of the return value actually lands in R2.
Quoted 9 lines
This isn't a bug, but should this say the previous patch added the shared R0:R2 helpers, rather than that it taught the verifier the convention? Looking at
882cac43a2cef("bpf: Add helpers to describe the R0:R2 return register pair"), its changelog states "No functional change,bpf_ret_reg_pair()has no callers yet" and "Later patches teach the JIT, precision backtracking, live register analysis and the verifier itself about that convention." The verifier actually learns the convention ind280e9c973c59("bpf: Add verifier support for 16-byte returns in R0:R2"), two patches later.
Okay, will update the commit message.
Quoted 18 lines
Quoted 18 lines
arch/x86/net/bpf_jit_comp.c +9
-0@@ -2647,6 +2647,22 @@ st: insn_off = insn->off;return -EINVAL ;if (priv_frame_ptr)pop_r9 (&prog); +if (src_reg ==BPF_PSEUDO_KFUNC_CALL ) { +const struct btf_func_model *fm; + +/* +* A kfunc returning a >8 byte aggregate hands the +* second half back in RDX (the native ABI's second +* return reg), but BPF expects it in R0:R2. BPF R0 +* is RAX (no move needed), while BPF R2 is RSI, so +* copy RDX into RSI.
Quoted 8 lines
arch/x86/net/bpf_jit_comp.c st: insn_off = insn->off; · line 2659
Quoted 1 line
arch/x86/net/bpf_jit_comp.c +1
-0+*/ This isn't a bug, but since struct/union returns are still rejected by
btf_distill_func_proto()at this point in the series (kernel/bpf/btf.cchecks "if (ret < 0 ||btf_type_is_struct(t))"), would "a kfunc returning more than 8 bytes" read more accurately here than ">8 byte aggregate"? Aggregates only become reachable later inb12c58bbc5e59("bpf: Enable aggregate return types up to 16 bytes"), so the only return type that can reachfm->ret_size> 8 at this commit is__int128.
Will update commit message.
Quoted 17 lines
arch/x86/net/bpf_jit_comp.c st: insn_off = insn->off; · line 2667
Quoted 9 lines
arch/x86/net/bpf_jit_comp.c +6
-0+ fm =bpf_jit_find_kfunc_model (bpf_prog, insn); +if (!fm) +return -EFAULT ; +if (fm->ret_size >8 ) +emit_mov_reg (&prog, true,BPF_REG_2 ,BPF_REG_3 ); + }break ; }This isn't a bug, but
emit_kfunc_arena_args()already does this samebpf_jit_find_kfunc_model()lookup earlier indo_jit()and bails out on NULL. Looking atemit_kfunc_arena_args()inarch/x86/net/bpf_jit_comp.c:fm = bpf_jit_find_kfunc_model(prog, insn); if (!fm) return -EINVAL;
Quoted 5 lines
Since
bpf_jit_find_kfunc_model()is a pure bsearch over an immutable table, if the first lookup succeeded the second cannot fail. Could the twoBPF_PSEUDO_KFUNC_CALLblocks share one lookup, so the second bsearch and the unreachable "if (!fm)" guard go away? The two sites also disagree on the errno for the identical condition (-EINVALvs-EFAULT).
Good point. Let us just one 'fm = bpf_jit_find_kfunc_model(prog, insn)'. Will change.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Re: [PATCH bpf-next v4 03/13] bpf: Wire up JIT support for 16-byte kfunc returns
In reply to Yonghong SongQuoted 32 lines · Yonghong Song
LLVM 23 returns an
__int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch taught the verifier about that convention; wire up the JIT side so that the second half of the return value actually lands in R2.A kfunc returning more than 8 bytes hands the second half of the result back in RDX, the native x86-64 ABI's second return register. BPF R0 maps to RAX so it needs no move, but BPF R2 maps to RSI, so emit a
RDX->RSImove after aBPF_PSEUDO_KFUNC_CALLwhose function model reportsret_size> 8.Placing the second return half into R2 is possible on any JIT, but it needs architecture-specific JIT work. Rather than requiring every JIT to implement it at once, add a
bpf_jit_supports_kfunc_ret_reg_pair()capability, defaulting to false in the generic core; an architecture opts in once its JIT handles the R0:R2 pair, and the remaining ones are left for future work. The verifier enforces it inbpf_add_kfunc_call(), rejecting a kfunc whose return is larger than 8 bytes with-EOPNOTSUPPwhen the JIT lacks the capability. Only x86, arm64 and riscv are supported so far.On arm64 and riscv the native second return register is already BPF R2 (x1 in bpf2a64[] and a1 in regmap[] respectively), so the value is in the R0:R2 register pair on return with no extra move, unlike x86 (
RDX->RSI). This has been tested on x86 and arm64. The riscv path is expected to work by the same register-mapping reasoning as arm64 but has not been tested.
bpf_add_kfunc_call()also rejects a kfunc that is markedKF_FASTCALLand returns more than 8 bytes. Thebpf_fastcallcontract implemented bymark_fastcall_pattern_for_call()assumes a call clobbers R0 plus the registers holding its arguments, so a return in the R0:R2 pair would clobber an R2 the caller expects the fastcall pattern to preserve. Such a kfunc is rejected with-EOPNOTSUPPas well.
Quoted 2 lines
Signed-off-by Yonghong Song <[email protected]>---
...
Yonghong Song
reviewerPATCH v4 4/13 bpf: Track R2 of register-pair returns in precision backtracking
In reply to Yonghong SongA function returning a value larger than 8 bytes (a struct/union, or an __int128) uses R2 as a second return register alongside R0. Precision backtracking treats only R0 as a return register at a call/return boundary, so once the verifier starts modeling R2 that way, marking the second half of such a return precise would trip the "unexpected regs" checks in backtrack_insn() and reject a valid program with -EFAULT. Handle it here, ahead of the patch that introduces the modeling.
Marking the upper half precise, for example by branching on it after a call to a static subprogram, walks backtracking into the callee and reaches its BPF_EXIT with R2 still set in the mask. R2 is part of BPF_REGMASK_ARGS, so this hits "backtracking exit unexpected regs". Returning the pair from a global subprogram or from a kfunc instead hits the equivalent check at the call site.
Handle R2 like R0 in the three boundaries where a call defines the return registers:
- static subprog exit (BPF_EXIT): when the callee returns a pair, R2 is a return register rather than a clobbered argument, so its precision has to cross the frame boundary just like R0's: clear it from the caller's mask before the R1-R5 check, then set it again in the callee's mask after bt_subprog_enter().
The clear has to be conditional, which is why the subprogram containing the exit insn is looked up and queried. For a callee that does not return a pair, check_func_call() has already invalidated the caller's R1-R5 and prepare_func_exit() copies back only R0, so nothing after the call can depend on R2 and backtracking should never still be asking for it here. Clearing it unconditionally would turn that into a silent no-op instead of reporting it through the existing "backtracking exit unexpected regs" check.
- global subprog call: a global subprog returning >8 bytes also sets R2; clear it before the args check.
- kfunc call (BPF_CALL): a kfunc returning >8 bytes (model ret_size > 8) also sets R2; clear it like R0.
All three are gated on R2 actually being in the mask, so the extra BTF and kfunc descriptor lookups stay off the common backtracking path.
--- include/linux/bpf_verifier.h | 2 ++ kernel/bpf/backtrack.c | 63 +++++++++++++++++++++++++++++------- kernel/bpf/verifier.c | 13 ++++++++ 3 files changed, 66 insertions(+), 12 deletions(-)
include/linux/bpf_verifier.h +2 -0
@@ -1447,6 +1447,8 @@ int bpf_jmp_offset(struct bpf_insn *insn);struct bpf_iarray *bpf_insn_successors (struct bpf_verifier_env *env,u32 idx);void bpf_fmt_stack_mask (char *buf,ssize_t buf_sz,u64 stack_mask);bool bpf_subprog_is_global (const struct bpf_verifier_env *env,int subprog); +int bpf_get_kfunc_ret_size (const struct bpf_prog *prog,u32 func_id, +u16 btf_fd_idx,u8 *ret_size);int bpf_find_subprog (struct bpf_verifier_env *env,int off);bool bpf_is_throw_kfunc (struct bpf_insn *insn);
kernel/bpf/backtrack.c +51 -12
99 lines not on this page · raw message
kernel/bpf/verifier.c +13 -0
@@ -2500,6 +2500,19 @@ int bpf_get_kfunc_addr(const struct bpf_prog *prog, u32 func_id,return 0 ; } +int bpf_get_kfunc_ret_size (const struct bpf_prog *prog,u32 func_id, +u16 btf_fd_idx,u8 *ret_size) +{ +const struct bpf_kfunc_desc *desc; + + desc =find_kfunc_desc (prog, func_id, btf_fd_idx); +if (!desc) +return -EFAULT ; + + *ret_size = desc->func_model.ret_size; +return 0 ; +} +#define BPF_FD_SLOT_BTF 1ULstatic void fd_slot_set_map (struct bpf_fd_array *slot,struct bpf_map *map)
Signature · 1 line
2.53.0-Meta
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 04/13] bpf: Track R2 of register-pair returns in precision backtracking
On Mon, 2026-08-10 at 17:09 -0700, Yonghong Song wrote:
...
Quoted 20 lines
include/linux/bpf_verifier.h +2
-0@@ -1447,6 +1447,8 @@ int bpf_jmp_offset(struct bpf_insn *insn); struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx); void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask); bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog); +int bpf_get_kfunc_ret_size (const struct bpf_prog *prog,u32 func_id, + u16 btf_fd_idx, u8 *ret_size);
intbpf_find_subprog(struct bpf_verifier_env *env, int off); boolbpf_is_throw_kfunc(struct bpf_insn *insn);kernel/bpf/backtrack.c +0
-0@@ -425,6 +425,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,*/ verifier_bug_if(idx + 1 != subseq_idx, env,
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 436
Quoted 10 lines
kernel/bpf/backtrack.c +9
-0"extra insn from subprog"); +/* +* a global subprog returning more than 8 bytes +* sets R2 as well. R2 is part of the args mask +* checked just below, so it has to be cleared +* here rather than next to R0. +*/ +if (bt_is_reg_set (bt,BPF_REG_2 ) && + bpf_ret_reg_pair(env, subprog)) +bt_clear_reg (bt,BPF_REG_2 );
Nit: Tbh, I don't see a need in the bpf_ret_reg_pair() check.
I'd call bt_clear_reg(r2) unconditionally.
Don't find verifier_bug() justification all that compelling.
Quoted 10 lines
kernel/bpf/backtrack.c +3
-0/* r1-r5 are invalidated after subprog call,* so for global func call it shouldn't be set * anymore @@ -508,6 +517,19 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, return -ENOTSUPP; /* regular helper call sets R0 */ bt_clear_reg(bt, BPF_REG_0); +/* a kfunc returning more than 8 bytes also sets R2 */ +if (insn->src_reg ==BPF_PSEUDO_KFUNC_CALL && + bt_is_reg_set(bt, BPF_REG_2)) {
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 532
Quoted 10 lines
kernel/bpf/backtrack.c +10
-0+u8 ret_size; +int err; + + err =bpf_get_kfunc_ret_size (env->prog, insn->imm, insn->off, + &ret_size); +if (verifier_bug_if (err, env,"no kfunc desc for insn %d" , idx)) +return -EFAULT ; +if (ret_size >8 ) +bt_clear_reg (bt,BPF_REG_2 ); + }
And same here, too much code for bug detection that fired once or twice in my memory.
Quoted 22 lines
kernel/bpf/backtrack.c +14
-1if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) { /* if backtracking was looking for registers R1-R5* they should have been found already. @@ -522,7 +544,30 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, return -EFAULT; } } else if (opcode == BPF_EXIT) { -bool r0_precise; +bool from_subprog_call, r0_precise, r2_precise = false; + +/* +* BPF_EXIT in subprog or callback always returns +* right after the call instruction, so by checking +* whether the instruction at subseq_idx-1 is subprog +* call or not we can distinguish actual exit from +* *subprog* from exit from *callback*. In the former +* case, we need to propagate the precision of the +* return registers, if necessary. In the latter we +* never do that. +*/ + from_subprog_call = subseq_idx -1 >=0 && + bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 570
Quoted 10 lines
kernel/bpf/backtrack.c +10
-0+if (from_subprog_call &&bt_is_reg_set (bt,BPF_REG_2 )) { +struct bpf_subprog_info *callee; + +/* 'idx' is the exit insn, so it is in the callee */ + callee =bpf_find_containing_subprog (env, idx); +if (verifier_bug_if (!callee, env, + "no subprog contains exit insn %d", idx)) +return -EFAULT ; + r2_precise =bpf_ret_reg_pair (env, callee - env->subprog_info); + }
And here, if the verifier_bug check is ignored r0/r2 can be handled together.
Quoted 3 lines
/* Backtracking to a nested function call, 'idx' is a part of
* the inner frame 'subseq_idx' is a part of the outer frame.
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 04/13] bpf: Track R2 of register-pair returns in precision backtracking
Quoted 29 lines · Eduard Zingerman
On Mon, 2026-08-10 at 17:09 -0700, Yonghong Song wrote:
...
Quoted 25 lines
include/linux/bpf_verifier.h +2
-0@@ -1447,6 +1447,8 @@ int bpf_jmp_offset(struct bpf_insn *insn); struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx); void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask); bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog); +int bpf_get_kfunc_ret_size (const struct bpf_prog *prog,u32 func_id, + u16 btf_fd_idx, u8 *ret_size); int bpf_find_subprog(struct bpf_verifier_env *env, int off); bool bpf_is_throw_kfunc(struct bpf_insn *insn);kernel/bpf/backtrack.c +4
-0@@ -425,6 +425,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,*/ verifier_bug_if(idx + 1 != subseq_idx, env, "extra insn from subprog"); +/* +* a global subprog returning more than 8 bytes +* sets R2 as well. R2 is part of the args mask +* checked just below, so it has to be cleared
Quoted 8 lines
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 436
Quoted 5 lines
kernel/bpf/backtrack.c +5
-0+* here rather than next to R0. +*/ +if (bt_is_reg_set (bt,BPF_REG_2 ) && + bpf_ret_reg_pair(env, subprog)) +bt_clear_reg (bt,BPF_REG_2 );Nit: Tbh, I don't see a need in the
bpf_ret_reg_pair()check.
I'd callbt_clear_reg(r2)unconditionally.
Don't findverifier_bug()justification all that compelling.
Okay, will remove the above 'if' condition.
Quoted 15 lines
Quoted 15 lines
kernel/bpf/backtrack.c +7
-0/* r1-r5 are invalidated after subprog call,* so for global func call it shouldn't be set * anymore @@ -508,6 +517,19 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, return -ENOTSUPP; /* regular helper call sets R0 */ bt_clear_reg(bt, BPF_REG_0); +/* a kfunc returning more than 8 bytes also sets R2 */ +if (insn->src_reg ==BPF_PSEUDO_KFUNC_CALL && + bt_is_reg_set(bt, BPF_REG_2)) { +u8 ret_size; +int err; + + err = bpf_get_kfunc_ret_size(env->prog, insn->imm, insn->off,
Quoted 8 lines
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 532
Quoted 6 lines
kernel/bpf/backtrack.c +6
-0+ &ret_size); +if (verifier_bug_if (err, env,"no kfunc desc for insn %d" , idx)) +return -EFAULT ; +if (ret_size >8 ) +bt_clear_reg (bt,BPF_REG_2 ); + }And same here, too much code for bug detection that fired once or twice in my memory.
Will remove verifier_bug_if().
Quoted 26 lines
Quoted 26 lines
kernel/bpf/backtrack.c +17
-1if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) { /* if backtracking was looking for registers R1-R5* they should have been found already. @@ -522,7 +544,30 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, return -EFAULT; } } else if (opcode == BPF_EXIT) { -bool r0_precise; +bool from_subprog_call, r0_precise, r2_precise = false; + +/* +* BPF_EXIT in subprog or callback always returns +* right after the call instruction, so by checking +* whether the instruction at subseq_idx-1 is subprog +* call or not we can distinguish actual exit from +* *subprog* from exit from *callback*. In the former +* case, we need to propagate the precision of the +* return registers, if necessary. In the latter we +* never do that. +*/ + from_subprog_call = subseq_idx -1 >=0 && + bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]); +if (from_subprog_call &&bt_is_reg_set (bt,BPF_REG_2 )) { +struct bpf_subprog_info *callee; +
Quoted 8 lines
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 570
Quoted 7 lines
kernel/bpf/backtrack.c +7
-0+/* 'idx' is the exit insn, so it is in the callee */ + callee =bpf_find_containing_subprog (env, idx); +if (verifier_bug_if (!callee, env, + "no subprog contains exit insn %d", idx)) +return -EFAULT ; + r2_precise =bpf_ret_reg_pair (env, callee - env->subprog_info); + }And here, if the
verifier_bugcheck is ignored r0/r2 can be handled together.
Yes, will remove verifier_bug_if().
Quoted 5 lines
Quoted 4 lines
kernel/bpf/backtrack.c +0
-0/* Backtracking to a nested function call, 'idx' is a part of* the inner frame 'subseq_idx' is a part of the outer frame. ...
Yonghong Song
reviewerPATCH v4 5/13 bpf: Account R2 of register-pair returns in live register analysis
In reply to Yonghong SongA BPF_EXIT of a subprogram returning a value larger than 8 bytes (a struct/union or an __int128) reads R2 as well as R0, since the second half of the return value is passed back in R2. compute_insn_live_regs() only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.
Mark R2 as read at the BPF_EXIT of a subprogram that does return a register pair. bpf_compute_live_registers() walks the instructions in order and env->subprog_info[] is sorted by subprogram start, so the containing subprogram is tracked with a running index and its return convention is queried once per subprogram through bpf_ret_reg_pair().
Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.
--- kernel/bpf/liveness.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)
kernel/bpf/liveness.c +18 -7
@@ -2062,10 +2062,15 @@ static inline u32 mask_widen(u32 m) { return m | (m << 16); }static inline u16 mask_lo (u32 m) {return (u16 )m; }static inline u16 mask_hi (u32 m) {return (u16)(m >>16 ); } -/* Compute info->{use,def} fields for the instruction */ +/* +* Compute info->{use,def} fields for the instruction. @ret_reg_pair tells +* whether the subprogram containing @insn returns its value in the R0:R2 +* register pair, which matters for BPF_EXIT. +*/ static void compute_insn_live_regs (struct bpf_verifier_env *env,struct bpf_insn *insn, -struct insn_live_regs *info) +struct insn_live_regs *info, +bool ret_reg_pair) {struct bpf_call_summary cs;const u8 class =BPF_CLASS (insn->code);
@@ -2196,7 +2201,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,break ;case BPF_EXIT : def =0 ; - use = r0; + use = ret_reg_pair ? (r0 |reg64_mask (BPF_REG_2 )) : r0;break ;case BPF_CALL : def =ALL_CALLER_SAVED_REGS ;
@@ -2233,8 +2238,8 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)struct insn_live_regs *state;int insn_cnt = env->prog->len;u64 pos, insn_pos; -int err =0 , i, j; -bool changed; +int err =0 , i, j, subprog, start, end; +bool changed, ret_reg_pair;/* Use the following algorithm: * - define the following:
@@ -2261,8 +2266,14 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env) goto out; } -for (i =0 ; i < insn_cnt; ++i) -compute_insn_live_regs (env, &insns[i], &state[i]); +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) { + start = env->subprog_info[subprog].start; + end = env->subprog_info[subprog +1 ].start; + ret_reg_pair =bpf_ret_reg_pair (env, subprog); + +for (i = start; i < end; ++i) +compute_insn_live_regs (env, &insns[i], &state[i], ret_reg_pair); + }/* Forward pass: resolve stack access through FP-derived pointers */ err =bpf_compute_subprog_arg_access (env);
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 05/13] bpf: Account R2 of register-pair returns in live register analysis
This isn't a bug, but the description talks about tracking the containing subprogram with a running index while walking instructions in order. The code that landed uses a plain nested loop iterating…
Quoted 11 lines
A
BPF_EXITof a subprogram returning a value larger than 8 bytes (a struct/union or an__int128) reads R2 as well as R0, since the second half of the return value is passed back in R2.compute_insn_live_regs()only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.Mark R2 as read at the
BPF_EXITof a subprogram that does return a register pair.bpf_compute_live_registers()walks the instructions in order and env->subprog_info[] is sorted by subprogram start, so the containing subprogram is tracked with a running index and its return convention is queried once per subprogram throughbpf_ret_reg_pair().
This isn't a bug, but the description talks about tracking the containing subprogram with a running index while walking instructions in order. The code that landed uses a plain nested loop iterating over subprog_info[] directly:
for (subprog = 0; subprog < env->subprog_cnt; subprog++) {
start = env->subprog_info[subprog].start;
end = env->subprog_info[subprog + 1].start;
ret_reg_pair = bpf_ret_reg_pair(env, subprog);
for (i = start; i < end; ++i)
compute_insn_live_regs(env, &insns[i], &state[i], ret_reg_pair);
}This nested form gets its [start, end) ranges straight from subprog_info[] (the same idiom mark_fastcall_patterns() uses), so neither "walks the instructions in order" nor the sortedness reasoning is what makes the loop correct. Would it be worth adjusting that paragraph to describe the nested loop form that landed?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 05/13] bpf: Account R2 of register-pair returns in live register analysis
Quoted 25 lines · On 8/10/26 6:09 PM, [email protected] wrote:
Quoted 11 lines
A
BPF_EXITof a subprogram returning a value larger than 8 bytes (a struct/union or an__int128) reads R2 as well as R0, since the second half of the return value is passed back in R2.compute_insn_live_regs()only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.Mark R2 as read at the
BPF_EXITof a subprogram that does return a register pair.bpf_compute_live_registers()walks the instructions in order and env->subprog_info[] is sorted by subprogram start, so the containing subprogram is tracked with a running index and its return convention is queried once per subprogram throughbpf_ret_reg_pair().This isn't a bug, but the description talks about tracking the containing subprogram with a running index while walking instructions in order. The code that landed uses a plain nested loop iterating over subprog_info[] directly:
for (subprog = 0; subprog < env->subprog_cnt; subprog++) { start = env->subprog_info[subprog].start; end = env->subprog_info[subprog + 1].start; ret_reg_pair = bpf_ret_reg_pair(env, subprog); for (i = start; i < end; ++i) compute_insn_live_regs(env, &insns[i], &state[i], ret_reg_pair); }
Quoted 5 lines
This nested form gets its [start, end) ranges straight from subprog_info[] (the same idiom
mark_fastcall_patterns()uses), so neither "walks the instructions in order" nor the sortedness reasoning is what makes the loop correct. Would it be worth adjusting that paragraph to describe the nested loop form that landed?
Okay, will update the commit message.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Re: [PATCH bpf-next v4 05/13] bpf: Account R2 of register-pair returns in live register analysis
In reply to Yonghong SongQuoted 12 lines · Yonghong Song
A
BPF_EXITof a subprogram returning a value larger than 8 bytes (a struct/union or an__int128) reads R2 as well as R0, since the second half of the return value is passed back in R2.compute_insn_live_regs()only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.Mark R2 as read at the
BPF_EXITof a subprogram that does return a register pair.bpf_compute_live_registers()walks the instructions in order and env->subprog_info[] is sorted by subprogram start, so the containing subprogram is tracked with a running index and its return convention is queried once per subprogram throughbpf_ret_reg_pair().
Quoted 6 lines
Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.
Signed-off-by Yonghong Song <[email protected]>---
Quoted 11 lines
kernel/bpf/liveness.c | 25 ++++++++++++++++++-------
1 file changed, 18 insertions(+), 7 deletions(-)kernel/bpf/liveness.c +0
-0@@ -2062,10 +2062,15 @@ static inline u32 mask_widen(u32 m) { return m | (m << 16); } static inline u16 mask_lo(u32 m) { return (u16)m; } static inline u16 mask_hi(u32 m) { return (u16)(m >> 16); }
kernel/bpf/liveness.c static inline u32 mask_widen(u32 m) { return m | (m << 16); } · line 2069
Quoted 6 lines
kernel/bpf/liveness.c +5
-1-/* Compute info->{use,def} fields for the instruction */ +/* +* Compute info->{use,def} fields for the instruction. @ret_reg_pair tells +* whether the subprogram containing @insn returns its value in the R0:R2 +* register pair, which matters for BPF_EXIT. +*/
Nit: this comment change is unnecessary.
Quoted 12 lines
static void compute_insn_live_regs(struct
bpf_verifier_env*env, structbpf_insn*insn,kernel/bpf/liveness.c +2
-1- struct insn_live_regs *info) + struct insn_live_regs *info, + bool ret_reg_pair){
structbpf_call_summarycs;
const u8 class =BPF_CLASS(insn->code);kernel/bpf/liveness.c +0
-0@@ -2196,7 +2201,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env, break;case
BPF_EXIT:
def = 0;
kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2204
Quoted 2 lines
kernel/bpf/liveness.c +1
-1- use = r0; + use = ret_reg_pair ? (r0 |reg64_mask (BPF_REG_2 )) : r0;
Nit: let's add an r2 variable.
Quoted 3 lines
break;
caseBPF_CALL:
def =ALL_CALLER_SAVED_REGS;
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 05/13] bpf: Account R2 of register-pair returns in live register analysis
Quoted 32 lines · Eduard Zingerman
Quoted 12 lines · Yonghong Song
A
BPF_EXITof a subprogram returning a value larger than 8 bytes (a struct/union or an__int128) reads R2 as well as R0, since the second half of the return value is passed back in R2.compute_insn_live_regs()only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.Mark R2 as read at the
BPF_EXITof a subprogram that does return a register pair.bpf_compute_live_registers()walks the instructions in order and env->subprog_info[] is sorted by subprogram start, so the containing subprogram is tracked with a running index and its return convention is queried once per subprogram throughbpf_ret_reg_pair().Quoted 6 lines
Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.
Signed-off-by Yonghong Song <[email protected]>---
Acked-by Eduard Zingerman <[email protected]>Quoted 11 lines
kernel/bpf/liveness.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)kernel/bpf/liveness.c +0
-0@@ -2062,10 +2062,15 @@ static inline u32 mask_widen(u32 m) { return m | (m << 16); } static inline u16 mask_lo(u32 m) { return (u16)m; } static inline u16 mask_hi(u32 m) { return (u16)(m >> 16); }
Quoted 7 lines
kernel/bpf/liveness.c static inline u32 mask_widen(u32 m) { return m | (m << 16); } · line 2069
Quoted 6 lines
kernel/bpf/liveness.c +5
-1-/* Compute info->{use,def} fields for the instruction */ +/* +* Compute info->{use,def} fields for the instruction. @ret_reg_pair tells +* whether the subprogram containing @insn returns its value in the R0:R2 +* register pair, which matters for BPF_EXIT. +*/ Nit: this comment change is unnecessary.
Okay, will revert to the original comment.
Quoted 16 lines
Quoted 10 lines
kernel/bpf/liveness.c +2
-1static void compute_insn_live_regs(struct bpf_verifier_env *env, struct bpf_insn *insn, - struct insn_live_regs *info) + struct insn_live_regs *info, + bool ret_reg_pair) { struct bpf_call_summary cs; const u8 class = BPF_CLASS(insn->code);kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2204
Quoted 5 lines
kernel/bpf/liveness.c +1
-1@@ -2196,7 +2201,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env, break; case BPF_EXIT: def = 0; - use = r0; + use = ret_reg_pair ? (r0 |reg64_mask (BPF_REG_2 )) : r0;Nit: let's add an r2 variable.
Good point. Will do.
Quoted 5 lines
Quoted 4 lines
kernel/bpf/liveness.c +0
-0break; case BPF_CALL: def = ALL_CALLER_SAVED_REGS;...
Yonghong Song
reviewerPATCH v4 6/13 bpf: Reject callbacks returning more than 8 bytes
In reply to Yonghong SongA callback handed to a helper or a kfunc (bpf_loop(), bpf_timer_set_callback(), bpf_for_each_map_elem(), ...) is invoked through bpf_callback_t, and an exception callback is invoked by bpf_throw() through
u64 (*bpf_exception_cb)(u64 cookie, u64 sp, u64 bp, u64, u64);
Both prototypes yield a single u64 in R0, and neither caller has any notion of a second return register, so a callback returning a value in the R0:R2 pair would have the upper half of its return value silently dropped.
Reject both at load time:
- check_ld_imm(): a callback is materialized as PTR_TO_FUNC by an ld_imm64 pointing at its subprogram, so the subprogram's return convention can be checked where the callback pointer is created, before it ever reaches a helper or kfunc argument.
- do_check_common(): an exception callback is not referenced by a PTR_TO_FUNC, it is named by a BTF decl_tag and verified on its own, so check it as its frame is set up, next to the existing "cannot return void" and single-argument checks.
--- kernel/bpf/verifier.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
kernel/bpf/verifier.c +11 -0
@@ -16306,6 +16306,11 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)verbose (env,"callback function not static\n" );return -EINVAL ; } +if (bpf_ret_reg_pair (env, subprogno)) { +verbose (env, +"callback function with >8-byte return value is not supported\n" ); +return -EINVAL ; + } dst_reg->type =PTR_TO_FUNC ; dst_reg->subprogno = subprogno;
@@ -18528,6 +18533,12 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) ret = -EINVAL ; goto out; } +if (bpf_ret_reg_pair (env, subprog)) { +verbose (env, +"exception cb cannot return value larger than 8 bytes\n" ); + ret = -EINVAL ; + goto out; + }/* Also ensure the callback only has a single scalar argument. */ if (sub->arg_cnt !=1 || sub->args[0 ].arg_type !=ARG_ANYTHING ) {
Signature · 1 line
2.53.0-Meta
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 06/13] bpf: Reject callbacks returning more than 8 bytes
Quoted 19 lines · Yonghong Song
A callback handed to a helper or a kfunc (
bpf_loop(),bpf_timer_set_callback(),bpf_for_each_map_elem(), ...) is invoked throughbpf_callback_t, and an exception callback is invoked bybpf_throw()throughu64 (*bpf_exception_cb)(u64 cookie, u64 sp, u64 bp, u64, u64);
Both prototypes yield a single u64 in R0, and neither caller has any notion of a second return register, so a callback returning a value in the R0:R2 pair would have the upper half of its return value silently dropped.
Reject both at load time:
-
check_ld_imm(): a callback is materialized asPTR_TO_FUNCby anld_imm64pointing at its subprogram, so the subprogram's return convention can be checked where the callback pointer is created, before it ever reaches a helper or kfunc argument.
Quoted 7 lines
-
do_check_common(): an exception callback is not referenced by aPTR_TO_FUNC, it is named by a BTFdecl_tagand verified on its own, so check it as its frame is set up, next to the existing "cannot return void" and single-argument checks.Signed-off-by Yonghong Song <[email protected]>---
The code itself makes sense to me, but do we really need to check this?
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 06/13] bpf: Reject callbacks returning more than 8 bytes
Quoted 20 lines · Eduard Zingerman
Quoted 19 lines · Yonghong Song
A callback handed to a helper or a kfunc (
bpf_loop(),bpf_timer_set_callback(),bpf_for_each_map_elem(), ...) is invoked throughbpf_callback_t, and an exception callback is invoked bybpf_throw()throughu64 (*bpf_exception_cb)(u64 cookie, u64 sp, u64 bp, u64, u64);
Both prototypes yield a single u64 in R0, and neither caller has any notion of a second return register, so a callback returning a value in the R0:R2 pair would have the upper half of its return value silently dropped.
Reject both at load time:
-
check_ld_imm(): a callback is materialized asPTR_TO_FUNCby anld_imm64pointing at its subprogram, so the subprogram's return convention can be checked where the callback pointer is created, before it ever reaches a helper or kfunc argument.
Quoted 8 lines
Quoted 7 lines
-
do_check_common(): an exception callback is not referenced by aPTR_TO_FUNC, it is named by a BTFdecl_tagand verified on its own, so check it as its frame is set up, next to the existing "cannot return void" and single-argument checks.Signed-off-by Yonghong Song <[email protected]>---
The code itself makes sense to me, but do we really need to check this?
Okay, I will remove this patch. The verifier has some definition for callback function with returning R0. If callback function (prog) intends to return two registers, the return register R2 will be ignored.
Quoted 2 lines
...
Yonghong Song
reviewerPATCH v4 7/13 bpf: Add verifier support for 16-byte returns in R0:R2
In reply to Yonghong SongLLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].
Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an __int128 return failed later in the backend with:
fatal error: error in backend: unable to allocate function return #1
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch adds handling for returns greater than 8 bytes in several
places: BPF subprogram returns (the main program, and both global and static subprograms) and kfunc returns.
The R0:R2 convention is only implemented in the JIT. The BPF interpreter has no notion of a second return register: a BPF-to-BPF call goes through JMP_CALL_ARGS and a BPF_EXIT hands back BPF_R0 alone, so a caller reading R2 would see a stale value. Force the JIT wherever a caller can observe the pair, that is at the call to a global subprogram in check_func_call() and at the return from a static subprogram in prepare_func_exit(). Kfunc calls need no separate handling since bpf_add_kfunc_call() already sets jit_required for every kfunc call.
A by-value struct or union returned by a kfunc must be composed only of scalars, since the verifier models the returned register bits as an unknown scalar and a pointer field would otherwise be laundered into one, escaping provenance and reference tracking.
A global subprogram must return a scalar in every return register. The existing exemption for arena pointers now applies only when the return value fits in R0 alone: both halves of a register pair carry a piece of a
Quoted 1 line
8 byte scalar, so an arena pointer in either of them is a leak rather than
a legitimate return value. A subprogram whose whole return value is an arena pointer is unaffected.
A static subprogram is handled differently. The verifier walks into its frame, so prepare_func_exit() propagates the return register(s) to the caller. R0 holding a stack pointer has long been rejected outright there, but R2 is deliberately not treated the same way. LLVM owns both sides of a static call and is not bound by the ABI, so even with a 9..16 byte declared return type it may leave R2 untouched when the caller only consumes the low half; R2 can then hold an incidental stack pointer that is not a return value at all, and rejecting the program would be a false positive. Propagating the register as is would be worse: the callee frame is freed immediately afterwards, leaving the caller with a PTR_TO_STACK that refers to a frame which no longer exists. So the caller's R2 is marked uninitialized instead, and only a caller that actually reads the returned upper half fails. As with R0, a pointer into the caller's own frame is scrubbed too, which is conservative but keeps the two registers consistent.
Once callers read R0:R2, an extension program can no longer replace a function with a >8 byte return value: an extension's own return is capped at 8 bytes by the program-exit check above, so it would leave R2 stale for the target's callers. btf_check_type_match() cannot catch this, as it compares return types by btf_type->info only and an int carries no vlen, so a 16-byte __int128 and an 8-byte long compare equal. Reject such an attach in bpf_check_attach_target() instead.
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876
--- kernel/bpf/verifier.c | 122 ++++++++++++++++++++++++++++++++++++++---- 1 file changed, 112 insertions(+), 10 deletions(-)
kernel/bpf/verifier.c +112 -10
@@ -419,6 +419,9 @@ static u32 ret_regs_cnt(u32 size)return size >8 && size <=16 ?2 :1 ; } +/* Registers holding a function return value, in order. See ret_regs_cnt(). */ +static const int ret_regs[] = {BPF_REG_0 ,BPF_REG_2 }; +/* * Resolve the return convention of every subprogram once, so that * bpf_ret_reg_pair() is a plain flag test on the hot paths that use it.
@@ -9444,6 +9447,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,u16 callee_incoming, stack_arg_cnt;struct bpf_func_state *caller;int err, subprog, target_insn; +u32 i, nregs; target_insn = *insn_idx + insn->imm +1 ; subprog =bpf_find_subprog (env, target_insn);
@@ -9486,9 +9490,22 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,clear_caller_saved_regs (env, caller->regs);invalidate_outgoing_stack_args (env,cur_func (env)); -/* All non-void global functions return a 64-bit SCALAR_VALUE. */ +/* +* A non-void global function returns a 64-bit SCALAR_VALUE in +* R0, or a >8 byte SCALAR_VALUE in the R0:R2 register pair. +*/ if (!subprog_returns_void (env, subprog)) { -mark_reg_unknown (env, caller->regs,BPF_REG_0 ); + nregs =bpf_ret_reg_pair (env, subprog) ?2 :1 ; +/* +* The R0:R2 return convention is only implemented in the +* JIT: the interpreter propagates BPF_R0 alone out of a +* subprogram, so a caller reading R2 would see a stale +* value. Force the JIT once a caller can observe the pair. +*/ +if (nregs >1 ) + env->prog->jit_required =1 ; +for (i =0 ; i < nregs; i++) +mark_reg_unknown (env, caller->regs, ret_regs[i]); }if (env->subprog_info[subprog].might_throw) {
@@ -9810,10 +9827,14 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)struct bpf_func_state *caller, *callee;struct bpf_reg_state *r0;bool in_callback_fn; +u32 i, nregs;int err; callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0 ]; + nregs =bpf_ret_reg_pair (env, callee->subprogno) ?2 :1 ; +if (nregs >1 ) + env->prog->jit_required =1 ;if (r0->type ==PTR_TO_STACK ) {/* technically it's ok to return caller's stack pointer * (or caller's caller's pointer) back to the caller,
@@ -9849,8 +9870,23 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)return -EFAULT ; } }else { -/* return to the caller whatever r0 had in the callee */ - caller->regs[BPF_REG_0 ] = *r0; +/* +* return to the caller whatever the callee had in the +* return register(s) +*/ +for (i =0 ; i < nregs; i++) + caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]]; + +/* +* R2 carries only the upper half of a register pair return +* value. A stack pointer must not escape the callee (see the +* R0 case above), but there is no need to reject the whole +* program for it: hand the caller an uninitialized R2 instead, +* so that only a caller actually using the returned pointer +* fails. +*/ +if (nregs >1 && caller->regs[BPF_REG_2 ].type ==PTR_TO_STACK ) +bpf_mark_reg_not_init (env, &caller->regs[BPF_REG_2 ]); }/* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,
@@ -10747,6 +10783,19 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insnreturn 0 ; } +/* +* Mark the register(s) holding a @size byte kfunc return value as unknown +* scalars. Both halves of a register pair are treated the same way. +*/ +static void mark_kfunc_ret_regs (struct bpf_verifier_env *env, +struct bpf_reg_state *regs,u32 size) +{ +u32 i, nregs =ret_regs_cnt (size); + +for (i =0 ; i < nregs; i++) +mark_reg_unknown (env, regs, ret_regs[i]); +} +static bool is_kfunc_acquire (struct bpf_call_arg_meta *meta) {return meta->kfunc_flags &KF_ACQUIRE ;
@@ -13223,10 +13272,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, }if (btf_type_is_scalar (t)) { -mark_reg_unknown (env, regs,BPF_REG_0 ); +mark_kfunc_ret_regs (env, regs, t->size);if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] || meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))__mark_reg_const_zero (env, ®s[BPF_REG_0 ]); + }else if (btf_type_is_struct (t)) { +/* +* The returned struct comes back as raw register bits modeled +* as an unknown scalar, so it must contain only scalars: +* otherwise a pointer field would be laundered into a scalar +* and escape provenance and reference tracking. +*/ +if (!__btf_type_is_scalar_struct (env, desc_btf, t,0 )) { +verbose (env, +"kernel function %s returns %s %s that is not composed of scalars\n" , + func_name,btf_type_str (t), +btf_name_by_offset (desc_btf, t->name_off)); +return -EINVAL ; + } +mark_kfunc_ret_regs (env, regs, t->size); }else if (btf_type_is_ptr (t)) { ptr_type =btf_type_skip_modifiers (desc_btf, t->type, &ptr_type_id); err =check_special_kfunc (env, &meta, regs, insn_aux, ptr_type, desc_btf);
@@ -16678,7 +16742,8 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charreturn 0 ; } -static int check_global_ret_scalar_reg (struct bpf_verifier_env *env,u32 regno) +static int check_global_ret_scalar_reg (struct bpf_verifier_env *env,u32 regno, +bool allow_arena_ptr_return) {struct bpf_reg_state *reg;int err;
@@ -16688,7 +16753,7 @@ static int check_global_ret_scalar_reg(struct bpf_verifier_env *env, u32 regno)return err;/* Pointers to arena are safe to pass between subprograms. */ -if (is_arena_reg (env, regno)) +if (allow_arena_ptr_return &&is_arena_reg (env, regno))return 0 ;if (is_pointer_value (env, regno)) {
@@ -16710,11 +16775,26 @@ static int check_global_subprog_return_code(struct bpf_verifier_env *env) {struct bpf_func_state *cur_frame =cur_func (env);u32 subprog = cur_frame->subprogno; +u32 i, nregs; +int err;if (subprog_returns_void (env, subprog))return 0 ; -return check_global_ret_scalar_reg (env,BPF_REG_0 ); +/* +* An arena pointer is only a legitimate return value when it is the +* whole of it, that is when it is returned in R0 alone. Both halves of +* a register pair carry a piece of a >8 byte scalar, so an arena +* pointer in either of them is a leak. +*/ + nregs =bpf_ret_reg_pair (env, subprog) ?2 :1 ; +for (i =0 ; i < nregs; i++) { + err =check_global_ret_scalar_reg (env, ret_regs[i], nregs ==1 ); +if (err) +return err; + } + +return 0 ; }/* Bitmask with 1s for all caller saved registers */
@@ -17203,10 +17283,16 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,*/ if (cur_frame->subprogno && !cur_frame->in_async_callback_fn && - !cur_frame->in_exception_callback_fn) + !cur_frame->in_exception_callback_fn) { err =check_global_subprog_return_code (env); -else + }else { +if (!cur_frame->subprogno &&bpf_ret_reg_pair (env,0 )) { +verbose (env, +"return value larger than 8 bytes is not supported at program exit\n" ); +return -EINVAL ; + } err =check_return_code (env,BPF_REG_0 ,"R0" ); + }if (err)return err;return PROCESS_BPF_EXIT ;
@@ -19366,6 +19452,22 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,return -EOPNOTSUPP ; } +/* +* An extension replaces the target outright, so it has to match +* the target's return convention. Its own return value is capped +* at 8 bytes (a >8 byte program return is rejected at BPF_EXIT), +* so it can never fill the R0:R2 pair the target's callers read. +* This cannot be left to btf_check_type_match() above, which +* compares return types by btf_type->info only: an int carries no +* vlen, so a 16-byte __int128 and an 8-byte long compare equal. +*/ +if (prog_extension && tgt_info->fmodel.ret_size >8 ) { +bpf_log (log, +"Cannot replace function %s with a >8 byte return value\n" , + tname); +return -EOPNOTSUPP ; + } +/* * *.multi programs don't need an address during program * verification, we just take the module ref if needed.
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 07/13] bpf: Add verifier support for 16-byte returns in R0:R2
The main program is listed among the places that gain handling for a >8 byte return:
Quoted 67 lines
bpf: Add verifier support for 16-byte returns in R0:R2
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an
__int128return failed later in the backend with:fatal error: error in backend: unable to allocate function return #1
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch adds handling for returns greater than 8 bytes in several
places: BPF subprogram returns (the main program, and both global and static subprograms) and kfunc returns.The R0:R2 convention is only implemented in the JIT. The BPF interpreter has no notion of a second return register: a BPF-to-BPF call goes through
JMP_CALL_ARGSand aBPF_EXIThands backBPF_R0alone, so a caller reading R2 would see a stale value. Force the JIT wherever a caller can observe the pair, that is at the call to a global subprogram incheck_func_call()and at the return from a static subprogram inprepare_func_exit(). Kfunc calls need no separate handling sincebpf_add_kfunc_call()already setsjit_requiredfor every kfunc call.A by-value struct or union returned by a kfunc must be composed only of scalars, since the verifier models the returned register bits as an unknown scalar and a pointer field would otherwise be laundered into one, escaping provenance and reference tracking.
A global subprogram must return a scalar in every return register. The existing exemption for arena pointers now applies only when the return value fits in R0 alone: both halves of a register pair carry a piece of a
Quoted 1 line
8 byte scalar, so an arena pointer in either of them is a leak rather than
a legitimate return value. A subprogram whose whole return value is an arena pointer is unaffected.
A static subprogram is handled differently. The verifier walks into its frame, so
prepare_func_exit()propagates the return register(s) to the caller. R0 holding a stack pointer has long been rejected outright there, but R2 is deliberately not treated the same way. LLVM owns both sides of a static call and is not bound by the ABI, so even with a 9..16 byte declared return type it may leave R2 untouched when the caller only consumes the low half; R2 can then hold an incidental stack pointer that is not a return value at all, and rejecting the program would be a false positive. Propagating the register as is would be worse: the callee frame is freed immediately afterwards, leaving the caller with aPTR_TO_STACKthat refers to a frame which no longer exists. So the caller's R2 is marked uninitialized instead, and only a caller that actually reads the returned upper half fails. As with R0, a pointer into the caller's own frame is scrubbed too, which is conservative but keeps the two registers consistent.Once callers read R0:R2, an extension program can no longer replace a function with a >8 byte return value: an extension's own return is capped at 8 bytes by the program-exit check above, so it would leave R2 stale for the target's callers.
btf_check_type_match()cannot catch this, as it compares return types bybtf_type->infoonly and an int carries no vlen, so a 16-byte__int128and an 8-byte long compare equal. Reject such an attach inbpf_check_attach_target()instead.
Quoted 4 lines
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876Signed-off-by Yonghong Song <[email protected]>
The main program is listed among the places that gain handling for a >8 byte return:
Quoted 3 lines
This patch adds handling for returns greater than 8 bytes in several
places: BPF subprogram returns (the main program, and both global and static subprograms) and kfunc returns.
For the main program the patch does the opposite, though. process_bpf_exit_full() gains an outright rejection:
if (!cur_frame->subprogno && bpf_ret_reg_pair(env, 0)) {
verbose(env,
"return value larger than 8 bytes is not supported at program exit\n");
return -EINVAL;
}That rejection and its error message are not described anywhere in the changelog. The only trace of it is the later backward reference:
Quoted 2 lines
an extension's own return is
capped at 8 bytes by the program-exit check above
which points at a check the changelog never introduced, and the extension reject added in bpf_check_attach_target() rests entirely on that cap.
Could the changelog say directly that a >8 byte return from the main program is rejected at BPF_EXIT, rather than listing the main program as a place where such returns are now handled?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 07/13] bpf: Add verifier support for 16-byte returns in R0:R2
Quoted 89 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 67 lines
bpf: Add verifier support for 16-byte returns in R0:R2
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an
__int128return failed later in the backend with:fatal error: error in backend: unable to allocate function return #1
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch adds handling for returns greater than 8 bytes in several
places: BPF subprogram returns (the main program, and both global and static subprograms) and kfunc returns.The R0:R2 convention is only implemented in the JIT. The BPF interpreter has no notion of a second return register: a BPF-to-BPF call goes through
JMP_CALL_ARGSand aBPF_EXIThands backBPF_R0alone, so a caller reading R2 would see a stale value. Force the JIT wherever a caller can observe the pair, that is at the call to a global subprogram incheck_func_call()and at the return from a static subprogram inprepare_func_exit(). Kfunc calls need no separate handling sincebpf_add_kfunc_call()already setsjit_requiredfor every kfunc call.A by-value struct or union returned by a kfunc must be composed only of scalars, since the verifier models the returned register bits as an unknown scalar and a pointer field would otherwise be laundered into one, escaping provenance and reference tracking.
A global subprogram must return a scalar in every return register. The existing exemption for arena pointers now applies only when the return value fits in R0 alone: both halves of a register pair carry a piece of a
Quoted 1 line
8 byte scalar, so an arena pointer in either of them is a leak rather than
a legitimate return value. A subprogram whose whole return value is an arena pointer is unaffected.
A static subprogram is handled differently. The verifier walks into its frame, so
prepare_func_exit()propagates the return register(s) to the caller. R0 holding a stack pointer has long been rejected outright there, but R2 is deliberately not treated the same way. LLVM owns both sides of a static call and is not bound by the ABI, so even with a 9..16 byte declared return type it may leave R2 untouched when the caller only consumes the low half; R2 can then hold an incidental stack pointer that is not a return value at all, and rejecting the program would be a false positive. Propagating the register as is would be worse: the callee frame is freed immediately afterwards, leaving the caller with aPTR_TO_STACKthat refers to a frame which no longer exists. So the caller's R2 is marked uninitialized instead, and only a caller that actually reads the returned upper half fails. As with R0, a pointer into the caller's own frame is scrubbed too, which is conservative but keeps the two registers consistent.Once callers read R0:R2, an extension program can no longer replace a function with a >8 byte return value: an extension's own return is capped at 8 bytes by the program-exit check above, so it would leave R2 stale for the target's callers.
btf_check_type_match()cannot catch this, as it compares return types bybtf_type->infoonly and an int carries no vlen, so a 16-byte__int128and an 8-byte long compare equal. Reject such an attach inbpf_check_attach_target()instead.Quoted 4 lines
[1] https://github.com/llvm/llvm-project/pull/190894 [2] https://github.com/llvm/llvm-project/pull/206876Signed-off-by Yonghong Song <[email protected]>The main program is listed among the places that gain handling for a >8 byte return:
Quoted 3 lines
This patch adds handling for returns greater than 8 bytes in several
places: BPF subprogram returns (the main program, and both global and static subprograms) and kfunc returns.For the main program the patch does the opposite, though.
process_bpf_exit_full()gains an outright rejection:if (!cur_frame->subprogno && bpf_ret_reg_pair(env, 0)) { verbose(env, "return value larger than 8 bytes is not supported at program exit\n"); return -EINVAL; }That rejection and its error message are not described anywhere in the changelog. The only trace of it is the later backward reference:
Quoted 8 lines
Quoted 2 lines
an extension's own return is
capped at 8 bytes by the program-exit check abovewhich points at a check the changelog never introduced, and the extension reject added in
bpf_check_attach_target()rests entirely on that cap.Could the changelog say directly that a >8 byte return from the main program is rejected at
BPF_EXIT, rather than listing the main program as a place where such returns are now handled?
Okay, will update commit message as you suggested in the above.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 07/13] bpf: Add verifier support for 16-byte returns in R0:R2
In reply to Yonghong SongOn Mon, 2026-08-10 at 17:09 -0700, Yonghong Song wrote:
...
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9837
Quoted 12 lines
kernel/bpf/verifier.c +1
-0@@ -9810,10 +9827,14 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) struct bpf_func_state *caller, *callee; struct bpf_reg_state *r0; bool in_callback_fn; +u32 i, nregs; int err;
callee = state->frame[state->curframe];
r0 = &callee->regs[BPF_REG_0];kernel/bpf/verifier.c +3
-0+ nregs =bpf_ret_reg_pair (env, callee->subprogno) ?2 :1 ; +if (nregs >1 ) + env->prog->jit_required =1 ;
Definitely let's move this to bpf_compute_subprog_ret_regs() instead of setting it in multiple places.
Quoted 16 lines
if (
r0->type==PTR_TO_STACK) {
/* technically it's ok to return caller's stack pointer
* (or caller's caller's pointer) back to the caller,kernel/bpf/verifier.c +7
-2@@ -9849,8 +9870,23 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) return -EFAULT; } } else { -/* return to the caller whatever r0 had in the callee */ - caller->regs[BPF_REG_0 ] = *r0; +/* +* return to the caller whatever the callee had in the +* return register(s) +*/ +for (i =0 ; i < nregs; i++) + caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]]; +
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9889
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+/* +* R2 carries only the upper half of a register pair return +* value. A stack pointer must not escape the callee (see the +* R0 case above), but there is no need to reject the whole +* program for it: hand the caller an uninitialized R2 instead, +* so that only a caller actually using the returned pointer +* fails. +*/ +if (nregs >1 && caller->regs[BPF_REG_2 ].type ==PTR_TO_STACK ) +bpf_mark_reg_not_init (env, &caller->regs[BPF_REG_2 ]);
Why special casing this? What if caller does not use r0, should r0 be reset in such a case as well? Let's handle both r0 and r2 in one place.
Quoted 3 lines
(quoted diff) +0
-0}
/* for callbacks likebpf_looporbpf_for_each_map_elemgo back to callsite,
...
Quoted 10 lines
kernel/bpf/verifier.c +2
-0@@ -16710,11 +16775,26 @@ static int check_global_subprog_return_code(struct bpf_verifier_env *env) { struct bpf_func_state *cur_frame = cur_func(env); u32 subprog = cur_frame->subprogno; +u32 i, nregs; +int err;
if (subprog_returns_void(env, subprog))
return 0;
kernel/bpf/verifier.c static int check_global_subprog_return_code(struct bpf_verifier_env *env) · line 16789
Quoted 7 lines
kernel/bpf/verifier.c +6
-1-return check_global_ret_scalar_reg (env,BPF_REG_0 ); +/* +* An arena pointer is only a legitimate return value when it is the +* whole of it, that is when it is returned in R0 alone. Both halves of +* a register pair carry a piece of a >8 byte scalar, so an arena +* pointer in either of them is a leak. +*/
Why forbidding returning two arena pointers?
Quoted 18 lines
kernel/bpf/verifier.c +8
-0+ nregs =bpf_ret_reg_pair (env, subprog) ?2 :1 ; +for (i =0 ; i < nregs; i++) { + err =check_global_ret_scalar_reg (env, ret_regs[i], nregs ==1 ); +if (err) +return err; + } + +return 0 ;}
/* Bitmask with 1s for all caller saved registers */(quoted diff) +0
-0@@ -17203,10 +17283,16 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env, */if (
cur_frame->subprogno&&
!cur_frame->in_async_callback_fn &&kernel/bpf/verifier.c +1
-1- !cur_frame->in_exception_callback_fn) + !cur_frame->in_exception_callback_fn) {err =
check_global_subprog_return_code(env);
kernel/bpf/verifier.c static int process_bpf_exit_full(struct bpf_verifier_env *env, · line 17293
Quoted 7 lines
kernel/bpf/verifier.c +6
-1-else + }else { +if (!cur_frame->subprogno &&bpf_ret_reg_pair (env,0 )) { +verbose (env, +"return value larger than 8 bytes is not supported at program exit\n" ); +return -EINVAL ; + }
Same as with callbacks, I don't see a reason to check this. The purpose of the verifier is to avoid loading a program that would accidentally bring down the kernel, this check does not contribute towards this goal.
Quoted 9 lines
err =
check_return_code(env, BPF_REG_0, "R0");(quoted diff) +1
-0+ }if (err)
return err;
returnPROCESS_BPF_EXIT;kernel/bpf/verifier.c +0
-0@@ -19366,6 +19452,22 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, return -EOPNOTSUPP; }
kernel/bpf/verifier.c int bpf_check_attach_target(struct bpf_verifier_log *log, · line 19463
Quoted 9 lines
kernel/bpf/verifier.c +9
-0+/* +* An extension replaces the target outright, so it has to match +* the target's return convention. Its own return value is capped +* at 8 bytes (a >8 byte program return is rejected at BPF_EXIT), +* so it can never fill the R0:R2 pair the target's callers read. +* This cannot be left to btf_check_type_match() above, which +* compares return types by btf_type->info only: an int carries no +* vlen, so a 16-byte __int128 and an 8-byte long compare equal. +*/
Should the btf_check_type_match() be fixed?
Quoted 10 lines
kernel/bpf/verifier.c +7
-0+if (prog_extension && tgt_info->fmodel.ret_size >8 ) { +bpf_log (log, +"Cannot replace function %s with a >8 byte return value\n" , + tname); +return -EOPNOTSUPP ; + } +/*
* *.multi programs don't need an address during program
* verification, we just take the module ref if needed.
Yonghong Song
reviewerRe: [PATCH bpf-next v4 07/13] bpf: Add verifier support for 16-byte returns in R0:R2
Quoted 11 lines · Eduard Zingerman
On Mon, 2026-08-10 at 17:09 -0700, Yonghong Song wrote:
...
Quoted 7 lines
kernel/bpf/verifier.c +1
-0@@ -9810,10 +9827,14 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) struct bpf_func_state *caller, *callee; struct bpf_reg_state *r0; bool in_callback_fn; +u32 i, nregs; int err;
Quoted 7 lines
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9837
Quoted 5 lines
kernel/bpf/verifier.c +3
-0callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0]; + nregs =bpf_ret_reg_pair (env, callee->subprogno) ?2 :1 ; +if (nregs >1 ) + env->prog->jit_required =1 ;Definitely let's move this to
bpf_compute_subprog_ret_regs()instead of setting it in multiple places.
Indeed, 'env->prog->jit_required = 1' will be in bpf_compute_subprog_ret_regs().
Quoted 22 lines
Quoted 22 lines
kernel/bpf/verifier.c +12
-2if (r0->type == PTR_TO_STACK) { /* technically it's ok to return caller's stack pointer* (or caller's caller's pointer) back to the caller, @@ -9849,8 +9870,23 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) return -EFAULT; } } else { -/* return to the caller whatever r0 had in the callee */ - caller->regs[BPF_REG_0 ] = *r0; +/* +* return to the caller whatever the callee had in the +* return register(s) +*/ +for (i =0 ; i < nregs; i++) + caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]]; + +/* +* R2 carries only the upper half of a register pair return +* value. A stack pointer must not escape the callee (see the +* R0 case above), but there is no need to reject the whole +* program for it: hand the caller an uninitialized R2 instead,
Quoted 8 lines
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9889
Quoted 5 lines
kernel/bpf/verifier.c +5
-0+* so that only a caller actually using the returned pointer +* fails. +*/ +if (nregs >1 && caller->regs[BPF_REG_2 ].type ==PTR_TO_STACK ) +bpf_mark_reg_not_init (env, &caller->regs[BPF_REG_2 ]);Why special casing this? What if caller does not use r0, should r0 be reset in such a case as well? Let's handle both r0 and r2 in one place.
Will handle PTR_TO_STACK for both r0 and r2.
Quoted 16 lines
Quoted 4 lines
kernel/bpf/verifier.c +0
-0} /* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,...
Quoted 10 lines
kernel/bpf/verifier.c +2
-0@@ -16710,11 +16775,26 @@ static int check_global_subprog_return_code(struct bpf_verifier_env *env) { struct bpf_func_state *cur_frame = cur_func(env); u32 subprog = cur_frame->subprogno; +u32 i, nregs; +int err; if (subprog_returns_void(env, subprog)) return 0;
Quoted 8 lines
kernel/bpf/verifier.c static int check_global_subprog_return_code(struct bpf_verifier_env *env) · line 16789
Quoted 7 lines
kernel/bpf/verifier.c +6
-1-return check_global_ret_scalar_reg (env,BPF_REG_0 ); +/* +* An arena pointer is only a legitimate return value when it is the +* whole of it, that is when it is returned in R0 alone. Both halves of +* a register pair carry a piece of a >8 byte scalar, so an arena +* pointer in either of them is a leak. +*/ Why forbidding returning two arena pointers?
Okay, will relax this. Indeed, two arena pointers are supported.
Quoted 22 lines
Quoted 22 lines
kernel/bpf/verifier.c +11
-2+ nregs =bpf_ret_reg_pair (env, subprog) ?2 :1 ; +for (i =0 ; i < nregs; i++) { + err =check_global_ret_scalar_reg (env, ret_regs[i], nregs ==1 ); +if (err) +return err; + } + +return 0 ; } /* Bitmask with 1s for all caller saved registers */@@ -17203,10 +17283,16 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,*/ if (cur_frame->subprogno && !cur_frame->in_async_callback_fn && - !cur_frame->in_exception_callback_fn) + !cur_frame->in_exception_callback_fn) { err = check_global_subprog_return_code(env); -else + }else { +if (!cur_frame->subprogno &&bpf_ret_reg_pair (env,0 )) {
Quoted 8 lines
kernel/bpf/verifier.c static int process_bpf_exit_full(struct bpf_verifier_env *env, · line 17293
Quoted 4 lines
kernel/bpf/verifier.c +4
-0+verbose (env, +"return value larger than 8 bytes is not supported at program exit\n" ); +return -EINVAL ; + }Same as with callbacks, I don't see a reason to check this. The purpose of the verifier is to avoid loading a program that would accidentally bring down the kernel, this check does not contribute towards this goal.
Okay, will remove it.
Quoted 10 lines
Quoted 10 lines
kernel/bpf/verifier.c +1
-0err = check_return_code(env, BPF_REG_0, "R0"); + } if (err) return err; return PROCESS_BPF_EXIT;@@ -19366,6 +19452,22 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, return -EOPNOTSUPP; }
Quoted 10 lines
kernel/bpf/verifier.c int bpf_check_attach_target(struct bpf_verifier_log *log, · line 19463
Quoted 9 lines
kernel/bpf/verifier.c +9
-0+/* +* An extension replaces the target outright, so it has to match +* the target's return convention. Its own return value is capped +* at 8 bytes (a >8 byte program return is rejected at BPF_EXIT), +* so it can never fill the R0:R2 pair the target's callers read. +* This cannot be left to btf_check_type_match() above, which +* compares return types by btf_type->info only: an int carries no +* vlen, so a 16-byte __int128 and an 8-byte long compare equal. +*/ Should the
btf_check_type_match()be fixed?
The function btf_check_type_match() calls btf_check_func_type_match(). In btf_check_func_type_match(), we have
t1 = btf_type_skip_modifiers(btf1, t1->type, NULL);
t2 = btf_type_skip_modifiers(btf2, t2->type, NULL);
if (t1->info != t2->info) {
bpf_log(log,
"Return type %s of %s() doesn't match type %s of %s()\n",
btf_type_str(t1), fn1,
btf_type_str(t2), fn2);
return -EINVAL;
}
for (i = 0; i < nargs1; i++) {
...
}It only checked the t1->info vs. t2->info. For example t1->info and t2->info both have kind INT. But t1 and t2 may have different INT type (e.g. int vs. long) and this is allowed in btf_check_func_type_match().
The same thing it also allows int vs. int128.
The same for other kinds e.g. struct (some struct has smaller size and some struct has larger size).
So I didn't use btf_check_type_match() and rather use tgt_info->fmodel.ret_size > 8 where reject any prog returning more than 8 bytes.
Quoted 11 lines
Quoted 11 lines
kernel/bpf/verifier.c +7
-0+if (prog_extension && tgt_info->fmodel.ret_size >8 ) { +bpf_log (log, +"Cannot replace function %s with a >8 byte return value\n" , + tname); +return -EOPNOTSUPP ; + } + /** *.multi programs don't need an address during program * verification, we just take the module ref if needed.
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 07/13] bpf: Add verifier support for 16-byte returns in R0:R2
On Thu, 2026-08-13 at 11:20 -0700, Yonghong Song wrote:
...
Quoted 40 lines
Quoted 4 lines
Quoted 4 lines
kernel/bpf/verifier.c +0
-0@@ -19366,6 +19452,22 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, return -EOPNOTSUPP; }Quoted 10 lines
kernel/bpf/verifier.c int bpf_check_attach_target(struct bpf_verifier_log *log, · line 19463
Quoted 9 lines
kernel/bpf/verifier.c +9
-0+/* +* An extension replaces the target outright, so it has to match +* the target's return convention. Its own return value is capped +* at 8 bytes (a >8 byte program return is rejected at BPF_EXIT), +* so it can never fill the R0:R2 pair the target's callers read. +* This cannot be left to btf_check_type_match() above, which +* compares return types by btf_type->info only: an int carries no +* vlen, so a 16-byte __int128 and an 8-byte long compare equal. +*/ Should the
btf_check_type_match()be fixed?The function btf_check_type_match() calls
btf_check_func_type_match(). Inbtf_check_func_type_match(), we havet1 = btf_type_skip_modifiers(btf1, t1->type, NULL); t2 = btf_type_skip_modifiers(btf2, t2->type, NULL); if (t1->info != t2->info) { bpf_log(log, "Return type %s of %s() doesn't match type %s of %s()\n", btf_type_str(t1), fn1, btf_type_str(t2), fn2); return -EINVAL; } for (i = 0; i < nargs1; i++) { ... }It only checked the
t1->infovs.t2->info. For examplet1->infoandt2->infoboth have kind INT. But t1 and t2 may have different INT type (e.g. int vs. long) and this is allowed inbtf_check_func_type_match().
The same thing it also allows int vs. int128.
The same for other kinds e.g. struct (some struct has smaller size and some struct has larger size).
Quoted 2 lines
So I didn't use
btf_check_type_match()and rather usetgt_info->fmodel.ret_size> 8 where reject any prog returning more than 8 bytes.
Yes, I understand the mechanics. The question is whether btf_check_type_match() needs adjustment. This function is used by a single caller, bpf_check_attach_target():
if (prog_extension && btf_check_type_match(log, prog, btf, t)) return -EINVAL;
And just a few lines below this (and v5 of the series) adds:
+ if (prog_extension && tgt_info->fmodel.ret_size > 8) {
+ bpf_log(log,
+ "Cannot replace function %s with a >8 byte return value\n",
+ tname);
+ return -EOPNOTSUPP;
+ }
Given that the sole reason for btf_check_type_match() to exist is to answer the question if an extension BPF subprogram is compatible with the subprogram being extended, I thing this new check has to be moved inside btf_check_type_match(). (E.g. by deeming that integers below 8-bytes are compatible between each other, but larger integers are not).
Yonghong Song
reviewerPATCH v4 8/13 bpf: Reject register-pair returns when the subprog BTF is unreliable
In reply to Yonghong SongThe R0:R2 return convention is derived from the BTF function prototype:bpf_compute_subprog_ret_regs() inspects the return type of every subprogram and records whether its value comes back in a register pair.
btf_check_subprog_call() can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. From that point on the verifier falls back to conservative, R0-only, semantics for the subprogram, while the compiled code keeps returning a pair and leaves the upper half in R2 behind the verifier's back.
Rather than silently mistracking R2, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add subprog_ret_pair_unreliable() and test it at the two places that can observe the flag: check_func_call(), for the call itself, and prepare_func_exit(), for the return from an inlined static subprogram.
Note that the main program needs no such check: a >8 byte return from subprog 0 is rejected at BPF_EXIT regardless of whether its BTF is reliable. Callbacks need none either: a callback address only becomes a PTR_TO_FUNC through check_ld_imm(), which already rejects any callback returning more than 8 bytes.
--- kernel/bpf/verifier.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
kernel/bpf/verifier.c +25 -0
@@ -438,6 +438,21 @@ static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) } } +/* +* A >8 byte BPF return changes the calling convention to R0:R2, so the +* verifier can only allow it while the subprogram's prototype remains +* reliable. Once BTF is marked unreliable, reject the feature instead of +* silently falling back to R0-only semantics. +*/ +static bool subprog_ret_pair_unreliable (struct bpf_verifier_env *env,int subprog) +{ +struct bpf_prog_aux *aux = env->prog->aux; + +return bpf_ret_reg_pair (env, subprog) && + aux->func_info_aux && + aux->func_info_aux[subprog].unreliable; +} +static const char *subprog_name (const struct bpf_verifier_env *env,int subprog) {struct bpf_func_info *info;
@@ -9459,6 +9474,11 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, err =btf_check_subprog_call (env, subprog, caller->regs);if (err == -EFAULT )return err; +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + }if (bpf_subprog_is_global (env, subprog)) {const char *sub_name =subprog_name (env, subprog);
@@ -9832,6 +9852,11 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0 ]; +if (subprog_ret_pair_unreliable (env, callee->subprogno)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + callee->subprogno,subprog_name (env, callee->subprogno)); +return -EINVAL ; + } nregs =bpf_ret_reg_pair (env, callee->subprogno) ?2 :1 ;if (nregs >1 ) env->prog->jit_required =1 ;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 08/13] bpf: Reject register-pair returns when the subprog BTF is unreliable
This looks like a bug fix for register-pair return support. Should this include:
Quoted 24 lines
commit
cb2551cf6e866ae6692739f7cbb75ff97e2d1a3bAuthor: Yonghong Song <[email protected]> Date: Mon Aug 10 17:09:52 2026 -0700bpf: Reject register-pair returns when the subprog BTF is unreliable
The R0:R2 return convention is derived from the BTF function prototype:
bpf_compute_subprog_ret_regs()inspects the return type of every subprogram and records whether its value comes back in a register pair.
btf_check_subprog_call()can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. From that point on the verifier falls back to conservative, R0-only, semantics for the subprogram, while the compiled code keeps returning a pair and leaves the upper half in R2 behind the verifier's back.Rather than silently mistracking R2, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add
subprog_ret_pair_unreliable()and test it at the two places that can observe the flag:check_func_call(), for the call itself, andprepare_func_exit(), for the return from an inlined static subprogram.
Quoted 7 lines
Note that the main program needs no such check: a >8 byte return from subprog 0 is rejected at
BPF_EXITregardless of whether its BTF is reliable. Callbacks need none either: a callback address only becomes aPTR_TO_FUNCthroughcheck_ld_imm(), which already rejects any callback returning more than 8 bytes.Signed-off-by Yonghong Song <[email protected]>
This looks like a bug fix for register-pair return support. Should this include:
Quoted 5 lines
kernel/bpf/verifier.c +0
-0
kernel/bpf/verifier.c static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) · line 446
Quoted 9 lines
kernel/bpf/verifier.c +6
-0@@ -438,6 +438,21 @@ static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) } } +/* +* A >8 byte BPF return changes the calling convention to R0:R2, so the +* verifier can only allow it while the subprogram's prototype remains +* reliable. Once BTF is marked unreliable, reject the feature instead of +* silently falling back to R0-only semantics. +*/
Does the verifier actually fall back to R0-only semantics? The return convention is read through bpf_ret_reg_pair():
include/linux/bpf_verifier.h:
static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
{
return subprog_info(env, subprog)->ret_reg_pair;
}and ret_reg_pair is computed once in bpf_compute_subprog_ret_regs() from the BTF return type. It is never recomputed or cleared when unreliable becomes true.
Grepping every reader of func_info_aux[].unreliable finds exactly three, none of which touches the return convention:
- btf_check_subprog_call(): returns -EINVAL early on a later call site
- btf_prepare_func_args(): verifier_bug() + -EFAULT
- bpf_check_attach_target(): sets 'conservative' for fentry/fexit
So when unreliable becomes true, prepare_func_exit() still computes nregs = 2 and still models R2. For a static subprogram the verifier walks the callee, prepare_func_exit() propagates the callee's real R0/R2, and clear_caller_saved_regs() marks R0-R5 NOT_INIT at every call, so a caller that reads R2 without the callee having written it gets 'R2 !read_ok'.
Worth either citing the concrete path where R2 is mistracked, or dropping the claim and stating the real motivation (unreliable BTF means the prototype the convention was derived from cannot be trusted at all)?
Quoted 13 lines
kernel/bpf/verifier.c +9
-0+static bool subprog_ret_pair_unreliable (struct bpf_verifier_env *env,int subprog) +{ +struct bpf_prog_aux *aux = env->prog->aux; + +return bpf_ret_reg_pair (env, subprog) && + aux->func_info_aux && + aux->func_info_aux[subprog].unreliable; +} +static const char *subprog_name (const struct bpf_verifier_env *env,int subprog) {struct bpf_func_info *info;
kernel/bpf/verifier.c static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 9481
Quoted 8 lines
kernel/bpf/verifier.c +5
-0@@ -9459,6 +9474,11 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, err =btf_check_subprog_call (env, subprog, caller->regs);if (err == -EFAULT )return err; +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + }
Does this break git bisect? At this commit the new check makes every static (local) subprogram that returns a 9..16 byte struct or union unloadable, which the parent commit d280e9c973c59 accepted.
The two conditions in subprog_ret_pair_unreliable() are guaranteed to be true together for such a subprogram at this point in the series:
1. bpf_compute_subprog_ret_regs() at verifier.c:429 sets ret_reg_pair from the BTF return type for structs as well as scalars:
if (type && (btf_type_is_struct(type) || btf_type_is_scalar(type)))
subprog_info(env, subprog)->ret_reg_pair = ret_regs_cnt(type->size) > 1; so a 16-byte struct return gets ret_reg_pair = 1.
2. btf_validate_return_type() in kernel/bpf/btf.c still ends with 'return -EOPNOTSUPP;' for anything that is not void/int/enum (or an arena pointer), i.e. for every struct/union return. That -EOPNOTSUPP propagates through btf_prepare_func_args() and btf_check_func_arg_match() to btf_check_subprog_call(), which sets prog->aux->func_info_aux[subprog].unreliable = true precisely because it deliberately swallows any error other than -EFAULT for static subprograms.
So btf_check_subprog_call() marks the subprogram unreliable, returns -EOPNOTSUPP (not -EFAULT, so the early return above does not fire), and the new check then rejects the call with 'Func#N (...) returns >8 bytes, which requires reliable BTF'. Before this patch the swallowed error was
harmless: the verifier walked into the static callee and prepare_func_exit() copied the real R0 and R2 state back to the caller.
The series' own selftest proves the intended behaviour. The file tools/testing/selftests/bpf/progs/aggregate_ret_func.c (added later in the series by f40812d018cc3) has:
static __naked __noinline struct ptr_pair static_ret_ptr_pair(void) /* 16 bytes */
...
SEC("tc") __success __retval(0)
__naked int aggregate_ret_static_ptr_pair(void)
tagged __success, with the comment 'A static subprogram is verified inline, so prepare_func_exit() hands the caller the callee's actual R0:R2 register state ... which is why btf_validate_return_type() does not apply the scalar-only restriction to a local function.' Built at this commit that program would be rejected.
The later commit b12c58bbc5e59 'bpf: Enable aggregate return types up to 16 bytes' relaxes btf_validate_return_type() to accept a struct of size <= 16 for a local function, and its own comment states the reason: 'What it avoids is needlessly marking the subprogram's BTF unreliable.' Only __int128 returns are unaffected (int passes btf_validate_return_type()).
Consider reordering so the return-type relaxation lands before this rejection, or noting the dependency in this commit's changelog?
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9859
Quoted 9 lines
kernel/bpf/verifier.c +5
-0@@ -9832,6 +9852,11 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0 ]; +if (subprog_ret_pair_unreliable (env, callee->subprogno)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + callee->subprogno,subprog_name (env, callee->subprogno)); +return -EINVAL ; + }
Can this check ever fire? The changelog claims it is needed 'for the return from an inlined static subprogram', but the check appears to be unreachable.
unreliable is monotonic (only ever set to true) and has exactly two writers:
- btf_check_subprog_call() at verifier.c:9357, called from check_func_call() and push_callback_call()
- do_check_common() at verifier.c:18591, which sets func_info_aux[0].unreliable for subprog 0 only
For a frame to reach prepare_func_exit() it must have been pushed by setup_func_entry() via one of these paths:
- check_func_call(): the new hunk above runs immediately after btf_check_subprog_call() and before setup_func_entry(), so a subprog that just became unreliable is rejected there and never gets a frame. Since the flag cannot be set later for that subprog without going through check_func_call() again (which would reject again), the exit check can never see it true.
- push_callback_call(): check_ld_imm() is the only producer of PTR_TO_FUNC and already rejects bpf_ret_reg_pair() callbacks ('callback function with >8-byte return value is not supported'), so a callback frame never has ret_reg_pair set.
- subprog 0 is frame 0 and is never a callee frame; check_cfg() rejects a call back into subprog 0.
The two other callers of prepare_func_exit() are equally excluded: the tail-call path at verifier.c:10797 and the LD_ABS path at verifier.c:16533 can only run in a subprog whose BTF return type is a small int, because check_btf_func() in kernel/bpf/check_btf.c rejects BPF_LD_ABS and tail calls in any subprog that fails btf_type_is_small_int() (int of size <= 8), which excludes every ret_reg_pair subprog.
If the check is intended as defence in depth for a future caller of btf_check_subprog_call(), saying so in the changelog would help; otherwise the hunk can be dropped?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 08/13] bpf: Reject register-pair returns when the subprog BTF is unreliable
Quoted 30 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 30 lines
commit
cb2551cf6e866ae6692739f7cbb75ff97e2d1a3bAuthor: Yonghong Song <[email protected]> Date: Mon Aug 10 17:09:52 2026 -0700bpf: Reject register-pair returns when the subprog BTF is unreliable
The R0:R2 return convention is derived from the BTF function prototype:
bpf_compute_subprog_ret_regs()inspects the return type of every subprogram and records whether its value comes back in a register pair.
btf_check_subprog_call()can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. From that point on the verifier falls back to conservative, R0-only, semantics for the subprogram, while the compiled code keeps returning a pair and leaves the upper half in R2 behind the verifier's back.Rather than silently mistracking R2, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add
subprog_ret_pair_unreliable()and test it at the two places that can observe the flag:check_func_call(), for the call itself, andprepare_func_exit(), for the return from an inlined static subprogram.Note that the main program needs no such check: a >8 byte return from subprog 0 is rejected at
BPF_EXITregardless of whether its BTF is reliable. Callbacks need none either: a callback address only becomes aPTR_TO_FUNCthroughcheck_ld_imm(), which already rejects any callback returning more than 8 bytes.
Quoted 5 lines
Quoted 1 line
Signed-off-by Yonghong Song <[email protected]>This looks like a bug fix for register-pair return support. Should this include:
Fixes d280e9c973c5 ("bpf: Add verifier support for 16-byte returns in R0:R2")
Typically we don't do 'Fixes' within the same series.
Quoted 41 lines
Quoted 6 lines
kernel/bpf/verifier.c +0
-0kernel/bpf/verifier.c static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) · line 446
Quoted 9 lines
kernel/bpf/verifier.c +6
-0@@ -438,6 +438,21 @@ static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) } } +/* +* A >8 byte BPF return changes the calling convention to R0:R2, so the +* verifier can only allow it while the subprogram's prototype remains +* reliable. Once BTF is marked unreliable, reject the feature instead of +* silently falling back to R0-only semantics. +*/ Does the verifier actually fall back to R0-only semantics? The return convention is read through
bpf_ret_reg_pair():include/linux/bpf_verifier.h: static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog) { return subprog_info(env, subprog)->ret_reg_pair; }and
ret_reg_pairis computed once inbpf_compute_subprog_ret_regs()from the BTF return type. It is never recomputed or cleared when unreliable becomes true.Grepping every reader of func_info_aux[].unreliable finds exactly three, none of which touches the return convention:
-
btf_check_subprog_call(): returns-EINVALearly on a later call site
-btf_prepare_func_args():verifier_bug()+-EFAULT
-bpf_check_attach_target(): sets 'conservative' for fentry/fexitSo when unreliable becomes true,
prepare_func_exit()still computes nregs = 2 and still models R2. For a static subprogram the verifier walks the callee,prepare_func_exit()propagates the callee's real R0/R2, andclear_caller_saved_regs()marks R0-R5NOT_INITat every call, so a caller that reads R2 without the callee having written it gets 'R2 !read_ok'.
Quoted 3 lines
Worth either citing the concrete path where R2 is mistracked, or dropping the claim and stating the real motivation (unreliable BTF means the prototype the convention was derived from cannot be trusted at all)?
I will change to mean that for unreliable BTF, the convention was derived from the prototype cannot be trusted.
Quoted 68 lines
Quoted 14 lines
kernel/bpf/verifier.c +9
-0+static bool subprog_ret_pair_unreliable (struct bpf_verifier_env *env,int subprog) +{ +struct bpf_prog_aux *aux = env->prog->aux; + +return bpf_ret_reg_pair (env, subprog) && + aux->func_info_aux && + aux->func_info_aux[subprog].unreliable; +} +static const char *subprog_name (const struct bpf_verifier_env *env,int subprog) {struct bpf_func_info *info;kernel/bpf/verifier.c static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 9481
Quoted 8 lines
kernel/bpf/verifier.c +5
-0@@ -9459,6 +9474,11 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, err =btf_check_subprog_call (env, subprog, caller->regs);if (err == -EFAULT )return err; +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + }Does this break git bisect? At this commit the new check makes every static (local) subprogram that returns a 9..16 byte struct or union unloadable, which the parent commit
d280e9c973c59accepted.The two conditions in
subprog_ret_pair_unreliable()are guaranteed to be true together for such a subprogram at this point in the series:1.
bpf_compute_subprog_ret_regs()at verifier.c:429 setsret_reg_pairfrom the BTF return type for structs as well as scalars:if (type && (
btf_type_is_struct(type)||btf_type_is_scalar(type)))
subprog_info(env, subprog)->ret_reg_pair =ret_regs_cnt(type->size)> 1;so a 16-byte struct return gets
ret_reg_pair= 1.2.
btf_validate_return_type()inkernel/bpf/btf.cstill ends with 'return-EOPNOTSUPP;' for anything that is not void/int/enum (or an arena pointer), i.e. for every struct/union return. That-EOPNOTSUPPpropagates throughbtf_prepare_func_args()andbtf_check_func_arg_match()tobtf_check_subprog_call(), which sets prog->aux->func_info_aux[subprog].unreliable = true precisely because it deliberately swallows any error other than-EFAULTfor static subprograms.So
btf_check_subprog_call()marks the subprogram unreliable, returns-EOPNOTSUPP(not-EFAULT, so the early return above does not fire), and the new check then rejects the call with 'Func#N (...) returns >8 bytes, which requires reliable BTF'. Before this patch the swallowed error was
harmless: the verifier walked into the static callee andprepare_func_exit()copied the real R0 and R2 state back to the caller.The series' own selftest proves the intended behaviour. The file
tools/testing/selftests/bpf/progs/aggregate_ret_func.c(added later in the series byf40812d018cc3) has:static
__naked__noinlinestructptr_pairstatic_ret_ptr_pair(void)/* 16 bytes */
...
SEC("tc")__success__retval(0)
__nakedintaggregate_ret_static_ptr_pair(void)tagged
__success, with the comment 'A static subprogram is verified inline, soprepare_func_exit()hands the caller the callee's actual R0:R2 register state ... which is whybtf_validate_return_type()does not apply the scalar-only restriction to a local function.' Built at this commit that program would be rejected.
Quoted 8 lines
The later commit
b12c58bbc5e59'bpf: Enable aggregate return types up to 16 bytes' relaxesbtf_validate_return_type()to accept a struct of size <= 16 for a local function, and its own comment states the reason: 'What it avoids is needlessly marking the subprogram's BTF unreliable.' Only__int128returns are unaffected (int passesbtf_validate_return_type()).Consider reordering so the return-type relaxation lands before this rejection, or noting the dependency in this commit's changelog?
I will do reordering between patch 8 and patch 9.
Quoted 47 lines
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9859
Quoted 10 lines
kernel/bpf/verifier.c +5
-0@@ -9832,6 +9852,11 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0 ]; +if (subprog_ret_pair_unreliable (env, callee->subprogno)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + callee->subprogno,subprog_name (env, callee->subprogno)); +return -EINVAL ; + }Can this check ever fire? The changelog claims it is needed 'for the return from an inlined static subprogram', but the check appears to be unreachable.
unreliable is monotonic (only ever set to true) and has exactly two writers:
-
btf_check_subprog_call()at verifier.c:9357, called fromcheck_func_call()andpush_callback_call()
-do_check_common()at verifier.c:18591, which sets func_info_aux[0].unreliable for subprog 0 onlyFor a frame to reach
prepare_func_exit()it must have been pushed bysetup_func_entry()via one of these paths:-
check_func_call(): the new hunk above runs immediately afterbtf_check_subprog_call()and beforesetup_func_entry(), so a subprog that just became unreliable is rejected there and never gets a frame. Since the flag cannot be set later for that subprog without going throughcheck_func_call()again (which would reject again), the exit check can never see it true.-
push_callback_call():check_ld_imm()is the only producer ofPTR_TO_FUNCand already rejectsbpf_ret_reg_pair()callbacks ('callback function with >8-byte return value is not supported'), so a callback frame never hasret_reg_pairset.- subprog 0 is frame 0 and is never a callee frame;
check_cfg()rejects a call back into subprog 0.The two other callers of
prepare_func_exit()are equally excluded: the tail-call path at verifier.c:10797 and theLD_ABSpath at verifier.c:16533 can only run in a subprog whose BTF return type is a small int, becausecheck_btf_func()inkernel/bpf/check_btf.crejectsBPF_LD_ABSand tail calls in any subprog that failsbtf_type_is_small_int()(int of size <= 8), which excludes everyret_reg_pairsubprog.
Quoted 3 lines
If the check is intended as defence in depth for a future caller of
btf_check_subprog_call(), saying so in the changelog would help; otherwise the hunk can be dropped?
Thanks. I will drop this.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 08/13] bpf: Reject register-pair returns when the subprog BTF is unreliable
In reply to Yonghong SongQuoted 18 lines · Yonghong Song
The R0:R2 return convention is derived from the BTF function prototype:
bpf_compute_subprog_ret_regs()inspects the return type of every subprogram and records whether its value comes back in a register pair.
btf_check_subprog_call()can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. From that point on the verifier falls back to conservative, R0-only, semantics for the subprogram, while the compiled code keeps returning a pair and leaves the upper half in R2 behind the verifier's back.Rather than silently mistracking R2, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add
subprog_ret_pair_unreliable()and test it at the two places that can observe the flag:check_func_call(), for the call itself, andprepare_func_exit(), for the return from an inlined static subprogram.
Quoted 8 lines
Note that the main program needs no such check: a >8 byte return from subprog 0 is rejected at
BPF_EXITregardless of whether its BTF is reliable. Callbacks need none either: a callback address only becomes aPTR_TO_FUNCthroughcheck_ld_imm(), which already rejects any callback returning more than 8 bytes.Signed-off-by Yonghong Song <[email protected]>---
I think this is a good example on why r0:r2 has to be inferred from register liveness. The user has not real control over reliability of BTF for static functions. Hence, this limitation might be quite inconvenient in practice.
Another option is to assume that static subprograms always return two registers. Then if for some of them static liveness analysis shows that second register is never used -- skip the register move in jit. This latter option seem appealing to me, as it appears to simplify the patch-set.
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 08/13] bpf: Reject register-pair returns when the subprog BTF is unreliable
Quoted 32 lines · Eduard Zingerman
Quoted 18 lines · Yonghong Song
The R0:R2 return convention is derived from the BTF function prototype:
bpf_compute_subprog_ret_regs()inspects the return type of every subprogram and records whether its value comes back in a register pair.
btf_check_subprog_call()can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. From that point on the verifier falls back to conservative, R0-only, semantics for the subprogram, while the compiled code keeps returning a pair and leaves the upper half in R2 behind the verifier's back.Rather than silently mistracking R2, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add
subprog_ret_pair_unreliable()and test it at the two places that can observe the flag:check_func_call(), for the call itself, andprepare_func_exit(), for the return from an inlined static subprogram.Quoted 8 lines
Note that the main program needs no such check: a >8 byte return from subprog 0 is rejected at
BPF_EXITregardless of whether its BTF is reliable. Callbacks need none either: a callback address only becomes aPTR_TO_FUNCthroughcheck_ld_imm(), which already rejects any callback returning more than 8 bytes.Signed-off-by Yonghong Song <[email protected]>---
I think this is a good example on why r0:r2 has to be inferred from register liveness. The user has not real control over reliability of BTF for static functions. Hence, this limitation might be quite inconvenient in practice.
Quoted 5 lines
Another option is to assume that static subprograms always return two registers. Then if for some of them static liveness analysis shows that second register is never used -- skip the register move in jit. This latter option seem appealing to me, as it appears to simplify the patch-set.
For >= llvm23 we have true signatures so return types (registers) should match actual code.
Quoted 2 lines
...
Yonghong Song
reviewerPATCH v4 9/13 bpf: Enable aggregate return types up to 16 bytes
In reply to Yonghong SongRelax btf_distill_func_proto() to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry:
- a struct or union larger than 8 and up to 16 bytes, returned in the R0:R2 register pair, matching what LLVM emits for the BPF target;
- a struct or union up to 8 bytes, returned in R0 alone.
A >8 byte scalar (__int128) was already accepted and is unchanged. Everything else stays rejected: a return type larger than 16 bytes, and any type that __get_type_size() cannot return in registers at all (e.g. an array), which it already reports as ret < 0.
btf_distill_func_proto() also builds the trampoline (fentry/fexit/fmod_ret) and struct_ops function models, so relaxing it widens what those can attach to. A >8 byte return stays rejected on every path that reads the target's return value: commit c48796aa6c39 ("bpf: Reject >8 byte return values on return-reading trampoline paths") covers fexit, fmod_ret and fsession plus their _multi variants, and struct_ops, and an fentry-only trampoline never sets BPF_TRAMP_F_CALL_ORIG so it does not touch the return value at all. A struct or union of 8 bytes or less is newly accepted for those paths; its single eightbyte is returned in R0 like any other scalar.
btf_validate_return_type() is relaxed as well, so that it accepts a by-value struct or union up to 16 bytes in addition to void and scalars.
With btf_distill_func_proto() and btf_validate_return_type() relaxed, the verifier, JIT, precision-backtracking and live-register support from the earlier patches becomes reachable: this final patch enables <=16 byte aggregate return values end to end.
--- include/linux/bpf_verifier.h | 2 + kernel/bpf/btf.c | 44 ++++++++++++++++--- kernel/bpf/verifier.c | 6 +-- .../selftests/bpf/progs/exceptions_fail.c | 2 +- 4 files changed, 45 insertions(+), 9 deletions(-)
include/linux/bpf_verifier.h +2 -0
@@ -1449,6 +1449,8 @@ void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);bool bpf_subprog_is_global (const struct bpf_verifier_env *env,int subprog);int bpf_get_kfunc_ret_size (const struct bpf_prog *prog,u32 func_id,u16 btf_fd_idx,u8 *ret_size); +bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env,const struct btf *btf, +const struct btf_type *t,int rec);int bpf_find_subprog (struct bpf_verifier_env *env,int off);bool bpf_is_throw_kfunc (struct bpf_insn *insn);
kernel/bpf/btf.c +39 -5
71 lines not on this page · raw message
kernel/bpf/verifier.c +3 -3
@@ -11088,9 +11088,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_ }/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ -static bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, -const struct btf *btf, -const struct btf_type *t,int rec) +bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, +const struct btf *btf, +const struct btf_type *t,int rec) {const struct btf_type *member_type;const struct btf_member *member;
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a) SEC("?tc") __exception_cb(exception_cb_bad_ret_type1) -__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.") +__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")int reject_exception_cb_type_1 (struct __sk_buff *ctx) {bpf_throw (0 );
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 09/13] bpf: Enable aggregate return types up to 16 bytes
convention end-to-end.
Quoted 1 line
8 bytes, R0 alone for <=8 bytes), enabling the R0:R2 register-pair return
convention end-to-end.
Quoted 4 lines
kernel/bpf/btf.c +0
-0
[ ... ]
Quoted 34 lines
kernel/bpf/btf.c +22
-1@@ -7965,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,/* Check whether the type is a valid return type. */ static int btf_validate_return_type (struct bpf_verifier_env *env,struct btf *btf, -const struct btf_type *t,int subprog) +const struct btf_type *t,int subprog,bool is_global) {u32 tags =0 ;int err;@@ -7988,6 +7993,35 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btif (btf_type_is_void (t) ||btf_type_is_int (t) ||btf_is_any_enum (t))return 0 ; +if (btf_type_is_struct (t) && t->size <=16 ) { +/* +* A >8 byte struct/union is returned in the R0:R2 register pair. +* A global function is verified in isolation, so its caller models +* the return as an opaque R0:R2 scalar pair; it must therefore +* contain only scalars, otherwise a pointer field would be +* laundered into a scalar and escape provenance and reference +* tracking. That requirement is enforced here: do_check_common() +* propagates the error for global functions and for the main +* program. +* +* A local (static) function is verified inline and its R0:R2 are +* copied as precise register state (with the JIT forced on when +* the pair is consumed), so a pointer field stays tracked and needs +* no such restriction. Accepting it here is not by itself what +* makes it legal: btf_check_subprog_call() drops any error other +* than -EFAULT. What it avoids is needlessly marking the +* subprogram's BTF unreliable. +* +* The main program (subprog 0) takes the scalar-only path as well, +* but its return value is the program's exit code, so a >8 byte
kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8026
Quoted 10 lines
kernel/bpf/btf.c +8
-0+* return is rejected separately at BPF_EXIT. +*/ +bool local_func = subprog && !is_global; + +if (local_func ||__btf_type_is_scalar_struct (env, btf, t,0 )) +return 0 ; + } +return -EOPNOTSUPP ; }
Does the comment accurately describe the enforcement mechanism?
The comment states "do_check_common() propagates the error for global functions and for the main program." That's true for global functions but appears incorrect for the main program. Looking at do_check_common() in kernel/bpf/verifier.c:18624:
if (subprog || env->prog->type == BPF_PROG_TYPE_EXT) {
...
ret = btf_prepare_func_args(env, subprog);
if (ret)
goto out; /* <- error propagated */
} else {
if (env->prog->aux->func_info_aux) {
ret = btf_prepare_func_args(env, 0);
if (ret || sub->arg_cnt != 1 || ...) {
env->prog->aux->func_info_aux[0].unreliable = true;
...
}
}
...
}
ret = do_check(env);The main-program arm (subprog 0 of a non-EXT program) uses the return value only to set func_info_aux[0].unreliable, then unconditionally overwrites ret with do_check(env). The -EOPNOTSUPP from btf_validate_return_type() is therefore discarded.
Concretely: a program whose entry function returns a pointer-containing struct like 'struct { void *p; }' (size 8) now takes the new struct branch, fails __btf_type_is_scalar_struct(), but the resulting error is silently dropped. The program continues to be verified.
No memory-safety hole results: check_return_code() still refuses a pointer in R0 at exit ('R0 leaks addr as return value'), so the pointer-laundering scenario the comment worries about does not actually occur for the main program. The issue is that the comment states an enforcement mechanism that does not exist.
Should the comment drop "and for the main program" and point at the BPF_EXIT check instead?
Also: "btf_check_subprog_call() drops any error other than -EFAULT" is slightly inaccurate. btf_check_subprog_call() actually returns the error and marks the BTF unreliable; its callers, check_func_call() and push_callback_call(), do 'if (err == -EFAULT) return err;' and otherwise continue.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 09/13] bpf: Enable aggregate return types up to 16 bytes
Quoted 87 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 1 line
8 bytes, R0 alone for <=8 bytes), enabling the R0:R2 register-pair return
convention end-to-end.
Quoted 4 lines
kernel/bpf/btf.c +0
-0[ ... ]
Quoted 34 lines
kernel/bpf/btf.c +22
-1@@ -7965,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,/* Check whether the type is a valid return type. */ static int btf_validate_return_type (struct bpf_verifier_env *env,struct btf *btf, -const struct btf_type *t,int subprog) +const struct btf_type *t,int subprog,bool is_global) {u32 tags =0 ;int err;@@ -7988,6 +7993,35 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btif (btf_type_is_void (t) ||btf_type_is_int (t) ||btf_is_any_enum (t))return 0 ; +if (btf_type_is_struct (t) && t->size <=16 ) { +/* +* A >8 byte struct/union is returned in the R0:R2 register pair. +* A global function is verified in isolation, so its caller models +* the return as an opaque R0:R2 scalar pair; it must therefore +* contain only scalars, otherwise a pointer field would be +* laundered into a scalar and escape provenance and reference +* tracking. That requirement is enforced here: do_check_common() +* propagates the error for global functions and for the main +* program. +* +* A local (static) function is verified inline and its R0:R2 are +* copied as precise register state (with the JIT forced on when +* the pair is consumed), so a pointer field stays tracked and needs +* no such restriction. Accepting it here is not by itself what +* makes it legal: btf_check_subprog_call() drops any error other +* than -EFAULT. What it avoids is needlessly marking the +* subprogram's BTF unreliable. +* +* The main program (subprog 0) takes the scalar-only path as well, +* but its return value is the program's exit code, so a >8 byte kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8026
Quoted 10 lines
kernel/bpf/btf.c +8
-0+* return is rejected separately at BPF_EXIT. +*/ +bool local_func = subprog && !is_global; + +if (local_func ||__btf_type_is_scalar_struct (env, btf, t,0 )) +return 0 ; + } +return -EOPNOTSUPP ; }Does the comment accurately describe the enforcement mechanism?
The comment states "
do_check_common()propagates the error for global functions and for the main program." That's true for global functions but appears incorrect for the main program. Looking atdo_check_common()inkernel/bpf/verifier.c:18624:if (subprog || env->prog->type == BPF_PROG_TYPE_EXT) { ... ret = btf_prepare_func_args(env, subprog); if (ret) goto out; /* <- error propagated */ } else { if (env->prog->aux->func_info_aux) { ret = btf_prepare_func_args(env, 0); if (ret || sub->arg_cnt != 1 || ...) { env->prog->aux->func_info_aux[0].unreliable = true; ... } } ... } ret = do_check(env);The main-program arm (subprog 0 of a non-EXT program) uses the return value only to set func_info_aux[0].unreliable, then unconditionally overwrites ret with
do_check(env). The-EOPNOTSUPPfrombtf_validate_return_type()is therefore discarded.Concretely: a program whose entry function returns a pointer-containing struct like 'struct { void *p; }' (size 8) now takes the new struct branch, fails
__btf_type_is_scalar_struct(), but the resulting error is silently dropped. The program continues to be verified.
Quoted 8 lines
No memory-safety hole results:
check_return_code()still refuses a pointer in R0 at exit ('R0 leaks addr as return value'), so the pointer-laundering scenario the comment worries about does not actually occur for the main program. The issue is that the comment states an enforcement mechanism that does not exist.Should the comment drop "and for the main program" and point at the
BPF_EXITcheck instead?
Yes, will do.
Quoted 13 lines
Also: "
btf_check_subprog_call()drops any error other than-EFAULT" is slightly inaccurate.btf_check_subprog_call()actually returns the error and marks the BTF unreliable; its callers,check_func_call()andpush_callback_call(), do 'if (err ==-EFAULT) return err;' and otherwise continue.---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Re: [PATCH bpf-next v4 09/13] bpf: Enable aggregate return types up to 16 bytes
In reply to Yonghong SongQuoted 25 lines · Yonghong Song
Relax
btf_distill_func_proto()to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry:- a struct or union larger than 8 and up to 16 bytes, returned in the R0:R2 register pair, matching what LLVM emits for the BPF target;
- a struct or union up to 8 bytes, returned in R0 alone.A >8 byte scalar (
__int128) was already accepted and is unchanged. Everything else stays rejected: a return type larger than 16 bytes, and any type that__get_type_size()cannot return in registers at all (e.g. an array), which it already reports as ret < 0.
btf_distill_func_proto()also builds the trampoline (fentry/fexit/fmod_ret) andstruct_opsfunction models, so relaxing it widens what those can attach to. A >8 byte return stays rejected on every path that reads the target's return value: commitc48796aa6c39("bpf: Reject >8 byte return values on return-reading trampoline paths") covers fexit,fmod_retand fsession plus their_multivariants, andstruct_ops, and an fentry-only trampoline never setsBPF_TRAMP_F_CALL_ORIGso it does not touch the return value at all. A struct or union of 8 bytes or less is newly accepted for those paths; its single eightbyte is returned in R0 like any other scalar.
btf_validate_return_type()is relaxed as well, so that it accepts a by-value struct or union up to 16 bytes in addition to void and scalars.
Quoted 7 lines
With
btf_distill_func_proto()andbtf_validate_return_type()relaxed, the verifier, JIT, precision-backtracking and live-register support from the earlier patches becomes reachable: this final patch enables <=16 byte aggregate return values end to end.Signed-off-by Yonghong Song <[email protected]>---
...
Quoted 5 lines
kernel/bpf/btf.c +0
-0
kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 7599
Quoted 9 lines
kernel/bpf/btf.c +5
-1@@ -7592,7 +7592,12 @@ int btf_distill_func_proto(struct bpf_verifier_log *log, return -EINVAL; } ret = __get_type_size(btf, func->type, &t); -if (ret <0 ||btf_type_is_struct (t)) { +/* +* __get_type_size() already restricts a non-negative ret to void, a +* pointer, an int, an enum or a struct/union, so only the size is checked +* here. +*/
Nit: I'd drop this comment.
Quoted 4 lines
kernel/bpf/btf.c +1
-0+if (ret <0 || ret >16 ) { bpf_log(log, "The function %s return type %s is unsupported.\n", tname, btf_type_str(t));
...
Quoted 17 lines
kernel/bpf/btf.c +0
-0@@ -7988,6 +7993,35 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t)) return 0;kernel/bpf/btf.c +13
-0+if (btf_type_is_struct (t) && t->size <=16 ) { +/* +* A >8 byte struct/union is returned in the R0:R2 register pair. +* A global function is verified in isolation, so its caller models +* the return as an opaque R0:R2 scalar pair; it must therefore +* contain only scalars, otherwise a pointer field would be +* laundered into a scalar and escape provenance and reference +* tracking. That requirement is enforced here: do_check_common() +* propagates the error for global functions and for the main +* program. +* +* A local (static) function is verified inline and its R0:R2 are +* copied as precise register state (with the JIT forced on when
kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8018
Quoted 10 lines
kernel/bpf/btf.c +10
-0+* the pair is consumed), so a pointer field stays tracked and needs +* no such restriction. Accepting it here is not by itself what +* makes it legal: btf_check_subprog_call() drops any error other +* than -EFAULT. What it avoids is needlessly marking the +* subprogram's BTF unreliable. +* +* The main program (subprog 0) takes the scalar-only path as well, +* but its return value is the program's exit code, so a >8 byte +* return is rejected separately at BPF_EXIT. +*/
Nit: the comment is way too long, I'd drop the justification.
Especially given that at the moment btf_validate_return_type() would only be called from main/global subprograms/callback subprograms.
Quoted 9 lines
kernel/bpf/btf.c +6
-0+bool local_func = subprog && !is_global; + +if (local_func ||__btf_type_is_scalar_struct (env, btf, t,0 )) +return 0 ; + } +return
-EOPNOTSUPP;
}
...
Quoted 8 lines
kernel/bpf/verifier.c +0
-0@@ -11088,9 +11088,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_ }
/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */
kernel/bpf/verifier.c static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_ · line 11093
Quoted 6 lines
kernel/bpf/verifier.c +3
-3-static bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, -const struct btf *btf, -const struct btf_type *t,int rec) +bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, +const struct btf *btf, +const struct btf_type *t,int rec)
Nit: since this function is now exported, let's drop the '__' prefix.
Quoted 3 lines
{
const structbtf_type*member_type;
const structbtf_member*member;
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 09/13] bpf: Enable aggregate return types up to 16 bytes
Quoted 37 lines · Eduard Zingerman
Quoted 25 lines · Yonghong Song
Relax
btf_distill_func_proto()to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry:- a struct or union larger than 8 and up to 16 bytes, returned in the R0:R2 register pair, matching what LLVM emits for the BPF target;
- a struct or union up to 8 bytes, returned in R0 alone.A >8 byte scalar (
__int128) was already accepted and is unchanged. Everything else stays rejected: a return type larger than 16 bytes, and any type that__get_type_size()cannot return in registers at all (e.g. an array), which it already reports as ret < 0.
btf_distill_func_proto()also builds the trampoline (fentry/fexit/fmod_ret) andstruct_opsfunction models, so relaxing it widens what those can attach to. A >8 byte return stays rejected on every path that reads the target's return value: commitc48796aa6c39("bpf: Reject >8 byte return values on return-reading trampoline paths") covers fexit,fmod_retand fsession plus their_multivariants, andstruct_ops, and an fentry-only trampoline never setsBPF_TRAMP_F_CALL_ORIGso it does not touch the return value at all. A struct or union of 8 bytes or less is newly accepted for those paths; its single eightbyte is returned in R0 like any other scalar.
btf_validate_return_type()is relaxed as well, so that it accepts a by-value struct or union up to 16 bytes in addition to void and scalars.Quoted 7 lines
With
btf_distill_func_proto()andbtf_validate_return_type()relaxed, the verifier, JIT, precision-backtracking and live-register support from the earlier patches becomes reachable: this final patch enables <=16 byte aggregate return values end to end.Signed-off-by Yonghong Song <[email protected]>---
Acked-by Eduard Zingerman <[email protected]>...
Quoted 15 lines
Quoted 5 lines
kernel/bpf/btf.c +0
-0kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 7599
Quoted 9 lines
kernel/bpf/btf.c +5
-1@@ -7592,7 +7592,12 @@ int btf_distill_func_proto(struct bpf_verifier_log *log, return -EINVAL; } ret = __get_type_size(btf, func->type, &t); -if (ret <0 ||btf_type_is_struct (t)) { +/* +* __get_type_size() already restricts a non-negative ret to void, a +* pointer, an int, an enum or a struct/union, so only the size is checked +* here. +*/ Nit: I'd drop this comment.
Will do.
Quoted 29 lines
Quoted 5 lines
kernel/bpf/btf.c +1
-0+if (ret <0 || ret >16 ) { bpf_log(log, "The function %s return type %s is unsupported.\n", tname, btf_type_str(t));...
Quoted 22 lines
kernel/bpf/btf.c +18
-0@@ -7988,6 +7993,35 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t)) return 0; +if (btf_type_is_struct (t) && t->size <=16 ) { +/* +* A >8 byte struct/union is returned in the R0:R2 register pair. +* A global function is verified in isolation, so its caller models +* the return as an opaque R0:R2 scalar pair; it must therefore +* contain only scalars, otherwise a pointer field would be +* laundered into a scalar and escape provenance and reference +* tracking. That requirement is enforced here: do_check_common() +* propagates the error for global functions and for the main +* program. +* +* A local (static) function is verified inline and its R0:R2 are +* copied as precise register state (with the JIT forced on when +* the pair is consumed), so a pointer field stays tracked and needs +* no such restriction. Accepting it here is not by itself what +* makes it legal: btf_check_subprog_call() drops any error other +* than -EFAULT. What it avoids is needlessly marking the +* subprogram's BTF unreliable.
Quoted 8 lines
kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8018
Quoted 5 lines
kernel/bpf/btf.c +5
-0+* +* The main program (subprog 0) takes the scalar-only path as well, +* but its return value is the program's exit code, so a >8 byte +* return is rejected separately at BPF_EXIT. +*/ Nit: the comment is way too long, I'd drop the justification.
Especially given that at the momentbtf_validate_return_type()would only be called from main/global subprograms/callback subprograms.
Will do.
Quoted 19 lines
Quoted 10 lines
kernel/bpf/btf.c +6
-0+bool local_func = subprog && !is_global; + +if (local_func ||__btf_type_is_scalar_struct (env, btf, t,0 )) +return 0 ; + } + return -EOPNOTSUPP; }...
Quoted 7 lines
kernel/bpf/verifier.c +0
-0@@ -11088,9 +11088,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_ }
Quoted 8 lines
kernel/bpf/verifier.c static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_ · line 11093
Quoted 7 lines
kernel/bpf/verifier.c +3
-3/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ -static bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, -const struct btf *btf, -const struct btf_type *t,int rec) +bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, +const struct btf *btf, +const struct btf_type *t,int rec)Nit: since this function is now exported, let's drop the '__' prefix.
Will change.
Quoted 5 lines
Quoted 4 lines
kernel/bpf/verifier.c +0
-0{ const struct btf_type *member_type; const struct btf_member *member;...
Yonghong Song
reviewerPATCH v4 10/13 selftests/bpf: Add C tests for 16-byte returns in R0:R2
In reply to Yonghong SongAdd selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return. Covered are an __int128 return, a 16-byte struct return (from a static and from a global subprogram) and a 16-byte union return, plus __int128 and 16-byte struct returns from a kfunc, for which bpf_kfunc_call_test_i128() and bpf_kfunc_call_test_ret_pair() are added to bpf_testmod.
The R0:R2 convention is only emitted by LLVM 23 and newer. Each object records in a read-only has_reg_pair_ret flag which compiler built it; where that is false the programs are stubs and subtests report a skip rather than a pass.
The two kfunc subtests further depend on the JIT: bpf_add_kfunc_call() rejects a kfunc returning more than 8 bytes with -EOPNOTSUPP where bpf_jit_supports_kfunc_ret_reg_pair() is false. Those calls therefore live in an object of their own, and that load failing with -EOPNOTSUPP is what turns the two subtests into skips, so no list of the JITs implementing the pair needs to be kept here. A register-pair return from a BPF subprogram needs no JIT support, so the remaining subtests run everywhere.
Both kfuncs are restricted to x86_64 and arm64, and the two subtests report a skip elsewhere. pahole only BTF-encodes a function whose declared arguments sit in the ABI's argument registers, and an architecture that returns a value larger than 8 bytes through a hidden pointer (sret) shifts every one of them by a register. s390x is such an architecture: there pahole drops the function, resolve_btfids leaves the kfunc ID at 0, and register_btf_kfunc_id_set() then fails at module init, so bpf_testmod does not load at all and every test that needs it fails. This is not a property of the compiler -- __SIZEOF_INT128__ is defined by gcc on s390x, and a by-value struct return has nothing to do with __int128 in the first place -- so the guard is on the architecture. prog_tests/tracing_failure.c already restricts a __int128 return the same way.
--- .../selftests/bpf/prog_tests/aggregate_ret.c | 148 ++++++++++++++++++ .../bpf/progs/aggregate_ret_int128_c.c | 48 ++++++ .../bpf/progs/aggregate_ret_kfunc_c.c | 66 ++++++++ .../bpf/progs/aggregate_ret_struct_c.c | 82 ++++++++++ .../bpf/progs/aggregate_ret_union_c.c | 58 +++++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 32 ++++ .../bpf/test_kmods/bpf_testmod_kfunc.h | 13 ++ 7 files changed, 447 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +148 -0
@@ -0,0 +1,148 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <test_progs.h> +#include "aggregate_ret_int128_c.skel.h" +#include "aggregate_ret_struct_c.skel.h" +#include "aggregate_ret_union_c.skel.h" +#include "aggregate_ret_kfunc_c.skel.h" + +/* +* The bpf_testmod kfuncs returning more than 8 bytes are only built on x86_64 +* and arm64 (see bpf_testmod.c); everywhere else the tests calling them are +* skipped. +*/ +static bool has_ret_pair_kfuncs (void ) +{ +#if defined(__x86_64__) || defined(__aarch64__) +return true; +#else +return false; +#endif +} + +static void run_prog (struct bpf_program *prog) +{ +char buf[64 ] = {}; +int err, prog_fd; +LIBBPF_OPTS (bpf_test_run_opts, topts, + .data_in = buf, + .data_size_in = sizeof(buf), + .repeat = 1, + ); + + prog_fd =bpf_program__fd (prog); + err =bpf_prog_test_run_opts (prog_fd, &topts); +if (!ASSERT_OK (err,"test_run" )) +return ; + +ASSERT_EQ (topts.retval,0 ,"aggregate_ret_result" ); +} + +/* +* Run @prog as subtest @name. Where the register-pair return is unsupported +* the subtest reports a skip instead, so that the list of subtests does not +* depend on the compiler or on the architecture; @prog is unused then and may +* be NULL, for a caller that could not even open its object. +*/ +static void run_subtest (const char *name,struct bpf_program *prog,bool supported) +{ +if (!test__start_subtest (name)) +return ; + +if (!supported) { +test__skip (); +return ; + } + +run_prog (prog); +} + +static void test_int128_c (void ) +{ +struct aggregate_ret_int128_c *skel; + + skel =aggregate_ret_int128_c__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_int128_c_open_load" )) +return ; + +run_subtest ("int128_c" , skel->progs.aggregate_ret_int128_c_test, + skel->rodata->has_reg_pair_ret); + +aggregate_ret_int128_c__destroy (skel); +} + +static void test_struct_c (void ) +{ +struct aggregate_ret_struct_c *skel; + + skel =aggregate_ret_struct_c__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_struct_c_open_load" )) +return ; + +run_subtest ("struct_c" , skel->progs.aggregate_ret_struct_c_test, + skel->rodata->has_reg_pair_ret); + +run_subtest ("global_struct_c" , skel->progs.aggregate_ret_global_struct_c_test, + skel->rodata->has_reg_pair_ret); + +aggregate_ret_struct_c__destroy (skel); +} + +static void test_union_c (void ) +{ +struct aggregate_ret_union_c *skel; + + skel =aggregate_ret_union_c__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_union_c_open_load" )) +return ; + +run_subtest ("union_c" , skel->progs.aggregate_ret_union_c_test, + skel->rodata->has_reg_pair_ret); + +aggregate_ret_union_c__destroy (skel); +} + +static void test_kfunc_c (void ) +{ +struct aggregate_ret_kfunc_c *skel; +bool supported; +int err; + + skel =aggregate_ret_kfunc_c__open (); +if (!ASSERT_OK_PTR (skel,"skel_kfunc_c_open" )) +return ; + + supported = skel->rodata->has_reg_pair_ret &&has_ret_pair_kfuncs (); + +if (supported) { +/* +* Where the JIT cannot hand the second half of a >8-byte kfunc +* return back in R0:R2, bpf_add_kfunc_call() rejects the call +* with -EOPNOTSUPP. Asking the kernel keeps this test free of a +* list of the JITs that can, which would have to be updated as +* the rest of them learn. +*/ + err =aggregate_ret_kfunc_c__load (skel); +if (err == -EOPNOTSUPP ) + supported = false; +else if (!ASSERT_OK (err,"skel_kfunc_c_load" )) + goto out; + } + +run_subtest ("kfunc_int128_c" , skel->progs.aggregate_ret_kfunc_int128_c_test, + supported); + +run_subtest ("kfunc_struct_c" , skel->progs.aggregate_ret_kfunc_struct_c_test, + supported); + +out: +aggregate_ret_kfunc_c__destroy (skel); +} + +void test_aggregate_ret (void ) +{ +test_int128_c (); +test_struct_c (); +test_union_c (); +test_kfunc_c (); +}
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c new file +48 -0
48 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c new file +66 -0
66 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +82 -0
82 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c new file +58 -0
58 lines not on this page · raw message
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +32 -0
@@ -939,6 +939,34 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)return 0 ; } +/* +* A kfunc is only usable where the ABI hands its return value back in +* registers. s390x, for example, returns a by-value struct or union through a +* hidden pointer argument (sret) whatever its size. That pointer shifts every +* declared argument by one register, and pahole, which maps parameters to +* registers positionally, then skips the function with "unexpected register +* usage for parameter". resolve_btfids reports "no BTF func for kfunc" and +* leaves the ID at 0, which makes register_btf_kfunc_id_set() fail at module +* init, so the module does not load at all. +* +* Restrict these kfuncs to the architectures where the return value comes back +* in registers. A kfunc taking no argument has nothing for the sret pointer to +* displace and needs no guard, whatever it returns. +*/ +#if defined(__x86_64__) || defined(__aarch64__) +__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b) +{ +return (__int128)(((unsigned __int128 )(a + b) <<64 ) | (a - b)); +} + +__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) +{ +struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b }; + +return r; +} +#endif /* __x86_64__ || __aarch64__ */ + __bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,u64 e,u64 f,u64 g,u64 h,u64 i,u64 j)
@@ -1472,6 +1500,10 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2) BTF_ID_FLAGS(func, bpf_kfunc_call_test3) BTF_ID_FLAGS(func, bpf_kfunc_call_test4) BTF_ID_FLAGS(func, bpf_kfunc_call_test5) +#if defined(__x86_64__) || defined(__aarch64__) +BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128) +BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair) +#endif BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg) BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)BTF_ID_FLAGS (func, bpf_kfunc_call_stack_arg_mix)
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +13 -0
@@ -55,6 +55,15 @@ struct prog_test_big_arg {__u64 b; }; +/* +* A 16-byte struct returned by value from a kfunc: .hi comes back in R0 and +* .lo in R2. +*/ +struct prog_test_ret_pair { +__u64 hi; +__u64 lo; +}; +struct prog_test_fail1 {void *p;int x;
@@ -130,6 +139,10 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;struct sock *bpf_kfunc_call_test3 (struct sock *sk) __ksym; long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym; int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym; +#ifdef __SIZEOF_INT128__ +__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym; +#endif +struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym; __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 g, __u64 h, __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 10/13] selftests/bpf: Add C tests for 16-byte returns in R0:R2
Can the remaining subtests run everywhere? Looking at the same series, kernel/bpf/verifier.c makes a register-pair subprogram return JIT-only.
Quoted 2 lines
A register-pair return from a BPF subprogram needs no JIT support, so the remaining subtests run everywhere.
Can the remaining subtests run everywhere? Looking at the same series, kernel/bpf/verifier.c makes a register-pair subprogram return JIT-only.
In check_func_call() for global subprogs:
nregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;
/* The R0:R2 return convention is only implemented in the JIT ... */
if (nregs > 1)
env->prog->jit_required = 1;and in prepare_func_exit() for static subprogs:
nregs = bpf_ret_reg_pair(env, callee->subprogno) ? 2 : 1;
if (nregs > 1)
env->prog->jit_required = 1;Then kernel/bpf/fixups.c rejects such a program outright when the JIT did not run:
#ifndef CONFIG_BPF_JIT_ALWAYS_ON
if (prog->jit_required) {
verbose(env, "program requires BPF JIT compiler but it is not available\n");
return -EINVAL;So on a kernel built with CONFIG_BPF_JIT=y but with net.core.bpf_jit_enable=0, or with no eBPF JIT at all, would aggregate_ret_int128_c__open_and_load() fail with -EINVAL?
Both kfuncs are restricted to x86_64 and arm64, and the two subtests report a skip elsewhere. pahole only BTF-encodes a function whose declared arguments sit in the ABI's argument registers, and an architecture that returns a value larger than 8 bytes through a hidden pointer (sret) shifts every one of them by a register. s390x is such an
architecture: there pahole drops the function, resolve_btfids leaves the kfunc ID at 0, and register_btf_kfunc_id_set() then fails at module init, so bpf_testmod does not load at all and every test that needs it fails. This is not a property of the compiler -- __SIZEOF_INT128__ is defined by gcc on s390x, and a by-value struct return has nothing to do with __int128 in the first place -- so the guard is on the architecture. prog_tests/tracing_failure.c already restricts a __int128 return the same way.
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +0
-0
[ ... ]
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c line 72
Quoted 13 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +13
-0+static void test_int128_c (void ) +{ +struct aggregate_ret_int128_c *skel; + + skel =aggregate_ret_int128_c__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_int128_c_open_load" )) +return ; + +run_subtest ("int128_c" , skel->progs.aggregate_ret_int128_c_test, + skel->rodata->has_reg_pair_ret); + +aggregate_ret_int128_c__destroy (skel); +}
Following up on the commit-message question: the four non-kfunc subtests (test_int128_c, test_struct_c, test_union_c with five run_subtest calls total) gate only on skel->rodata->has_reg_pair_ret, the compiler flag.
Where the kernel has CONFIG_BPF_JIT=y but net.core.bpf_jit_enable=0, or no JIT at all, wouldn't aggregate_ret_int128_c__open_and_load() fail with -EINVAL because fixups.c rejects prog->jit_required, causing the !ASSERT_OK_PTR check to fire? That would be a hard test failure rather than a skip, and because it happens before test__start_subtest() the subtest wouldn't even register.
The test already has is_jit_enabled() available (declared in testing_helpers.h and cached in env.jit_enabled), so folding that into the 'supported' argument of run_subtest() would turn these into skips.
test_struct_c() and test_union_c() have the same shape.
[ ... ]
tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c new file +0
-0
A subsystem pattern flags this as potentially concerning: a new selftest file added for a narrow variant (union vs struct return) of behavior an existing test already covers. The union and struct objects are both added in this commit and follow identical scaffolding (same compiler-version split, same has_reg_pair_ret flag, same MIX_A/MIX_B constants, a pair type, a make_pair() helper, one SEC("tc") program plus stub).
On the kernel side bpf_compute_subprog_ret_regs() keys off btf_type_is_struct(), which covers BTF_KIND_UNION and BTF_KIND_STRUCT alike, so the verifier path is shared. aggregate_ret_struct_c.c already hosts two programs behind one has_reg_pair_ret flag.
Would the union case fit as a third SEC("tc") program in the struct object, or does it benefit from being separate? The kfunc object has a stated reason to be separate (its load can fail with -EOPNOTSUPP), but the union vs struct distinction is less clear.
Quoted 25 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +17
-0@@ -939,6 +939,34 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)return 0 ; } +/* +* A kfunc is only usable where the ABI hands its return value back in +* registers. s390x, for example, returns a by-value struct or union through a +* hidden pointer argument (sret) whatever its size. That pointer shifts every +* declared argument by one register, and pahole, which maps parameters to +* registers positionally, then skips the function with "unexpected register +* usage for parameter". resolve_btfids reports "no BTF func for kfunc" and +* leaves the ID at 0, which makes register_btf_kfunc_id_set() fail at module +* init, so the module does not load at all. +* +* Restrict these kfuncs to the architectures where the return value comes back +* in registers. A kfunc taking no argument has nothing for the sret pointer to +* displace and needs no guard, whatever it returns. +*/ +#if defined(__x86_64__) || defined(__aarch64__) +__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b) +{
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c) · line 968
Quoted 10 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +10
-0+return (__int128)(((unsigned __int128 )(a + b) <<64 ) | (a - b)); +} + +__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) +{ +struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b }; + +return r; +} +#endif /* __x86_64__ || __aarch64__ */
The comment states the criterion as "Restrict these kfuncs to the architectures where the return value comes back in registers", but the guard implements a two-entry allowlist. Does riscv64 also meet this criterion?
The LP64 RISC-V ABI returns a 16-byte two-integer aggregate (and an __int128) in a0:a1, so nothing displaces the declared u64 arguments and pahole would encode the function. In this same series, arch/riscv/net/bpf_jit_comp64.c:2114 already opts in:
bool bpf_jit_supports_kfunc_ret_reg_pair(void)
{
return true;
}On riscv64 the module does not define these two kfuncs, so has_ret_pair_kfuncs() returns false and both kfunc subtests skip. That means the one architecture whose R0:R2 kfunc path the series describes as untested ("The riscv path is expected to work by the same register-mapping reasoning as arm64 but has not been tested") is also the one the new coverage silently never reaches.
Either the comment should explain why the allowlist is narrower than the stated criterion (e.g. only the arches the pair return has been tested on), or __riscv (with a 64-bit pointer-width check if needed) should be added so the new subtests actually exercise the riscv JIT path.
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 10/13] selftests/bpf: Add C tests for 16-byte returns in R0:R2
Quoted 21 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 2 lines
A register-pair return from a BPF subprogram needs no JIT support, so the remaining subtests run everywhere.
Can the remaining subtests run everywhere? Looking at the same series,
kernel/bpf/verifier.cmakes a register-pair subprogram return JIT-only.In
check_func_call()for global subprogs:nregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1; /* The R0:R2 return convention is only implemented in the JIT ... */ if (nregs > 1) env->prog->jit_required = 1;and in
prepare_func_exit()for static subprogs:nregs = bpf_ret_reg_pair(env, callee->subprogno) ? 2 : 1; if (nregs > 1) env->prog->jit_required = 1;Then
kernel/bpf/fixups.crejects such a program outright when the JIT did not run:
Quoted 8 lines
#ifndef CONFIG_BPF_JIT_ALWAYS_ON if (prog->jit_required) { verbose(env, "program requires BPF JIT compiler but it is not available\n"); return -EINVAL;So on a kernel built with
CONFIG_BPF_JIT=ybut with net.core.bpf_jit_enable=0, or with no eBPF JIT at all, wouldaggregate_ret_int128_c__open_and_load()fail with-EINVAL?
I missed this case (net.core.bpf_jit_enable=0). Let me add this matrix for selftest run.
Quoted 46 lines
Both kfuncs are restricted to
x86_64and arm64, and the two subtests report a skip elsewhere. pahole only BTF-encodes a function whose declared arguments sit in the ABI's argument registers, and an architecture that returns a value larger than 8 bytes through a hidden pointer (sret) shifts every one of them by a register. s390x is such an
architecture: there pahole drops the function,resolve_btfidsleaves the kfunc ID at 0, andregister_btf_kfunc_id_set()then fails at module init, sobpf_testmoddoes not load at all and every test that needs it fails. This is not a property of the compiler --__SIZEOF_INT128__is defined by gcc on s390x, and a by-value struct return has nothing to do with__int128in the first place -- so the guard is on the architecture.prog_tests/tracing_failure.calready restricts a__int128return the same way.Signed-off-by Yonghong Song <[email protected]>Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +0
-0[ ... ]
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c line 72
Quoted 13 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +13
-0+static void test_int128_c (void ) +{ +struct aggregate_ret_int128_c *skel; + + skel =aggregate_ret_int128_c__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_int128_c_open_load" )) +return ; + +run_subtest ("int128_c" , skel->progs.aggregate_ret_int128_c_test, + skel->rodata->has_reg_pair_ret); + +aggregate_ret_int128_c__destroy (skel); +}Following up on the commit-message question: the four non-kfunc subtests (
test_int128_c,test_struct_c,test_union_cwith fiverun_subtestcalls total) gate only onskel->rodata->has_reg_pair_ret, the compiler flag.Where the kernel has
CONFIG_BPF_JIT=ybut net.core.bpf_jit_enable=0, or no JIT at all, wouldn'taggregate_ret_int128_c__open_and_load()fail with-EINVALbecausefixups.crejectsprog->jit_required, causing the !ASSERT_OK_PTR check to fire? That would be a hard test failure rather than a skip, and because it happens beforetest__start_subtest()the subtest wouldn't even register.
Quoted 5 lines
The test already has
is_jit_enabled()available (declared intesting_helpers.hand cached in env.jit_enabled), so folding that into the 'supported' argument ofrun_subtest()would turn these into skips.
test_struct_c()andtest_union_c()have the same shape.
Okay, I will check net.core.bpf_jit_enable=0 and env.jit_enabled. I missed net.core.bpf_jit_enable case.
Quoted 18 lines
[ ... ]
tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c new file +0
-0A subsystem pattern flags this as potentially concerning: a new selftest file added for a narrow variant (union vs struct return) of behavior an existing test already covers. The union and struct objects are both added in this commit and follow identical scaffolding (same compiler-version split, same
has_reg_pair_retflag, same MIX_A/MIX_B constants, a pair type, amake_pair()helper, oneSEC("tc")program plus stub).On the kernel side
bpf_compute_subprog_ret_regs()keys offbtf_type_is_struct(), which coversBTF_KIND_UNIONandBTF_KIND_STRUCTalike, so the verifier path is shared.aggregate_ret_struct_c.calready hosts two programs behind onehas_reg_pair_retflag.
Quoted 4 lines
Would the union case fit as a third
SEC("tc")program in the struct object, or does it benefit from being separate? The kfunc object has a stated reason to be separate (its load can fail with-EOPNOTSUPP), but the union vs struct distinction is less clear.
Okay, I will move union case to struct case. So we just do struct case collectively (where union cases will be in).
Quoted 58 lines
Quoted 26 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +17
-0@@ -939,6 +939,34 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)return 0 ; } +/* +* A kfunc is only usable where the ABI hands its return value back in +* registers. s390x, for example, returns a by-value struct or union through a +* hidden pointer argument (sret) whatever its size. That pointer shifts every +* declared argument by one register, and pahole, which maps parameters to +* registers positionally, then skips the function with "unexpected register +* usage for parameter". resolve_btfids reports "no BTF func for kfunc" and +* leaves the ID at 0, which makes register_btf_kfunc_id_set() fail at module +* init, so the module does not load at all. +* +* Restrict these kfuncs to the architectures where the return value comes back +* in registers. A kfunc taking no argument has nothing for the sret pointer to +* displace and needs no guard, whatever it returns. +*/ +#if defined(__x86_64__) || defined(__aarch64__) +__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b) +{tools/testing/selftests/bpf/test_kmods/bpf_testmod.c __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c) · line 968
Quoted 10 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +10
-0+return (__int128)(((unsigned __int128 )(a + b) <<64 ) | (a - b)); +} + +__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) +{ +struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b }; + +return r; +} +#endif /* __x86_64__ || __aarch64__ */ The comment states the criterion as "Restrict these kfuncs to the architectures where the return value comes back in registers", but the guard implements a two-entry allowlist. Does riscv64 also meet this criterion?
The LP64 RISC-V ABI returns a 16-byte two-integer aggregate (and an
__int128) in a0:a1, so nothing displaces the declared u64 arguments and pahole would encode the function. In this same series,arch/riscv/net/bpf_jit_comp64.c:2114already opts in:bool bpf_jit_supports_kfunc_ret_reg_pair(void) { return true; }On riscv64 the module does not define these two kfuncs, so
has_ret_pair_kfuncs()returns false and both kfunc subtests skip. That means the one architecture whose R0:R2 kfunc path the series describes as untested ("The riscv path is expected to work by the same register-mapping reasoning as arm64 but has not been tested") is also the one the new coverage silently never reaches.
Quoted 4 lines
Either the comment should explain why the allowlist is narrower than the stated criterion (e.g. only the arches the pair return has been tested on), or
__riscv(with a 64-bit pointer-width check if needed) should be added so the new subtests actually exercise the riscv JIT path.
Okay, I will skip riscv64 for now.
Quoted 9 lines
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 10/13] selftests/bpf: Add C tests for 16-byte returns in R0:R2
In reply to Yonghong SongOn Mon, 2026-08-10 at 17:10 -0700, Yonghong Song wrote:
...
Quoted 1 line
tools/testing/selftests/bpf/prog_tests renamed from tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +0
-0
...
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests renamed from tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +4
-0+static void run_subtest(const char *name, struct bpf_program *prog, bool supported) +{ + if (!test__start_subtest(name)) + return;
tools/testing/selftests/bpf/prog_tests
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests renamed from tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +10
-0+ + if (!supported) { + test__skip(); + return; + } + + run_prog(prog); +} + +static void test_int128_c(void)
Can this set of runner functions be replaced with RUN_TEST() macro and __retval() annotations on the BPF side?
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 10/13] selftests/bpf: Add C tests for 16-byte returns in R0:R2
Quoted 7 lines · Eduard Zingerman
On Mon, 2026-08-10 at 17:10 -0700, Yonghong Song wrote:
...
Quoted 1 line
tools/testing/selftests/bpf/prog_tests renamed from tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +0
-0...
Quoted 16 lines
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests renamed from tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +4
-0+static void run_subtest(const char *name, struct bpf_program *prog, bool supported) +{ + if (!test__start_subtest(name)) + return;tools/testing/selftests/bpf/prog_tests
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests renamed from tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +10
-0+ + if (!supported) { + test__skip(); + return; + } + + run_prog(prog); +} + +static void test_int128_c(void)Can this set of runner functions be replaced with
RUN_TEST()macro and__retval()annotations on the BPF side?
Will use RUN_TEST().
Quoted 2 lines
...
Yonghong Song
reviewerPATCH v4 11/13 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
In reply to Yonghong SongAdd inline-asm tests, which do not depend on the compiler lowering a register-pair return and so run regardless of the LLVM version, covering what the C tests cannot reach. aggregate_ret_func.c exercises BPF-to-BPF
returns: a global subprogram whose R2 the caller may read, ones that leave R2 uninitialised or holding a pointer, a static subprogram whose R2 stays precise under backtracking, R2 liveness across a call, and a >8 byte return at program exit. Six kfuncs returning aggregates by value are added to bpf_testmod, and aggregate_ret_run.c calls them from inline asm to check what comes back in R0:R2.
A negative arena test is added as well: a global subprogram with a register-pair return that leaves an arena pointer in R2 is rejected, since an arena pointer is only a valid return value when it is returned in R0 alone.
Three cases cover the boundaries of the new convention:
- A return value larger than 16 bytes does not fit in R0:R2 and is rejected by btf_distill_func_proto(), ahead of the KF_FASTCALL and JIT-capability checks; one of the new kfuncs returns a 24-byte struct for this. The equivalent for a BPF subprogram cannot be written in C:
from LLVM 23 on, a by-value return larger than 16 bytes is lowered to an sret pointer argument and the BTF the verifier reads says the function returns void, so the size bound in btf_validate_return_type() only guards hand-crafted BTF.
- A static subprogram returning a struct that contains a pointer is accepted, and the caller can use the returned pointer. Unlike a global subprogram, whose caller models the return as an opaque scalar pair, a static one is verified inline, so prepare_func_exit() hands the caller real register state and the pointer stays tracked.
- An extension cannot replace a function returning more than 8 bytes. btf_check_type_match() does not catch this, since it compares return types by btf_type->info alone and both an __int128 and a __u64 are BTF_KIND_INT with no vlen, so the rejection has to come from bpf_check_attach_target(). The test reuses the freplace failure harness in fexit_bpf2bpf.c, with aggregate_ret_target.c providing a target whose global subprogram returns in R0:R2.
The kfunc tests need the JIT to place the second half of a return value into R2, which bpf_add_kfunc_call() only allows where bpf_jit_supports_kfunc_ret_reg_pair() is true. In aggregate_ret_kfunc.c the two tests that depend on getting past that check are tagged __arch_x86_64/__arch_arm64/__arch_riscv64; the others are rejected earlier (on KF_FASTCALL, on a >16 byte return, and on reading R2 after an 8-byte struct return). In aggregate_ret_run.c the kfunc-calling programs are dropped from the object when the load reports -EOPNOTSUPP and their subtests are skipped, and the __int128 inline-asm test is split into a BPF-to-BPF half, which needs no JIT capability and runs everywhere, and a kfunc half.
Five of the six new kfuncs return a struct or union by value and take arguments, so they join the x86_64/arm64 guard added in the previous patch; see the comment there. That includes the one returning only 8 bytes: s390x hands back a by-value aggregate through an sret pointer whatever its size, so its declared arguments are displaced just the same and pahole skips the function. The sixth returns 24 bytes but takes no argument, leaving nothing for the sret pointer to displace, and pahole does encode it there. Both objects calling the guarded kfuncs reference every one of them from __kfunc_btf_root(), so neither can load at all where those kfuncs are absent; the aggregate_ret_kfunc tests and the four aggregate_ret_run subtests are skipped as a group elsewhere. aggregate_ret_func.c calls no kfunc and keeps running everywhere.
R0 holds bytes 0..7 of a return value and R2 bytes 8..15, so where a member sits inside a register depends on the endianness of the target. The checks in aggregate_ret_run.c that read a member out of half a register are built for a little-endian target only; arm64 is the one JIT implementing the pair that can be built big endian, and that configuration is left for later.
--- .../selftests/bpf/prog_tests/aggregate_ret.c | 58 +++ .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 15 + .../selftests/bpf/progs/aggregate_ret_func.c | 420 ++++++++++++++++++ .../selftests/bpf/progs/aggregate_ret_kfunc.c | 126 ++++++ .../selftests/bpf/progs/aggregate_ret_run.c | 178 ++++++++ .../bpf/progs/aggregate_ret_target.c | 29 ++ .../selftests/bpf/progs/freplace_ret_pair.c | 20 + .../selftests/bpf/progs/verifier_arena.c | 38 ++ .../selftests/bpf/test_kmods/bpf_testmod.c | 55 +++ .../bpf/test_kmods/bpf_testmod_kfunc.h | 35 ++ 10 files changed, 974 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_run.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +58 -0
@@ -5,6 +5,9 @@#include "aggregate_ret_struct_c.skel.h" #include "aggregate_ret_union_c.skel.h" #include "aggregate_ret_kfunc_c.skel.h" +#include "aggregate_ret_run.skel.h" +#include "aggregate_ret_func.skel.h" +#include "aggregate_ret_kfunc.skel.h" /* * The bpf_testmod kfuncs returning more than 8 bytes are only built on x86_64
@@ -139,10 +142,65 @@ static void test_kfunc_c(void)aggregate_ret_kfunc_c__destroy (skel); } +static void test_run (void ) +{ +struct aggregate_ret_run *skel; +bool kfunc_ok = true; +int err; + +/* +* Every program in this object shares __kfunc_btf_root(), so where the +* testmod kfuncs are absent the object cannot load at all -- including +* for the kfunc-free "asm" subtest. +*/ +if (!has_ret_pair_kfuncs ()) { +run_subtest ("asm" ,NULL , false); +run_subtest ("asm_kfunc" ,NULL , false); +run_subtest ("struct" ,NULL , false); +run_subtest ("union" ,NULL , false); +return ; + } + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_open" )) +return ; + + err =aggregate_ret_run__load (skel); +if (err == -EOPNOTSUPP ) { + kfunc_ok = false; +aggregate_ret_run__destroy (skel); + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_reopen" )) +return ; + +bpf_program__set_autoload (skel->progs.aggregate_ret_asm_kfunc_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_struct_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_union_test, false); + + err =aggregate_ret_run__load (skel); + } +if (!ASSERT_OK (err,"skel_run_load" )) + goto out; + +run_subtest ("asm" , skel->progs.aggregate_ret_asm_test, true); +run_subtest ("asm_kfunc" , skel->progs.aggregate_ret_asm_kfunc_test, kfunc_ok); +run_subtest ("struct" , skel->progs.aggregate_ret_struct_test, kfunc_ok); +run_subtest ("union" , skel->progs.aggregate_ret_union_test, kfunc_ok); + +out: +aggregate_ret_run__destroy (skel); +} +void test_aggregate_ret (void ) {test_int128_c ();test_struct_c ();test_union_c ();test_kfunc_c (); +test_run (); + +RUN_TESTS (aggregate_ret_func); +if (has_ret_pair_kfuncs ()) +RUN_TESTS (aggregate_ret_kfunc); }
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +15 -0
@@ -441,6 +441,19 @@ static void test_func_replace_int_with_void(void) " doesn't match type INT of global_func2()"); } +static void test_func_replace_ret_pair (void ) +{ +const char *msg ="Cannot replace function agg_ret_target_func with a >8 byte return" ; + +/* +* An extension cannot replace a function whose return value comes back +* in the R0:R2 pair: the extension's own return is capped at 8 bytes, +* so it would leave R2 stale for the target's callers. +*/ +test_obj_load_failure_common ("freplace_ret_pair.bpf.o" , +"./aggregate_ret_target.bpf.o" , msg); +} +static int find_prog_btf_id (const char *name,__u32 attach_prog_fd) {struct bpf_prog_info info = {};
@@ -660,6 +673,8 @@ void serial_test_fexit_bpf2bpf(void)test_func_replace_progmap ();if (test__start_subtest ("freplace_int_with_void" ))test_func_replace_int_with_void (); +if (test__start_subtest ("freplace_ret_pair" )) +test_func_replace_ret_pair ();if (test__start_subtest ("freplace_void" ))test_func_replace_void ();if (test__start_subtest ("sleepable_fentry_to_xdp" ))
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +420 -0
420 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +126 -0
126 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +178 -0
178 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_target.c new file +29 -0
@@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" + +/* freplace target: a global subprogram returning 16 bytes in R0:R2. */ +__naked unsigned __int128 agg_ret_target_func (void ) +{ + asmvolatile ( +"r0 = 0x1234;" +"r2 = 0x5678;" +"exit;" + ); +} + +SEC("tc") +__naked int agg_ret_target (void ) +{ + asmvolatile ( +"call %[agg_ret_target_func];" +"r0 = 0;" +"exit;" + : + : __imm(agg_ret_target_func) + : __clobber_all); +} + +char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/freplace_ret_pair.c new file +20 -0
@@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> + +/* +* An extension replaces its target outright, so it has to match the target's +* return convention. Its own return value is capped at 8 bytes, so it can +* never fill the R0:R2 pair that the target's callers read, and the attach is +* rejected. btf_check_type_match() cannot catch this: it compares return types +* by btf_type->info only, and an int carries no vlen, so the __u64 here and +* the target's __int128 compare equal. +*/ +SEC("freplace/agg_ret_target_func") +__u64 new_agg_ret_target_func (void ) +{ +return 0 ; +} + +char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/verifier_arena.c +38 -0
@@ -704,4 +704,42 @@ int check_arena_arg_ret(void *ctx)return 0 ; } +struct arena_ret_pair { +__u64 lo; +__u64 hi; +}; + +/* +* A 16-byte value is returned in the R0:R2 register pair. A global subprogram +* may return an arena pointer in R0, but R2 holds the upper half of a scalar +* pair, so an arena pointer there is not a valid return value. The ld_imm64 of +* the arena map is what links the arena to the program, without which the +* addr_space_cast insn is not allowed. +*/ +__naked struct arena_ret_pair global_ret_arena_ptr_in_r2 (void ) +{ + asmvolatile ( +"r1 = %[arena] ll;" +"r2 = 8192;" +"r2 = addr_space_cast(r2, 0x0, 0x1);" +"r0 = 0;" +"exit;" + : + : __imm_addr(arena) + : __clobber_all); +} + +SEC("syscall") +__failure __msg("At subprogram exit the register R2 is not a scalar value (arena)") +__naked int check_global_ret_arena_ptr_in_r2(void) +{ + asmvolatile ( +"call %[global_ret_arena_ptr_in_r2];" +"r0 = 0;" +"exit;" + : + : __imm(global_ret_arena_ptr_in_r2) + : __clobber_all); +} + char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +55 -0
70 lines not on this page · raw message
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +35 -0
@@ -64,6 +64,35 @@ struct prog_test_ret_pair {__u64 lo; }; +struct prog_test_ret_li {/* 16 bytes: R0:R2 */ +__u64 a; +int b; +}; + +struct prog_test_ret_ii {/* 8 bytes: R0 only */ +int a; +int b; +}; + +union prog_test_ret_uu {/* 16 bytes: R0:R2 */ +__u64 halves[2 ]; +struct { +__u64 lo; +__u64 hi; + } parts; +}; + +struct prog_test_ret_ptr {/* 16 bytes: contains a pointer */ +void *p; +__u64 tag; +}; + +struct prog_test_ret_big {/* 24 bytes: too large for R0:R2 */ +__u64 a; +__u64 b; +__u64 c; +}; +struct prog_test_fail1 {void *p;int x;
@@ -143,6 +172,12 @@ int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym; __int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;#endif struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym; +struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym; +struct prog_test_ret_li bpf_kfunc_call_test_ret_li(__u64 a, int b) __ksym; +struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym; +union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(__u64 a, __u64 b) __ksym; +struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym; +struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void) __ksym; __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 g, __u64 h, __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 11/13] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
A subsystem pattern flags this as potentially concerning: has_ret_pair_kfuncs() checks only for x86_64 or arm64 architecture at compile time. On those architectures, if bpf_testmod.ko is not…
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +3
-0@@ -5,6 +5,9 @@#include "aggregate_ret_struct_c.skel.h" #include "aggregate_ret_union_c.skel.h" #include "aggregate_ret_kfunc_c.skel.h" +#include "aggregate_ret_run.skel.h" +#include "aggregate_ret_func.skel.h" +#include "aggregate_ret_kfunc.skel.h"
[ ... ]
Quoted 34 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +30
-0@@ -139,10 +142,65 @@ static void test_kfunc_c(void)aggregate_ret_kfunc_c__destroy (skel); } +static void test_run (void ) +{ +struct aggregate_ret_run *skel; +bool kfunc_ok = true; +int err; + +/* +* Every program in this object shares __kfunc_btf_root(), so where the +* testmod kfuncs are absent the object cannot load at all -- including +* for the kfunc-free "asm" subtest. +*/ +if (!has_ret_pair_kfuncs ()) { +run_subtest ("asm" ,NULL , false); +run_subtest ("asm_kfunc" ,NULL , false); +run_subtest ("struct" ,NULL , false); +run_subtest ("union" ,NULL , false); +return ; + } + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_open" )) +return ; + + err =aggregate_ret_run__load (skel); +if (err == -EOPNOTSUPP ) { + kfunc_ok = false; +aggregate_ret_run__destroy (skel); + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_reopen" ))
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c static void test_kfunc_c(void) · line 184
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +10
-0+return ; + +bpf_program__set_autoload (skel->progs.aggregate_ret_asm_kfunc_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_struct_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_union_test, false); + + err =aggregate_ret_run__load (skel); + } +if (!ASSERT_OK (err,"skel_run_load" )) + goto out;
A subsystem pattern flags this as potentially concerning:has_ret_pair_kfuncs() checks only for x86_64 or arm64 architecture at compile time. On those architectures, if bpf_testmod.ko is not actually loaded at runtime, aggregate_ret_run__load() returns -ESRCH rather than -EOPNOTSUPP, causing the reopen branch to be skipped and the test to report a hard FAIL rather than skipping. The comment states "where the testmod kfuncs are absent the object cannot load at all" but the code only guards one form of "absent" (architecture) and not the other (module not loaded).
test_progs tracks whether bpf_testmod.ko is loaded in env.has_testmod and sets it to false if load_bpf_testmod() fails. Every other testmod-dependent test in the tree checks both the architecture capability and env.has_testmod before attempting to use testmod kfuncs. Examples include bpf_cookie.c, iters.c, ksyms_module.c, btf_module.c, btf_tag.c, tp_btf_nullable.c, module_fentry_shadow.c, and core_reloc.c.
Should test_run() fold env.has_testmod into its guard to match the established pattern, so that the test skips cleanly in all environments where bpf_testmod is unavailable?
Quoted 12 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +10
-0+ +run_subtest ("asm" , skel->progs.aggregate_ret_asm_test, true); +run_subtest ("asm_kfunc" , skel->progs.aggregate_ret_asm_kfunc_test, kfunc_ok); +run_subtest ("struct" , skel->progs.aggregate_ret_struct_test, kfunc_ok); +run_subtest ("union" , skel->progs.aggregate_ret_union_test, kfunc_ok); + +out: +aggregate_ret_run__destroy (skel); +} +void test_aggregate_ret (void ) {
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c static void test_kfunc_c(void) · line 206
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +5
-0test_int128_c ();test_struct_c ();test_union_c ();test_kfunc_c (); +test_run (); + +RUN_TESTS (aggregate_ret_func); +if (has_ret_pair_kfuncs ()) +RUN_TESTS (aggregate_ret_kfunc); }
A subsystem pattern flags this as potentially concerning: the four test_run() subtests are registered as named skips when kfuncs are unavailable, with a comment stating "the list of subtests does not depend on the compiler or on the architecture." RUN_TESTS(aggregate_ret_kfunc) conditionally disappears, so the five aggregate_ret_kfunc_* subtests do not appear at all outside x86_64/arm64 - the subtest list does depend on the architecture. Anyone diffing test_progs output across architectures sees subtests vanish rather than skip. The two halves of the change implement opposite policies about whether an unavailable prerequisite should produce a visible skip.
Should RUN_TESTS(aggregate_ret_kfunc) use run_subtest() to produce named skips, matching the approach a few lines earlier?
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +0
-0
[ ... ]
Quoted 21 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +14
-0@@ -704,4 +704,42 @@ int check_arena_arg_ret(void *ctx)return 0 ; } +struct arena_ret_pair { +__u64 lo; +__u64 hi; +}; + +/* +* A 16-byte value is returned in the R0:R2 register pair. A global subprogram +* may return an arena pointer in R0, but R2 holds the upper half of a scalar +* pair, so an arena pointer there is not a valid return value. The ld_imm64 of +* the arena map is what links the arena to the program, without which the +* addr_space_cast insn is not allowed. +*/ +__naked struct arena_ret_pair global_ret_arena_ptr_in_r2 (void ) +{
tools/testing/selftests/bpf/progs/verifier_arena.c int check_arena_arg_ret(void *ctx) · line 730
Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +10
-0+ asmvolatile ( +"r1 = %[arena] ll;" +"r2 = 8192;" +"r2 = addr_space_cast(r2, 0x0, 0x1);" +"r0 = 0;" +"exit;" + : + : __imm_addr(arena) + : __clobber_all); +}
Does this inline-asm instruction build on clang older than LLVM 18? The addr_space_cast mnemonic was introduced in LLVM 18 along with __BPF_FEATURE_ADDR_SPACE_CAST. Every other inline-asm use of addr_space_cast in tools/testing/selftests/bpf/progs/ is guarded:
verifier_arena.c at lines 511-638 wraps six uses inside '#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)'.
arena_atomics.c guards its uses with '#if defined(ENABLE_ATOMICS_TESTS) && defined(__BPF_FEATURE_ADDR_SPACE_CAST)'.
verifier_zext.c guards with '#ifdef __BPF_FEATURE_ADDR_SPACE_CAST'.
compute_live_registers.c guards with '#ifdef__BPF_FEATURE_ADDR_SPACE_CAST'.
verifier_ldsx.c guards with '__clang_major__ >= 18'.
The new code here is the only unguarded occurrence in the tree. On older clang this asm string will not parse and verifier_arena.bpf.o will fail to build.
Quoted 3 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +0
-0
[ ... ]
Quoted 4 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +0
-0@@ -0,0 +1,420 @@
[ ... ]
Quoted 46 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +46
-0+#if __clang_major__ >=23 + +struct pair { +__u64 hi; +__u64 lo; +}; + +union upair { +__u64 halves[2 ]; +struct { +__u64 lo; +__u64 hi; + } parts; +}; + +/* A by-value struct that smuggles a pointer, which must be rejected. */ +struct with_ptr { +void *p; +__u64 x; +}; + +/* A by-value union that smuggles a pointer, which must be rejected too. */ +union upair_with_ptr { +void *p; +__u64 halves[2 ]; +}; + +/* Global subprogram returning a scalar-only 16-byte struct in R0:R2. */ +__naked struct pair global_ret_struct (void ) +{ + asmvolatile ( +"r0 = 0x1234;" /* struct's first half */ +"r2 = 0x5678;" /* struct's second half */ +"exit;" + ); +} + +/* Global subprogram returning a scalar-only 16-byte union in R0:R2. */ +__naked union upair global_ret_union (void ) +{ + asmvolatile ( +"r0 = 0x1234;" +"r2 = 0x5678;" +"exit;" + ); +}
[ ... ]
Quoted 54 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +54
-0+__naked struct with_ptr global_ret_struct_ptr (void ) +{ + asmvolatile ( +"r0 = 0;" +"r2 = 0;" +"exit;" + ); +} + +SEC("tc") +__failure __msg("Global function global_ret_struct_ptr() has unsupported return type") +__naked int aggregate_ret_global_struct_ptr_fail(void) +{ + asmvolatile ( +"call %[global_ret_struct_ptr];" +"r0 = 0;" +"exit;" + : + : __imm(global_ret_struct_ptr) + : __clobber_all); +} + +__naked union upair_with_ptr global_ret_union_ptr (void ) +{ + asmvolatile ( +"r0 = 0;" +"r2 = 0;" +"exit;" + ); +} + +SEC("tc") +__failure __msg("Global function global_ret_union_ptr() has unsupported return type") +__naked int aggregate_ret_global_union_ptr_fail(void) +{ + asmvolatile ( +"call %[global_ret_union_ptr];" +"r0 = 0;" +"exit;" + : + : __imm(global_ret_union_ptr) + : __clobber_all); +} + +#endif /* __clang_major__ >= 23 */ + +static __naked u128 agg_callee (void ) +{ + asmvolatile ( +"r0 = 1;" +"r2 = 2;" +"exit;" + ); +}
[ ... ]
Quoted 17 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +17
-0+struct ptr_pair { +void *p; +__u64 x; +}; + +static __naked __noinline struct ptr_pair static_ret_ptr_pair(void) +{ + asmvolatile ( +"r0 = 0;" +"r2 = r1;" +"exit;" + ); +} + +SEC("tc") +__success __retval(0) +__naked int aggregate_ret_static_ptr_pair(void)
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 418
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +10
-0+{ + asmvolatile ( +"call %[static_ret_ptr_pair];" +"r1 = *(u32 *)(r2 + 0);" /* deref the returned ctx pointer */ +"r0 = 0;" +"exit;" + : + : __imm(static_ret_ptr_pair) + : __clobber_all); +}
The file treats the same construct - a __naked subprogram whose C prototype returns a 16-byte aggregate - two different ways, and exactly one of the two must be wrong.
Inside the guard at lines 218-349 sit global_ret_struct (struct pair), global_ret_union (union upair), global_ret_struct_ptr (struct with_ptr) and global_ret_union_ptr (union upair_with_ptr), plus their four test programs. Outside the guard, after #endif, lines 392-418 add struct ptr_pair, static_ret_ptr_pair() returning struct ptr_pair, and aggregate_ret_static_ptr_pair() - an unguarded naked subprogram returning a 16-byte struct by value, in the same file.
If the guard is unnecessary, then the four guarded tests are dead on every toolchain in use today. That costs test coverage:
'aggregate_ret_global_struct_ptr_fail' and 'aggregate_ret_global_union_ptr_fail' are the only coverage anywhere in the tree for the __btf_type_is_scalar_struct() rejection added by this series to btf_validate_return_type() in kernel/bpf/btf.c. That check is the security-relevant half of the new convention - it stops a global subprogram from laundering a pointer to its caller through half of an "opaque scalar pair". Under this interpretation that check ships with zero test coverage until clang 23 is widely deployed, and the two positive tests 'aggregate_ret_global_struct' / 'aggregate_ret_global_union' (the only ones proving a struct/union - as opposed to __int128 - actually reaches R0:R2) are dead too.
If the guard is necessary, then static_ret_ptr_pair is broken on clang < 23. The commit message states "a by-value return larger than 16 bytes is lowered to an sret pointer argument and the BTF the verifier reads says the function returns void". If a pre-23 backend does the same for any by-value aggregate return, then on clang < 23 static_ret_ptr_pair's BTF says void (or gains a hidden sret argument), bpf_ret_reg_pair(env, subprog) is false, prepare_func_exit() copies only R0 back into the caller, and the caller's 'r1 = *(u32 *)(r2 + 0)' hits R2 == NOT_INIT - the naked entry never wrote it. That would be 'R2 !read_ok' at verifier.c, so a test tagged '__success __retval(0)' fails to load. test_aggregate_ret() calls RUN_TESTS(aggregate_ret_func) unconditionally - no arch or compiler gate - so this would be a hard FAIL for essentially every current toolchain.
Evidence points at the guard being the mistake: (1) the same commit adds '__naked struct arena_ret_pair global_ret_arena_ptr_in_r2(void)' to progs/verifier_arena.c unguarded; (2) the same commit adds '__naked u128 agg_ret_target_func(void)' in progs/aggregate_ret_target.c unguarded; (3) progs/aggregate_ret_run.c's own comment says taking the address rather than calling "avoids any dependence on the compiler lowering an __int128 or struct return value". A naked body has no return statement to lower, so the source-level prototype is what lands in BTF.
Either way the asymmetry needs resolving. If the guard is superfluous, could it be dropped or replaced with the '#if defined(__clang_major__) && __clang_major__ >= 23' + '#else' fallback shape used by aggregate_ret_struct_c.c? If it is not superfluous, should static_ret_ptr_pair and aggregate_ret_static_ptr_pair move inside it, and should progs/verifier_arena.c receive the same treatment?
The lore discussion with Eduard Zingerman on v1 suggested moving the liveness test (aggregate_ret_live) to compute_live_registers.c, where such tests are usually added. You agreed ('I can move the test to compute_live_registers.c') but the test remains in aggregate_ret_func.c in v4. Should aggregate_ret_live be moved to compute_live_registers.c to follow the established pattern?
The same v1 review suggested keeping only the C version of the global union test (not assembly) to reduce duplication. The assembly version aggregate_ret_global_union still exists in aggregate_ret_func.c. Would dropping the assembly version match the intent?
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +0
-0
[ ... ]
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +0
-0
[ ... ]
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_target.c new file +0
-0
[ ... ]
tools/testing/selftests/bpf/progs/freplace_ret_pair.c
Quoted 3 lines
tools/testing/selftests/bpf/progs/freplace_ret_pair.c new file +0
-0
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 11/13] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
Quoted 74 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +3
-0@@ -5,6 +5,9 @@#include "aggregate_ret_struct_c.skel.h" #include "aggregate_ret_union_c.skel.h" #include "aggregate_ret_kfunc_c.skel.h" +#include "aggregate_ret_run.skel.h" +#include "aggregate_ret_func.skel.h" +#include "aggregate_ret_kfunc.skel.h" [ ... ]
Quoted 34 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +30
-0@@ -139,10 +142,65 @@ static void test_kfunc_c(void)aggregate_ret_kfunc_c__destroy (skel); } +static void test_run (void ) +{ +struct aggregate_ret_run *skel; +bool kfunc_ok = true; +int err; + +/* +* Every program in this object shares __kfunc_btf_root(), so where the +* testmod kfuncs are absent the object cannot load at all -- including +* for the kfunc-free "asm" subtest. +*/ +if (!has_ret_pair_kfuncs ()) { +run_subtest ("asm" ,NULL , false); +run_subtest ("asm_kfunc" ,NULL , false); +run_subtest ("struct" ,NULL , false); +run_subtest ("union" ,NULL , false); +return ; + } + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_open" )) +return ; + + err =aggregate_ret_run__load (skel); +if (err == -EOPNOTSUPP ) { + kfunc_ok = false; +aggregate_ret_run__destroy (skel); + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_reopen" ))tools/testing/selftests/bpf/prog_tests/aggregate_ret.c static void test_kfunc_c(void) · line 184
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +10
-0+return ; + +bpf_program__set_autoload (skel->progs.aggregate_ret_asm_kfunc_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_struct_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_union_test, false); + + err =aggregate_ret_run__load (skel); + } +if (!ASSERT_OK (err,"skel_run_load" )) + goto out;A subsystem pattern flags this as potentially concerning:
has_ret_pair_kfuncs()checks only forx86_64or arm64 architecture at compile time. On those architectures, if bpf_testmod.ko is not actually loaded at runtime,aggregate_ret_run__load()returns-ESRCHrather than-EOPNOTSUPP, causing the reopen branch to be skipped and the test to report a hard FAIL rather than skipping. The comment states "where the testmod kfuncs are absent the object cannot load at all" but the code only guards one form of "absent" (architecture) and not the other (module not loaded).
test_progstracks whether bpf_testmod.ko is loaded in env.has_testmod and sets it to false ifload_bpf_testmod()fails. Every other testmod-dependent test in the tree checks both the architecture capability and env.has_testmod before attempting to use testmod kfuncs. Examples includebpf_cookie.c,iters.c,ksyms_module.c,btf_module.c,btf_tag.c,tp_btf_nullable.c,module_fentry_shadow.c, andcore_reloc.c.
Quoted 3 lines
Should
test_run()fold env.has_testmod into its guard to match the established pattern, so that the test skips cleanly in all environments wherebpf_testmodis unavailable?
Yes, we should add env.has_testmod.
Quoted 34 lines
Quoted 13 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +10
-0+ +run_subtest ("asm" , skel->progs.aggregate_ret_asm_test, true); +run_subtest ("asm_kfunc" , skel->progs.aggregate_ret_asm_kfunc_test, kfunc_ok); +run_subtest ("struct" , skel->progs.aggregate_ret_struct_test, kfunc_ok); +run_subtest ("union" , skel->progs.aggregate_ret_union_test, kfunc_ok); + +out: +aggregate_ret_run__destroy (skel); +} +void test_aggregate_ret (void ) {tools/testing/selftests/bpf/prog_tests/aggregate_ret.c static void test_kfunc_c(void) · line 206
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +5
-0test_int128_c ();test_struct_c ();test_union_c ();test_kfunc_c (); +test_run (); + +RUN_TESTS (aggregate_ret_func); +if (has_ret_pair_kfuncs ()) +RUN_TESTS (aggregate_ret_kfunc); }A subsystem pattern flags this as potentially concerning: the four
test_run()subtests are registered as named skips when kfuncs are unavailable, with a comment stating "the list of subtests does not depend on the compiler or on the architecture."RUN_TESTS(aggregate_ret_kfunc)conditionally disappears, so the five aggregate_ret_kfunc_* subtests do not appear at all outside x86_64/arm64 - the subtest list does depend on the architecture. Anyone diffingtest_progsoutput across architectures sees subtests vanish rather than skip. The two halves of the change implement opposite policies about whether an unavailable prerequisite should produce a visible skip.
Quoted 2 lines
Should
RUN_TESTS(aggregate_ret_kfunc)userun_subtest()to produce named skips, matching the approach a few lines earlier?
Right, we can do skip. In the future, when additional arch gets support, those 'skip' can be changed to actual run.
Quoted 50 lines
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +0
-0[ ... ]
Quoted 21 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +14
-0@@ -704,4 +704,42 @@ int check_arena_arg_ret(void *ctx)return 0 ; } +struct arena_ret_pair { +__u64 lo; +__u64 hi; +}; + +/* +* A 16-byte value is returned in the R0:R2 register pair. A global subprogram +* may return an arena pointer in R0, but R2 holds the upper half of a scalar +* pair, so an arena pointer there is not a valid return value. The ld_imm64 of +* the arena map is what links the arena to the program, without which the +* addr_space_cast insn is not allowed. +*/ +__naked struct arena_ret_pair global_ret_arena_ptr_in_r2 (void ) +{tools/testing/selftests/bpf/progs/verifier_arena.c int check_arena_arg_ret(void *ctx) · line 730
Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +10
-0+ asmvolatile ( +"r1 = %[arena] ll;" +"r2 = 8192;" +"r2 = addr_space_cast(r2, 0x0, 0x1);" +"r0 = 0;" +"exit;" + : + : __imm_addr(arena) + : __clobber_all); +}Does this inline-asm instruction build on clang older than LLVM 18? The
addr_space_castmnemonic was introduced in LLVM 18 along with__BPF_FEATURE_ADDR_SPACE_CAST. Every other inline-asm use ofaddr_space_castintools/testing/selftests/bpf/progs/is guarded:
verifier_arena.cat lines 511-638 wraps six uses inside '#ifdefined(__BPF_FEATURE_ADDR_SPACE_CAST)'.
arena_atomics.cguards its uses with '#ifdefined(ENABLE_ATOMICS_TESTS)&&defined(__BPF_FEATURE_ADDR_SPACE_CAST)'.
verifier_zext.cguards with '#ifdef__BPF_FEATURE_ADDR_SPACE_CAST'.
Quoted 8 lines
compute_live_registers.cguards with '#ifdef__BPF_FEATURE_ADDR_SPACE_CAST'.
verifier_ldsx.cguards with '__clang_major__>= 18'.The new code here is the only unguarded occurrence in the tree. On older clang this asm string will not parse and verifier_arena.bpf.o will fail to build.
The CI is using llvm21, so we should be okay.
Quoted 200 lines
Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +0
-0[ ... ]
Quoted 4 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +0
-0@@ -0,0 +1,420 @@[ ... ]
Quoted 46 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +46
-0+#if __clang_major__ >=23 + +struct pair { +__u64 hi; +__u64 lo; +}; + +union upair { +__u64 halves[2 ]; +struct { +__u64 lo; +__u64 hi; + } parts; +}; + +/* A by-value struct that smuggles a pointer, which must be rejected. */ +struct with_ptr { +void *p; +__u64 x; +}; + +/* A by-value union that smuggles a pointer, which must be rejected too. */ +union upair_with_ptr { +void *p; +__u64 halves[2 ]; +}; + +/* Global subprogram returning a scalar-only 16-byte struct in R0:R2. */ +__naked struct pair global_ret_struct (void ) +{ + asmvolatile ( +"r0 = 0x1234;" /* struct's first half */ +"r2 = 0x5678;" /* struct's second half */ +"exit;" + ); +} + +/* Global subprogram returning a scalar-only 16-byte union in R0:R2. */ +__naked union upair global_ret_union (void ) +{ + asmvolatile ( +"r0 = 0x1234;" +"r2 = 0x5678;" +"exit;" + ); +}[ ... ]
Quoted 54 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +54
-0+__naked struct with_ptr global_ret_struct_ptr (void ) +{ + asmvolatile ( +"r0 = 0;" +"r2 = 0;" +"exit;" + ); +} + +SEC("tc") +__failure __msg("Global function global_ret_struct_ptr() has unsupported return type") +__naked int aggregate_ret_global_struct_ptr_fail(void) +{ + asmvolatile ( +"call %[global_ret_struct_ptr];" +"r0 = 0;" +"exit;" + : + : __imm(global_ret_struct_ptr) + : __clobber_all); +} + +__naked union upair_with_ptr global_ret_union_ptr (void ) +{ + asmvolatile ( +"r0 = 0;" +"r2 = 0;" +"exit;" + ); +} + +SEC("tc") +__failure __msg("Global function global_ret_union_ptr() has unsupported return type") +__naked int aggregate_ret_global_union_ptr_fail(void) +{ + asmvolatile ( +"call %[global_ret_union_ptr];" +"r0 = 0;" +"exit;" + : + : __imm(global_ret_union_ptr) + : __clobber_all); +} + +#endif /* __clang_major__ >= 23 */ + +static __naked u128 agg_callee (void ) +{ + asmvolatile ( +"r0 = 1;" +"r2 = 2;" +"exit;" + ); +}[ ... ]
Quoted 17 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +17
-0+struct ptr_pair { +void *p; +__u64 x; +}; + +static __naked __noinline struct ptr_pair static_ret_ptr_pair(void) +{ + asmvolatile ( +"r0 = 0;" +"r2 = r1;" +"exit;" + ); +} + +SEC("tc") +__success __retval(0) +__naked int aggregate_ret_static_ptr_pair(void)tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 418
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +10
-0+{ + asmvolatile ( +"call %[static_ret_ptr_pair];" +"r1 = *(u32 *)(r2 + 0);" /* deref the returned ctx pointer */ +"r0 = 0;" +"exit;" + : + : __imm(static_ret_ptr_pair) + : __clobber_all); +}The file treats the same construct - a
__nakedsubprogram whose C prototype returns a 16-byte aggregate - two different ways, and exactly one of the two must be wrong.Inside the guard at lines 218-349 sit
global_ret_struct(struct pair),global_ret_union(union upair),global_ret_struct_ptr(structwith_ptr) andglobal_ret_union_ptr(unionupair_with_ptr), plus their four test programs. Outside the guard, after #endif, lines 392-418 add structptr_pair,static_ret_ptr_pair()returning structptr_pair, andaggregate_ret_static_ptr_pair()- an unguarded naked subprogram returning a 16-byte struct by value, in the same file.If the guard is unnecessary, then the four guarded tests are dead on every toolchain in use today. That costs test coverage:
'aggregate_ret_global_struct_ptr_fail' and 'aggregate_ret_global_union_ptr_fail' are the only coverage anywhere in the tree for the__btf_type_is_scalar_struct()rejection added by this series tobtf_validate_return_type()inkernel/bpf/btf.c. That check is the security-relevant half of the new convention - it stops a global subprogram from laundering a pointer to its caller through half of an "opaque scalar pair". Under this interpretation that check ships with zero test coverage until clang 23 is widely deployed, and the two positive tests 'aggregate_ret_global_struct' / 'aggregate_ret_global_union' (the only ones proving a struct/union - as opposed to__int128- actually reaches R0:R2) are dead too.If the guard is necessary, then
static_ret_ptr_pairis broken on clang < 23. The commit message states "a by-value return larger than 16 bytes is lowered to an sret pointer argument and the BTF the verifier reads says the function returns void". If a pre-23 backend does the same for any by-value aggregate return, then on clang < 23 static_ret_ptr_pair's BTF says void (or gains a hidden sret argument),bpf_ret_reg_pair(env, subprog)is false,prepare_func_exit()copies only R0 back into the caller, and the caller's 'r1 = *(u32 *)(r2 + 0)' hits R2 ==NOT_INIT- the naked entry never wrote it. That would be 'R2 !read_ok' atverifier.c, so a test tagged '__success__retval(0)' fails to load.test_aggregate_ret()callsRUN_TESTS(aggregate_ret_func)unconditionally - no arch or compiler gate - so this would be a hard FAIL for essentially every current toolchain.Evidence points at the guard being the mistake: (1) the same commit adds '
__nakedstructarena_ret_pairglobal_ret_arena_ptr_in_r2(void)' toprogs/verifier_arena.cunguarded; (2) the same commit adds '__nakedu128agg_ret_target_func(void)' inprogs/aggregate_ret_target.cunguarded; (3) progs/aggregate_ret_run.c's own comment says taking the address rather than calling "avoids any dependence on the compiler lowering an__int128or struct return value". A naked body has no return statement to lower, so the source-level prototype is what lands in BTF.Either way the asymmetry needs resolving. If the guard is superfluous, could it be dropped or replaced with the '#if
defined(__clang_major__)&&__clang_major__>= 23' + '#else' fallback shape used byaggregate_ret_struct_c.c? If it is not superfluous, shouldstatic_ret_ptr_pairandaggregate_ret_static_ptr_pairmove inside it, and shouldprogs/verifier_arena.creceive the same treatment?
Quoted 6 lines
The lore discussion with Eduard Zingerman on v1 suggested moving the liveness test (
aggregate_ret_live) tocompute_live_registers.c, where such tests are usually added. You agreed ('I can move the test tocompute_live_registers.c') but the test remains inaggregate_ret_func.cin v4. Shouldaggregate_ret_livebe moved tocompute_live_registers.cto follow the established pattern?
Okay, will do.
Quoted 5 lines
The same v1 review suggested keeping only the C version of the global union test (not assembly) to reduce duplication. The assembly version
aggregate_ret_global_unionstill exists inaggregate_ret_func.c. Would dropping the assembly version match the intent?
Let me check if indeed there are some redundancy.
Quoted 27 lines
Quoted 4 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +0
-0[ ... ]
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +0
-0[ ... ]
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_target.c new file +0
-0[ ... ]
tools/testing/selftests/bpf/progs/freplace_ret_pair.c
Quoted 3 lines
tools/testing/selftests/bpf/progs/freplace_ret_pair.c new file +0
-0[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 11/13] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
On Wed, 2026-08-12 at 14:08 -0700, Yonghong Song wrote:
...
Quoted 8 lines
Quoted 8 lines
test_progstracks whether bpf_testmod.ko is loaded in env.has_testmod and sets it to false ifload_bpf_testmod()fails. Every other testmod-dependent test in the tree checks both the architecture capability and env.has_testmod before attempting to use testmod kfuncs. Examples includebpf_cookie.c,iters.c,ksyms_module.c,btf_module.c,btf_tag.c,tp_btf_nullable.c,module_fentry_shadow.c, andcore_reloc.c.
Quoted 5 lines
Quoted 3 lines
Should
test_run()fold env.has_testmod into its guard to match the established pattern, so that the test skips cleanly in all environments wherebpf_testmodis unavailable?Yes, we should add env.has_testmod.
Note that env.has_testmod handling is inconsistent across the testsuite. If anything, I think this flag has to be dropped entirely. I'd suggest to ignore this llm suggestion.
Yonghong Song
reviewerRe: [PATCH bpf-next v4 11/13] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
Quoted 12 lines · Eduard Zingerman
On Wed, 2026-08-12 at 14:08 -0700, Yonghong Song wrote:
...
Quoted 8 lines
Quoted 8 lines
test_progstracks whether bpf_testmod.ko is loaded in env.has_testmod and sets it to false ifload_bpf_testmod()fails. Every other testmod-dependent test in the tree checks both the architecture capability and env.has_testmod before attempting to use testmod kfuncs. Examples includebpf_cookie.c,iters.c,ksyms_module.c,btf_module.c,btf_tag.c,tp_btf_nullable.c,module_fentry_shadow.c, andcore_reloc.c.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
Should
test_run()fold env.has_testmod into its guard to match the established pattern, so that the test skips cleanly in all environments wherebpf_testmodis unavailable?Yes, we should add env.has_testmod.
Note that env.has_testmod handling is inconsistent across the testsuite. If anything, I think this flag has to be dropped entirely. I'd suggest to ignore this llm suggestion.
Okay, I will ignore env.has_testmod thing. I will use RUN_TEST() for this patch as well.
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 11/13] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
In reply to Yonghong SongQuoted 65 lines · Yonghong Song
Add inline-asm tests, which do not depend on the compiler lowering a register-pair return and so run regardless of the LLVM version, covering what the C tests cannot reach.
aggregate_ret_func.cexercises BPF-to-BPF
returns: a global subprogram whose R2 the caller may read, ones that leave R2 uninitialised or holding a pointer, a static subprogram whose R2 stays precise under backtracking, R2 liveness across a call, and a >8 byte return at program exit. Six kfuncs returning aggregates by value are added tobpf_testmod, andaggregate_ret_run.ccalls them from inline asm to check what comes back in R0:R2.A negative arena test is added as well: a global subprogram with a register-pair return that leaves an arena pointer in R2 is rejected, since an arena pointer is only a valid return value when it is returned in R0 alone.
Three cases cover the boundaries of the new convention:
- A return value larger than 16 bytes does not fit in R0:R2 and is rejected by
btf_distill_func_proto(), ahead of theKF_FASTCALLand JIT-capability checks; one of the new kfuncs returns a 24-byte struct for this. The equivalent for a BPF subprogram cannot be written in C:
from LLVM 23 on, a by-value return larger than 16 bytes is lowered to an sret pointer argument and the BTF the verifier reads says the function returns void, so the size bound inbtf_validate_return_type()only guards hand-crafted BTF.- A static subprogram returning a struct that contains a pointer is accepted, and the caller can use the returned pointer. Unlike a global subprogram, whose caller models the return as an opaque scalar pair, a static one is verified inline, so
prepare_func_exit()hands the caller real register state and the pointer stays tracked.- An extension cannot replace a function returning more than 8 bytes.
btf_check_type_match()does not catch this, since it compares return types bybtf_type->infoalone and both an__int128and a__u64areBTF_KIND_INTwith no vlen, so the rejection has to come frombpf_check_attach_target(). The test reuses the freplace failure harness infexit_bpf2bpf.c, withaggregate_ret_target.cproviding a target whose global subprogram returns in R0:R2.The kfunc tests need the JIT to place the second half of a return value into R2, which
bpf_add_kfunc_call()only allows wherebpf_jit_supports_kfunc_ret_reg_pair()is true. Inaggregate_ret_kfunc.cthe two tests that depend on getting past that check are tagged __arch_x86_64/__arch_arm64/__arch_riscv64; the others are rejected earlier (onKF_FASTCALL, on a >16 byte return, and on reading R2 after an 8-byte struct return). Inaggregate_ret_run.cthe kfunc-calling programs are dropped from the object when the load reports-EOPNOTSUPPand their subtests are skipped, and the__int128inline-asm test is split into a BPF-to-BPF half, which needs no JIT capability and runs everywhere, and a kfunc half.Five of the six new kfuncs return a struct or union by value and take arguments, so they join the x86_64/arm64 guard added in the previous patch; see the comment there. That includes the one returning only 8 bytes: s390x hands back a by-value aggregate through an sret pointer whatever its size, so its declared arguments are displaced just the same and pahole skips the function. The sixth returns 24 bytes but takes no argument, leaving nothing for the sret pointer to displace, and pahole does encode it there. Both objects calling the guarded kfuncs reference every one of them from
__kfunc_btf_root(), so neither can load at all where those kfuncs are absent; theaggregate_ret_kfunctests and the fouraggregate_ret_runsubtests are skipped as a group elsewhere.aggregate_ret_func.ccalls no kfunc and keeps running everywhere.
Quoted 5 lines
R0 holds bytes 0..7 of a return value and R2 bytes 8..15, so where a member sits inside a register depends on the endianness of the target. The checks in
aggregate_ret_run.cthat read a member out of half a register are built for a little-endian target only; arm64 is the one JIT implementing the pair that can be built big endian, and that configuration is left for later.
Nit: is such long commit message really warranted?
commit messages seem to be too verbose all across this series.
Quoted 3 lines
Signed-off-by Yonghong Song <[email protected]>---
...
Quoted 25 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +25
-0+static void test_run (void ) +{ +struct aggregate_ret_run *skel; +bool kfunc_ok = true; +int err; + +/* +* Every program in this object shares __kfunc_btf_root(), so where the +* testmod kfuncs are absent the object cannot load at all -- including +* for the kfunc-free "asm" subtest. +*/ +if (!has_ret_pair_kfuncs ()) { +run_subtest ("asm" ,NULL , false); +run_subtest ("asm_kfunc" ,NULL , false); +run_subtest ("struct" ,NULL , false); +run_subtest ("union" ,NULL , false); +return ; + } + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_open" )) +return ; + + err =aggregate_ret_run__load (skel); +if (err == -EOPNOTSUPP ) {
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c static void test_kfunc_c(void) · line 179
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +10
-0+ kfunc_ok = false; +aggregate_ret_run__destroy (skel); + + skel =aggregate_ret_run__open (); +if (!ASSERT_OK_PTR (skel,"skel_run_reopen" )) +return ; + +bpf_program__set_autoload (skel->progs.aggregate_ret_asm_kfunc_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_struct_test, false); +bpf_program__set_autoload (skel->progs.aggregate_ret_union_test, false);
Please find a way to avoid these special cases and rely exclusively on RUN_TESTS(). We have other tests that rely on specific archs / compiler version and such handling is not usually necessary.
Quoted 15 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +15
-0+ + err =aggregate_ret_run__load (skel); + } +if (!ASSERT_OK (err,"skel_run_load" )) + goto out; + +run_subtest ("asm" , skel->progs.aggregate_ret_asm_test, true); +run_subtest ("asm_kfunc" , skel->progs.aggregate_ret_asm_kfunc_test, kfunc_ok); +run_subtest ("struct" , skel->progs.aggregate_ret_struct_test, kfunc_ok); +run_subtest ("union" , skel->progs.aggregate_ret_union_test, kfunc_ok); + +out: +aggregate_ret_run__destroy (skel); +} +
...
Quoted 1 line
tools/testing/selftests/bpf/progs/aggregate_ret_func.c +0
-0
...
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 161
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c +10
-0+SEC("tc") +__success __retval(0) +__log_level(2) +__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1") +__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8") +__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10") +__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7") +__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2") +__msg("mark_precise: frame0: regs=r2 stack= before 12: (95) exit") +__msg ("mark_precise: frame1: regs=r2 stack= before 11: (b7) r2 = 4" )
Maybe add a log line showing that propagation process stopped here?
Quoted 17 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c +17
-0+__naked int aggregate_ret_static_precise (void ) +{ + asmvolatile ( +"call %[static_agg_precise];" +"r6 = r2;" /* derived from the aggregate's second half */ +"r6 &= 7;" /* keep it in [0, 7] to index the stack */ +"r1 = r10;" +"r1 += -8;" +"r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */ +"r0 = 0;" +"*(u8 *)(r1 + 0) = r0;" +"r0 = 0;" +"exit;" + : + : __imm(static_agg_precise) + : __clobber_all); +}
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 188
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c +10
-0+ +SEC("tc") +__success __retval(0) +__log_level(2) +__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1") +__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8") +__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10") +__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7") +__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2") +__msg ("mark_precise: frame0: regs=r2 stack= before 0: (85) call pc+9" )
And here.
Quoted 17 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c +17
-0+__naked int aggregate_ret_global_precise (void ) +{ + asmvolatile ( +"call %[global_agg_good];" +"r6 = r2;" /* derived from the aggregate's second half */ +"r6 &= 7;" /* keep it in [0, 7] to index the stack */ +"r1 = r10;" +"r1 += -8;" +"r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */ +"r0 = 0;" +"*(u8 *)(r1 + 0) = r0;" +"r0 = 0;" +"exit;" + : + : __imm(global_agg_good) + : __clobber_all); +}
...
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 267
Quoted 3 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c +3
-0+SEC("tc") +__success __retval(0) +int aggregate_ret_global_struct (void *ctx)
Tbh, for this and similar tests I'd keep either C or asm versions, keeping both seem excessive.
Quoted 16 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c +16
-0+{ +__u64 lo, hi; + + asmvolatile ( +"call %[global_ret_struct];" +"%[lo] = r0;" +"%[hi] = r2;" + : [lo]"=r" (lo), [hi]"=r" (hi) + : __imm(global_ret_struct) + : "r0", "r1", "r2", "r3", "r4", "r5"); +if (lo !=0x1234 ) +return 1 ; +if (hi !=0x5678 ) +return 2 ; +return 0 ; +}
...
Quoted 37 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +31
-0@@ -0,0 +1,178 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" +#include "../test_kmods/bpf_testmod_kfunc.h" + +typedef unsigned __int128 u128 ; + +/* +* Reference kfunc addresses to force those BTF to be emitted. Taking the address +* (rather than calling) avoids any dependence on the compiler lowering an __int128 +* or struct return value, which the BPF backend only supports from LLVM 23 on. +*/ +void __kfunc_btf_root (void ) +{ + asmvolatile ("" + : + :"r" (&bpf_kfunc_call_test_i128), + "r"(&bpf_kfunc_call_test_ret_pair), + "r"(&bpf_kfunc_call_test_ret_li), + "r"(&bpf_kfunc_call_test_ret_ii), + "r"(&bpf_kfunc_call_test_ret_uu)); +} + +#define I128_ASM_LO 0xABCDabcd12345678ULL +#define I128_ASM_HI 0x1234567890abcdefULL + +static __naked __noinline u128 make_i128_asm(void) +{ + asm volatile (
tools/testing/selftests/bpf/progs/aggregate_ret_run.c line 41
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +10
-0+ "r0 = %[lo] ll;" /* low 64 bits */ + "r2 = %[hi] ll;" /* high 64 bits */ + "exit;" + : + : __imm_const(lo, I128_ASM_LO), __imm_const(hi, I128_ASM_HI) + ); +} + +SEC("tc") +int aggregate_ret_asm_test (struct __sk_buff *skb)
Same here, let's keep either C or an asm variant.
tools/testing/selftests/bpf/progs/aggregate_ret_run.c line 51
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +10
-0+{ +__u64 lo, hi; + + asmvolatile ( +"call %[callee];" +"%[lo] = r0;" +"%[hi] = r2;" + : [lo]"=r" (lo), [hi]"=r" (hi) + : [callee]"i" (make_i128_asm) + : "r0", "r1", "r2", "r3", "r4", "r5"
Nit: we have __clobber_common macro.
Quoted 8 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +8
-0+ ); +if (lo !=I128_ASM_LO ) +return 1 ; +if (hi !=I128_ASM_HI ) +return 2 ; + +return 0 ; +}
tools/testing/selftests/bpf/progs/aggregate_ret_run.c line 69
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +10
-0+ +/* +* R0 holds bytes 0..7 of a kfunc return value and R2 bytes 8..15, so where a +* member sits inside a register depends on the endianness of the target. +* Although arm64 supports both little and big endian, for simplicity, only +* do little endian for now.. +*/ +SEC("tc") +int aggregate_ret_asm_kfunc_test (struct __sk_buff *skb) +{
The tests below are very similar to those added in the patch "selftests/bpf: Add C tests for 16-byte returns in R0:R2", Maybe convert these to C and keep in the same file?
Quoted 23 lines
tools/testing/selftests/bpf/progs/aggregate_ret_run.c new file +23
-0+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ +__u64 a = skb->len; +__u64 b = skb->len ^0xdeadbeefULL ; +__u64 lo, hi; + + asmvolatile ( +"r1 = %[a];" +"r2 = %[b];" +"call %[kfunc];" +"%[lo] = r0;" +"%[hi] = r2;" + : [lo]"=r" (lo), [hi]"=r" (hi) + : [a]"r" (a), [b]"r" (b), [kfunc]"i" (bpf_kfunc_call_test_i128) + :"r0" ,"r1" ,"r2" ,"r3" ,"r4" ,"r5" + ); +if (hi != a + b) +return 1 ; +if (lo != a - b) +return 2 ; +#endif + +return 0 ; +}
...
Yonghong Song
reviewerPATCH v4 12/13 selftests/bpf: Add tests for callbacks returning more than 8 bytes
In reply to Yonghong SongAdd two __failure tests covering the callback return-size checks:
- timer_ret_pair_fail: a bpf_timer callback declared to return more than 8 bytes, rejected by check_ld_imm() where the callback's PTR_TO_FUNC is created, with "callback function with >8-byte return value is not supported".
- exceptions_ret_pair_fail: an exception callback declared to return more than 8 bytes, rejected by do_check_common() when the callback subprogram is verified, with "exception cb cannot return value larger than 8 bytes".
Both callback bodies are written in inline asm so that the tests do not depend on LLVM 23 R0:R2 codegen and run on any compiler. The verifier reads the return type from BTF rather than from the instructions, so the >8 byte return prototype is supplied through __btf_func_path(), pointing at a companion btf__*.c program that exists only to carry that BTF.
--- .../selftests/bpf/prog_tests/exceptions.c | 2 + .../testing/selftests/bpf/prog_tests/timer.c | 2 + .../bpf/progs/btf__exceptions_ret_pair_fail.c | 10 ++++ .../bpf/progs/btf__timer_ret_pair_fail.c | 10 ++++ .../bpf/progs/exceptions_ret_pair_fail.c | 30 ++++++++++++ .../selftests/bpf/progs/timer_ret_pair_fail.c | 49 +++++++++++++++++++ 6 files changed, 103 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c create mode 100644 tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c create mode 100644 tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c create mode 100644 tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c
tools/testing/selftests/bpf/prog_tests/exceptions.c +2 -0
@@ -5,6 +5,7 @@#include "exceptions.skel.h" #include "exceptions_ext.skel.h" #include "exceptions_fail.skel.h" +#include "exceptions_ret_pair_fail.skel.h" #include "exceptions_assert.skel.h" static char log_buf[1024 *1024 ];
@@ -12,6 +13,7 @@ static char log_buf[1024 * 1024];static void test_exceptions_failure (void ) {RUN_TESTS (exceptions_fail); +RUN_TESTS (exceptions_ret_pair_fail); }static void test_exceptions_success (void )
tools/testing/selftests/bpf/prog_tests/timer.c +2 -0
@@ -6,6 +6,7 @@#include <sys/syscall.h> #include "timer.skel.h" #include "timer_failure.skel.h" +#include "timer_ret_pair_fail.skel.h" #include "timer_interrupt.skel.h" #define NUM_THR 8
@@ -285,6 +286,7 @@ void serial_test_timer(void)test_timer (timer);RUN_TESTS (timer_failure); +RUN_TESTS (timer_ret_pair_fail); }void serial_test_timer_stress (void )
tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c new file +10 -0
@@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> + +unsigned __int128 exception_cb_bad_ret_type3 (u64 cookie) +{ +for (;;) + ; +}
tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c new file +10 -0
@@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> + +unsigned __int128 timer_cb_ret_pair (void *map,int *key,struct bpf_timer *timer) +{ +for (;;) + ; +}
tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c new file +30 -0
@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_helpers.h> + +#include "bpf_misc.h" +#include "bpf_experimental.h" + +__naked __noinline __used +unsigned __int128 exception_cb_bad_ret_type3(u64 cookie) +{ + asmvolatile ( +"r0 = r1;" +"r2 = 0;" +"exit;" + ::: __clobber_all); +} + +SEC("?tc") +__exception_cb (exception_cb_bad_ret_type3) +__failure __msg("exception cb cannot return value larger than 8 bytes") +__btf_func_path("btf__exceptions_ret_pair_fail.bpf.o") +int reject_exception_cb_ret_pair (void *ctx) +{ +bpf_throw (0 ); +return 0 ; +} + +char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c new file +49 -0
49 lines not on this page · raw message
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v4 12/13] selftests/bpf: Add tests for callbacks returning more than 8 bytes
Does __btf_func_path() actually supply the return type? Looking at fixup_btf_from_path() in tools/testing/selftests/bpf/test_loader.c, it appears to keep the prog's return type and parameter types…
Quoted 14 lines
selftests/bpf: Add tests for callbacks returning more than 8 bytes
Add two
__failuretests covering the callback return-size checks:-
timer_ret_pair_fail: abpf_timercallback declared to return more than 8 bytes, rejected bycheck_ld_imm()where the callback'sPTR_TO_FUNCis created, with "callback function with >8-byte return value is not supported".-
exceptions_ret_pair_fail: an exception callback declared to return more than 8 bytes, rejected bydo_check_common()when the callback subprogram is verified, with "exception cb cannot return value larger than 8 bytes".
Quoted 5 lines
Both callback bodies are written in inline asm so that the tests do not depend on LLVM 23 R0:R2 codegen and run on any compiler. The verifier reads the return type from BTF rather than from the instructions, so the >8 byte return prototype is supplied through
__btf_func_path(), pointing at a companion btf__*.c program that exists only to carry that BTF.
Does __btf_func_path() actually supply the return type? Looking at fixup_btf_from_path() in tools/testing/selftests/bpf/test_loader.c, it appears to keep the prog's return type and parameter types while substituting only parameter names:
ret_type_id = fp->type; /* fp is the prog FUNC_PROTO */
for (j = 0; j < vlen; j++)
prog_param_types[j] = params[j].type; /* prog param types */
new_proto_id = btf__add_func_proto(prog_btf, ret_type_id);
...
pname = btf__name_by_offset(custom_btf, custom_params[j].name_off);
err = btf__add_func_param(prog_btf, pname ?: "", prog_param_types[j]);The comment at the call site says "Fix up __naked subprog BTF using a separate file with named params", and the >8 byte return type already comes from each test object's own BTF: exceptions_ret_pair_fail.c declares 'unsigned __int128 exception_cb_bad_ret_type3(u64 cookie)' and timer_ret_pair_fail.c declares 'static unsigned __int128 timer_cb_ret_pair(...)'.
The companion btf__*.c objects supply parameter names, which are needed because clang emits anonymous parameters for __naked functions and btf_func_check() in kernel/bpf/btf.c rejects a BTF_KIND_FUNC whose FUNC_PROTO has a typed-but-unnamed argument, which would make the whole object's BTF fail to load.
Could the explanation be more precise about what __btf_func_path() actually provides?
Quoted 43 lines
Signed-off-by Yonghong Song <[email protected]>tools/testing/selftests/bpf/prog_tests/exceptions.c +2
-0@@ -5,6 +5,7 @@#include "exceptions.skel.h" #include "exceptions_ext.skel.h" #include "exceptions_fail.skel.h" +#include "exceptions_ret_pair_fail.skel.h" #include "exceptions_assert.skel.h" static char log_buf[1024 *1024 ];@@ -12,6 +13,7 @@ static char log_buf[1024 * 1024];static void test_exceptions_failure (void ) {RUN_TESTS (exceptions_fail); +RUN_TESTS (exceptions_ret_pair_fail); }static void test_exceptions_success (void )tools/testing/selftests/bpf/prog_tests/timer.c +2
-0@@ -6,6 +6,7 @@#include <sys/syscall.h> #include "timer.skel.h" #include "timer_failure.skel.h" +#include "timer_ret_pair_fail.skel.h" #include "timer_interrupt.skel.h" #define NUM_THR 8@@ -285,6 +286,7 @@ void serial_test_timer(void)test_timer (timer);RUN_TESTS (timer_failure); +RUN_TESTS (timer_ret_pair_fail); }void serial_test_timer_stress (void )
[ ... ]
Quoted 26 lines
tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c new file +20
-0@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_helpers.h> + +#include "bpf_misc.h" +#include "bpf_experimental.h" + +__naked __noinline __used +unsigned __int128 exception_cb_bad_ret_type3(u64 cookie) +{ + asmvolatile ( +"r0 = r1;" +"r2 = 0;" +"exit;" + ::: __clobber_all); +} + +SEC ("?tc" )
tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c line 30
Quoted 10 lines
tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c new file +10
-0+__exception_cb (exception_cb_bad_ret_type3) +__failure __msg("exception cb cannot return value larger than 8 bytes") +__btf_func_path("btf__exceptions_ret_pair_fail.bpf.o") +int reject_exception_cb_ret_pair (void *ctx) +{ +bpf_throw (0 ); +return 0 ; +} + +char _license[] SEC("license") = "GPL";
A subsystem pattern flags this as potentially concerning:exceptions_ret_pair_fail.c is a new prog file added for one narrow variant of behaviour tools/testing/selftests/bpf/progs/exceptions_fail.c already covers: rejected exception-callback return types.
exceptions_fail.c already holds exception_cb_bad_ret_type1(), exception_cb_bad_ret_type2() and the corresponding '__failure __msg("exception cb cannot return void")' case. The new callback is even named exception_cb_bad_ret_type3, continuing that file's numbering while living in a different file, so a reader grepping exceptions_fail.c for the series will not find it next to its siblings.
There is a concrete technical reason the exceptions case cannot simply be appended to exceptions_fail.c: __btf_func_path also sets bpf_object_open_opts.btf_custom_path, and libbpf uses that as the CO-RE target BTF for the whole object. exceptions_fail.c includes <vmlinux.h>, which applies preserve_access_index to all records, and it dereferences vmlinux structs (ctx->len, ctx->protocol, ctx->tstamp), so it does have CO-RE relocations that would then be resolved against the 10-line companion BTF. That makes the split legitimate here; the remaining point is only that the shared naming (exception_cb_bad_ret_type3) does not reflect the split.
Is the naming pattern worth revisiting given that the test lives in a separate file?
Quoted 45 lines
tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c new file +39
-0@@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ + +#include <linux/bpf.h> +#include <time.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct elem { +struct bpf_timer t; +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, int); + __type(value, struct elem); +} timer_map SEC(".maps"); + +__naked __noinline __used +static unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer) +{ + asmvolatile ( +"r0 = 0;" +"r2 = 0;" +"exit;" + ::: __clobber_all + ); +} + +SEC("fentry/bpf_fentry_test1") +__failure __msg("callback function with >8-byte return value is not supported") +__btf_func_path("btf__timer_ret_pair_fail.bpf.o") +long BPF_PROG2 (test_bad_ret_pair ,int ,a ) +{ +int key =0 ;
tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c line 49
Quoted 10 lines
tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c new file +10
-0+struct bpf_timer *timer; + + timer =bpf_map_lookup_elem (&timer_map, &key); +if (timer) { +bpf_timer_init (timer, &timer_map,CLOCK_BOOTTIME ); +bpf_timer_set_callback (timer, timer_cb_ret_pair); + } + +return 0 ; +}
A subsystem pattern flags this as potentially concerning:timer_ret_pair_fail.c is a new prog file added for one narrow variant (a callback whose declared return value is larger than 8 bytes) of behaviour that tools/testing/selftests/bpf/progs/timer_failure.c already covers more generally: bad bpf_timer callback return values.
The two files are near-identical scaffolding - the same '#include <linux/bpf.h> / <time.h> / bpf_helpers.h / bpf_tracing.h / bpf_misc.h' set, the same 'struct elem { struct bpf_timer t; }', the same BPF_MAP_TYPE_ARRAY 'timer_map' with max_entries 1, the same SEC("fentry/bpf_fentry_test1") + BPF_PROG2 entry program, and the same '__naked __noinline __used static' callback idiom already used by timer_cb_ret_bad().
The new program adds ~10 lines of actual test content on top of ~35 lines of copied setup, plus a new skeleton, plus a companion btf__timer_ret_pair_fail.c, plus the RUN_TESTS()/#include churn in prog_tests/timer.c. Adding it as a third program inside timer_failure.c would reuse all of that. Since __btf_func_path is a per-program tag consumed per-subtest in run_subtest(), and timer_failure.c has no CO-RE relocations (it includes uapi <linux/bpf.h>, not <vmlinux.h>), the btf_custom_path side effect of the tag would not affect the existing subtests in that object.
A separate file is defensible for symmetry with the exceptions half of the patch, where a separate file IS required due to CO-RE relocations. Should the new timer case be a new subtest in timer_failure.c, or does the symmetry with exceptions_ret_pair_fail.c make the separate file structure clearer?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Yonghong Song
reviewerRe: [PATCH bpf-next v4 12/13] selftests/bpf: Add tests for callbacks returning more than 8 bytes
Quoted 39 lines · On 8/10/26 6:24 PM, [email protected] wrote:
Quoted 14 lines
selftests/bpf: Add tests for callbacks returning more than 8 bytes
Add two
__failuretests covering the callback return-size checks:-
timer_ret_pair_fail: abpf_timercallback declared to return more than 8 bytes, rejected bycheck_ld_imm()where the callback'sPTR_TO_FUNCis created, with "callback function with >8-byte return value is not supported".-
exceptions_ret_pair_fail: an exception callback declared to return more than 8 bytes, rejected bydo_check_common()when the callback subprogram is verified, with "exception cb cannot return value larger than 8 bytes".Quoted 5 lines
Both callback bodies are written in inline asm so that the tests do not depend on LLVM 23 R0:R2 codegen and run on any compiler. The verifier reads the return type from BTF rather than from the instructions, so the >8 byte return prototype is supplied through
__btf_func_path(), pointing at a companion btf__*.c program that exists only to carry that BTF.Does
__btf_func_path()actually supply the return type? Looking atfixup_btf_from_path()intools/testing/selftests/bpf/test_loader.c, it appears to keep the prog's return type and parameter types while substituting only parameter names:ret_type_id = fp->type; /* fp is the prog FUNC_PROTO */ for (j = 0; j < vlen; j++) prog_param_types[j] = params[j].type; /* prog param types */ new_proto_id = btf__add_func_proto(prog_btf, ret_type_id); ... pname = btf__name_by_offset(custom_btf, custom_params[j].name_off); err = btf__add_func_param(prog_btf, pname ?: "", prog_param_types[j]);The comment at the call site says "Fix up
__nakedsubprog BTF using a separate file with named params", and the >8 byte return type already comes from each test object's own BTF:exceptions_ret_pair_fail.cdeclares 'unsigned__int128exception_cb_bad_ret_type3(u64 cookie)' andtimer_ret_pair_fail.cdeclares 'static unsigned__int128timer_cb_ret_pair(...)'.
Quoted 8 lines
The companion btf__*.c objects supply parameter names, which are needed because clang emits anonymous parameters for
__nakedfunctions andbtf_func_check()inkernel/bpf/btf.crejects aBTF_KIND_FUNCwhoseFUNC_PROTOhas a typed-but-unnamed argument, which would make the whole object's BTF fail to load.Could the explanation be more precise about what
__btf_func_path()actually provides?
__btf_func_path() provides precise function BTF which won't be available for inline asm code.
Quoted 103 lines
Quoted 42 lines
Signed-off-by Yonghong Song <[email protected]>tools/testing/selftests/bpf/prog_tests/exceptions.c +2
-0@@ -5,6 +5,7 @@#include "exceptions.skel.h" #include "exceptions_ext.skel.h" #include "exceptions_fail.skel.h" +#include "exceptions_ret_pair_fail.skel.h" #include "exceptions_assert.skel.h" static char log_buf[1024 *1024 ];@@ -12,6 +13,7 @@ static char log_buf[1024 * 1024];static void test_exceptions_failure (void ) {RUN_TESTS (exceptions_fail); +RUN_TESTS (exceptions_ret_pair_fail); }static void test_exceptions_success (void )tools/testing/selftests/bpf/prog_tests/timer.c +2
-0@@ -6,6 +6,7 @@#include <sys/syscall.h> #include "timer.skel.h" #include "timer_failure.skel.h" +#include "timer_ret_pair_fail.skel.h" #include "timer_interrupt.skel.h" #define NUM_THR 8@@ -285,6 +286,7 @@ void serial_test_timer(void)test_timer (timer);RUN_TESTS (timer_failure); +RUN_TESTS (timer_ret_pair_fail); }void serial_test_timer_stress (void )[ ... ]
Quoted 26 lines
tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c new file +20
-0@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_helpers.h> + +#include "bpf_misc.h" +#include "bpf_experimental.h" + +__naked __noinline __used +unsigned __int128 exception_cb_bad_ret_type3(u64 cookie) +{ + asmvolatile ( +"r0 = r1;" +"r2 = 0;" +"exit;" + ::: __clobber_all); +} + +SEC ("?tc" )tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c line 30
Quoted 10 lines
tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c new file +10
-0+__exception_cb (exception_cb_bad_ret_type3) +__failure __msg("exception cb cannot return value larger than 8 bytes") +__btf_func_path("btf__exceptions_ret_pair_fail.bpf.o") +int reject_exception_cb_ret_pair (void *ctx) +{ +bpf_throw (0 ); +return 0 ; +} + +char _license[] SEC("license") = "GPL";A subsystem pattern flags this as potentially concerning:
exceptions_ret_pair_fail.cis a new prog file added for one narrow variant of behaviourtools/testing/selftests/bpf/progs/exceptions_fail.calready covers: rejected exception-callback return types.
exceptions_fail.calready holdsexception_cb_bad_ret_type1(),exception_cb_bad_ret_type2()and the corresponding '__failure__msg("exception cb cannot return void")' case. The new callback is even namedexception_cb_bad_ret_type3, continuing that file's numbering while living in a different file, so a reader greppingexceptions_fail.cfor the series will not find it next to its siblings.There is a concrete technical reason the exceptions case cannot simply be appended to
exceptions_fail.c:__btf_func_pathalso sets bpf_object_open_opts.btf_custom_path, and libbpf uses that as the CO-RE target BTF for the whole object.exceptions_fail.cincludes <vmlinux.h>, which appliespreserve_access_indexto all records, and it dereferences vmlinux structs (ctx->len,ctx->protocol,ctx->tstamp), so it does have CO-RE relocations that would then be resolved against the 10-line companion BTF. That makes the split legitimate here; the remaining point is only that the shared naming (exception_cb_bad_ret_type3) does not reflect the split.
Quoted 2 lines
Is the naming pattern worth revisiting given that the test lives in a separate file?
Right, the prog itself is in a dedicated file. But user space part is in prog_tests/exceptions.c.
Quoted 80 lines
Quoted 46 lines
tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c new file +39
-0@@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ + +#include <linux/bpf.h> +#include <time.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct elem { +struct bpf_timer t; +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __type(key, int); + __type(value, struct elem); +} timer_map SEC(".maps"); + +__naked __noinline __used +static unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer) +{ + asmvolatile ( +"r0 = 0;" +"r2 = 0;" +"exit;" + ::: __clobber_all + ); +} + +SEC("fentry/bpf_fentry_test1") +__failure __msg("callback function with >8-byte return value is not supported") +__btf_func_path("btf__timer_ret_pair_fail.bpf.o") +long BPF_PROG2 (test_bad_ret_pair ,int ,a ) +{ +int key =0 ;tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c line 49
Quoted 10 lines
tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c new file +10
-0+struct bpf_timer *timer; + + timer =bpf_map_lookup_elem (&timer_map, &key); +if (timer) { +bpf_timer_init (timer, &timer_map,CLOCK_BOOTTIME ); +bpf_timer_set_callback (timer, timer_cb_ret_pair); + } + +return 0 ; +}A subsystem pattern flags this as potentially concerning:
timer_ret_pair_fail.cis a new prog file added for one narrow variant (a callback whose declared return value is larger than 8 bytes) of behaviour thattools/testing/selftests/bpf/progs/timer_failure.calready covers more generally: badbpf_timercallback return values.The two files are near-identical scaffolding - the same '#include <
linux/bpf.h> / <time.h> /bpf_helpers.h/bpf_tracing.h/bpf_misc.h' set, the same 'struct elem { structbpf_timert; }', the sameBPF_MAP_TYPE_ARRAY'timer_map' withmax_entries1, the sameSEC("fentry/bpf_fentry_test1")+BPF_PROG2entry program, and the same '__naked__noinline__usedstatic' callback idiom already used bytimer_cb_ret_bad().The new program adds ~10 lines of actual test content on top of ~35 lines of copied setup, plus a new skeleton, plus a companion
btf__timer_ret_pair_fail.c, plus the RUN_TESTS()/#include churn inprog_tests/timer.c. Adding it as a third program insidetimer_failure.cwould reuse all of that. Since__btf_func_pathis a per-program tag consumed per-subtest inrun_subtest(), andtimer_failure.chas no CO-RE relocations (it includes uapi <linux/bpf.h>, not <vmlinux.h>), thebtf_custom_pathside effect of the tag would not affect the existing subtests in that object.
Quoted 5 lines
A separate file is defensible for symmetry with the exceptions half of the patch, where a separate file IS required due to CO-RE relocations. Should the new timer case be a new subtest in
timer_failure.c, or does the symmetry withexceptions_ret_pair_fail.cmake the separate file structure clearer?
In this case, the prog is in timer_failure.c for failure testing.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31446101762
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 12/13] selftests/bpf: Add tests for callbacks returning more than 8 bytes
In reply to Yonghong SongQuoted 19 lines · Yonghong Song
Add two
__failuretests covering the callback return-size checks:-
timer_ret_pair_fail: abpf_timercallback declared to return more than 8 bytes, rejected bycheck_ld_imm()where the callback'sPTR_TO_FUNCis created, with "callback function with >8-byte return value is not supported".-
exceptions_ret_pair_fail: an exception callback declared to return more than 8 bytes, rejected bydo_check_common()when the callback subprogram is verified, with "exception cb cannot return value larger than 8 bytes".Both callback bodies are written in inline asm so that the tests do not depend on LLVM 23 R0:R2 codegen and run on any compiler. The verifier reads the return type from BTF rather than from the instructions, so the >8 byte return prototype is supplied through
__btf_func_path(), pointing at a companion btf__*.c program that exists only to carry that BTF.Signed-off-by Yonghong Song <[email protected]>
Quoted 8 lines
---
.../selftests/bpf/prog_tests/exceptions.c | 2 + .../testing/selftests/bpf/prog_tests/timer.c | 2 +.../bpf/progs/btf__exceptions_ret_pair_fail.c| 10 ++++ .../bpf/progs/btf__timer_ret_pair_fail.c | 10 ++++ .../bpf/progs/exceptions_ret_pair_fail.c | 30 ++++++++++++.../selftests/bpf/progs/timer_ret_pair_fail.c| 49 +++++++++++++++++++6 files changed, 103 insertions(+)
Dropping callback checks avoids adding all this code.
...
Yonghong Song
reviewerRe: [PATCH bpf-next v4 12/13] selftests/bpf: Add tests for callbacks returning more than 8 bytes
Quoted 28 lines · Eduard Zingerman
Quoted 19 lines · Yonghong Song
Add two
__failuretests covering the callback return-size checks:-
timer_ret_pair_fail: abpf_timercallback declared to return more than 8 bytes, rejected bycheck_ld_imm()where the callback'sPTR_TO_FUNCis created, with "callback function with >8-byte return value is not supported".-
exceptions_ret_pair_fail: an exception callback declared to return more than 8 bytes, rejected bydo_check_common()when the callback subprogram is verified, with "exception cb cannot return value larger than 8 bytes".Both callback bodies are written in inline asm so that the tests do not depend on LLVM 23 R0:R2 codegen and run on any compiler. The verifier reads the return type from BTF rather than from the instructions, so the >8 byte return prototype is supplied through
__btf_func_path(), pointing at a companion btf__*.c program that exists only to carry that BTF.Signed-off-by Yonghong Song <[email protected]>Quoted 8 lines
--- .../selftests/bpf/prog_tests/exceptions.c | 2 + .../testing/selftests/bpf/prog_tests/timer.c | 2 + .../bpf/progs/btf__exceptions_ret_pair_fail.c | 10 ++++ .../bpf/progs/btf__timer_ret_pair_fail.c | 10 ++++ .../bpf/progs/exceptions_ret_pair_fail.c | 30 ++++++++++++ .../selftests/bpf/progs/timer_ret_pair_fail.c | 49 +++++++++++++++++++ 6 files changed, 103 insertions(+)
Quoted 1 line
Dropping callback checks avoids adding all this code.
Yes, I removed 'bpf: Reject callbacks returning more than 8 bytes' so this patch will be removed as well.
Quoted 2 lines
...
Yonghong Song
reviewerPATCH v4 13/13 Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2
In reply to Yonghong Songkfuncs may now return a value larger than 8 bytes and up to 16 bytes (a scalar-only struct or union, or an __int128), passed back in the R0:R2 register pair. Add a kfunc return-value section documenting this, including that a struct or union up to 8 bytes is returned in R0 alone, which struct and union members are accepted, that the R0:R2 register pair requires JIT support (bpf_jit_supports_kfunc_ret_reg_pair()), and that a return value larger than 16 bytes is unsupported.
Also note that the same convention applies to BPF subprogram returns, and document the consequence for a global subprogram: it must assign both halves of a register-pair return, since an unassigned R2 may be left holding a pointer argument and is then rejected as a leak.
--- Documentation/bpf/kfuncs.rst | 62 ++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+)
Documentation/bpf/kfuncs.rst +62 -0
@@ -575,6 +575,68 @@ is also covered by this recovery. A kfunc handed an arena pointer may therefore access up to ``GUARD_SZ / 2`` past it without bounds-checking against the arena. Larger accesses must verify the range explicitly. +2.9 kfunc Return Values +----------------------- + +A kfunc may return a scalar, a pointer, or a small struct or union by +value. A scalar or pointer of up to 8 bytes is returned in R0, as usual. + +A struct or union returned by value must be composed only of scalars +(recursively), where a scalar is an integer or an enum; arrays of scalars are +allowed as members. Its bytes are handed back to the program as the raw +contents of R0 (and R2), so a pointer field would be laundered into a scalar +and escape the verifier's pointer provenance and reference tracking. A struct +or union with a pointer member is therefore rejected at load time, and so is +one with a floating-point member, which the ABI may not return in R0:R2 at +all. + +A kfunc may also return a value larger than 8 bytes and up to 16 bytes -- a +scalar-only struct or union, or an ``__int128``. Such a value is returned +in the register pair R0:R2, matching the convention LLVM uses for the BPF +target: the first 8 bytes in R0 and the second 8 bytes in R2. A struct or +union of 8 bytes or less is returned in R0 alone. + +:: + + struct bpf_pair { __u64 a, b; }; /* 16 bytes */ + + __bpf_kfunc struct bpf_pair bpf_kfunc_get_pair(void) + { + struct bpf_pair p = { .a = 1, .b = 2 }; + + return p; /* p.a in R0, p.b in R2 */ + } + +Returning a value in the R0:R2 pair requires the JIT to place the second +half of the return value into R2, which not every architecture supports +right now. A kfunc with a return value larger than 8 bytes is therefore +rejected at load time on a JIT that does not advertise this capability (see +``bpf_jit_supports_kfunc_ret_reg_pair()``), and such a program is never run +by the interpreter. A return value larger than 16 bytes is not supported. + +The same R0:R2 convention applies to a BPF subprogram, global or static, +that returns an ``__int128`` or a struct or union larger than 8 bytes. Such a +program also requires the JIT, since the interpreter propagates only R0 out +of a subprogram. A global subprogram is verified in isolation, so its +by-value struct or union return is restricted to scalars just like a kfunc's; +a static subprogram is verified inline and has no such restriction. The main +program cannot return more than 8 bytes, as its return value is the program's +exit code. + +A global subprogram must leave a scalar in *every* register of the pair, so +both halves of the returned value have to be assigned. Leaving the upper half +uninitialized is not merely untidy: the compiler is then free to leave R2 +holding whatever it happened to hold, which for a subprogram taking a pointer +argument is typically that pointer. Handing the caller an unknown scalar built +from a pointer is a leak, so the verifier rejects it with:: + + At subprogram exit the register R2 is not a scalar value (...) + +Initialize the whole return value, for example ``struct pair p = {};``, to +avoid this. A static subprogram is exempt: it is verified inline, so an +unassigned R2 is simply passed back to the caller as uninitialized and only a +caller that reads it fails. + .. _BPF_kfunc_lifecycle_expectations: 3. kfunc lifecycle expectations
Signature · 1 line
2.53.0-Meta