bpf: Support aggregate return values up to 16 bytes
Subsystems (12)
Yonghong Song
reviewerv5 cover bpf: Support aggregate return values up to 16 bytes
LLVM 23 can return an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair [1][2]. Before that the BPF backend could not return such values at all: a by-value aggregate return was rejected at compile time with "aggregate returns are not supported", and an __int128 return failed in the backend with "unable to allocate function return #1".
This series teaches the kernel the same convention, so that BPF programs and kfuncs can return these values. The first 8 bytes of the value come back in R0 and the second 8 bytes in R2. It applies to kfunc returns and to BPF-to-BPF subprogram returns, both global and static. The main program is unchanged: its return value is the program's exit code, so a return larger than 8 bytes is still rejected at BPF_EXIT.
Patches 1-2 are preparation: patch 1 factors out the per-register check used by the global return path, and patch 2 adds the shared helpers that answer "does this subprogram return a register pair", so that the patches which follow can be ordered independently. Patch 3 wires up the JIT side. Patches 4-5 teach precision backtracking and live register analysis about R2 as a second return register, ahead of the patch that starts modeling it. Patch 6 adds the verifier support proper. Patch 7 relaxes btf_distill_func_proto() and btf_validate_return_type(), which is what makes the whole thing reachable, and patch 8 rejects a pair return once the subprogram's BTF has been marked unreliable. Patches 9-10 add selftests and patch 11 documents the convention.
Constraints worth calling out:
- A by-value struct or union returned by a kfunc or by a global subprogram must be composed only of scalars. The verifier models the returned register bits as an unknown scalar, so a pointer member would be laundered into one and escape provenance and reference tracking. A static subprogram is verified inline and is not restricted this way.
- Returning the pair from a kfunc needs the JIT to place the second half into R2, which is architecture-specific work. Architectures opt in through bpf_jit_supports_kfunc_ret_reg_pair(); x86_64, arm64 and riscv64 do so here, and elsewhere bpf_add_kfunc_call() rejects such a kfunc with -EOPNOTSUPP. A register-pair return from a BPF subprogram needs no such capability.
- The interpreter propagates R0 alone out of a subprogram, so the JIT is forced wherever a caller can observe the pair.
- The compiler side requires LLVM 23 or newer. The selftests written in C record which compiler built them in a read-only flag and report a skip rather than a pass when built by anything older; the inline-asm tests do not depend on the compiler and run everywhere.
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876
Changelog:
v4 -> v5:
- v4: https://lore.kernel.org/bpf/[email protected]/
- Removed R0:R2 restriction on callback functions and its related tests.
- Simplify the code for backtracking.
- Reduce comments and verify R0/R2 together.
- Use RUN_TESTS for selftests.
v3 -> v4:
- v3: https://lore.kernel.org/bpf/[email protected]/
- Fix a few kernel comment format.
- Guard more kfunc's with x86_64/arm64 only to avoid s390x failure.
v2 -> v3:
- v2: https://lore.kernel.org/bpf/[email protected]/
- Add additional guard with __SIZEOF_INT128__ for 32bit kernel.
- Guard little endian to avoid arm64 big endian for selftests.
- Fix test failures for gcc15.
- Rebase to avoid conflict with latest master branch.
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- Split the R0:R2 helpers out of the verifier patch into their own preparation patch, and reordered the series so the core verifier patch comes after the infrastructure it depends on.
- New patch rejecting callbacks that return more than 8 bytes, both helper/kfunc callbacks and exception callbacks, with selftests.
- New patch rejecting a register-pair return once btf_check_subprog_call() has marked the subprogram's BTF unreliable, rather than silently mistracking R2.
- Folded "bpf: Force JIT for programs using the R0:R2 register pair" into the verifier patch.
- Dropped "bpf: Reject >8 byte return values on return-reading trampoline paths" and its selftests; that went in separately as commit c48796aa6c39.
- Described the register mapping as the first and second 8 bytes rather than the low and high 64 bits, which is only correct on little-endian, and reworded "16-byte" to "up to 16 bytes" where the range 9..16 was meant.
Yonghong Song (11):
bpf: Factor check_global_ret_scalar_reg() out of the global return
check
bpf: Add helpers to describe the R0:R2 return register pair
bpf: Wire up JIT support for 16-byte kfunc returns
bpf: Track R2 of register-pair returns in precision backtracking
bpf: Account R2 of register-pair returns in live register analysis
bpf: Add verifier support for 16-byte returns in R0:R2
bpf: Enable aggregate return types up to 16 bytes
bpf: Reject register-pair returns when the subprog BTF is unreliable selftests/bpf: Add C tests for 16-byte returns in R0:R2 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2
Documentation/bpf/kfuncs.rst | 63 +++++ arch/arm64/net/bpf_jit_comp.c | 5 + arch/riscv/net/bpf_jit_comp64.c | 5 + arch/x86/net/bpf_jit_comp.c | 27 +- include/linux/bpf_verifier.h | 11 + include/linux/filter.h | 1 + kernel/bpf/backtrack.c | 78 +++++- kernel/bpf/btf.c | 23 +- kernel/bpf/core.c | 5 + kernel/bpf/liveness.c | 20 +- kernel/bpf/verifier.c | 229 ++++++++++++--- .../selftests/bpf/prog_tests/aggregate_ret.c | 17 ++ .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 15 + .../selftests/bpf/progs/aggregate_ret_func.c | 260 ++++++++++++++++++ .../bpf/progs/aggregate_ret_int128_c.c | 49 ++++ .../selftests/bpf/progs/aggregate_ret_kfunc.c | 122 ++++++++ .../bpf/progs/aggregate_ret_kfunc_c.c | 125 +++++++++ .../bpf/progs/aggregate_ret_struct_c.c | 114 ++++++++ .../bpf/progs/aggregate_ret_target.c | 29 ++ .../bpf/progs/compute_live_registers.c | 30 ++ .../selftests/bpf/progs/exceptions_fail.c | 2 +- .../selftests/bpf/progs/freplace_ret_pair.c | 12 + .../selftests/bpf/test_kmods/bpf_testmod.c | 73 +++++ .../bpf/test_kmods/bpf_testmod_kfunc.h | 44 +++ 24 files changed, 1290 insertions(+), 69 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c
Signature · 1 line
2.53.0-Meta
Yonghong Song
reviewerPATCH v5 1/11 bpf: Factor check_global_ret_scalar_reg() out of the global return check
check_global_subprog_return_code() verifies that a global subprogram returns void, an arena pointer, or register R0 holding a scalar value. Later patches in this series add 16-byte aggregate return support, whose second half is returned in R2 and needs the same validation.
Factor the per-register check into check_global_ret_scalar_reg(env, regno) so that it can be reused for R2. No functional change.
--- kernel/bpf/verifier.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-)
kernel/bpf/verifier.c +20 -12
@@ -16654,37 +16654,45 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charreturn 0 ; } -static int check_global_subprog_return_code (struct bpf_verifier_env *env) +static int check_global_ret_scalar_reg (struct bpf_verifier_env *env,u32 regno) { -struct bpf_reg_state *reg =reg_state (env,BPF_REG_0 ); -struct bpf_func_state *cur_frame =cur_func (env); +struct bpf_reg_state *reg;int err; -if (subprog_returns_void (env, cur_frame->subprogno)) -return 0 ; - - err =check_reg_arg (env,BPF_REG_0 ,SRC_OP ); + err =check_reg_arg (env, regno,SRC_OP );if (err)return err;/* Pointers to arena are safe to pass between subprograms. */ -if (is_arena_reg (env,BPF_REG_0 )) +if (is_arena_reg (env, regno))return 0 ; -if (is_pointer_value (env,BPF_REG_0 )) { -verbose (env,"R%d leaks addr as return value\n" ,BPF_REG_0 ); +if (is_pointer_value (env, regno)) { +verbose (env,"R%d leaks addr as return value\n" , regno);return -EACCES ; } + reg =reg_state (env, regno);if (reg->type !=SCALAR_VALUE ) { -verbose (env,"At subprogram exit the register R0 is not a scalar value (%s)\n" , -reg_type_str (env, reg->type)); +verbose (env,"At subprogram exit the register R%d is not a scalar value (%s)\n" , + regno,reg_type_str (env, reg->type));return -EINVAL ; }return 0 ; } +static int check_global_subprog_return_code (struct bpf_verifier_env *env) +{ +struct bpf_func_state *cur_frame =cur_func (env); +u32 subprog = cur_frame->subprogno; + +if (subprog_returns_void (env, subprog)) +return 0 ; + +return check_global_ret_scalar_reg (env,BPF_REG_0 ); +} +/* Bitmask with 1s for all caller saved registers */ #define ALL_CALLER_SAVED_REGS ((1u << CALLER_SAVED_REGS) - 1)
Signature · 1 line
2.53.0-Meta
Re: [PATCH bpf-next v5 01/11] bpf: Factor check_global_ret_scalar_reg() out of the global return check
Quoted 10 lines · Yonghong Song
check_global_subprog_return_code()verifies that a global subprogram returns void, an arena pointer, or register R0 holding a scalar value. Later patches in this series add 16-byte aggregate return support, whose second half is returned in R2 and needs the same validation.Factor the per-register check into
check_global_ret_scalar_reg(env, regno)so that it can be reused for R2. No functional change.Signed-off-by Yonghong Song <[email protected]>---
...
Yonghong Song
reviewerPATCH v5 2/11 bpf: Add helpers to describe the R0:R2 return register pair
In reply to Yonghong SongLLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes: such a value comes back in the R0:R2 register pair, with R2 holding the upper half. See LLVM patches [1] and [2].
Later patches teach the JIT, precision backtracking, live register analysis and the verifier itself about that convention. All of them need to answer the same question: does this call return its value in a register pair? Add the shared helpers up front so they can be used in subsequent patches:
- bpf_ret_reg_pair() for a BPF subprogram, answered from a per-subprogram flag that bpf_compute_subprog_ret_regs() derives once from the BTF prototype;
- bpf_kfunc_ret_reg_pair() for a kfunc call site, answered from the btf_func_model that bpf_add_kfunc_call() already built, which is the same ret_size the JIT keys the second return register off, so the verifier and the generated code cannot disagree.
It also sets jit_required, from the same place the convention is decided rather than from each site that later comes to model R2, so that no such site can be missed.
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876
--- include/linux/bpf_verifier.h | 9 +++++ kernel/bpf/verifier.c | 73 ++++++++++++++++++++++++++++++------ 2 files changed, 71 insertions(+), 11 deletions(-)
include/linux/bpf_verifier.h +9 -0
@@ -814,6 +814,8 @@ struct bpf_subprog_info { bool is_async_cb: 1; bool is_exception_cb: 1; bool args_cached: 1; +/* true if the return value is passed in the R0:R2 register pair */ + bool ret_reg_pair: 1;/* true if bpf_fastcall stack region is used by functions that can't be inlined */ bool keep_fastcall_stack: 1; bool changes_pkt_data: 1;
@@ -1048,6 +1050,13 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *envreturn &env->subprog_info[subprog]; } +static inline bool bpf_ret_reg_pair (struct bpf_verifier_env *env,int subprog) +{ +return subprog_info (env, subprog)->ret_reg_pair; +} + +bool bpf_kfunc_ret_reg_pair (struct bpf_verifier_env *env,struct bpf_insn *insn); +struct bpf_call_summary {u8 num_params;bool is_void;
kernel/bpf/verifier.c +62 -11
101 lines not on this page · raw message
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 02/11] bpf: Add helpers to describe the R0:R2 return register pair
Does this loop need to start at subprog 0? The main program (subprog 0) delivers its exit code in R0 alone and never uses the R0:R2 calling convention, so checking its BTF return type can…
Quoted 72 lines
include/linux/bpf_verifier.h +9
-0@@ -814,6 +814,8 @@ struct bpf_subprog_info { bool is_async_cb: 1; bool is_exception_cb: 1; bool args_cached: 1; +/* true if the return value is passed in the R0:R2 register pair */ + bool ret_reg_pair: 1;/* true if bpf_fastcall stack region is used by functions that can't be inlined */ bool keep_fastcall_stack: 1; bool changes_pkt_data: 1;@@ -1048,6 +1050,13 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env, int subprog)return &env->subprog_info[subprog]; } +static inline bool bpf_ret_reg_pair (struct bpf_verifier_env *env,int subprog) +{ +return subprog_info (env, subprog)->ret_reg_pair; +} + +bool bpf_kfunc_ret_reg_pair (struct bpf_verifier_env *env,struct bpf_insn *insn); +struct bpf_call_summary {u8 num_params;bool is_void;kernel/bpf/verifier.c +18
-11@@ -382,29 +382,62 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)return aux && aux[subprog].linkage ==BTF_FUNC_GLOBAL ; } -static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +static const struct btf_type *subprog_ret_type (struct bpf_verifier_env *env,int subprog) { -const struct btf_type *type, *func, *func_proto; +const struct btf_type *func, *func_proto;const struct btf *btf = env->prog->aux->btf;u32 btf_id; +if (!btf || !env->prog->aux->func_info) +return NULL ; + btf_id = env->prog->aux->func_info[subprog].type_id; +/* Both already validated by prepare_btf_func() at prog load. */ func =btf_type_by_id (btf, btf_id); -if (verifier_bug_if (!func, env,"btf_id %u not found" , btf_id)) -return false; - func_proto =btf_type_by_id (btf, func->type); -if (!func_proto) -return false; - type =btf_type_skip_modifiers (btf, func_proto->type,NULL ); -if (!type) -return false; +return btf_type_skip_modifiers (btf, func_proto->type,NULL ); +} -return btf_type_is_void (type); +static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +{ +const struct btf_type *type =subprog_ret_type (env, subprog); + +return type &&btf_type_is_void (type); +} + +static u32 ret_regs_cnt (u32 size) +{ +return size >8 && size <=16 ?2 :1 ;
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 422
Quoted 10 lines
kernel/bpf/verifier.c +8
-0} +static void bpf_compute_subprog_ret_regs (struct bpf_verifier_env *env) +{ +const struct btf *btf = env->prog->aux->btf; +const struct btf_type *type; +int subprog; +u32 size; + +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) {
Does this loop need to start at subprog 0? The main program (subprog 0) delivers its exit code in R0 alone and never uses the R0:R2 calling convention, so checking its BTF return type can incorrectly trigger jit_required even though no R2 propagation is needed.
Quoted 7 lines
kernel/bpf/verifier.c +7
-0+ type =subprog_ret_type (env, subprog); +if (!type || !(btf_type_is_struct (type) ||btf_type_is_scalar (type))) +continue ; +if (IS_ERR (btf_resolve_size (btf, type, &size))) +continue ; +if (ret_regs_cnt (size) >1 ) { +subprog_info (env, subprog)->ret_reg_pair = true;
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+/* +* The R0:R2 return convention is only implemented in +* the JIT: the interpreter propagates BPF_R0 alone out +* of a subprogram, so a caller reading R2 would see a +* stale value. +*/ + env->prog->jit_required =1 ; + } + } +}
Additionally, should jit_required be set for subprogs whose BTF prototypes are marked unreliable? btf_check_subprog_call() can flag a mismatch with prog->aux->func_info_aux[subprog].unreliable = true, and check_func_call() only aborts on -EFAULT, so a static callee with an aggregate return can load today without using the BTF return type. Subprogs subsequently deleted by bpf_opt_remove_dead_code() (which runs after this pass) also don't need jit_required set.
This changes behavior for existing programs: on CONFIG_BPF_JIT=n or net.core.bpf_jit_enable=0, such programs would now fail in bpf_fixup_call_args() with "program requires BPF JIT compiler but it is not available", whereas they previously verified and ran on the interpreter.
[ ... ]
kernel/bpf/verifier.c line 20389
Quoted 10 lines
kernel/bpf/verifier.c +3
-0@@ -20333,6 +20381,9 @@if (ret <0 ) goto skip_full_check; +/* must precede the first bpf_ret_reg_pair() user below */ +bpf_compute_subprog_ret_regs (env); + ret =bpf_compute_live_registers (env);if (ret <0 ) goto skip_full_check;
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Yonghong Song
reviewerRe: [PATCH bpf-next v5 02/11] bpf: Add helpers to describe the R0:R2 return register pair
Quoted 74 lines · On 8/13/26 2:11 PM, [email protected] wrote:
Quoted 74 lines
include/linux/bpf_verifier.h +9
-0@@ -814,6 +814,8 @@ struct bpf_subprog_info { bool is_async_cb: 1; bool is_exception_cb: 1; bool args_cached: 1; +/* true if the return value is passed in the R0:R2 register pair */ + bool ret_reg_pair: 1;/* true if bpf_fastcall stack region is used by functions that can't be inlined */ bool keep_fastcall_stack: 1; bool changes_pkt_data: 1;@@ -1048,6 +1050,13 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env, int subprog)return &env->subprog_info[subprog]; } +static inline bool bpf_ret_reg_pair (struct bpf_verifier_env *env,int subprog) +{ +return subprog_info (env, subprog)->ret_reg_pair; +} + +bool bpf_kfunc_ret_reg_pair (struct bpf_verifier_env *env,struct bpf_insn *insn); +struct bpf_call_summary {u8 num_params;bool is_void;kernel/bpf/verifier.c +18
-11@@ -382,29 +382,62 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)return aux && aux[subprog].linkage ==BTF_FUNC_GLOBAL ; } -static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +static const struct btf_type *subprog_ret_type (struct bpf_verifier_env *env,int subprog) { -const struct btf_type *type, *func, *func_proto; +const struct btf_type *func, *func_proto;const struct btf *btf = env->prog->aux->btf;u32 btf_id; +if (!btf || !env->prog->aux->func_info) +return NULL ; + btf_id = env->prog->aux->func_info[subprog].type_id; +/* Both already validated by prepare_btf_func() at prog load. */ func =btf_type_by_id (btf, btf_id); -if (verifier_bug_if (!func, env,"btf_id %u not found" , btf_id)) -return false; - func_proto =btf_type_by_id (btf, func->type); -if (!func_proto) -return false; - type =btf_type_skip_modifiers (btf, func_proto->type,NULL ); -if (!type) -return false; +return btf_type_skip_modifiers (btf, func_proto->type,NULL ); +} -return btf_type_is_void (type); +static bool subprog_returns_void (struct bpf_verifier_env *env,int subprog) +{ +const struct btf_type *type =subprog_ret_type (env, subprog); + +return type &&btf_type_is_void (type); +} + +static u32 ret_regs_cnt (u32 size) +{ +return size >8 && size <=16 ?2 :1 ; }
Quoted 12 lines
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 422
Quoted 8 lines
kernel/bpf/verifier.c +8
-0+static void bpf_compute_subprog_ret_regs (struct bpf_verifier_env *env) +{ +const struct btf *btf = env->prog->aux->btf; +const struct btf_type *type; +int subprog; +u32 size; + +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) {Does this loop need to start at subprog 0? The main program (subprog 0) delivers its exit code in R0 alone and never uses the R0:R2 calling convention, so checking its BTF return type can incorrectly trigger
jit_requiredeven though no R2 propagation is needed.
You are right. We only need to start from subprog 1.
Quoted 17 lines
Quoted 17 lines
kernel/bpf/verifier.c +16
-0+ type =subprog_ret_type (env, subprog); +if (!type || !(btf_type_is_struct (type) ||btf_type_is_scalar (type))) +continue ; +if (IS_ERR (btf_resolve_size (btf, type, &size))) +continue ; +if (ret_regs_cnt (size) >1 ) { +subprog_info (env, subprog)->ret_reg_pair = true; +/* +* The R0:R2 return convention is only implemented in +* the JIT: the interpreter propagates BPF_R0 alone out +* of a subprogram, so a caller reading R2 would see a +* stale value. +*/ + env->prog->jit_required =1 ; + } + }
Quoted 8 lines
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 439
Quoted 1 line
kernel/bpf/verifier.c +1
-0+}Additionally, should
jit_requiredbe set for subprogs whose BTF prototypes are marked unreliable?btf_check_subprog_call()can flag a mismatch with prog->aux->func_info_aux[subprog].unreliable = true, andcheck_func_call()only aborts on-EFAULT, so a static callee with an aggregate return can load today without using the BTF return type. Subprogs subsequently deleted bybpf_opt_remove_dead_code()(which runs after this pass) also don't needjit_requiredset.
We cannot just mark jit_required if BTF prototypes is marked unreliable. This is too conservative. Marking unreliable may not have 2 return registers, e.g., if a argument is 'int *p', the subprog will mark unreliable. but it may not have 2 return registers.
Quoted 6 lines
This changes behavior for existing programs: on
CONFIG_BPF_JIT=nor net.core.bpf_jit_enable=0, such programs would now fail inbpf_fixup_call_args()with "program requires BPF JIT compiler but it is not available", whereas they previously verified and ran on the interpreter.
If jit is not enabled, let us just assume returning R0 only.
Quoted 19 lines
[ ... ]
kernel/bpf/verifier.c line 20389
Quoted 10 lines
kernel/bpf/verifier.c +3
-0@@ -20333,6 +20381,9 @@if (ret <0 ) goto skip_full_check; +/* must precede the first bpf_ret_reg_pair() user below */ +bpf_compute_subprog_ret_regs (env); + ret =bpf_compute_live_registers (env);if (ret <0 ) goto skip_full_check;---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Eduard Zingerman
maintainerRe: [PATCH bpf-next v5 02/11] bpf: Add helpers to describe the R0:R2 return register pair
In reply to Yonghong SongOn Thu, 2026-08-13 at 13:02 -0700, Yonghong Song wrote:
...
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 425
Quoted 12 lines
kernel/bpf/verifier.c +10
-1+static void bpf_compute_subprog_ret_regs (struct bpf_verifier_env *env) +{ +const struct btf *btf = env->prog->aux->btf; +const struct btf_type *type; +int subprog; +u32 size; -return btf_type_is_void (type); +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) { + type =subprog_ret_type (env, subprog); +if (!type || !(btf_type_is_struct (type) ||btf_type_is_scalar (type))) +continue ;
Is the check '!(btf_type_is_struct(type) || btf_type_is_scalar(type))' needed after v4->v5 migration?
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 427
Quoted 2 lines
kernel/bpf/verifier.c +2
-0+if (IS_ERR (btf_resolve_size (btf, type, &size))) +continue ;
Let's propagate this error instead of ignoring it. Or add WARN_ON_ONCE(). After BTF validation it shouldn't really happen.
Quoted 4 lines
kernel/bpf/verifier.c +4
-0+if (ret_regs_cnt (size) >1 ) { +subprog_info (env, subprog)->ret_reg_pair = true; +/* +* The R0:R2 return convention is only implemented in
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 441
Quoted 10 lines
kernel/bpf/verifier.c +7
-0+* the JIT: the interpreter propagates BPF_R0 alone out +* of a subprogram, so a caller reading R2 would see a +* stale value. +*/ + env->prog->jit_required =1 ; + } + } }static const char *subprog_name (const struct bpf_verifier_env *env,int subprog)
...
Regarding the bot comment:
Quoted 7 lines
Additionally, should
jit_requiredbe set for subprogs whose BTF prototypes are marked unreliable?btf_check_subprog_call()can flag a mismatch with prog->aux->func_info_aux[subprog].unreliable = true, andcheck_func_call()only aborts on-EFAULT, so a static callee with an aggregate return can load today without using the BTF return type. Subprogs subsequently deleted bybpf_opt_remove_dead_code()(which runs after this pass) also don't needjit_requiredset.
It seem to be correct. As prepare_func_exit() unconditionally copies r0 and r2 to caller's frame, the verifier would conclude that r2 is initialized; while interpreter won't copy it. One option is to force nregs == 1 if there is no jit.
(Or derive this information not from BTF, I'll comment on that later).
Yonghong Song
reviewerRe: [PATCH bpf-next v5 02/11] bpf: Add helpers to describe the R0:R2 return register pair
Quoted 11 lines · Eduard Zingerman
On Thu, 2026-08-13 at 13:02 -0700, Yonghong Song wrote:
...
Quoted 7 lines
kernel/bpf/verifier.c +6
-0+static void bpf_compute_subprog_ret_regs (struct bpf_verifier_env *env) +{ +const struct btf *btf = env->prog->aux->btf; +const struct btf_type *type; +int subprog; +u32 size;
Quoted 7 lines
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 425
Quoted 5 lines
kernel/bpf/verifier.c +4
-1-return btf_type_is_void (type); +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) { + type =subprog_ret_type (env, subprog); +if (!type || !(btf_type_is_struct (type) ||btf_type_is_scalar (type))) +continue ;Is the check '!(btf_type_is_struct(type) ||
btf_type_is_scalar(type))' needed afterv4->v5migration?
Yes, the condition is similar to btf_validate_return_type().
Quoted 5 lines
kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 427
Quoted 3 lines
kernel/bpf/verifier.c +2
-0+if (IS_ERR (btf_resolve_size (btf, type, &size))) +continue ;Let's propagate this error instead of ignoring it. Or add
WARN_ON_ONCE(). After BTF validation it shouldn't really happen.
Okay, will propagate the error.
Quoted 26 lines
Quoted 5 lines
kernel/bpf/verifier.c +4
-0+if (ret_regs_cnt (size) >1 ) { +subprog_info (env, subprog)->ret_reg_pair = true; +/* +* The R0:R2 return convention is only implemented in kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 441
Quoted 10 lines
kernel/bpf/verifier.c +7
-0+* the JIT: the interpreter propagates BPF_R0 alone out +* of a subprogram, so a caller reading R2 would see a +* stale value. +*/ + env->prog->jit_required =1 ; + } + } }static const char *subprog_name (const struct bpf_verifier_env *env,int subprog)...
Regarding the bot comment:
Quoted 7 lines
Additionally, should
jit_requiredbe set for subprogs whose BTF prototypes are marked unreliable?btf_check_subprog_call()can flag a mismatch with prog->aux->func_info_aux[subprog].unreliable = true, andcheck_func_call()only aborts on-EFAULT, so a static callee with an aggregate return can load today without using the BTF return type. Subprogs subsequently deleted bybpf_opt_remove_dead_code()(which runs after this pass) also don't needjit_requiredset.
Quoted 4 lines
It seem to be correct. As
prepare_func_exit()unconditionally copies r0 and r2 to caller's frame, the verifier would conclude that r2 is initialized; while interpreter won't copy it. One option is to force nregs == 1 if there is no jit.
As suggested, we can just do nregs = 1 (returning registers) if there is no jit.
Quoted 2 lines
(Or derive this information not from BTF, I'll comment on that later).
Yonghong Song
reviewerPATCH v5 3/11 bpf: Wire up JIT support for 16-byte kfunc returns
In reply to Yonghong SongLLVM 23 returns an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch added the shared helpers describing that convention; wire up the JIT side so that the second half of the return value actually lands in R2.
Placing the second return half into R2 is possible on any JIT, but it needs architecture-specific JIT work. Rather than requiring every JIT to implement it at once, add a bpf_jit_supports_kfunc_ret_reg_pair() capability, defaulting to false in the generic core; an architecture opts in once its JIT handles the R0:R2 pair, and the remaining ones are left for future work. Only the x86-64, arm64 and riscv64 JITs opt in so far.
On arm64 and riscv64 the native second return register is already BPF R2 (x1 in bpf2a64[] and a1 in regmap[] respectively), so the upper half needs no move at all, unlike x86-64's RDX->RSI. The lower half is covered by the move into BPF R0 that those JITs already emit after every call, from x0 into x8 and from a0 into a5. This has been tested on x86-64 and arm64. The riscv64 path is expected to work by the same register-mapping reasoning as arm64 but has not been tested.
bpf_add_kfunc_call() also rejects a kfunc that is marked KF_FASTCALL and returns more than 8 bytes. The bpf_fastcall contract implemented by mark_fastcall_pattern_for_call() assumes a call clobbers R0 plus the registers holding its arguments, so a return in the R0:R2 pair would clobber an R2 the caller expects the fastcall pattern to preserve. Such a kfunc is rejected with -EOPNOTSUPP as well.
--- arch/arm64/net/bpf_jit_comp.c | 5 +++++ arch/riscv/net/bpf_jit_comp64.c | 5 +++++ arch/x86/net/bpf_jit_comp.c | 27 ++++++++++++++++++++------- include/linux/filter.h | 1 + kernel/bpf/core.c | 5 +++++ kernel/bpf/verifier.c | 12 ++++++++++++ 6 files changed, 48 insertions(+), 7 deletions(-)
arch/arm64/net/bpf_jit_comp.c +5 -0
@@ -2346,6 +2346,11 @@ bool bpf_jit_supports_kfunc_call(void)return true; } +bool bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return true; +} +bool bpf_jit_supports_stack_args (void ) {return true;
arch/riscv/net/bpf_jit_comp64.c +5 -0
@@ -2120,6 +2120,11 @@ bool bpf_jit_supports_kfunc_call(void)return true; } +bool bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return true; +} +bool bpf_jit_supports_ptr_xchg (void ) {return true;
arch/x86/net/bpf_jit_comp.c +20 -7
@@ -1689,17 +1689,12 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,* arena NULL is offset 0. Return the number of emitted bytes. */ static int emit_kfunc_arena_args (struct bpf_prog *bpf_prog, -const struct bpf_insn *insn,u8 **pprog) +const struct btf_func_model *fm,u8 **pprog) { -const struct btf_func_model *fm;u8 *prog = *pprog;u8 *start = prog;int i; - fm =bpf_jit_find_kfunc_model (bpf_prog, insn); -if (!fm) -return -EINVAL ; -for (i =0 ; i <min_t (int, fm->nr_args,MAX_BPF_FUNC_REG_ARGS ); i++) {u8 flags = fm->arg_flags[i];u32 reg =BPF_REG_1 + i;
@@ -2644,6 +2639,8 @@ st: insn_off = insn->off;/* call */ case BPF_JMP |BPF_CALL : { +const struct btf_func_model *fm =NULL ; + func = (u8 *) __bpf_call_base + imm32;if (src_reg ==BPF_PSEUDO_CALL && tail_call_reachable) {LOAD_TAIL_CALL_CNT_PTR (stack_depth);
@@ -2652,7 +2649,10 @@ st: insn_off = insn->off;if (!imm32)return -EINVAL ;if (src_reg ==BPF_PSEUDO_KFUNC_CALL ) { - err =emit_kfunc_arena_args (bpf_prog, insn, &prog); + fm =bpf_jit_find_kfunc_model (bpf_prog, insn); +if (!fm) +return -EINVAL ; + err =emit_kfunc_arena_args (bpf_prog, fm, &prog);if (err <0 )return err; ip += err;
@@ -2666,6 +2666,14 @@ st: insn_off = insn->off;return -EINVAL ;if (priv_frame_ptr)pop_r9 (&prog); +/* +* A kfunc returning more than 8 bytes hands the second +* half back in RDX (the native ABI's second return reg), +* but BPF expects it in R0:R2. BPF R0 is RAX (no move +* needed), while BPF R2 is RSI, so copy RDX into RSI. +*/ +if (fm && fm->ret_size >8 ) +emit_mov_reg (&prog, true,BPF_REG_2 ,BPF_REG_3 );break ; }
@@ -4156,6 +4164,11 @@ bool bpf_jit_supports_kfunc_call(void)return true; } +bool bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return true; +} +bool bpf_jit_supports_stack_args (void ) {return true;
include/linux/filter.h +1 -0
@@ -1237,6 +1237,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);bool bpf_jit_supports_subprog_tailcalls (void );bool bpf_jit_supports_percpu_insn (void );bool bpf_jit_supports_kfunc_call (void ); +bool bpf_jit_supports_kfunc_ret_reg_pair (void );bool bpf_jit_supports_stack_args (void );bool bpf_jit_supports_arena_args (void );bool bpf_jit_supports_far_kfunc_call (void );
kernel/bpf/core.c +5 -0
@@ -3287,6 +3287,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)return false; } +bool __weak bpf_jit_supports_kfunc_ret_reg_pair (void ) +{ +return false; +} +bool __weak bpf_jit_supports_stack_args (void ) {return false;
kernel/bpf/verifier.c +12 -0
@@ -2854,6 +2854,18 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) err =btf_distill_func_proto (&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);if (err)return err; +if (func_model.ret_size >8 ) { +if (kfunc.flags && (*kfunc.flags &KF_FASTCALL )) { +verbose (env,"kfunc %s with >8-byte return is not supported with KF_FASTCALL\n" , + kfunc.name); +return -EOPNOTSUPP ; + } +if (!bpf_jit_supports_kfunc_ret_reg_pair ()) { +verbose (env,"kfunc %s with >8-byte return is not supported by JIT\n" , + kfunc.name); +return -EOPNOTSUPP ; + } + }memset (&meta,0 ,sizeof (meta)); meta.btf = kfunc.btf;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 03/11] bpf: Wire up JIT support for 16-byte kfunc returns
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The verifier fails to initialize BPF_REG_2 for 16-byte kfunc returns, and completely fails to…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The verifier fails to initialize BPF_REG_2 for 16-byte kfunc returns, and completely fails to initialize both BPF_REG_0 and BPF_REG_2 if the kfunc returns a 16-byte struct. --
--- Patch [3]: [PATCH bpf-next v5 03/11] bpf: Wire up JIT support for 16-byte kfunc returns --- commit fc27c41d2405fa63fb5ef7aa5e34586ccf98e0bd
Author: Yonghong Song <[email protected]>
bpf: Wire up JIT support for 16-byte kfunc returns
This commit adds JIT support for 16-byte kfunc returns by placing the second half of the return value in the BPF R2 register. It introduces architecture-specific capabilities and enables support for x86-64, arm64, and riscv64. The verifier is also updated to reject unsupported 16-byte returns.
Quoted 10 lines
kernel/bpf/verifier.c +2
-0@@ -2854,6 +2854,18 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) err =btf_distill_func_proto (&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);if (err)return err; +if (func_model.ret_size >8 ) { +if (kfunc.flags && (*kfunc.flags &KF_FASTCALL )) {
kernel/bpf/verifier.c int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) · line 2868
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+verbose (env,"kfunc %s with >8-byte return is not supported with KF_FASTCALL\n" , + kfunc.name); +return -EOPNOTSUPP ; + } +if (!bpf_jit_supports_kfunc_ret_reg_pair ()) { +verbose (env,"kfunc %s with >8-byte return is not supported by JIT\n" , + kfunc.name); +return -EOPNOTSUPP ; + } + }
[Severity: High]
Does the verifier need updates in check_kfunc_call() to track the initialization of BPF_REG_2 for these larger returns?
When evaluating the kfunc return type in check_kfunc_call(), the verifier currently appears to miss marking BPF_REG_2 as initialized for 16-byte scalar returns:
kernel/bpf/verifier.c:check_kfunc_call() {
if (btf_type_is_scalar(t)) {
mark_reg_unknown(env, regs, BPF_REG_0);
...
} else if (btf_type_is_ptr(t)) {
...
} else if (btf_type_is_void(t)) {
...
}
}Furthermore, if the kfunc returns a 16-byte struct, the code falls through completely without initializing either BPF_REG_0 or BPF_REG_2.
Since all CALLER_SAVED_REGS are cleared to NOT_INIT earlier during the call, won't this cause the verifier to incorrectly reject valid BPF programs that attempt to read these return values?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Yonghong Song
reviewerRe: [PATCH bpf-next v5 03/11] bpf: Wire up JIT support for 16-byte kfunc returns
Quoted 55 lines · On 8/13/26 6:20 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The verifier fails to initializeBPF_REG_2for 16-byte kfunc returns, and completely fails to initialize bothBPF_REG_0andBPF_REG_2if the kfunc returns a 16-byte struct. ----- Patch [3]: [PATCH bpf-next v5 03/11] bpf: Wire up JIT support for 16-byte kfunc returns --- commit
fc27c41d2405fa63fb5ef7aa5e34586ccf98e0bd
Author: Yonghong Song <[email protected]>bpf: Wire up JIT support for 16-byte kfunc returns
This commit adds JIT support for 16-byte kfunc returns by placing the second half of the return value in the BPF R2 register. It introduces architecture-specific capabilities and enables support for x86-64, arm64, and riscv64. The verifier is also updated to reject unsupported 16-byte returns.
Quoted 10 lines
kernel/bpf/verifier.c +2
-0@@ -2854,6 +2854,18 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) err =btf_distill_func_proto (&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);if (err)return err; +if (func_model.ret_size >8 ) { +if (kfunc.flags && (*kfunc.flags &KF_FASTCALL )) {kernel/bpf/verifier.c int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) · line 2868
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+verbose (env,"kfunc %s with >8-byte return is not supported with KF_FASTCALL\n" , + kfunc.name); +return -EOPNOTSUPP ; + } +if (!bpf_jit_supports_kfunc_ret_reg_pair ()) { +verbose (env,"kfunc %s with >8-byte return is not supported by JIT\n" , + kfunc.name); +return -EOPNOTSUPP ; + } + }[Severity: High]
Does the verifier need updates incheck_kfunc_call()to track the initialization ofBPF_REG_2for these larger returns?When evaluating the kfunc return type in
check_kfunc_call(), the verifier currently appears to miss markingBPF_REG_2as initialized for 16-byte scalar returns:kernel/bpf/verifier.c:check_kfunc_call() { if (btf_type_is_scalar(t)) { mark_reg_unknown(env, regs, BPF_REG_0); ... } else if (btf_type_is_ptr(t)) { ... } else if (btf_type_is_void(t)) { ... } }
Quoted 7 lines
Furthermore, if the kfunc returns a 16-byte struct, the code falls through completely without initializing either
BPF_REG_0orBPF_REG_2.Since all
CALLER_SAVED_REGSare cleared toNOT_INITearlier during the call, won't this cause the verifier to incorrectly reject valid BPF programs that attempt to read these return values?
We should be okay here. The actual aggregate returning 16-byte struct is enabled in later patch "bpf: Enable aggregate return types up to 16 bytes".
Yonghong Song
reviewerPATCH v5 4/11 bpf: Track R2 of register-pair returns in precision backtracking
In reply to Yonghong SongPrecision backtracking treats only R0 as a return register at a call/return boundary, so once the verifier starts modeling R2 that way, marking the second half of such a return precise would trip the "unexpected regs" checks in backtrack_insn() and reject a valid program.
Marking the upper half precise, for example by branching on it after a call to a static subprogram, walks backtracking into the callee and reaches its BPF_EXIT with R2 still set in the mask. Handle R2 like R0 in boundaries where a call defines the return registers.
R2 differs from R0 in that it is an argument register as well, which makes two things worth spelling out:
- R2 is only excused from the BPF_REGMASK_ARGS check where the callee or kfunc really does return a pair, as reported by bpf_ret_reg_pair()/bpf_kfunc_ret_reg_pair(). Any other call leaves R2 uninitialized, so a request for its precision cannot be legitimate, and clearing it unconditionally would turn that verifier bug from a loud -EFAULT into a silently dropped precision request.
- at BPF_EXIT the return registers are sampled before the callback path clears R1-R5. That clear does not touch R0, but it does cover R2, and running it first would drop a pair return whenever the instruction following the call happens to be one that invokes a callback.
--- kernel/bpf/backtrack.c | 78 +++++++++++++++++++++++++++++++++++------- 1 file changed, 65 insertions(+), 13 deletions(-)
kernel/bpf/backtrack.c +65 -13
@@ -423,6 +423,16 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,*/ verifier_bug_if (idx +1 != subseq_idx, env,"extra insn from subprog" ); +/* a global subprog returning more than 8 bytes +* sets R2 as well. R2 is part of the args mask +* checked just below, so clear it here rather +* than next to R0. Only a subprog that does +* return a pair defines R2, so leave the mask +* alone otherwise and let the check below catch +* an R2 that has no business being set. +*/ +if (bpf_ret_reg_pair (env, subprog)) +bt_clear_reg (bt,BPF_REG_2 );/* r1-r5 are invalidated after subprog call, * so for global func call it shouldn't be set * anymore
@@ -506,6 +516,17 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -ENOTSUPP ;/* regular helper call sets R0 */ bt_clear_reg (bt,BPF_REG_0 ); +/* a kfunc returning more than 8 bytes also sets R2. +* R2 is part of the args mask checked just below, so +* clear it here rather than next to R0. The prototype +* lookup is only worth doing when R2 is requested at +* all; any other call leaves R2 uninitialized, so a +* request for it is caught by the check below. +*/ +if (bt_is_reg_set (bt,BPF_REG_2 ) && + insn->src_reg ==BPF_PSEUDO_KFUNC_CALL && +bpf_kfunc_ret_reg_pair (env, insn)) +bt_clear_reg (bt,BPF_REG_2 );if (bt_reg_mask (bt) &BPF_REGMASK_ARGS ) {/* if backtracking was looking for registers R1-R5 * they should have been found already.
@@ -520,7 +541,41 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -EFAULT ; } }else if (opcode ==BPF_EXIT ) { -bool r0_precise; +bool from_subprog_call, r0_precise, r2_precise; +struct bpf_insn *call; +int subprog; + +/* BPF_EXIT in subprog or callback always returns +* right after the call instruction, so by checking +* whether the instruction at subseq_idx-1 is subprog +* call or not we can distinguish actual exit from +* *subprog* from exit from *callback*. In the former +* case, we need to propagate the precision of the +* return registers, if necessary. In the latter we +* never do that. +*/ + from_subprog_call = subseq_idx -1 >=0 && +bpf_pseudo_call (&env->prog->insnsi[subseq_idx -1 ]); + +/* Sample the return registers before the callback +* handling below clears R1-R5: unlike R0, R2 is an +* argument register as well, so that clear would drop +* a pair return on the floor. +*/ + r0_precise = from_subprog_call &&bt_is_reg_set (bt,BPF_REG_0 ); + r2_precise = false; +if (from_subprog_call &&bt_is_reg_set (bt,BPF_REG_2 )) { + call = &env->prog->insnsi[subseq_idx -1 ]; + subprog =bpf_find_subprog (env, subseq_idx + call->imm); +if (subprog <0 ) +return -EFAULT ; +/* Only a callee that does return a pair defines +* R2. Leave the mask alone otherwise, so that +* the check below still catches an R2 that has +* no business being set. +*/ + r2_precise =bpf_ret_reg_pair (env, subprog); + }/* Backtracking to a nested function call, 'idx' is a part of * the inner frame 'subseq_idx' is a part of the outer frame.
@@ -533,30 +588,27 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,if (subseq_idx >=0 &&bpf_calls_callback (env, subseq_idx))for (i =BPF_REG_1 ; i <=BPF_REG_5 ; i++)bt_clear_reg (bt, i); + +/* a callee returning more than 8 bytes sets R2 as well; +* R2 is part of the args mask checked just below, so +* clear it here rather than next to R0. +*/ +if (r2_precise) +bt_clear_reg (bt,BPF_REG_2 );if (bt_reg_mask (bt) &BPF_REGMASK_ARGS ) {verifier_bug (env,"backtracking exit unexpected regs %x" ,bt_reg_mask (bt));return -EFAULT ; } -/* BPF_EXIT in subprog or callback always returns -* right after the call instruction, so by checking -* whether the instruction at subseq_idx-1 is subprog -* call or not we can distinguish actual exit from -* *subprog* from exit from *callback*. In the former -* case, we need to propagate r0 precision, if -* necessary. In the former we never do that. -*/ - r0_precise = subseq_idx -1 >=0 && -bpf_pseudo_call (&env->prog->insnsi[subseq_idx -1 ]) && -bt_is_reg_set (bt,BPF_REG_0 ); -bt_clear_reg (bt,BPF_REG_0 );if (bt_subprog_enter (bt))return -EFAULT ;if (r0_precise)bt_set_reg (bt,BPF_REG_0 ); +if (r2_precise) +bt_set_reg (bt,BPF_REG_2 );/* r6-r9 and stack slots will stay set in caller frame * bitmasks until we return back from callee(s) */
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 04/11] bpf: Track R2 of register-pair returns in precision backtracking
This isn't a bug, but the "R2 is part of the args mask checked just below, so clear it here rather than next to R0" rationale appears at each of the three call boundaries (global subprog, kfunc, and…
Quoted 18 lines
kernel/bpf/backtrack.c +10
-0@@ -423,6 +423,16 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,*/ verifier_bug_if (idx +1 != subseq_idx, env,"extra insn from subprog" ); +/* a global subprog returning more than 8 bytes +* sets R2 as well. R2 is part of the args mask +* checked just below, so clear it here rather +* than next to R0. Only a subprog that does +* return a pair defines R2, so leave the mask +* alone otherwise and let the check below catch +* an R2 that has no business being set. +*/ +if (bpf_ret_reg_pair (env, subprog)) +bt_clear_reg (bt,BPF_REG_2 );
[ ... ]
Quoted 15 lines
kernel/bpf/backtrack.c +11
-0@@ -506,6 +516,17 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -ENOTSUPP ;/* regular helper call sets R0 */ bt_clear_reg (bt,BPF_REG_0 ); +/* a kfunc returning more than 8 bytes also sets R2. +* R2 is part of the args mask checked just below, so +* clear it here rather than next to R0. The prototype +* lookup is only worth doing when R2 is requested at +* all; any other call leaves R2 uninitialized, so a +* request for it is caught by the check below. +*/ +if (bt_is_reg_set (bt,BPF_REG_2 ) && + insn->src_reg ==BPF_PSEUDO_KFUNC_CALL && +bpf_kfunc_ret_reg_pair (env, insn)) +bt_clear_reg (bt,BPF_REG_2 );
[ ... ]
Quoted 4 lines
kernel/bpf/backtrack.c +0
-0@@ -520,7 +541,41 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -EFAULT ; } }else if (opcode ==BPF_EXIT ) {
[ ... ]
Quoted 14 lines
kernel/bpf/backtrack.c +14
-0+ r0_precise = from_subprog_call &&bt_is_reg_set (bt,BPF_REG_0 ); + r2_precise = false; +if (from_subprog_call &&bt_is_reg_set (bt,BPF_REG_2 )) { + call = &env->prog->insnsi[subseq_idx -1 ]; + subprog =bpf_find_subprog (env, subseq_idx + call->imm); +if (subprog <0 ) +return -EFAULT ; +/* Only a callee that does return a pair defines +* R2. Leave the mask alone otherwise, so that +* the check below still catches an R2 that has +* no business being set. +*/ + r2_precise =bpf_ret_reg_pair (env, subprog); + }
[ ... ]
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 597
Quoted 11 lines
kernel/bpf/backtrack.c +7
-0@@ -533,30 +588,27 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,if (subseq_idx >=0 &&bpf_calls_callback (env, subseq_idx))for (i =BPF_REG_1 ; i <=BPF_REG_5 ; i++)bt_clear_reg (bt, i); + +/* a callee returning more than 8 bytes sets R2 as well; +* R2 is part of the args mask checked just below, so +* clear it here rather than next to R0. +*/ +if (r2_precise) +bt_clear_reg (bt,BPF_REG_2 );
This isn't a bug, but the "R2 is part of the args mask checked just below, so clear it here rather than next to R0" rationale appears at each of the three call boundaries (global subprog, kfunc, and BPF_EXIT). Would stating it once - say at the first site, or in a note above the BPF_EXIT block - and keeping the other two to a one-liner read more easily?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Yonghong Song
reviewerRe: [PATCH bpf-next v5 04/11] bpf: Track R2 of register-pair returns in precision backtracking
Quoted 68 lines · On 8/13/26 1:49 PM, [email protected] wrote:
Quoted 18 lines
kernel/bpf/backtrack.c +10
-0@@ -423,6 +423,16 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,*/ verifier_bug_if (idx +1 != subseq_idx, env,"extra insn from subprog" ); +/* a global subprog returning more than 8 bytes +* sets R2 as well. R2 is part of the args mask +* checked just below, so clear it here rather +* than next to R0. Only a subprog that does +* return a pair defines R2, so leave the mask +* alone otherwise and let the check below catch +* an R2 that has no business being set. +*/ +if (bpf_ret_reg_pair (env, subprog)) +bt_clear_reg (bt,BPF_REG_2 );[ ... ]
Quoted 15 lines
kernel/bpf/backtrack.c +11
-0@@ -506,6 +516,17 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -ENOTSUPP ;/* regular helper call sets R0 */ bt_clear_reg (bt,BPF_REG_0 ); +/* a kfunc returning more than 8 bytes also sets R2. +* R2 is part of the args mask checked just below, so +* clear it here rather than next to R0. The prototype +* lookup is only worth doing when R2 is requested at +* all; any other call leaves R2 uninitialized, so a +* request for it is caught by the check below. +*/ +if (bt_is_reg_set (bt,BPF_REG_2 ) && + insn->src_reg ==BPF_PSEUDO_KFUNC_CALL && +bpf_kfunc_ret_reg_pair (env, insn)) +bt_clear_reg (bt,BPF_REG_2 );[ ... ]
Quoted 4 lines
kernel/bpf/backtrack.c +0
-0@@ -520,7 +541,41 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -EFAULT ; } }else if (opcode ==BPF_EXIT ) {[ ... ]
Quoted 14 lines
kernel/bpf/backtrack.c +14
-0+ r0_precise = from_subprog_call &&bt_is_reg_set (bt,BPF_REG_0 ); + r2_precise = false; +if (from_subprog_call &&bt_is_reg_set (bt,BPF_REG_2 )) { + call = &env->prog->insnsi[subseq_idx -1 ]; + subprog =bpf_find_subprog (env, subseq_idx + call->imm); +if (subprog <0 ) +return -EFAULT ; +/* Only a callee that does return a pair defines +* R2. Leave the mask alone otherwise, so that +* the check below still catches an R2 that has +* no business being set. +*/ + r2_precise =bpf_ret_reg_pair (env, subprog); + }[ ... ]
Quoted 9 lines
kernel/bpf/backtrack.c +5
-0@@ -533,30 +588,27 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,if (subseq_idx >=0 &&bpf_calls_callback (env, subseq_idx))for (i =BPF_REG_1 ; i <=BPF_REG_5 ; i++)bt_clear_reg (bt, i); + +/* a callee returning more than 8 bytes sets R2 as well; +* R2 is part of the args mask checked just below, so +* clear it here rather than next to R0. +*/
Quoted 8 lines
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 597
Quoted 2 lines
kernel/bpf/backtrack.c +2
-0+if (r2_precise) +bt_clear_reg (bt,BPF_REG_2 );This isn't a bug, but the "R2 is part of the args mask checked just below, so clear it here rather than next to R0" rationale appears at each of the three call boundaries (global subprog, kfunc, and
BPF_EXIT). Would stating it once - say at the first site, or in a note above theBPF_EXITblock - and keeping the other two to a one-liner read more easily?
Eduard suggested to remove the condition "r2_precise" for bt_clear_reg(bt, BPF_REG_2) which should solve the problem.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Eduard Zingerman
maintainerRe: [PATCH bpf-next v5 04/11] bpf: Track R2 of register-pair returns in precision backtracking
In reply to Yonghong SongOn Thu, 2026-08-13 at 13:02 -0700, Yonghong Song wrote:
...
Quoted 33 lines
(quoted diff) +28
-1@@ -520,7 +541,41 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, return -EFAULT; } } else if (opcode == BPF_EXIT) { - bool r0_precise; + bool from_subprog_call, r0_precise, r2_precise; + struct bpf_insn *call; + int subprog; + + /* BPF_EXIT in subprog or callback always returns + * right after the call instruction, so by checking + * whether the instruction at subseq_idx-1 is subprog + * call or not we can distinguish actual exit from + * *subprog* from exit from *callback*. In the former + * case, we need to propagate the precision of the + * return registers, if necessary. In the latter we + * never do that. + */ + from_subprog_call = subseq_idx - 1 >= 0 && + bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]); + + /* Sample the return registers before the callback + * handling below clears R1-R5: unlike R0, R2 is an + * argument register as well, so that clear would drop + * a pair return on the floor. + */ + r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0); + r2_precise = false; + if (from_subprog_call && bt_is_reg_set(bt, BPF_REG_2)) { + call = &env->prog->insnsi[subseq_idx - 1]; + subprog = bpf_find_subprog(env, subseq_idx + call->imm); + if (subprog < 0) + return -EFAULT;
Review static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 581
Quoted 10 lines
(quoted diff) +7
-0+ /* Only a callee that does return a pair defines + * R2. Leave the mask alone otherwise, so that + * the check below still catches an R2 that has + * no business being set. + */ + r2_precise = bpf_ret_reg_pair(env, subprog); + } /* Backtracking to a nested function call, 'idx' is a part of * the inner frame 'subseq_idx' is a part of the outer frame.
I still think that the above complications are unnecessary. The patch could be simplified by assuming that R2 always propagates w/o loosing verification safety. E.g. as in the attachment.
kernel/bpf/backtrack.c +30 -16
@@ -423,6 +423,10 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,*/ verifier_bug_if (idx +1 != subseq_idx, env,"extra insn from subprog" ); +/* global subprog always sets R0 */ +bt_clear_reg (bt,BPF_REG_0 ); +/* and if it does not set R2, main pass would catch it */ +bt_clear_reg (bt,BPF_REG_2 );/* r1-r5 are invalidated after subprog call, * so for global func call it shouldn't be set * anymore
@@ -432,8 +436,6 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, bt_reg_mask(bt));return -EFAULT ; } -/* global subprog always sets R0 */ -bt_clear_reg (bt,BPF_REG_0 );return 0 ; }else {/* static subprog call instruction, which
@@ -506,6 +508,8 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -ENOTSUPP ;/* regular helper call sets R0 */ bt_clear_reg (bt,BPF_REG_0 ); +/* kfunc might aslo set R2 */ +bt_clear_reg (bt,BPF_REG_2 );if (bt_reg_mask (bt) &BPF_REGMASK_ARGS ) {/* if backtracking was looking for registers R1-R5 * they should have been found already.
@@ -520,7 +524,25 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -EFAULT ; } }else if (opcode ==BPF_EXIT ) { -bool r0_precise; +bool from_subprog_call, r0_precise, r2_precise; + +/* BPF_EXIT in subprog or callback always returns +* right after the call instruction, so by checking +* whether the instruction at subseq_idx-1 is subprog +* call or not we can distinguish actual exit from +* *subprog* from exit from *callback*. In the former +* case, we need to propagate the precision of the +* return registers, if necessary. In the latter we +* never do that. +*/ + from_subprog_call = subseq_idx -1 >=0 && +bpf_pseudo_call (&env->prog->insnsi[subseq_idx -1 ]); + +/* Sample the return registers before the callback +* handling below clears R1-R5. +*/ + r0_precise = from_subprog_call &&bt_is_reg_set (bt,BPF_REG_0 ); + r2_precise = from_subprog_call &&bt_is_reg_set (bt,BPF_REG_2 );/* Backtracking to a nested function call, 'idx' is a part of * the inner frame 'subseq_idx' is a part of the outer frame.
@@ -533,30 +555,22 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,if (subseq_idx >=0 &&bpf_calls_callback (env, subseq_idx))for (i =BPF_REG_1 ; i <=BPF_REG_5 ; i++)bt_clear_reg (bt, i); + +bt_clear_reg (bt,BPF_REG_0 ); +bt_clear_reg (bt,BPF_REG_2 );if (bt_reg_mask (bt) &BPF_REGMASK_ARGS ) {verifier_bug (env,"backtracking exit unexpected regs %x" ,bt_reg_mask (bt));return -EFAULT ; } -/* BPF_EXIT in subprog or callback always returns -* right after the call instruction, so by checking -* whether the instruction at subseq_idx-1 is subprog -* call or not we can distinguish actual exit from -* *subprog* from exit from *callback*. In the former -* case, we need to propagate r0 precision, if -* necessary. In the former we never do that. -*/ - r0_precise = subseq_idx -1 >=0 && -bpf_pseudo_call (&env->prog->insnsi[subseq_idx -1 ]) && -bt_is_reg_set (bt,BPF_REG_0 ); - -bt_clear_reg (bt,BPF_REG_0 );if (bt_subprog_enter (bt))return -EFAULT ;if (r0_precise)bt_set_reg (bt,BPF_REG_0 ); +if (r2_precise) +bt_set_reg (bt,BPF_REG_2 );/* r6-r9 and stack slots will stay set in caller frame * bitmasks until we return back from callee(s) */
Yonghong Song
reviewerRe: [PATCH bpf-next v5 04/11] bpf: Track R2 of register-pair returns in precision backtracking
Quoted 45 lines · Eduard Zingerman
On Thu, 2026-08-13 at 13:02 -0700, Yonghong Song wrote:
...
Quoted 41 lines
kernel/bpf/backtrack.c +35
-1@@ -520,7 +541,41 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,return -EFAULT ; } }else if (opcode ==BPF_EXIT ) { -bool r0_precise; +bool from_subprog_call, r0_precise, r2_precise; +struct bpf_insn *call; +int subprog; + +/* BPF_EXIT in subprog or callback always returns +* right after the call instruction, so by checking +* whether the instruction at subseq_idx-1 is subprog +* call or not we can distinguish actual exit from +* *subprog* from exit from *callback*. In the former +* case, we need to propagate the precision of the +* return registers, if necessary. In the latter we +* never do that. +*/ + from_subprog_call = subseq_idx -1 >=0 && +bpf_pseudo_call (&env->prog->insnsi[subseq_idx -1 ]); + +/* Sample the return registers before the callback +* handling below clears R1-R5: unlike R0, R2 is an +* argument register as well, so that clear would drop +* a pair return on the floor. +*/ + r0_precise = from_subprog_call &&bt_is_reg_set (bt,BPF_REG_0 ); + r2_precise = false; +if (from_subprog_call &&bt_is_reg_set (bt,BPF_REG_2 )) { + call = &env->prog->insnsi[subseq_idx -1 ]; + subprog =bpf_find_subprog (env, subseq_idx + call->imm); +if (subprog <0 ) +return -EFAULT ; +/* Only a callee that does return a pair defines +* R2. Leave the mask alone otherwise, so that +* the check below still catches an R2 that has +* no business being set. +*/ + r2_precise =bpf_ret_reg_pair (env, subprog); + }
Quoted 5 lines
kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 548
Quoted 2 lines
kernel/bpf/backtrack.c +0
-0/* Backtracking to a nested function call, 'idx' is a part of * the inner frame 'subseq_idx' is a part of the outer frame. I still think that the above complications are unnecessary. The patch could be simplified by assuming that R2 always propagates w/o loosing verification safety. E.g. as in the attachment.
Ack. Thanks and will adopt your suggestions.
Yonghong Song
reviewerPATCH v5 5/11 bpf: Account R2 of register-pair returns in live register analysis
In reply to Yonghong SongA BPF_EXIT of a subprogram returning a value larger than 8 bytes (a struct/union or an __int128) reads R2 as well as R0, since the second half of the return value is passed back in R2. compute_insn_live_regs() only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.
Mark R2 as read at the BPF_EXIT of a subprogram that does return a register pair. bpf_compute_live_registers() now loops over the subprograms and, for each, over the [start, end) instruction range from env->subprog_info[], so the return convention is queried once per subprogram through bpf_ret_reg_pair() rather than once per instruction.
Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.
--- kernel/bpf/liveness.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
kernel/bpf/liveness.c +14 -6
@@ -2060,7 +2060,8 @@ static inline u16 mask_hi(u32 m) { return (u16)(m >> 16); }/* Compute info->{use,def} fields for the instruction */ static void compute_insn_live_regs (struct bpf_verifier_env *env,struct bpf_insn *insn, -struct insn_live_regs *info) +struct insn_live_regs *info, +bool ret_reg_pair) {struct bpf_call_summary cs;const u8 class =BPF_CLASS (insn->code);
@@ -2072,6 +2073,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,const u32 src32 =mask_lo (src);const u32 dst32 =mask_lo (dst);const u32 r0 =reg64_mask (0 ); +const u32 r2 =reg64_mask (BPF_REG_2 );u32 def =0 ;u32 use =U32_MAX ;
@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,break ;case BPF_EXIT : def =0 ; - use = r0; + use = ret_reg_pair ? (r0 | r2) : r0;break ;case BPF_CALL : def =ALL_CALLER_SAVED_REGS ;
@@ -2228,8 +2230,8 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)struct insn_live_regs *state;int insn_cnt = env->prog->len;u64 pos, insn_pos; -int err =0 , i, j; -bool changed; +int err =0 , i, j, subprog, start, end; +bool changed, ret_reg_pair;/* Use the following algorithm: * - define the following:
@@ -2256,8 +2258,14 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env) goto out; } -for (i =0 ; i < insn_cnt; ++i) -compute_insn_live_regs (env, &insns[i], &state[i]); +for (subprog =0 ; subprog < env->subprog_cnt; subprog++) { + start = env->subprog_info[subprog].start; + end = env->subprog_info[subprog +1 ].start; + ret_reg_pair =bpf_ret_reg_pair (env, subprog); + +for (i = start; i < end; ++i) +compute_insn_live_regs (env, &insns[i], &state[i], ret_reg_pair); + }/* Forward pass: resolve stack access through FP-derived pointers */ err =bpf_compute_subprog_arg_access (env);
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 05/11] bpf: Account R2 of register-pair returns in live register analysis
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Static (non-global) BPF subprograms returning a register pair are functionally broken because the verifier does not propagate R2 back to the caller frame. --
--- Patch [5]: [PATCH bpf-next v5 05/11] bpf: Account R2 of register-pair returns in live register analysis --- commit 35ab629244a73b2ad2baa7b7f0fd5b365c1b35e9
Author: Yonghong Song <[email protected]>
bpf: Account R2 of register-pair returns in live register analysis
This commit updates the live register analysis to correctly mark R2 as read at the BPF_EXIT of a subprogram that returns a register pair. This prevents R2 from being considered dead and pruned incorrectly.
Quoted 4 lines
kernel/bpf/liveness.c +0
-0
[ ... ]
kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2197
Quoted 7 lines
kernel/bpf/liveness.c +1
-1@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,break ;case BPF_EXIT : def =0 ; - use = r0; + use = ret_reg_pair ? (r0 | r2) : r0;break ;
[Severity: High]
This is a pre-existing issue, and wasn't introduced by this patch, but while we are fixing liveness tracking for register-pair returns here, is the propagation of R2 back to the caller missing in the verifier?
Looking at prepare_func_exit() in kernel/bpf/verifier.c:
} else {
/* return to the caller whatever r0 had in the callee */
caller->regs[BPF_REG_0] = *r0;
}When a static BPF subprogram returning a 16-byte struct is called, check_func_call() sets all caller-saved registers (including R2) to NOT_INIT. When the callee exits, it seems we only copy BPF_REG_0 back to the caller's state, leaving BPF_REG_2 as NOT_INIT.
If the caller attempts to read the second half of the returned struct in R2, will the verifier reject it with an 'R2 !read_ok' error?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Yonghong Song
reviewerRe: [PATCH bpf-next v5 05/11] bpf: Account R2 of register-pair returns in live register analysis
Quoted 40 lines · On 8/13/26 6:50 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Static (non-global) BPF subprograms returning a register pair are functionally broken because the verifier does not propagate R2 back to the caller frame. ----- Patch [5]: [PATCH bpf-next v5 05/11] bpf: Account R2 of register-pair returns in live register analysis --- commit
35ab629244a73b2ad2baa7b7f0fd5b365c1b35e9
Author: Yonghong Song <[email protected]>bpf: Account R2 of register-pair returns in live register analysis
This commit updates the live register analysis to correctly mark R2 as read at the
BPF_EXITof a subprogram that returns a register pair. This prevents R2 from being considered dead and pruned incorrectly.Quoted 4 lines
kernel/bpf/liveness.c +0
-0[ ... ]
kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2197
Quoted 7 lines
kernel/bpf/liveness.c +1
-1@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,break ;case BPF_EXIT : def =0 ; - use = r0; + use = ret_reg_pair ? (r0 | r2) : r0;break ;[Severity: High]
This is a pre-existing issue, and wasn't introduced by this patch, but while we are fixing liveness tracking for register-pair returns here, is the propagation of R2 back to the caller missing in the verifier?Looking at
prepare_func_exit()inkernel/bpf/verifier.c:} else { /* return to the caller whatever r0 had in the callee */ caller->regs[BPF_REG_0] = *r0; }
Quoted 7 lines
When a static BPF subprogram returning a 16-byte struct is called,
check_func_call()sets all caller-saved registers (including R2) toNOT_INIT. When the callee exits, it seems we only copyBPF_REG_0back to the caller's state, leavingBPF_REG_2asNOT_INIT.If the caller attempts to read the second half of the returned struct in R2, will the verifier reject it with an 'R2 !read_ok' error?
We should be okay here. The next patch will actually introducing return value R2 in various places.
Quoted 1 line
Yonghong Song
reviewerPATCH v5 6/11 bpf: Add verifier support for 16-byte returns in R0:R2
In reply to Yonghong SongLLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].
Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an __int128 return failed later in the backend with:
fatal error: error in backend: unable to allocate function return #1
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch models that pair at calls to global and static BPF subprograms and at kfunc calls: R2 is marked alongside R0 at the call, propagated out of a callee at its exit, and held to the same scalar-only and no-stack- pointer rules that R0 already is. A struct returned by a kfunc must be composed of scalars, since its bytes reach the program as raw register contents and a pointer field would otherwise be laundered into a scalar.
An extension program is the one caller of the convention that cannot take part in it: its own return value is the program exit code, read out of R0 alone, so it has no way to hand back an upper half. Replacing a function whose return value is larger than 8 bytes is therefore rejected with -EOPNOTSUPP rather than supported.
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876
--- kernel/bpf/verifier.c | 75 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 68 insertions(+), 7 deletions(-)
kernel/bpf/verifier.c +68 -7
@@ -412,6 +412,9 @@ static u32 ret_regs_cnt(u32 size)return size >8 && size <=16 ?2 :1 ; } +/* Registers holding a function return value, in order. See ret_regs_cnt(). */ +static const int ret_regs[] = {BPF_REG_0 ,BPF_REG_2 }; +static void bpf_compute_subprog_ret_regs (struct bpf_verifier_env *env) {const struct btf *btf = env->prog->aux->btf;
@@ -9456,6 +9459,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,u16 callee_incoming, stack_arg_cnt;struct bpf_func_state *caller;int err, subprog, target_insn; +u32 i, nregs; target_insn = *insn_idx + insn->imm +1 ; subprog =bpf_find_subprog (env, target_insn);
@@ -9498,9 +9502,14 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,clear_caller_saved_regs (env, caller->regs);invalidate_outgoing_stack_args (env,cur_func (env)); -/* All non-void global functions return a 64-bit SCALAR_VALUE. */ +/* +* A non-void global function returns a 64-bit SCALAR_VALUE in +* R0, or a >8 byte SCALAR_VALUE in the R0:R2 register pair. +*/ if (!subprog_returns_void (env, subprog)) { -mark_reg_unknown (env, caller->regs,BPF_REG_0 ); + nregs =bpf_ret_reg_pair (env, subprog) ?2 :1 ; +for (i =0 ; i < nregs; i++) +mark_reg_unknown (env, caller->regs, ret_regs[i]); }if (env->subprog_info[subprog].might_throw) {
@@ -9858,11 +9867,15 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)struct bpf_func_state *caller, *callee;struct bpf_reg_state *r0;bool in_callback_fn; +u32 i, nregs;int err; callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0 ]; -if (r0->type ==PTR_TO_STACK ) { + nregs =bpf_ret_reg_pair (env, callee->subprogno) ?2 :1 ; +for (i =0 ; i < nregs; i++) { +if (callee->regs[ret_regs[i]].type !=PTR_TO_STACK ) +continue ;/* technically it's ok to return caller's stack pointer * (or caller's caller's pointer) back to the caller, * since these pointers are valid. Only current stack
@@ -9897,8 +9910,12 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)return -EFAULT ; } }else { -/* return to the caller whatever r0 had in the callee */ - caller->regs[BPF_REG_0 ] = *r0; +/* +* return to the caller whatever the callee had in the +* return register(s) +*/ +for (i =0 ; i < nregs; i++) + caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]]; }/* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,
@@ -10796,6 +10813,19 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insnreturn 0 ; } +/* +* Mark the register(s) holding a @size byte kfunc return value as unknown +* scalars. Both halves of a register pair are treated the same way. +*/ +static void mark_kfunc_ret_regs (struct bpf_verifier_env *env, +struct bpf_reg_state *regs,u32 size) +{ +u32 i, nregs =ret_regs_cnt (size); + +for (i =0 ; i < nregs; i++) +mark_reg_unknown (env, regs, ret_regs[i]); +} +static bool is_kfunc_acquire (struct bpf_call_arg_meta *meta) {return meta->kfunc_flags &KF_ACQUIRE ;
@@ -13265,10 +13295,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, }if (btf_type_is_scalar (t)) { -mark_reg_unknown (env, regs,BPF_REG_0 ); +mark_kfunc_ret_regs (env, regs, t->size);if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] || meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))__mark_reg_const_zero (env, ®s[BPF_REG_0 ]); + }else if (btf_type_is_struct (t)) { +/* +* The returned struct comes back as raw register bits modeled +* as an unknown scalar, so it must contain only scalars: +* otherwise a pointer field would be laundered into a scalar +* and escape provenance and reference tracking. +*/ +if (!__btf_type_is_scalar_struct (env, desc_btf, t,0 )) { +verbose (env, +"kernel function %s returns %s %s that is not composed of scalars\n" , + func_name,btf_type_str (t), +btf_name_by_offset (desc_btf, t->name_off)); +return -EINVAL ; + } +mark_kfunc_ret_regs (env, regs, t->size); }else if (btf_type_is_ptr (t)) { ptr_type =btf_type_skip_modifiers (desc_btf, t->type, &ptr_type_id); err =check_special_kfunc (env, &meta, regs, insn_aux, ptr_type, desc_btf);
@@ -16746,11 +16791,20 @@ static int check_global_subprog_return_code(struct bpf_verifier_env *env) {struct bpf_func_state *cur_frame =cur_func (env);u32 subprog = cur_frame->subprogno; +u32 i, nregs; +int err;if (subprog_returns_void (env, subprog))return 0 ; -return check_global_ret_scalar_reg (env,BPF_REG_0 ); + nregs =bpf_ret_reg_pair (env, subprog) ?2 :1 ; +for (i =0 ; i < nregs; i++) { + err =check_global_ret_scalar_reg (env, ret_regs[i]); +if (err) +return err; + } + +return 0 ; }/* Bitmask with 1s for all caller saved registers */
@@ -19404,6 +19458,13 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,return -EOPNOTSUPP ; } +if (prog_extension && tgt_info->fmodel.ret_size >8 ) { +bpf_log (log, +"Cannot replace function %s with a >8 byte return value\n" , + tname); +return -EOPNOTSUPP ; + } +/* * *.multi programs don't need an address during program * verification, we just take the module ref if needed.
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2
This isn't a bug, but could the changelog be reworded in imperative mood to match Documentation/process/submitting-patches.rst? The other commits in this series follow that convention, so this one…
Quoted 16 lines
bpf: Add verifier support for 16-byte returns in R0:R2
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an
__int128return failed later in the backend with:fatal error: error in backend: unable to allocate function return #1
Quoted 7 lines
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch models that pair at calls to global and static BPF subprograms and at kfunc calls: R2 is marked alongside R0 at the call, propagated out of a callee at its exit, and held to the same scalar-only and no-stack- pointer rules that R0 already is.
This isn't a bug, but could the changelog be reworded in imperative mood to match Documentation/process/submitting-patches.rst? The other commits in this series follow that convention, so this one reads inconsistently.
Quoted 7 lines
kernel/bpf/verifier.c +1
-0@@ -9858,11 +9867,15 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)struct bpf_func_state *caller, *callee;struct bpf_reg_state *r0;bool in_callback_fn; +u32 i, nregs;int err;
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9881
Quoted 10 lines
kernel/bpf/verifier.c +4
-1callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0 ]; -if (r0->type ==PTR_TO_STACK ) { + nregs =bpf_ret_reg_pair (env, callee->subprogno) ?2 :1 ; +for (i =0 ; i < nregs; i++) { +if (callee->regs[ret_regs[i]].type !=PTR_TO_STACK ) +continue ;/* technically it's ok to return caller's stack pointer * (or caller's caller's pointer) back to the caller, * since these pointers are valid. Only current stack
The new R0:R2 return-register loop is placed before the
'if (callee->in_callback_fn)' branch, so it runs for callback frames as well. A callback follows bpf_callback_t (return value in R0 only) and R2 holds an incoming argument: set_loop_callback_state() and set_user_ringbuf_callback_state() both do 'callee->regs[BPF_REG_2] = caller->regs[BPF_REG_3]', i.e. the callback_ctx, which is PTR_TO_STACK for the common 'bpf_loop(n, cb, &local_var, 0)' idiom.
bpf_compute_subprog_ret_regs() sets ret_reg_pair purely from the subprog's BTF return type and iterates every subprog, with no exclusion for subprogs used as callbacks. So a static subprog whose BTF declares a 9..16 byte return and which is passed to bpf_loop() or bpf_user_ringbuf_drain() gets ret_reg_pair = true, and at its BPF_EXIT the still-live callback_ctx in R2 is inspected as if it were the upper half of a return value.
Consequence: the callback exit is rejected with 'cannot return stack pointer to the caller', a diagnostic that points at a register holding an argument rather than a return value, and it fires before the in_callback_fn R0 checks that would otherwise produce the accurate message. Impact is limited (any such program is already ill-typed with respect to bpf_callback_t, and compiler-generated code for a 16-byte return does write R2), so this is a misleading-rejection / modelling-scope defect rather than a memory-safety bug.
Would it make sense to compute nregs after the in_callback_fn branch, or force nregs = 1 when callee->in_callback_fn, since a callback frame has no R0:R2 return convention?
kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9919
Quoted 13 lines
kernel/bpf/verifier.c +6
-2@@ -9897,8 +9910,12 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)return -EFAULT ; } }else { -/* return to the caller whatever r0 had in the callee */ - caller->regs[BPF_REG_0 ] = *r0; +/* +* return to the caller whatever the callee had in the +* return register(s) +*/ +for (i =0 ; i < nregs; i++) + caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]]; }
prepare_func_exit() has three call sites, and only the real BPF_EXIT one (process_bpf_exit_full()) establishes the callee's return registers. The other two are synthetic exits that explicitly set up R0 but not R2, because R1-R5 were already reset to NOT_INIT above them:
In check_helper_call(), BPF_FUNC_tail_call inside a subprog: the caller-saved reset loop sets R1-R5 to NOT_INIT, then only 'mark_reg_unknown(env, regs, BPF_REG_0)' runs before 'prepare_func_exit(env, &env->insn_idx)'.
In check_ld_abs() hidden BPF_EXIT inside a subprog: reset loop, then only 'mark_reg_known_zero(env, regs, BPF_REG_0)' before prepare_func_exit().
For a static subprog with ret_reg_pair set, the new copy loop therefore propagates a NOT_INIT R2 into caller->regs[BPF_REG_2] on those paths. This is fail-safe (no uninitialised value is treated as readable), but it means a caller that reads the upper half of the 16-byte return is rejected on that path with an opaque 'R2 !read_ok', so a subprog with a
Review static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9920
Quoted 1 line
(quoted diff) +0
-08 byte return that contains a bpf_tail_call() or LD_ABS is unverifiable,
and only R0 gets the conservative unknown-scalar treatment the commit message describes.
Would symmetric handling be to mark R2 unknown next to R0 at both sites (the tail-called program only defines R0, so R2 is genuinely undefined there), or to reject the combination with an explicit message?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Yonghong Song
reviewerRe: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2
Quoted 70 lines · On 8/13/26 2:11 PM, [email protected] wrote:
Quoted 16 lines
bpf: Add verifier support for 16-byte returns in R0:R2
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an
__int128return failed later in the backend with:fatal error: error in backend: unable to allocate function return #1
Quoted 7 lines
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch models that pair at calls to global and static BPF subprograms and at kfunc calls: R2 is marked alongside R0 at the call, propagated out of a callee at its exit, and held to the same scalar-only and no-stack- pointer rules that R0 already is.
This isn't a bug, but could the changelog be reworded in imperative mood to match
Documentation/process/submitting-patches.rst? The other commits in this series follow that convention, so this one reads inconsistently.Quoted 7 lines
kernel/bpf/verifier.c +1
-0@@ -9858,11 +9867,15 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)struct bpf_func_state *caller, *callee;struct bpf_reg_state *r0;bool in_callback_fn; +u32 i, nregs;int err;kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9881
Quoted 10 lines
kernel/bpf/verifier.c +4
-1callee = state->frame[state->curframe]; r0 = &callee->regs[BPF_REG_0 ]; -if (r0->type ==PTR_TO_STACK ) { + nregs =bpf_ret_reg_pair (env, callee->subprogno) ?2 :1 ; +for (i =0 ; i < nregs; i++) { +if (callee->regs[ret_regs[i]].type !=PTR_TO_STACK ) +continue ;/* technically it's ok to return caller's stack pointer * (or caller's caller's pointer) back to the caller, * since these pointers are valid. Only current stack The new R0:R2 return-register loop is placed before the
'if (callee->in_callback_fn)' branch, so it runs for callback frames as well. A callback followsbpf_callback_t(return value in R0 only) and R2 holds an incoming argument:set_loop_callback_state()andset_user_ringbuf_callback_state()both do 'callee->regs[BPF_REG_2] = caller->regs[BPF_REG_3]', i.e. thecallback_ctx, which isPTR_TO_STACKfor the common 'bpf_loop(n, cb, &local_var, 0)' idiom.
bpf_compute_subprog_ret_regs()setsret_reg_pairpurely from the subprog's BTF return type and iterates every subprog, with no exclusion for subprogs used as callbacks. So a static subprog whose BTF declares a 9..16 byte return and which is passed tobpf_loop()orbpf_user_ringbuf_drain()getsret_reg_pair= true, and at itsBPF_EXITthe still-livecallback_ctxin R2 is inspected as if it were the upper half of a return value.Consequence: the callback exit is rejected with 'cannot return stack pointer to the caller', a diagnostic that points at a register holding an argument rather than a return value, and it fires before the
in_callback_fnR0 checks that would otherwise produce the accurate message. Impact is limited (any such program is already ill-typed with respect tobpf_callback_t, and compiler-generated code for a 16-byte return does write R2), so this is a misleading-rejection / modelling-scope defect rather than a memory-safety bug.
Quoted 3 lines
Would it make sense to compute nregs after the
in_callback_fnbranch, or force nregs = 1 whencallee->in_callback_fn, since a callback frame has no R0:R2 return convention?
The current callback function only returns one register. If the bpf prog intends to return two registers, the upper R2 register is ignored. The result may be incorrect. It is bpf prog users' problem.
Quoted 32 lines
Quoted 4 lines
kernel/bpf/verifier.c +0
-0@@ -9897,8 +9910,12 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)return -EFAULT ; }kernel/bpf/verifier.c static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9919
Quoted 10 lines
kernel/bpf/verifier.c +6
-2}else { -/* return to the caller whatever r0 had in the callee */ - caller->regs[BPF_REG_0 ] = *r0; +/* +* return to the caller whatever the callee had in the +* return register(s) +*/ +for (i =0 ; i < nregs; i++) + caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]]; }
prepare_func_exit()has three call sites, and only the realBPF_EXITone (process_bpf_exit_full()) establishes the callee's return registers. The other two are synthetic exits that explicitly set up R0 but not R2, because R1-R5 were already reset toNOT_INITabove them:In
check_helper_call(),BPF_FUNC_tail_callinside a subprog: the caller-saved reset loop sets R1-R5 toNOT_INIT, then only 'mark_reg_unknown(env, regs, BPF_REG_0)' runs before 'prepare_func_exit(env, &env->insn_idx)'.In
check_ld_abs()hiddenBPF_EXITinside a subprog: reset loop, then only 'mark_reg_known_zero(env, regs, BPF_REG_0)' beforeprepare_func_exit().For a static subprog with
ret_reg_pairset, the new copy loop therefore propagates aNOT_INITR2 into caller->regs[BPF_REG_2] on those paths. This is fail-safe (no uninitialised value is treated as readable), but it means a caller that reads the upper half of the 16-byte return is rejected on that path with an opaque 'R2 !read_ok', so a subprog with a
Quoted 7 lines
Review static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) · line 9920
Quoted 1 line
(quoted diff) +0
-08 byte return that contains a bpf_tail_call() or LD_ABS is unverifiable,and only R0 gets the conservative unknown-scalar treatment the commit message describes.
Would symmetric handling be to mark R2 unknown next to R0 at both sites (the tail-called program only defines R0, so R2 is genuinely undefined there), or to reject the combination with an explicit message?
Okay, this is related to tail call. The tail call prog expects to return R0 only. If the bpf prog intends to return R0:R2 (similar to main program), it is user's problem.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
[email protected]
AI reviewRe: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2
In reply to Yonghong SongThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The newly added support for kfuncs and global subprograms returning structs up to 16 bytes by…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added support for kfuncs and global subprograms returning structs up to 16 bytes by value is unreachable dead code because existing BTF validation logic unconditionally rejects struct return types. --
--- Patch [6]: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2 --- commit 9e421b8a810e50b00fab2bcdb9c1dbd32e705d84
Author: Yonghong Song <[email protected]>
bpf: Add verifier support for 16-byte returns in R0:R2
This commit adds verifier support for returning values in both R0 and R2 for types greater than 8 bytes and up to 16 bytes. It models the register pair for global and static subprograms as well as kfunc calls.
Quoted 4 lines
kernel/bpf/verifier.c +0
-0
[ ... ]
kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 13302
Quoted 10 lines
kernel/bpf/verifier.c +2
-1@@ -13265,10 +13295,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, }if (btf_type_is_scalar (t)) { -mark_reg_unknown (env, regs,BPF_REG_0 ); +mark_kfunc_ret_regs (env, regs, t->size);if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] || meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))__mark_reg_const_zero (env, ®s[BPF_REG_0 ]); + }else if (btf_type_is_struct (t)) {
[Severity: Medium]
Is this new logic for kfuncs returning structs actually reachable?
Looking at btf_distill_func_proto() in kernel/bpf/btf.c, it unconditionally rejects struct return types, meaning programs would fail to load early on:
if (ret < 0 || btf_type_is_struct(t)) {
bpf_log(log,
"The function %s return type %s is unsupported.\n",
tname, btf_type_str(t));
return -EINVAL;
}Similarly, btf_validate_return_type() blocks global subprograms from returning structs:
/* We always accept void or scalars. */ if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t)) return 0; return -EOPNOTSUPP;
Does this mean the new struct return handling logic is dead code, and only static subprograms or __int128 returns can actually use the new convention?
Quoted 15 lines
kernel/bpf/verifier.c +14
-0+/* +* The returned struct comes back as raw register bits modeled +* as an unknown scalar, so it must contain only scalars: +* otherwise a pointer field would be laundered into a scalar +* and escape provenance and reference tracking. +*/ +if (!__btf_type_is_scalar_struct (env, desc_btf, t,0 )) { +verbose (env, +"kernel function %s returns %s %s that is not composed of scalars\n" , + func_name,btf_type_str (t), +btf_name_by_offset (desc_btf, t->name_off)); +return -EINVAL ; + } +mark_kfunc_ret_regs (env, regs, t->size); }else if (btf_type_is_ptr (t)) {
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Yonghong Song
reviewerRe: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2
Quoted 46 lines · On 8/13/26 7:26 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added support for kfuncs and global subprograms returning structs up to 16 bytes by value is unreachable dead code because existing BTF validation logic unconditionally rejects struct return types. ----- Patch [6]: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2 --- commit
9e421b8a810e50b00fab2bcdb9c1dbd32e705d84
Author: Yonghong Song <[email protected]>bpf: Add verifier support for 16-byte returns in R0:R2
This commit adds verifier support for returning values in both R0 and R2 for types greater than 8 bytes and up to 16 bytes. It models the register pair for global and static subprograms as well as kfunc calls.
Quoted 4 lines
kernel/bpf/verifier.c +0
-0[ ... ]
kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 13302
Quoted 10 lines
kernel/bpf/verifier.c +2
-1@@ -13265,10 +13295,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, }if (btf_type_is_scalar (t)) { -mark_reg_unknown (env, regs,BPF_REG_0 ); +mark_kfunc_ret_regs (env, regs, t->size);if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] || meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))__mark_reg_const_zero (env, ®s[BPF_REG_0 ]); + }else if (btf_type_is_struct (t)) {[Severity: Medium]
Is this new logic for kfuncs returning structs actually reachable?Looking at
btf_distill_func_proto()inkernel/bpf/btf.c, it unconditionally rejects struct return types, meaning programs would fail to load early on:if (ret < 0 || btf_type_is_struct(t)) { bpf_log(log, "The function %s return type %s is unsupported.\n", tname, btf_type_str(t)); return -EINVAL; }Similarly,
btf_validate_return_type()blocks global subprograms from returning structs:
Quoted 8 lines
/* We always accept void or scalars. */ if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t)) return 0; return -EOPNOTSUPP;Does this mean the new struct return handling logic is dead code, and only static subprograms or
__int128returns can actually use the new convention?
The next patch will actually enable "aggregate return types up to 16 bytes".
Quoted 16 lines
Quoted 16 lines
kernel/bpf/verifier.c +14
-0+/* +* The returned struct comes back as raw register bits modeled +* as an unknown scalar, so it must contain only scalars: +* otherwise a pointer field would be laundered into a scalar +* and escape provenance and reference tracking. +*/ +if (!__btf_type_is_scalar_struct (env, desc_btf, t,0 )) { +verbose (env, +"kernel function %s returns %s %s that is not composed of scalars\n" , + func_name,btf_type_str (t), +btf_name_by_offset (desc_btf, t->name_off)); +return -EINVAL ; + } +mark_kfunc_ret_regs (env, regs, t->size); }else if (btf_type_is_ptr (t)) {
Re: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2
In reply to Yonghong SongQuoted 30 lines · Yonghong Song
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an
__int128return failed later in the backend with:fatal error: error in backend: unable to allocate function return #1
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch models that pair at calls to global and static BPF subprograms and at kfunc calls: R2 is marked alongside R0 at the call, propagated out of a callee at its exit, and held to the same scalar-only and no-stack- pointer rules that R0 already is. A struct returned by a kfunc must be composed of scalars, since its bytes reach the program as raw register contents and a pointer field would otherwise be laundered into a scalar.
An extension program is the one caller of the convention that cannot take part in it: its own return value is the program exit code, read out of R0 alone, so it has no way to hand back an upper half. Replacing a function whose return value is larger than 8 bytes is therefore rejected with
-EOPNOTSUPPrather than supported.
Quoted 5 lines
[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876Signed-off-by Yonghong Song <[email protected]>---
kernel/bpf/verifier.c int bpf_check_attach_target(struct bpf_verifier_log *log, · line 19467
Quoted 11 lines
kernel/bpf/verifier.c +7
-0@@ -19404,6 +19458,13 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,return -EOPNOTSUPP ; } +if (prog_extension && tgt_info->fmodel.ret_size >8 ) { +bpf_log (log, +"Cannot replace function %s with a >8 byte return value\n" , + tname); +return -EOPNOTSUPP ; + } +
As commented in v4 [1], this check belongs to btf_check_type_match().
Quoted 3 lines
kernel/bpf/verifier.c +0
-0/* * *.multi programs don't need an address during program * verification, we just take the module ref if needed.
Yonghong Song
reviewerRe: [PATCH bpf-next v5 06/11] bpf: Add verifier support for 16-byte returns in R0:R2
Quoted 51 lines · Eduard Zingerman
Quoted 30 lines · Yonghong Song
LLVM 23 added support for returning a value in two registers for an
__int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:
error: aggregate returns are not supported
and an
__int128return failed later in the backend with:fatal error: error in backend: unable to allocate function return #1
Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.
This patch models that pair at calls to global and static BPF subprograms and at kfunc calls: R2 is marked alongside R0 at the call, propagated out of a callee at its exit, and held to the same scalar-only and no-stack- pointer rules that R0 already is. A struct returned by a kfunc must be composed of scalars, since its bytes reach the program as raw register contents and a pointer field would otherwise be laundered into a scalar.
An extension program is the one caller of the convention that cannot take part in it: its own return value is the program exit code, read out of R0 alone, so it has no way to hand back an upper half. Replacing a function whose return value is larger than 8 bytes is therefore rejected with
-EOPNOTSUPPrather than supported.Quoted 5 lines
[1] https://github.com/llvm/llvm-project/pull/190894 [2] https://github.com/llvm/llvm-project/pull/206876Signed-off-by Yonghong Song <[email protected]>---
Acked-by Eduard Zingerman <[email protected]>kernel/bpf/verifier.c int bpf_check_attach_target(struct bpf_verifier_log *log, · line 19467
Quoted 11 lines
kernel/bpf/verifier.c +7
-0@@ -19404,6 +19458,13 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,return -EOPNOTSUPP ; } +if (prog_extension && tgt_info->fmodel.ret_size >8 ) { +bpf_log (log, +"Cannot replace function %s with a >8 byte return value\n" , + tname); +return -EOPNOTSUPP ; + } +As commented in v4 [1], this check belongs to
btf_check_type_match().
Quoted 1 line
Okay, will do.
Quoted 4 lines
Quoted 4 lines
kernel/bpf/verifier.c +0
-0/* * *.multi programs don't need an address during program * verification, we just take the module ref if needed.
Yonghong Song
reviewerPATCH v5 7/11 bpf: Enable aggregate return types up to 16 bytes
In reply to Yonghong SongRelax btf_distill_func_proto() to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry:
- a struct or union larger than 8 and up to 16 bytes, returned in the R0:R2 register pair, matching what LLVM emits for the BPF target;
- a struct or union up to 8 bytes, returned in R0 alone.
A >8 byte scalar (__int128) was already accepted and is unchanged. Everything else stays rejected: a return type larger than 16 bytes, and any type that __get_type_size() cannot return in registers at all (e.g. an array), which it already reports as ret < 0.
btf_validate_return_type() is relaxed as well, so that it accepts a by-value struct or union up to 16 bytes in addition to void and scalars. For a local (static) function this also stops btf_check_subprog_call() from marking the subprogram's BTF unreliable just because it returns an aggregate.
With btf_distill_func_proto() and btf_validate_return_type() relaxed, the verifier, JIT, precision-backtracking and live-register support from the earlier patches becomes reachable: <=16 byte aggregate return values now work end to end.
--- include/linux/bpf_verifier.h | 2 ++ kernel/bpf/btf.c | 23 +++++++++++++++---- kernel/bpf/verifier.c | 18 +++++++-------- .../selftests/bpf/progs/exceptions_fail.c | 2 +- 4 files changed, 30 insertions(+), 15 deletions(-)
include/linux/bpf_verifier.h +2 -0
@@ -1448,6 +1448,8 @@ int bpf_jmp_offset(struct bpf_insn *insn);struct bpf_iarray *bpf_insn_successors (struct bpf_verifier_env *env,u32 idx);void bpf_fmt_stack_mask (char *buf,ssize_t buf_sz,u64 stack_mask);bool bpf_subprog_is_global (const struct bpf_verifier_env *env,int subprog); +bool btf_type_is_scalar_struct (struct bpf_verifier_env *env,const struct btf *btf, +const struct btf_type *t,int rec);int bpf_find_subprog (struct bpf_verifier_env *env,int off);bool bpf_is_throw_kfunc (struct bpf_insn *insn);
kernel/bpf/btf.c +18 -5
@@ -7591,7 +7591,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,return -EINVAL ; } ret =__get_type_size (btf, func->type, &t); -if (ret <0 ||btf_type_is_struct (t)) { +if (ret <0 || ret >16 ) {bpf_log (log,"The function %s return type %s is unsupported.\n" , tname,btf_type_str (t));
@@ -7964,7 +7964,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,/* Check whether the type is a valid return type. */ static int btf_validate_return_type (struct bpf_verifier_env *env,struct btf *btf, -const struct btf_type *t,int subprog) +const struct btf_type *t,int subprog,bool is_global) {u32 tags =0 ;int err;
@@ -7987,6 +7987,19 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btif (btf_type_is_void (t) ||btf_type_is_int (t) ||btf_is_any_enum (t))return 0 ; +if (btf_type_is_struct (t) && t->size <=16 ) { +/* +* A global function's caller models the return as an opaque +* scalar pair, so it may only return scalars by value. A local +* function is verified inline, so a pointer field stays tracked +* and needs no such restriction. +*/ +bool local_func = subprog && !is_global; + +if (local_func ||btf_type_is_scalar_struct (env, btf, t,0 )) +return 0 ; + } +return -EOPNOTSUPP ; }
@@ -8074,12 +8087,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)return -EINVAL ; } - err =btf_validate_return_type (env, btf, t, subprog); + err =btf_validate_return_type (env, btf, t, subprog, is_global);if (err) {if (is_global) {bpf_log (log, -"Global function %s() return value not void or scalar. " -"Only those are supported.\n" , +"Global function %s() has unsupported return type. " +"Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\n" , tname); }return err;
kernel/bpf/verifier.c +9 -9
@@ -11093,9 +11093,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_ }/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ -static bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, -const struct btf *btf, -const struct btf_type *t,int rec) +bool btf_type_is_scalar_struct (struct bpf_verifier_env *env, +const struct btf *btf, +const struct btf_type *t,int rec) {const struct btf_type *member_type;const struct btf_member *member;
@@ -11113,7 +11113,7 @@ static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,verbose (env,"max struct nesting depth exceeded\n" );return false; } -if (!__btf_type_is_scalar_struct (env, btf, member_type, rec +1 )) +if (!btf_type_is_scalar_struct (env, btf, member_type, rec +1 ))return false;continue ; }
@@ -11512,7 +11512,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta, (is_kfunc_arg_mem_size(meta->btf, &args[arg + 1]) || is_kfunc_arg_const_mem_size(meta->btf, &args[arg + 1]))) {if (!btf_type_is_void (ref_t) && !btf_type_is_scalar (ref_t) && - !__btf_type_is_scalar_struct (env, meta->btf, ref_t,0 )) { + !btf_type_is_scalar_struct (env, meta->btf, ref_t,0 )) {verbose (env,"%s pointer type %s %s must point to void, scalar, or struct with scalar\n" ,reg_arg_name (env, argno),btf_type_str (ref_t), ref_tname);return -EINVAL ;
@@ -11528,7 +11528,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,* scalars. The access size is derived from the pointed-to BTF type. */ if (!btf_type_is_scalar (ref_t) && - !__btf_type_is_scalar_struct (env, meta->btf, ref_t,0 )) { + !btf_type_is_scalar_struct (env, meta->btf, ref_t,0 )) {verbose (env,"%s pointer type %s %s must point to scalar, or struct with scalar\n" ,reg_arg_name (env, argno),btf_type_str (ref_t), ref_tname);return -EINVAL ;
@@ -12498,7 +12498,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_mebreak ; } -if (!__btf_type_is_scalar_struct (env, meta->btf, ref_t,0 )) { +if (!btf_type_is_scalar_struct (env, meta->btf, ref_t,0 )) {enum bpf_reg_type reg2btf_type =lookup_reg2btf_ids (ref_id); verbose(env, "%s is %s expected %s %s",
@@ -12962,7 +12962,7 @@ static int check_special_kfunc(struct bpf_verifier_env *env, struct bpf_call_arg struct_meta =btf_find_struct_meta (ret_btf, ret_btf_id);if (is_bpf_percpu_obj_new_kfunc (meta->func_id)) { -if (!__btf_type_is_scalar_struct (env, ret_btf, ret_t,0 )) { +if (!btf_type_is_scalar_struct (env, ret_btf, ret_t,0 )) {verbose (env,"bpf_percpu_obj_new type ID argument must be of a struct of scalars\n" );return -EINVAL ; }
@@ -13306,7 +13306,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,* otherwise a pointer field would be laundered into a scalar * and escape provenance and reference tracking. */ -if (!__btf_type_is_scalar_struct (env, desc_btf, t,0 )) { +if (!btf_type_is_scalar_struct (env, desc_btf, t,0 )) { verbose(env, "kernel function %s returns %s %s that is not composed of scalars\n", func_name, btf_type_str(t),
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a) SEC("?tc") __exception_cb(exception_cb_bad_ret_type1) -__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.") +__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")int reject_exception_cb_type_1 (struct __sk_buff *ctx) {bpf_throw (0 );
Signature · 1 line
2.53.0-Meta
Yonghong Song
reviewerPATCH v5 8/11 bpf: Reject register-pair returns when the subprog BTF is unreliable
In reply to Yonghong Songbtf_check_subprog_call() can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. Verification carries on, but the prototype the return convention was read from is one the verifier has already declared not to describe the compiled code.
Rather than keep tracking R2 on the strength of a discarded signature, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add subprog_ret_pair_unreliable() and test it in check_func_call() on the path to a static subprogram, which is where the flag can be observed while the call still proceeds. For a static callee this only triggers on a genuine argument mismatch, since the previous patch stopped btf_validate_return_type() from marking a local function unreliable for returning an aggregate.
No check is needed anywhere else:
- a global subprogram is already rejected by the existing "Caller passes invalid args into func#N" path, because btf_check_subprog_call() returns an error both when it marks the BTF unreliable and on every later call;
- the main program does not use the convention at all: its return value is the program's exit code, read out of R0, so nothing looks at R2 there;
--- kernel/bpf/verifier.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
kernel/bpf/verifier.c +23 -0
@@ -441,6 +441,23 @@ static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) } } +/* +* A >8 byte BPF return changes the calling convention to R0:R2, and the +* verifier derives that convention from the subprogram's BTF prototype +* alone. Once that prototype is marked unreliable it is known not to +* describe the compiled code, so the convention read from it cannot be +* trusted either: reject the call rather than keep tracking R2 on the +* strength of a signature the verifier has already discarded. +*/ +static bool subprog_ret_pair_unreliable (struct bpf_verifier_env *env,int subprog) +{ +struct bpf_prog_aux *aux = env->prog->aux; + +return bpf_ret_reg_pair (env, subprog) && + aux->func_info_aux && + aux->func_info_aux[subprog].unreliable; +} +static const char *subprog_name (const struct bpf_verifier_env *env,int subprog) {struct bpf_func_info *info;
@@ -9527,6 +9544,12 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,return 0 ; } +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + } +/* * Track caller's total stack arg count (incoming + max outgoing). * This is needed so the JIT knows how much stack arg space to allocate.
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 08/11] bpf: Reject register-pair returns when the subprog BTF is unreliable
Does this enumeration cover all call sites? btf_check_subprog_call() has two callers: check_func_call() and push_callback_call() in verifier.c.
Quoted 16 lines
btf_check_subprog_call()can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. Verification carries on, but the prototype the return convention was read from is one the verifier has already declared not to describe the compiled code.Rather than keep tracking R2 on the strength of a discarded signature, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add
subprog_ret_pair_unreliable()and test it incheck_func_call()on the path to a static subprogram, which is where the flag can be observed while the call still proceeds. For a static callee this only triggers on a genuine argument mismatch, since the previous patch stoppedbtf_validate_return_type()from marking a local function unreliable for returning an aggregate.
Quoted 8 lines
No check is needed anywhere else:
- a global subprogram is already rejected by the existing "Caller passes invalid args into func#N" path, becausebtf_check_subprog_call()returns an error both when it marks the BTF unreliable and on every later call;
- the main program does not use the convention at all: its return value is the program's exit code, read out of R0, so nothing looks at R2 there;
Does this enumeration cover all call sites? btf_check_subprog_call() has two callers: check_func_call() and push_callback_call() in verifier.c.
push_callback_call() also swallows every non-EFAULT error:
err = btf_check_subprog_call(env, subprog, caller->regs);
if (err == -EFAULT)
return err;so a callback subprogram's BTF can be marked unreliable while the call still proceeds, exactly the situation described above for the static check_func_call() path. The two bullets given (global subprogram, main program) do not cover it.
It interacts with the sticky nature of the flag: a subprogram marked unreliable by a callback call site will subsequently be rejected by the new check at an unrelated direct call site whose own arguments were fine, since btf_check_subprog_call() returns -EINVAL for every later call once the flag is set (the early return in btf_check_subprog_call()).
Quoted 25 lines
kernel/bpf/verifier.c +17
-0@@ -441,6 +441,23 @@ static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) } } +/* +* A >8 byte BPF return changes the calling convention to R0:R2, and the +* verifier derives that convention from the subprogram's BTF prototype +* alone. Once that prototype is marked unreliable it is known not to +* describe the compiled code, so the convention read from it cannot be +* trusted either: reject the call rather than keep tracking R2 on the +* strength of a signature the verifier has already discarded. +*/ +static bool subprog_ret_pair_unreliable (struct bpf_verifier_env *env,int subprog) +{ +struct bpf_prog_aux *aux = env->prog->aux; + +return bpf_ret_reg_pair (env, subprog) && + aux->func_info_aux && + aux->func_info_aux[subprog].unreliable; +} +
[ ... ]
kernel/bpf/verifier.c static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 9552
Quoted 10 lines
kernel/bpf/verifier.c +6
-0@@ -9527,6 +9544,12 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,return 0 ; } +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + } +
Can this gate reject valid programs? For a static callee, func_info_aux[] .unreliable is set for reasons that have nothing to do with an argument mismatch.
The path is: do_check_insn() calls check_func_call(), which calls btf_check_subprog_call(), which calls btf_check_func_arg_match().
btf_check_func_arg_match() in kernel/bpf/verifier.c starts with:
ret = btf_prepare_func_args(env, subprog);
if (ret) {
if (bpf_in_stack_arg_cnt(sub) > 0) ...
return ret;
}Then btf_check_subprog_call() does unconditionally on any non-zero err:
if (err)
prog->aux->func_info_aux[subprog].unreliable = true;btf_prepare_func_args() returns -EINVAL for a non-global subprogram in the ordinary case of an untagged pointer argument. In the argument loop (kernel/bpf/btf.c), a pointer arg that is not the prog ctx type, not a dynptr, and carries no __arg_trusted/__arg_untrusted/__arg_arena tag skips the generic-user-data-pointer branch and falls through to:
skip_pointer:
if (tags) { ... return -EINVAL; }
if (btf_type_is_int(t) || btf_is_any_enum(t)) { ... }
if (!is_global)
return -EINVAL;A by-value struct argument takes the same exit. This is expected, documented behaviour for static callees since btf_prepare_func_args() even comments "not fatal for static funcs" at kernel/bpf/btf.c, because a static subprogram is verified inline so the verifier never needs a usable BTF argument model for it.
Concrete example that verifies fine at the parent commit but is now rejected:
struct pair { __u64 a, b; };
static __noinline struct pair make_pair(__u64 *p)
{
struct pair r = { *p, 1 };
return r;
}
SEC("tc") int prog(struct __sk_buff *ctx)
{
__u64 v = 1;
struct pair p = make_pair(&v);
return p.a + p.b;
}This program is rejected: "Func#1 ('make_pair') returns >8 bytes, which requires reliable BTF".
The flow is: bpf_compute_subprog_ret_regs() sees struct pair, size 16, and sets ret_reg_pair = true. btf_validate_return_type() accepts it via the local_func path. The argument loop then returns -EINVAL for __u64 *p, setting unreliable = true. The new check fires.
The rejection is also unnecessary for soundness on this path: a static callee is verified inline, so prepare_func_exit() copies the callee's actual tracked R0 and R2 into the caller. R2 tracking does not rest on the BTF prototype the way it does for a global callee.
The commit message premise appears incorrect: "For a static callee this only triggers on a genuine argument mismatch, since the previous patch stopped btf_validate_return_type() from marking a local function unreliable for returning an aggregate." Relaxing btf_validate_return_type() removed only one of two independent -EINVAL exits inside btf_prepare_func_args(); the argument-conversion exit at kernel/bpf/btf.c is untouched and is the common one. The new comment in verifier.c encodes the same premise: "Once that prototype is marked unreliable it is known not to describe the compiled code" - but the flag also means merely "the verifier cannot express these argument types".
Should the gate be restricted to cases where the return convention itself is untrustworthy, rather than whenever any argument cannot be expressed in BTF?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Yonghong Song
reviewerRe: [PATCH bpf-next v5 08/11] bpf: Reject register-pair returns when the subprog BTF is unreliable
Quoted 161 lines · On 8/13/26 1:49 PM, [email protected] wrote:
Quoted 16 lines
btf_check_subprog_call()can decide, at a call site, that this BTF is not to be trusted and mark the subprogram unreliable, which happens when compiler optimizations remove arguments from a static function or when a mismatched type is passed to a global one. Verification carries on, but the prototype the return convention was read from is one the verifier has already declared not to describe the compiled code.Rather than keep tracking R2 on the strength of a discarded signature, reject a return value larger than 8 bytes as soon as the prototype it was derived from becomes unreliable. Add
subprog_ret_pair_unreliable()and test it incheck_func_call()on the path to a static subprogram, which is where the flag can be observed while the call still proceeds. For a static callee this only triggers on a genuine argument mismatch, since the previous patch stoppedbtf_validate_return_type()from marking a local function unreliable for returning an aggregate.Quoted 8 lines
No check is needed anywhere else:
- a global subprogram is already rejected by the existing "Caller passes invalid args into func#N" path, becausebtf_check_subprog_call()returns an error both when it marks the BTF unreliable and on every later call;
- the main program does not use the convention at all: its return value is the program's exit code, read out of R0, so nothing looks at R2 there;Does this enumeration cover all call sites?
btf_check_subprog_call()has two callers:check_func_call()andpush_callback_call()inverifier.c.
push_callback_call()also swallows every non-EFAULT error:err = btf_check_subprog_call(env, subprog, caller->regs); if (err == -EFAULT) return err;so a callback subprogram's BTF can be marked unreliable while the call still proceeds, exactly the situation described above for the static
check_func_call()path. The two bullets given (global subprogram, main program) do not cover it.It interacts with the sticky nature of the flag: a subprogram marked unreliable by a callback call site will subsequently be rejected by the new check at an unrelated direct call site whose own arguments were fine, since
btf_check_subprog_call()returns-EINVALfor every later call once the flag is set (the early return inbtf_check_subprog_call()).Quoted 25 lines
kernel/bpf/verifier.c +17
-0@@ -441,6 +441,23 @@ static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env) } } +/* +* A >8 byte BPF return changes the calling convention to R0:R2, and the +* verifier derives that convention from the subprogram's BTF prototype +* alone. Once that prototype is marked unreliable it is known not to +* describe the compiled code, so the convention read from it cannot be +* trusted either: reject the call rather than keep tracking R2 on the +* strength of a signature the verifier has already discarded. +*/ +static bool subprog_ret_pair_unreliable (struct bpf_verifier_env *env,int subprog) +{ +struct bpf_prog_aux *aux = env->prog->aux; + +return bpf_ret_reg_pair (env, subprog) && + aux->func_info_aux && + aux->func_info_aux[subprog].unreliable; +} +[ ... ]
kernel/bpf/verifier.c static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 9552
Quoted 10 lines
kernel/bpf/verifier.c +6
-0@@ -9527,6 +9544,12 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,return 0 ; } +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + } +Can this gate reject valid programs? For a static callee, func_info_aux[] .unreliable is set for reasons that have nothing to do with an argument mismatch.
The path is:
do_check_insn()callscheck_func_call(), which callsbtf_check_subprog_call(), which callsbtf_check_func_arg_match().
btf_check_func_arg_match()inkernel/bpf/verifier.cstarts with:ret = btf_prepare_func_args(env, subprog); if (ret) { if (bpf_in_stack_arg_cnt(sub) > 0) ... return ret; }Then
btf_check_subprog_call()does unconditionally on any non-zero err:if (err) prog->aux->func_info_aux[subprog].unreliable = true;
btf_prepare_func_args()returns-EINVALfor a non-global subprogram in the ordinary case of an untagged pointer argument. In the argument loop (kernel/bpf/btf.c), a pointer arg that is not the prog ctx type, not a dynptr, and carries no __arg_trusted/__arg_untrusted/__arg_arena tag skips the generic-user-data-pointer branch and falls through to:skip_pointer: if (tags) { ... return -EINVAL; } if (btf_type_is_int(t) || btf_is_any_enum(t)) { ... } if (!is_global) return -EINVAL;A by-value struct argument takes the same exit. This is expected, documented behaviour for static callees since
btf_prepare_func_args()even comments "not fatal for static funcs" atkernel/bpf/btf.c, because a static subprogram is verified inline so the verifier never needs a usable BTF argument model for it.Concrete example that verifies fine at the parent commit but is now rejected:
struct pair { __u64 a, b; }; static __noinline struct pair make_pair(__u64 *p) { struct pair r = { *p, 1 }; return r; } SEC("tc") int prog(struct __sk_buff *ctx) { __u64 v = 1; struct pair p = make_pair(&v); return p.a + p.b; }This program is rejected: "Func#1 ('
make_pair') returns >8 bytes, which requires reliable BTF".The flow is:
bpf_compute_subprog_ret_regs()sees struct pair, size 16, and setsret_reg_pair= true.btf_validate_return_type()accepts it via thelocal_funcpath. The argument loop then returns-EINVALfor__u64*p, setting unreliable = true. The new check fires.The rejection is also unnecessary for soundness on this path: a static callee is verified inline, so
prepare_func_exit()copies the callee's actual tracked R0 and R2 into the caller. R2 tracking does not rest on the BTF prototype the way it does for a global callee.The commit message premise appears incorrect: "For a static callee this only triggers on a genuine argument mismatch, since the previous patch stopped
btf_validate_return_type()from marking a local function unreliable for returning an aggregate." Relaxingbtf_validate_return_type()removed only one of two independent-EINVALexits insidebtf_prepare_func_args(); the argument-conversion exit atkernel/bpf/btf.cis untouched and is the common one. The new comment inverifier.cencodes the same premise: "Once that prototype is marked unreliable it is known not to describe the compiled code" - but the flag also means merely "the verifier cannot express these argument types".
Quoted 3 lines
Should the gate be restricted to cases where the return convention itself is untrustworthy, rather than whenever any argument cannot be expressed in BTF?
Good point. Let us remove this patch as it is too conservative.
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
[email protected]
AI reviewRe: [PATCH bpf-next v5 08/11] bpf: Reject register-pair returns when the subprog BTF is unreliable
In reply to Yonghong SongThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] push_callback_call() incorrectly passes helper arguments for BTF validation, tainting callback subprograms as unreliable. --
--- Patch [8]: [PATCH bpf-next v5 08/11] bpf: Reject register-pair returns when the subprog BTF is unreliable --- commit c1362d6d7ccc6806c96ffe9fc78804a2b8b6f1b9
Author: Yonghong Song <[email protected]>
bpf: Reject register-pair returns when the subprog BTF is unreliable
This patch rejects return values larger than 8 bytes as soon as the subprogram's BTF prototype is marked unreliable. It adds subprog_ret_pair_unreliable() and uses it in check_func_call() to prevent tracking R2 based on a discarded signature.
Quoted 4 lines
kernel/bpf/verifier.c +0
-0
[ ... ]
kernel/bpf/verifier.c static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 9551
Quoted 9 lines
kernel/bpf/verifier.c +5
-0@@ -9527,6 +9544,12 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,return 0 ; } +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + }
[Severity: High]
This is a pre-existing issue, but will this new check incorrectly reject valid BPF programs because of how push_callback_call() marks subprograms as unreliable?
Looking at push_callback_call():
caller = state->frame[state->curframe];
err = btf_check_subprog_call(env, subprog, caller->regs);It passes the caller's registers (which contain the helper arguments, e.g., the PTR_TO_FUNC for bpf_loop) instead of the actual arguments the callback will receive. Because these arguments don't match the callback's prototype, btf_check_subprog_call() fails, sets the unreliable flag to true on the subprogram, and returns -EINVAL.
push_callback_call() ignores this error, leaving the callback permanently tainted as unreliable.
If a callback subprogram returns >8 bytes, won't it be incorrectly rejected here if it is also called directly elsewhere in the program, since the unreliability flag was set mistakenly?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Yonghong Song
reviewerRe: [PATCH bpf-next v5 08/11] bpf: Reject register-pair returns when the subprog BTF is unreliable
Quoted 48 lines · On 8/13/26 8:02 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High]push_callback_call()incorrectly passes helper arguments for BTF validation, tainting callback subprograms as unreliable. ----- Patch [8]: [PATCH bpf-next v5 08/11] bpf: Reject register-pair returns when the subprog BTF is unreliable --- commit
c1362d6d7ccc6806c96ffe9fc78804a2b8b6f1b9
Author: Yonghong Song <[email protected]>bpf: Reject register-pair returns when the subprog BTF is unreliable
This patch rejects return values larger than 8 bytes as soon as the subprogram's BTF prototype is marked unreliable. It adds
subprog_ret_pair_unreliable()and uses it incheck_func_call()to prevent tracking R2 based on a discarded signature.Quoted 4 lines
kernel/bpf/verifier.c +0
-0[ ... ]
kernel/bpf/verifier.c static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 9551
Quoted 9 lines
kernel/bpf/verifier.c +5
-0@@ -9527,6 +9544,12 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,return 0 ; } +if (subprog_ret_pair_unreliable (env, subprog)) { +verbose (env,"Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n" , + subprog,subprog_name (env, subprog)); +return -EINVAL ; + }[Severity: High]
This is a pre-existing issue, but will this new check incorrectly reject valid BPF programs because of howpush_callback_call()marks subprograms as unreliable?Looking at
push_callback_call():caller = state->frame[state->curframe]; err = btf_check_subprog_call(env, subprog, caller->regs);It passes the caller's registers (which contain the helper arguments, e.g., the
PTR_TO_FUNCforbpf_loop) instead of the actual arguments the callback will receive. Because these arguments don't match the callback's prototype,btf_check_subprog_call()fails, sets the unreliable flag to true on the subprogram, and returns-EINVAL.
Quoted 6 lines
push_callback_call()ignores this error, leaving the callback permanently tainted as unreliable.If a callback subprogram returns >8 bytes, won't it be incorrectly rejected here if it is also called directly elsewhere in the program, since the unreliability flag was set mistakenly?
Yes, let us remove this patch as the above unreliable rejection may actually reject legal prog's.
Yonghong Song
reviewerPATCH v5 9/11 selftests/bpf: Add C tests for 16-byte returns in R0:R2
In reply to Yonghong SongAdd selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return. Covered are an __int128 return, a 16-byte struct return (from a static and from a global subprogram) and a 16-byte union return, plus __int128 and 16-byte struct returns from a kfunc. The union program shares an object with the struct ones.
The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a __clang_major__ guard. An older compiler builds the dummy test in the #else branch instead, which keeps each object non-empty and says in its description why nothing was exercised.
The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), which bpf_add_kfunc_call() requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.
--- .../selftests/bpf/prog_tests/aggregate_ret.c | 13 ++ .../bpf/progs/aggregate_ret_int128_c.c | 49 ++++++++ .../bpf/progs/aggregate_ret_kfunc_c.c | 65 ++++++++++ .../bpf/progs/aggregate_ret_struct_c.c | 114 ++++++++++++++++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 18 +++ .../bpf/test_kmods/bpf_testmod_kfunc.h | 9 ++ 6 files changed, 268 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +13 -0
@@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <test_progs.h> +#include "aggregate_ret_int128_c.skel.h" +#include "aggregate_ret_struct_c.skel.h" +#include "aggregate_ret_kfunc_c.skel.h" + +void test_aggregate_ret (void ) +{ +RUN_TESTS (aggregate_ret_int128_c); +RUN_TESTS (aggregate_ret_struct_c); +RUN_TESTS (aggregate_ret_kfunc_c); +}
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c new file +49 -0
49 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c new file +65 -0
65 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +114 -0
114 lines not on this page · raw message
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +18 -0
@@ -939,6 +939,20 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)return 0 ; } +#if defined(__x86_64__) || defined(__aarch64__) +__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b) +{ +return (__int128)(((unsigned __int128 )(a + b) <<64 ) | (a - b)); +} + +__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) +{ +struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b }; + +return r; +} +#endif /* __x86_64__ || __aarch64__ */ + __bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,u64 e,u64 f,u64 g,u64 h,u64 i,u64 j)
@@ -1472,6 +1486,10 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2) BTF_ID_FLAGS(func, bpf_kfunc_call_test3) BTF_ID_FLAGS(func, bpf_kfunc_call_test4) BTF_ID_FLAGS(func, bpf_kfunc_call_test5) +#if defined(__x86_64__) || defined(__aarch64__) +BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128) +BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair) +#endif BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg) BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)BTF_ID_FLAGS (func, bpf_kfunc_call_stack_arg_mix)
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +9 -0
@@ -55,6 +55,11 @@ struct prog_test_big_arg {__u64 b; }; +struct prog_test_ret_pair { +__u64 hi; +__u64 lo; +}; +struct prog_test_fail1 {void *p;int x;
@@ -130,6 +135,10 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;struct sock *bpf_kfunc_call_test3 (struct sock *sk) __ksym; long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym; int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym; +#ifdef __SIZEOF_INT128__ +__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym; +#endif +struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym; __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 g, __u64 h, __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 09/11] selftests/bpf: Add C tests for 16-byte returns in R0:R2
This isn't a bug, but struct pair puts hi in R0 while union upair's parts puts lo in R0 - was the inverted naming in the union deliberate, or would matching the struct's order (or first/second) make…
Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +0
-0
[ ... ]
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c line 23
Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +12
-0+struct pair { +__u64 hi;/* R0 */ +__u64 lo;/* R2 */ +}; + +union upair { +__u64 halves[2 ]; +struct { +__u64 lo;/* R0 */ +__u64 hi;/* R2 */ + } parts; +};
This isn't a bug, but struct pair puts hi in R0 while union upair's parts puts lo in R0 - was the inverted naming in the union deliberate, or would matching the struct's order (or first/second) make the R0:R2 mapping easier to follow?
Quoted 25 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +25
-0+ +static __noinline struct pair make_pair(__u64 a, __u64 b) +{ +struct pair p = { .hi = a + b, .lo = a - b }; + +return p; +} + +SEC("tc") +__load_if_JITed() +__success __retval(0) +int aggregate_ret_struct_c_test(struct __sk_buff *skb) +{ +__u64 a = skb->len ^MIX_A ; +__u64 b = skb->len ^MIX_B ; +struct pair p; + + p =make_pair (a, b); +if (p.hi != a + b) +return 1 ; +if (p.lo != a - b) +return 2 ; + +return 0 ; +}
[ ... ]
Quoted 16 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +16
-0+static __noinline union upair make_upair(__u64 a, __u64 b) +{ +union upair p; + + p.halves[0 ] = a + b; + p.halves[1 ] = a - b; +return p; +} + +SEC("tc") +__load_if_JITed() +__success __retval(0) +int aggregate_ret_union_c_test(struct __sk_buff *skb) +{ +__u64 a = skb->len ^MIX_A ; +__u64 b = skb->len ^MIX_B ;
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c line 100
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +10
-0+union upair p; + + p =make_upair (a, b); +if (p.parts.lo != a + b) +return 1 ; +if (p.parts.hi != a - b) +return 2 ; + +return 0 ; +}
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Yonghong Song
reviewerRe: [PATCH bpf-next v5 09/11] selftests/bpf: Add C tests for 16-byte returns in R0:R2
Quoted 7 lines · On 8/13/26 2:11 PM, [email protected] wrote:
Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +0
-0[ ... ]
Quoted 16 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c line 23
Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +12
-0+struct pair { +__u64 hi;/* R0 */ +__u64 lo;/* R2 */ +}; + +union upair { +__u64 halves[2 ]; +struct { +__u64 lo;/* R0 */ +__u64 hi;/* R2 */ + } parts; +};This isn't a bug, but struct pair puts hi in R0 while union upair's parts puts lo in R0 - was the inverted naming in the union deliberate, or would matching the struct's order (or first/second) make the R0:R2 mapping easier to follow?
Thanks. This is an oversight. Will fix 'struct pair' with proper ordering.
Quoted 60 lines
Quoted 26 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +25
-0+ +static __noinline struct pair make_pair(__u64 a, __u64 b) +{ +struct pair p = { .hi = a + b, .lo = a - b }; + +return p; +} + +SEC("tc") +__load_if_JITed() +__success __retval(0) +int aggregate_ret_struct_c_test(struct __sk_buff *skb) +{ +__u64 a = skb->len ^MIX_A ; +__u64 b = skb->len ^MIX_B ; +struct pair p; + + p =make_pair (a, b); +if (p.hi != a + b) +return 1 ; +if (p.lo != a - b) +return 2 ; + +return 0 ; +}[ ... ]
Quoted 16 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +16
-0+static __noinline union upair make_upair(__u64 a, __u64 b) +{ +union upair p; + + p.halves[0 ] = a + b; + p.halves[1 ] = a - b; +return p; +} + +SEC("tc") +__load_if_JITed() +__success __retval(0) +int aggregate_ret_union_c_test(struct __sk_buff *skb) +{ +__u64 a = skb->len ^MIX_A ; +__u64 b = skb->len ^MIX_B ;tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c line 100
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c new file +10
-0+union upair p; + + p =make_upair (a, b); +if (p.parts.lo != a + b) +return 1 ; +if (p.parts.hi != a - b) +return 2 ; + +return 0 ; +}---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Re: [PATCH bpf-next v5 09/11] selftests/bpf: Add C tests for 16-byte returns in R0:R2
In reply to Yonghong SongQuoted 13 lines · Yonghong Song
Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return. Covered are an
__int128return, a 16-byte struct return (from a static and from a global subprogram) and a 16-byte union return, plus__int128and 16-byte struct returns from a kfunc. The union program shares an object with the struct ones.The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a
__clang_major__guard. An older compiler builds the dummy test in the #else branch instead, which keeps each object non-empty and says in its description why nothing was exercised.
Quoted 8 lines
The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises
bpf_jit_supports_kfunc_ret_reg_pair(), whichbpf_add_kfunc_call()requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.Signed-off-by Yonghong Song <[email protected]>---
Quoted 4 lines
.../selftests/bpf/prog_tests/aggregate_ret.c | 13 ++ .../bpf/progs/aggregate_ret_int128_c.c | 49 ++++++++ .../bpf/progs/aggregate_ret_kfunc_c.c | 65 ++++++++++ .../bpf/progs/aggregate_ret_struct_c.c | 114 ++++++++++++++++++
Nit: I'd merge all three files into one and move loader to prog_tests/verifier.c.
Also not sure about union vs struct tests.
Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c new file +0
-0
...
Quoted 1 line
+#if
defined(__clang_major__)&&__clang_major__>= 23
...
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c line 45
Quoted 9 lines
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c new file +9
-0+#else + +SEC("socket") +__description("aggregate_ret_int128_c: needs LLVM 23, dummy test") +__success +int dummy_test (void ) +{ +return 0 ; +}
Q: what would happen w/o #else branch? Is loading an empty object an error?
Quoted 1 line
+#endif
...
Yonghong Song
reviewerRe: [PATCH bpf-next v5 09/11] selftests/bpf: Add C tests for 16-byte returns in R0:R2
Quoted 22 lines · Eduard Zingerman
Quoted 13 lines · Yonghong Song
Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return. Covered are an
__int128return, a 16-byte struct return (from a static and from a global subprogram) and a 16-byte union return, plus__int128and 16-byte struct returns from a kfunc. The union program shares an object with the struct ones.The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a
__clang_major__guard. An older compiler builds the dummy test in the #else branch instead, which keeps each object non-empty and says in its description why nothing was exercised.Quoted 8 lines
The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises
bpf_jit_supports_kfunc_ret_reg_pair(), whichbpf_add_kfunc_call()requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.Signed-off-by Yonghong Song <[email protected]>---
Quoted 8 lines
Acked-by Eduard Zingerman <[email protected]>Quoted 4 lines
.../selftests/bpf/prog_tests/aggregate_ret.c | 13 ++ .../bpf/progs/aggregate_ret_int128_c.c | 49 ++++++++ .../bpf/progs/aggregate_ret_kfunc_c.c | 65 ++++++++++ .../bpf/progs/aggregate_ret_struct_c.c | 114 ++++++++++++++++++Nit: I'd merge all three files into one and move loader to
prog_tests/verifier.c.
Also not sure about union vs struct tests.
Okay, will do.
Quoted 11 lines
Quoted 6 lines
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c new file +0
-0...
Quoted 1 line
+#if
defined(__clang_major__)&&__clang_major__>= 23...
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c line 45
Quoted 9 lines
tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c new file +9
-0+#else + +SEC("socket") +__description("aggregate_ret_int128_c: needs LLVM 23, dummy test") +__success +int dummy_test (void ) +{ +return 0 ; +}Q: what would happen w/o #else branch? Is loading an empty object an error?
With #else branch, we will have:
#1/1 aggregate_ret/aggregate_ret_int128_c: needs LLVM 23, dummy test:OK #1/2 aggregate_ret/aggregate_ret_struct_c: needs LLVM 23, dummy test:OK #1/3 aggregate_ret/aggregate_ret_kfunc_c: needs LLVM 23, dummy test:OK
Without #else branch, all the above three subtests will be gone. I probably will keep it so user knows the current limitation. Once CI compiler becomes llvm23, the above three will go through real tests.
Quoted 3 lines
Quoted 2 lines
+#endif
...
Yonghong Song
reviewerPATCH v5 10/11 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
In reply to Yonghong SongAdd inline-asm tests covering what the C tests cannot reach. aggregate_ret_func.c exercises BPF-to-BPF returns: subprograms that leave R2 uninitialised or holding a stack pointer, ones returning a struct or a union that smuggles a pointer, static and global subprograms whose R2 stays precise under backtracking, and liveness at the exit of a subprogram returning a pair.
In addition, a negative freplace test checks that an extension cannot replace a function returning R0:R2: the extension's own return is capped at 8 bytes, so it would leave R2 stale for the target's callers, and the load is expected to fail.
--- .../selftests/bpf/prog_tests/aggregate_ret.c | 4 + .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 15 + .../selftests/bpf/progs/aggregate_ret_func.c | 260 ++++++++++++++++++ .../selftests/bpf/progs/aggregate_ret_kfunc.c | 122 ++++++++ .../bpf/progs/aggregate_ret_kfunc_c.c | 60 ++++ .../bpf/progs/aggregate_ret_target.c | 29 ++ .../bpf/progs/compute_live_registers.c | 30 ++ .../selftests/bpf/progs/freplace_ret_pair.c | 12 + .../selftests/bpf/test_kmods/bpf_testmod.c | 55 ++++ .../bpf/test_kmods/bpf_testmod_kfunc.h | 35 +++ 10 files changed, 622 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +4 -0
@@ -4,10 +4,14 @@#include "aggregate_ret_int128_c.skel.h" #include "aggregate_ret_struct_c.skel.h" #include "aggregate_ret_kfunc_c.skel.h" +#include "aggregate_ret_func.skel.h" +#include "aggregate_ret_kfunc.skel.h" void test_aggregate_ret (void ) {RUN_TESTS (aggregate_ret_int128_c);RUN_TESTS (aggregate_ret_struct_c);RUN_TESTS (aggregate_ret_kfunc_c); +RUN_TESTS (aggregate_ret_func); +RUN_TESTS (aggregate_ret_kfunc); }
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +15 -0
@@ -441,6 +441,19 @@ static void test_func_replace_int_with_void(void) " doesn't match type INT of global_func2()"); } +static void test_func_replace_ret_pair (void ) +{ +const char *msg ="Cannot replace function agg_ret_target_func with a >8 byte return" ; + +/* +* An extension cannot replace a function whose return value comes back +* in the R0:R2 pair: the extension's own return is capped at 8 bytes, +* so it would leave R2 stale for the target's callers. +*/ +test_obj_load_failure_common ("freplace_ret_pair.bpf.o" , +"./aggregate_ret_target.bpf.o" , msg); +} +static int find_prog_btf_id (const char *name,__u32 attach_prog_fd) {struct bpf_prog_info info = {};
@@ -660,6 +673,8 @@ void serial_test_fexit_bpf2bpf(void)test_func_replace_progmap ();if (test__start_subtest ("freplace_int_with_void" ))test_func_replace_int_with_void (); +if (test__start_subtest ("freplace_ret_pair" )) +test_func_replace_ret_pair ();if (test__start_subtest ("freplace_void" ))test_func_replace_void ();if (test__start_subtest ("sleepable_fentry_to_xdp" ))
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +260 -0
260 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +122 -0
122 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c +60 -0
66 lines not on this page · raw message
tools/testing/selftests/bpf/progs/aggregate_ret_target.c new file +29 -0
@@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" + +/* freplace target: a global subprogram returning 16 bytes in R0:R2. */ +__naked unsigned __int128 agg_ret_target_func (void ) +{ + asmvolatile ( +"r0 = 0x1234;" +"r2 = 0x5678;" +"exit;" + ); +} + +SEC("tc") +__naked int agg_ret_target (void ) +{ + asmvolatile ( +"call %[agg_ret_target_func];" +"r0 = 0;" +"exit;" + : + : __imm(agg_ret_target_func) + : __clobber_all); +} + +char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/compute_live_registers.c +30 -0
@@ -431,6 +431,36 @@ __naked void subprog1(void) ::: __clobber_all); } +static __used __naked unsigned __int128 aux2 (void ) +{ + asmvolatile ( +"r0 = 1;" +"r2 = 2;" +"exit;" + ::: __clobber_all); +} + +SEC ("socket" ) +/* A program observing the pair needs the JIT; see bpf_compute_subprog_ret_regs(). */ +__load_if_JITed() +__log_level(2) +__msg("0: .12345.... (85) call pc+2") +__msg ("1: ..2....... (bf) r0 = r2" ) +/* R2 is not read at the exit of this program, which returns an int, ... */ +__msg("2: 0......... (95) exit") +__msg("3: .......... (b7) r0 = 1") +__msg ("4: 0......... (b7) r2 = 2" ) +/* ... but it is at the exit of aux2(), which returns a register pair. */ +__msg("5: 0.2....... (95) exit") +__naked void subprog_ret_reg_pair (void ) +{ + asmvolatile ( +"call aux2;" +"r0 = r2;" +"exit;" + ::: __clobber_all); +} +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)SEC ("socket" )
tools/testing/selftests/bpf/progs/freplace_ret_pair.c new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> + +SEC("freplace/agg_ret_target_func") +__u64 new_agg_ret_target_func (void ) +{ +return 0 ; +} + +char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +55 -0
70 lines not on this page · raw message
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +35 -0
@@ -60,6 +60,35 @@ struct prog_test_ret_pair {__u64 lo; }; +struct prog_test_ret_li {/* 16 bytes: R0:R2 */ +__u64 a; +int b; +}; + +struct prog_test_ret_ii {/* 8 bytes: R0 only */ +int a; +int b; +}; + +union prog_test_ret_uu {/* 16 bytes: R0:R2 */ +__u64 halves[2 ]; +struct { +__u64 lo; +__u64 hi; + } parts; +}; + +struct prog_test_ret_ptr {/* 16 bytes: contains a pointer */ +void *p; +__u64 tag; +}; + +struct prog_test_ret_big {/* 24 bytes: too large for R0:R2 */ +__u64 a; +__u64 b; +__u64 c; +}; +struct prog_test_fail1 {void *p;int x;
@@ -139,6 +168,12 @@ int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym; __int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;#endif struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym; +struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym; +struct prog_test_ret_li bpf_kfunc_call_test_ret_li(__u64 a, int b) __ksym; +struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym; +union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(__u64 a, __u64 b) __ksym; +struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym; +struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void) __ksym; __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e, __u64 f, __u64 g, __u64 h, __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewRe: [PATCH bpf-next v5 10/11] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
A subsystem pattern flags these two tests as potentially concerning. The aggregate_ret_static_precise() and aggregate_ret_global_precise() tests in the new aggregate_ret_func.c file are the R2…
Quoted 19 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +4
-0@@ -4,10 +4,14 @@#include "aggregate_ret_int128_c.skel.h" #include "aggregate_ret_struct_c.skel.h" #include "aggregate_ret_kfunc_c.skel.h" +#include "aggregate_ret_func.skel.h" +#include "aggregate_ret_kfunc.skel.h" void test_aggregate_ret (void ) {RUN_TESTS (aggregate_ret_int128_c);RUN_TESTS (aggregate_ret_struct_c);RUN_TESTS (aggregate_ret_kfunc_c); +RUN_TESTS (aggregate_ret_func); +RUN_TESTS (aggregate_ret_kfunc); }
[ ... ]
Quoted 11 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +5
-0@@ -0,0 +1,260 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h"
[ ... ]
Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +12
-0+SEC("tc") +__load_if_JITed() +__success __retval(0) +__log_level(2) +__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1") +__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8") +__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10") +__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7") +__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2") +__msg("mark_precise: frame0: regs=r2 stack= before 12: (95) exit") +__msg("mark_precise: frame1: regs=r2 stack= before 11: (b7) r2 = 4") +__naked int aggregate_ret_static_precise (void )
[ ... ]
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 157
Quoted 11 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +11
-0+SEC("tc") +__load_if_JITed() +__success __retval(0) +__log_level(2) +__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1") +__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8") +__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10") +__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7") +__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2") +__msg("mark_precise: frame0: regs=r2 stack= before 0: (85) call pc+9") +__naked int aggregate_ret_global_precise (void )
A subsystem pattern flags these two tests as potentially concerning. The aggregate_ret_static_precise() and aggregate_ret_global_precise() tests in the new aggregate_ret_func.c file are the R2 variant of behavior that tools/testing/selftests/bpf/progs/verifier_subprog_precision.c already covers: subprog_result_precise() and global_subprog_result_precise() in that file assert the same mark_precise log shape across the call/exit boundary for static and global subprogs. The two new tests differ only in which register (R2 vs R0) is being tracked, and in the final one or two __msg lines.
This same commit did extend an existing file for its liveness test case rather than adding a new one (it added aux2() and subprog_ret_reg_pair() to compute_live_registers.c).
Would it be more natural to add these two precision tests as new cases in verifier_subprog_precision.c, where a reader looks for precision- backtracking coverage? The rest of aggregate_ret_func.c (the __failure cases and the accepted pointer-pair case) is genuinely new-feature coverage with no existing home, so perhaps that part stays separate.
[ ... ]
Quoted 20 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +12
-0@@ -431,6 +431,36 @@ __naked void subprog1(void) ::: __clobber_all); } +static __used __naked unsigned __int128 aux2 (void ) +{ + asmvolatile ( +"r0 = 1;" +"r2 = 2;" +"exit;" + ::: __clobber_all); +} + +SEC ("socket" ) +/* A program observing the pair needs the JIT; see bpf_compute_subprog_ret_regs(). */ +__load_if_JITed ()
tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void subprog1(void) · line 455
Quoted 10 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +10
-0+__log_level(2) +__msg("0: .12345.... (85) call pc+2") +__msg ("1: ..2....... (bf) r0 = r2" ) +/* R2 is not read at the exit of this program, which returns an int, ... */ +__msg("2: 0......... (95) exit") +__msg("3: .......... (b7) r0 = 1") +__msg ("4: 0......... (b7) r2 = 2" ) +/* ... but it is at the exit of aux2(), which returns a register pair. */ +__msg("5: 0.2....... (95) exit") +__naked void subprog_ret_reg_pair (void )
The changelog describes aggregate_ret_func.c covering "liveness at the exit of a subprogram returning a pair," but that test is not in that file. aggregate_ret_func.c has no liveness test at all - its only __log_level(2) tests match "mark_precise: ..." lines.
The liveness-at-exit test is here, in compute_live_registers.c: aux2() and subprog_ret_reg_pair(), with the register-mask __msg expectations like "0: .12345.... (85) call pc+2" and "5: 0.2....... (95) exit".
Conversely, the changelog does not mention the accepted-case test aggregate_ret_static_ptr_pair() in aggregate_ret_func.c (a static subprog legitimately returning a pointer in R2, whose result is then dereferenced), and it never mentions the aggregate_ret_kfunc.c / aggregate_ret_kfunc_c.c additions that make up roughly 180 of the commit's added lines. New kfuncs in the test module are the kind of addition a reviewer looks for in the changelog.
Could the message be adjusted so that aggregate_ret_func.c's description matches what the file actually contains, the liveness test is attributed to compute_live_registers.c, and the kfunc side of the patch gets a sentence?
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Yonghong Song
reviewerRe: [PATCH bpf-next v5 10/11] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
Quoted 73 lines · On 8/13/26 2:11 PM, [email protected] wrote:
Quoted 19 lines
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c +4
-0@@ -4,10 +4,14 @@#include "aggregate_ret_int128_c.skel.h" #include "aggregate_ret_struct_c.skel.h" #include "aggregate_ret_kfunc_c.skel.h" +#include "aggregate_ret_func.skel.h" +#include "aggregate_ret_kfunc.skel.h" void test_aggregate_ret (void ) {RUN_TESTS (aggregate_ret_int128_c);RUN_TESTS (aggregate_ret_struct_c);RUN_TESTS (aggregate_ret_kfunc_c); +RUN_TESTS (aggregate_ret_func); +RUN_TESTS (aggregate_ret_kfunc); }[ ... ]
Quoted 11 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +5
-0@@ -0,0 +1,260 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" [ ... ]
Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +12
-0+SEC("tc") +__load_if_JITed() +__success __retval(0) +__log_level(2) +__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1") +__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8") +__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10") +__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7") +__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2") +__msg("mark_precise: frame0: regs=r2 stack= before 12: (95) exit") +__msg("mark_precise: frame1: regs=r2 stack= before 11: (b7) r2 = 4") +__naked int aggregate_ret_static_precise (void )[ ... ]
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 157
Quoted 11 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +11
-0+SEC("tc") +__load_if_JITed() +__success __retval(0) +__log_level(2) +__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1") +__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8") +__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10") +__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7") +__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2") +__msg("mark_precise: frame0: regs=r2 stack= before 0: (85) call pc+9") +__naked int aggregate_ret_global_precise (void )A subsystem pattern flags these two tests as potentially concerning. The
aggregate_ret_static_precise()andaggregate_ret_global_precise()tests in the newaggregate_ret_func.cfile are the R2 variant of behavior thattools/testing/selftests/bpf/progs/verifier_subprog_precision.calready covers:subprog_result_precise()andglobal_subprog_result_precise()in that file assert the samemark_preciselog shape across the call/exit boundary for static and global subprogs. The two new tests differ only in which register (R2 vs R0) is being tracked, and in the final one or two__msglines.This same commit did extend an existing file for its liveness test case rather than adding a new one (it added
aux2()andsubprog_ret_reg_pair()tocompute_live_registers.c).
Quoted 5 lines
Would it be more natural to add these two precision tests as new cases in
verifier_subprog_precision.c, where a reader looks for precision- backtracking coverage? The rest ofaggregate_ret_func.c(the__failurecases and the accepted pointer-pair case) is genuinely new-feature coverage with no existing home, so perhaps that part stays separate.
I don't have a strong opinion. But current file aggregate_ret_func.c seems okay as well since there are some other returning R0:R1 tests.
Quoted 50 lines
[ ... ]
Quoted 20 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +12
-0@@ -431,6 +431,36 @@ __naked void subprog1(void) ::: __clobber_all); } +static __used __naked unsigned __int128 aux2 (void ) +{ + asmvolatile ( +"r0 = 1;" +"r2 = 2;" +"exit;" + ::: __clobber_all); +} + +SEC ("socket" ) +/* A program observing the pair needs the JIT; see bpf_compute_subprog_ret_regs(). */ +__load_if_JITed ()tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void subprog1(void) · line 455
Quoted 10 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +10
-0+__log_level(2) +__msg("0: .12345.... (85) call pc+2") +__msg ("1: ..2....... (bf) r0 = r2" ) +/* R2 is not read at the exit of this program, which returns an int, ... */ +__msg("2: 0......... (95) exit") +__msg("3: .......... (b7) r0 = 1") +__msg ("4: 0......... (b7) r2 = 2" ) +/* ... but it is at the exit of aux2(), which returns a register pair. */ +__msg("5: 0.2....... (95) exit") +__naked void subprog_ret_reg_pair (void )The changelog describes
aggregate_ret_func.ccovering "liveness at the exit of a subprogram returning a pair," but that test is not in that file.aggregate_ret_func.chas no liveness test at all - its only__log_level(2)tests match "mark_precise: ..." lines.The liveness-at-exit test is here, in
compute_live_registers.c:aux2()andsubprog_ret_reg_pair(), with the register-mask__msgexpectations like "0: .12345.... (85) call pc+2" and "5: 0.2....... (95) exit".Conversely, the changelog does not mention the accepted-case test
aggregate_ret_static_ptr_pair()inaggregate_ret_func.c(a static subprog legitimately returning a pointer in R2, whose result is then dereferenced), and it never mentions theaggregate_ret_kfunc.c/aggregate_ret_kfunc_c.cadditions that make up roughly 180 of the commit's added lines. New kfuncs in the test module are the kind of addition a reviewer looks for in the changelog.
Quoted 3 lines
Could the message be adjusted so that aggregate_ret_func.c's description matches what the file actually contains, the liveness test is attributed to
compute_live_registers.c, and the kfunc side of the patch gets a sentence?
I think this is unnecessary. The code itself will show which files have which tests.
Quoted 9 lines
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31740414277
Eduard Zingerman
maintainerRe: [PATCH bpf-next v5 10/11] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
In reply to Yonghong SongOn Thu, 2026-08-13 at 13:03 -0700, Yonghong Song wrote:
...
Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +0
-0
...
Quoted 8 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +8
-0+struct ptr_pair { +void *p; +__u64 x; +}; + +static __naked __noinline struct ptr_pair static_ret_ptr_pair(void) +{ + asm volatile (
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 248
Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +10
-0+"r0 = 0;" +"r2 = r1;" +"exit;" + ); +} + +SEC("tc") +__load_if_JITed() +__success __retval(0) +__naked int aggregate_ret_static_ptr_pair(void)
This test is redundant, already covered by a previous patch.
Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +12
-0+{ + asmvolatile ( +"call %[static_ret_ptr_pair];" +"r1 = *(u32 *)(r2 + 0);" /* deref the returned ctx pointer */ +"r0 = 0;" +"exit;" + : + : __imm(static_ret_ptr_pair) + : __clobber_all); +} + +char _license[] SEC("license") = "GPL";
...
Quoted 7 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c +1
-0@@ -50,6 +50,66 @@ int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb) +int aggregate_ret_kfunc_li_c_test (struct __sk_buff *skb)
...
Quoted 1 line
+int
aggregate_ret_kfunc_ii_c_test(struct __sk_buff *skb)
...
Quoted 1 line
+int
aggregate_ret_kfunc_uu_c_test(struct __sk_buff *skb)
I don't think three tests above a warranted, the previous patch already tested returning 16-byte structs. Maybe a test with a small structs of two ints, but I'd skip even that.
This would drop a ton of code from the back of the patch.
...
Yonghong Song
reviewerRe: [PATCH bpf-next v5 10/11] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
Quoted 22 lines · Eduard Zingerman
On Thu, 2026-08-13 at 13:03 -0700, Yonghong Song wrote:
...
Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +0
-0...
Quoted 11 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +11
-0+struct ptr_pair { +void *p; +__u64 x; +}; + +static __naked __noinline struct ptr_pair static_ret_ptr_pair(void) +{ + asmvolatile ( +"r0 = 0;" +"r2 = r1;" + "exit;"
Quoted 8 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 248
Quoted 7 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +7
-0+ ); +} + +SEC("tc") +__load_if_JITed() +__success __retval(0) +__naked int aggregate_ret_static_ptr_pair(void)This test is redundant, already covered by a previous patch.
Ack.
Quoted 22 lines
Quoted 13 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +12
-0+{ + asmvolatile ( +"call %[static_ret_ptr_pair];" +"r1 = *(u32 *)(r2 + 0);" /* deref the returned ctx pointer */ +"r0 = 0;" +"exit;" + : + : __imm(static_ret_ptr_pair) + : __clobber_all); +} + +char _license[] SEC("license") = "GPL";...
Quoted 6 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c +1
-0@@ -50,6 +50,66 @@ int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb) +int aggregate_ret_kfunc_li_c_test (struct __sk_buff *skb)...
Quoted 7 lines
Quoted 1 line
+int
aggregate_ret_kfunc_ii_c_test(struct __sk_buff *skb)...
Quoted 1 line
+int
aggregate_ret_kfunc_uu_c_test(struct __sk_buff *skb)I don't think three tests above a warranted, the previous patch already tested returning 16-byte structs. Maybe a test with a small structs of two ints, but I'd skip even that.
This would drop a ton of code from the back of the patch.
Ack. Will do.
Quoted 2 lines
...
Yonghong Song
reviewerPATCH v5 11/11 Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2
In reply to Yonghong Songkfuncs may now return a value larger than 8 bytes and up to 16 bytes (a scalar-only struct or union, or an __int128), passed back in the R0:R2 register pair. Add a kfunc return-value section documenting this, including that a struct or union up to 8 bytes is returned in R0 alone, which struct and union members are accepted, that the R0:R2 register pair requires JIT support (bpf_jit_supports_kfunc_ret_reg_pair()), and that a return value larger than 16 bytes is unsupported.
Also note that the same convention applies to BPF subprogram returns, and document the consequence for a global subprogram: it must assign both halves of a register-pair return, since an unassigned R2 may be left holding a pointer argument and is then rejected as a leak.
--- Documentation/bpf/kfuncs.rst | 63 ++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+)
Documentation/bpf/kfuncs.rst +63 -0
@@ -575,6 +575,69 @@ is also covered by this recovery. A kfunc handed an arena pointer may therefore access up to ``GUARD_SZ / 2`` past it without bounds-checking against the arena. Larger accesses must verify the range explicitly. +2.9 kfunc Return Values +----------------------- + +A kfunc may return a scalar, a pointer, or a small struct or union by +value. A scalar or pointer of up to 8 bytes is returned in R0, as usual. + +A struct or union returned by value must be composed only of scalars +(recursively), where a scalar is an integer or an enum; arrays of scalars are +allowed as members. Its bytes are handed back to the program as the raw +contents of R0 (and R2), so a pointer field would be laundered into a scalar +and escape the verifier's pointer provenance and reference tracking. A struct +or union with a pointer member is therefore rejected at load time, and so is +one with a floating-point member, which the ABI may not return in R0:R2 at +all. + +A kfunc may also return a value larger than 8 bytes and up to 16 bytes -- a +scalar-only struct or union, or an ``__int128``. Such a value is returned +in the register pair R0:R2, matching the convention LLVM uses for the BPF +target: the first 8 bytes in R0 and the second 8 bytes in R2. A struct or +union of 8 bytes or less is returned in R0 alone. + +:: + + struct bpf_pair { __u64 a, b; }; /* 16 bytes */ + + __bpf_kfunc struct bpf_pair bpf_kfunc_get_pair(void) + { + struct bpf_pair p = { .a = 1, .b = 2 }; + + return p; /* p.a in R0, p.b in R2 */ + } + +Returning a value in the R0:R2 pair requires the JIT to place the second +half of the return value into R2, which not every architecture supports +right now. A kfunc with a return value larger than 8 bytes is therefore +rejected at load time on a JIT that does not advertise this capability (see +``bpf_jit_supports_kfunc_ret_reg_pair()``), and such a program is never run +by the interpreter. A return value larger than 16 bytes is not supported. + +The same R0:R2 convention applies to a BPF subprogram, global or static, +that returns an ``__int128`` or a struct or union larger than 8 bytes. Such a +program also requires the JIT, since the interpreter propagates only R0 out +of a subprogram. A global subprogram is verified in isolation, so its +by-value struct or union return is restricted to scalars just like a kfunc's; +a static subprogram is verified inline and has no such restriction. The main +program is not covered: its return value is the program's exit code, read out +of R0 alone, so a declared upper half is never looked at. + +A global subprogram must leave a scalar in *every* register of the pair, so +both halves of the returned value have to be assigned. Leaving the upper half +uninitialized is not merely untidy: the compiler is then free to leave R2 +holding whatever it happened to hold, which for a subprogram taking a pointer +argument is typically that pointer. Handing the caller an unknown scalar built +from a pointer is a leak, so the verifier rejects it with:: + + At subprogram exit the register R2 is not a scalar value (...) + +Initialize the whole return value, for example ``struct pair p = {};``, to +avoid this. A static subprogram is exempt from the scalar-only rule: it is +verified inline, so an unassigned R2 is simply passed back to the caller as +uninitialized and only a caller that reads it fails. A stack pointer left in +R2 is still rejected there, just as one in R0 is. + .. _BPF_kfunc_lifecycle_expectations: 3. kfunc lifecycle expectations
Signature · 1 line
2.53.0-Meta