Skip to contentSkip to conversation

bpf: Support aggregate return values up to 16 bytes

PATCH Superseded Yonghong Song 4 participants 36 messages Aug 17 – Aug 19

Subsystems (12)
Documentation x86 Kselftest BPF +8 more subsystems

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Yonghong Song

author reviewer
#0 lore raw

v6 cover bpf: Support aggregate return values up to 16 bytes

LLVM 23 can return an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair [1][2]. Before that the BPF backend could not return such values at all: a by-value aggregate return was rejected at compile time with "aggregate returns are not supported", and an __int128 return failed in the backend with "unable to allocate function return #1".

This series teaches the kernel the same convention, so that BPF programs and kfuncs can return these values. The first 8 bytes of the value come back in R0 and the second 8 bytes in R2. It applies to kfunc returns and to BPF-to-BPF subprogram returns, both global and static. The main program is unchanged: its return value is the program's exit code, so a return larger than 8 bytes is still rejected at BPF_EXIT.

Patches 1-2 are preparation: patch 1 factors out the per-register check used by the global return path, and patch 2 adds the shared helpers that answer "does this subprogram return a register pair", so that the patches which follow can be ordered independently. Patch 3 wires up the JIT side. Patches 4-5 teach precision backtracking and live register analysis about R2 as a second return register, ahead of the patch that starts modeling it. Patch 6 adds the verifier support proper. Patch 7 relaxes btf_distill_func_proto() and btf_validate_return_type(), which is what makes the whole thing reachable. Patches 8-9 add selftests and patch 10 documents the convention.

Constraints worth calling out:

- A by-value struct or union returned by a kfunc or by a global subprogram must be composed only of scalars. The verifier models the returned register bits as an unknown scalar, so a pointer member would be laundered into one and escape provenance and reference tracking. A static subprogram is verified inline and is not restricted this way.

- Returning the pair from a kfunc needs the JIT to place the second half into R2, which is architecture-specific work. Architectures opt in through bpf_jit_supports_kfunc_ret_reg_pair(); x86_64, arm64 and riscv64 do so here, and elsewhere bpf_add_kfunc_call() rejects such a kfunc with -EOPNOTSUPP. A register-pair return from a BPF subprogram needs no such capability.

- The pair is modelled only when the JIT is enabled, since the interpreter propagates R0 alone out of a subprogram. Modelling it sets jit_required, so a JIT fallback to the interpreter becomes a load failure rather than a silent divergence from what was verified. With the JIT off, the pair is not modelled and a caller reading R2 fails verification.

- The compiler side requires LLVM 23 or newer. The selftests written in C sit behind a __clang_major__ guard; an older compiler builds a dummy test instead, whose description says why nothing was exercised. The inline-asm tests run everywhere, apart from the few whose callee prototype returns a struct or union by value, which are guarded the same way.

[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876

Changelog:
v5 -> v6:
- v5: https://lore.kernel.org/bpf/[email protected]/
- Returning R0 only if jit is not enabled. If jit is enabled, main prog will not force jit even if main prog may return R0:R2.
- Simplify precision backtracking.
- Check returning R0:R2 in btf_check_func_type_match().
- Remove some redundant tests.
v4 -> v5:
- v4: https://lore.kernel.org/bpf/[email protected]/
- Removed R0:R2 restriction on callback functions and its related tests.
- Simplify the code for backtracking.
- Reduce comments and verify R0/R2 together.
- Use RUN_TESTS for selftests.
v3 -> v4:
- v3: https://lore.kernel.org/bpf/[email protected]/
- Fix a few kernel comment format.
- Guard more kfunc's with x86_64/arm64 only to avoid s390x failure.
v2 -> v3:
- v2: https://lore.kernel.org/bpf/[email protected]/
- Add additional guard with __SIZEOF_INT128__ for 32bit kernel.
- Guard little endian to avoid arm64 big endian for selftests.
- Fix test failures for gcc15.
- Rebase to avoid conflict with latest master branch.
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- Split the R0:R2 helpers out of the verifier patch into their own preparation patch, and reordered the series so the core verifier patch comes after the infrastructure it depends on.
- New patch rejecting callbacks that return more than 8 bytes, both helper/kfunc callbacks and exception callbacks, with selftests.
- New patch rejecting a register-pair return once btf_check_subprog_call() has marked the subprogram's BTF unreliable, rather than silently mistracking R2.
- Folded "bpf: Force JIT for programs using the R0:R2 register pair" into the verifier patch.
- Dropped "bpf: Reject >8 byte return values on return-reading trampoline paths" and its selftests; that went in separately as commit c48796aa6c39.
- Described the register mapping as the first and second 8 bytes rather than the low and high 64 bits, which is only correct on little-endian, and reworded "16-byte" to "up to 16 bytes" where the range 9..16 was meant.

Yonghong Song (10):
bpf: Factor check_global_ret_scalar_reg() out of the global return
check
bpf: Add helpers to describe the R0:R2 return register pair
bpf: Wire up JIT support for 16-byte kfunc returns
bpf: Handle R2 as a return register in precision backtracking
bpf: Account R2 of register-pair returns in live register analysis
bpf: Add verifier support for 16-byte returns in R0:R2
bpf: Enable aggregate return types up to 16 bytes selftests/bpf: Add C tests for 16-byte returns in R0:R2 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2

 Documentation/bpf/kfuncs.rst                  |  65 +++++
 arch/arm64/net/bpf_jit_comp.c                 |   5 +
 arch/riscv/net/bpf_jit_comp64.c               |   5 +
 arch/x86/net/bpf_jit_comp.c                   |  27 +-
 include/linux/bpf_verifier.h                  |   9 +
 include/linux/filter.h                        |   1 +
 kernel/bpf/backtrack.c                        |  46 ++--
 kernel/bpf/btf.c                              |  29 ++-
 kernel/bpf/core.c                             |   5 +
 kernel/bpf/liveness.c                         |  20 +-
 kernel/bpf/verifier.c                         | 196 ++++++++++++---
 .../selftests/bpf/prog_tests/aggregate_ret.c  |  11 +
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |  16 ++
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/aggregate_ret_func.c  | 235 ++++++++++++++++++
 .../selftests/bpf/progs/aggregate_ret_kfunc.c | 122 +++++++++
 .../bpf/progs/aggregate_ret_target.c          |  29 +++
 .../bpf/progs/compute_live_registers.c        |  30 +++
 .../selftests/bpf/progs/exceptions_fail.c     |   2 +-
 .../selftests/bpf/progs/freplace_ret_pair.c   |  12 +
 .../bpf/progs/verifier_aggregate_ret.c        | 178 +++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  55 ++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  29 +++
 23 files changed, 1057 insertions(+), 72 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c
 create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c
Signature · 1 line
2.53.0-Meta
author reviewer
#1 lore raw

PATCH v6 1/10 bpf: Factor check_global_ret_scalar_reg() out of the global return check

check_global_subprog_return_code() verifies that a global subprogram returns void, an arena pointer, or register R0 holding a scalar value. Later patches in this series add 16-byte aggregate return support, whose second half is returned in R2 and needs the same validation.

Factor the per-register check into check_global_ret_scalar_reg(env, regno) so that it can be reused for R2. No functional change.

---
 kernel/bpf/verifier.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)
kernel/bpf/verifier.c +20 -12
index d17f14b35b79..b3c474ba7140 100644
@@ -17451,37 +17451,45 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
 	return 0;
 }
 
-static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+static int check_global_ret_scalar_reg(struct bpf_verifier_env *env, u32 regno)
 {
-	struct bpf_reg_state *reg = reg_state(env, BPF_REG_0);
-	struct bpf_func_state *cur_frame = cur_func(env);
+	struct bpf_reg_state *reg;
 	int err;
 
-	if (subprog_returns_void(env, cur_frame->subprogno))
-		return 0;
-
-	err = check_reg_arg(env, BPF_REG_0, SRC_OP);
+	err = check_reg_arg(env, regno, SRC_OP);
 	if (err)
 		return err;
 
 	/* Pointers to arena are safe to pass between subprograms. */
-	if (is_arena_reg(env, BPF_REG_0))
+	if (is_arena_reg(env, regno))
 		return 0;
 
-	if (is_pointer_value(env, BPF_REG_0)) {
-		verbose(env, "R%d leaks addr as return value\n", BPF_REG_0);
+	if (is_pointer_value(env, regno)) {
+		verbose(env, "R%d leaks addr as return value\n", regno);
 		return -EACCES;
 	}
 
+	reg = reg_state(env, regno);
 	if (reg->type != SCALAR_VALUE) {
-		verbose(env, "At subprogram exit the register R0 is not a scalar value (%s)\n",
-			reg_type_str(env, reg->type));
+		verbose(env, "At subprogram exit the register R%d is not a scalar value (%s)\n",
+			regno, reg_type_str(env, reg->type));
 		return -EINVAL;
 	}
 
 	return 0;
 }
 
+static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+{
+	struct bpf_func_state *cur_frame = cur_func(env);
+	u32 subprog = cur_frame->subprogno;
+
+	if (subprog_returns_void(env, subprog))
+		return 0;
+
+	return check_global_ret_scalar_reg(env, BPF_REG_0);
+}
+
 /* Bitmask with 1s for all caller saved registers */
 #define ALL_CALLER_SAVED_REGS ((1u << CALLER_SAVED_REGS) - 1)
 
Signature · 1 line
2.53.0-Meta
author reviewer
#2 lore raw

PATCH v6 2/10 bpf: Add helpers to describe the R0:R2 return register pair

In reply to Yonghong Song

LLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes: such a value comes back in the R0:R2 register pair, with R2 holding the upper half. See LLVM patches [1] and [2].

Later patches teach the JIT, live register analysis and the verifier itself about that convention. They need to answer the same question: does this subprogram return its value in a register pair? Add bpf_ret_reg_pair() up front so it can be used in subsequent patches. It is answered from a per-subprogram flag that bpf_compute_subprog_ret_regs() derives once from the BTF prototype.

jit_requested only says that the JIT is enabled, not that it succeeded:
bpf_fixup_call_args() falls back to the interpreter when bpf_jit_subprogs() fails with anything other than -EFAULT. So jit_required is still set once the pair is modelled, which turns that fallback into a load failure rather than a silent divergence from what was verified.

[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876

---
 include/linux/bpf_verifier.h |  7 ++++
 kernel/bpf/verifier.c        | 70 ++++++++++++++++++++++++++++++------
 2 files changed, 66 insertions(+), 11 deletions(-)
include/linux/bpf_verifier.h +7 -0
index bc2af02547fe..f70d5878fbff 100644
@@ -819,6 +819,8 @@ struct bpf_subprog_info {
 	bool is_async_cb: 1;
 	bool is_exception_cb: 1;
 	bool args_cached: 1;
+	/* true if the return value is passed in the R0:R2 register pair */
+	bool ret_reg_pair: 1;
 	/* true if bpf_fastcall stack region is used by functions that can't be inlined */
 	bool keep_fastcall_stack: 1;
 	bool changes_pkt_data: 1;
@@ -1055,6 +1057,11 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env
 	return &env->subprog_info[subprog];
 }
 
+static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
+{
+	return subprog_info(env, subprog)->ret_reg_pair;
+}
+
 struct bpf_call_summary {
 	u8 num_params;
 	bool is_void;
kernel/bpf/verifier.c +59 -11

92 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#3 lore raw

Re: [PATCH bpf-next v6 02/10] bpf: Add helpers to describe the R0:R2 return register pair

derive this from BTF prototypes.
Quoted 1 line

8 but <=16 bytes). It introduces bpf_compute_subprog_ret_regs() to

derive this from BTF prototypes.

Quoted 22 lines
include/linux/bpf_verifier.h +7 -0
index bc2af02547fe..f70d5878fbff 100644
@@ -819,6 +819,8 @@ struct bpf_subprog_info {
 	bool is_async_cb: 1;
 	bool is_exception_cb: 1;
 	bool args_cached: 1;
+	/* true if the return value is passed in the R0:R2 register pair */
+	bool ret_reg_pair: 1;
 	/* true if bpf_fastcall stack region is used by functions that can't be inlined */
 	bool keep_fastcall_stack: 1;
 	bool changes_pkt_data: 1;
@@ -1055,6 +1057,11 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env
 	return &env->subprog_info[subprog];
 }
 
+static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
+{
+	return subprog_info(env, subprog)->ret_reg_pair;
+}
+

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index b3c474ba7140..f1f1268d29c6 100644

[ ... ]

Quoted 59 lines
kernel/bpf/verifier.c +34 -11
@@ -385,27 +385,70 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)
 	return aux && aux[subprog].linkage == BTF_FUNC_GLOBAL;
 }
 
-static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+static const struct btf_type *subprog_ret_type(struct bpf_verifier_env *env, int subprog)
 {
-	const struct btf_type *type, *func, *func_proto;
+	const struct btf_type *func, *func_proto;
 	const struct btf *btf = env->prog->aux->btf;
 	u32 btf_id;
 
+	if (!btf || !env->prog->aux->func_info)
+		return NULL;
+
 	btf_id = env->prog->aux->func_info[subprog].type_id;
 
+	/* Both already validated by prepare_btf_func() at prog load. */
 	func = btf_type_by_id(btf, btf_id);
-	if (verifier_bug_if(!func, env, "btf_id %u not found", btf_id))
-		return false;
-
 	func_proto = btf_type_by_id(btf, func->type);
-	if (!func_proto)
-		return false;
 
-	type = btf_type_skip_modifiers(btf, func_proto->type, NULL);
-	if (!type)
-		return false;
+	return btf_type_skip_modifiers(btf, func_proto->type, NULL);
+}
 
-	return btf_type_is_void(type);
+static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+{
+	const struct btf_type *type = subprog_ret_type(env, subprog);
+
+	return type && btf_type_is_void(type);
+}
+
+static u32 ret_regs_cnt(u32 size)
+{
+	return size > 8 && size <= 16 ? 2 : 1;
+}
+
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 441

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
+		type = subprog_ret_type(env, subprog);
+		/*
+		 * This runs before btf_validate_return_type(), so apply the same
+		 * type filter here.
+		 */
+		if (!type || !(btf_type_is_struct(type) || btf_type_is_scalar(type)))
+			continue;

Does the comment's claim about applying "the same type filter" hold?

btf_validate_return_type() accepts void, int, or enum types via:

kernel/bpf/btf.c:btf_validate_return_type() {
    /* We always accept void or scalars. */
    if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
        return 0;

    return -EOPNOTSUPP;
}

The new filter instead accepts struct/union or scalar (int or enum, but not ENUM64):

    if (!type || !(btf_type_is_struct(type) || btf_type_is_scalar(type)))

btf_type_is_scalar() is defined as btf_type_is_int() || btf_type_is_enum(), and btf_type_is_enum() excludes BTF_KIND_ENUM64.

The filters differ in several ways. btf_validate_return_type() accepts void and ENUM64 (via btf_is_any_enum()) but rejects struct/union at this commit. The new filter accepts struct/union but filters out void and ENUM64.

The divergence is harmless since every dropped type is at most 8 bytes and would need only one register anyway. But the comment implies an equivalence that does not exist, and specifically suggests structs are validated elsewhere when btf_validate_return_type() at this commit still rejects them with -EOPNOTSUPP.

Would describing what the filter actually needs be clearer? Something like "only struct/union and integer/enum returns can exceed 8 bytes and need sizing" rather than claiming parity with btf_validate_return_type()?

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 452

Quoted 11 lines
kernel/bpf/verifier.c +10 -0
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;
 }

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

author reviewer
#4 lore raw

Re: [PATCH bpf-next v6 02/10] bpf: Add helpers to describe the R0:R2 return register pair

Quoted 133 lines · On 8/16/26 10:17 PM, [email protected] wrote:
Quoted 1 line

8 but <=16 bytes). It introduces bpf_compute_subprog_ret_regs() to

derive this from BTF prototypes.

Quoted 22 lines
include/linux/bpf_verifier.h +7 -0
index bc2af02547fe..f70d5878fbff 100644
@@ -819,6 +819,8 @@ struct bpf_subprog_info {
  	bool is_async_cb: 1;
  	bool is_exception_cb: 1;
  	bool args_cached: 1;
+	/* true if the return value is passed in the R0:R2 register pair */
+	bool ret_reg_pair: 1;
  	/* true if bpf_fastcall stack region is used by functions that can't be inlined */
  	bool keep_fastcall_stack: 1;
  	bool changes_pkt_data: 1;
@@ -1055,6 +1057,11 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env
  	return &env->subprog_info[subprog];
  }
 
+static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
+{
+	return subprog_info(env, subprog)->ret_reg_pair;
+}
+

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index b3c474ba7140..f1f1268d29c6 100644

[ ... ]

Quoted 59 lines
kernel/bpf/verifier.c +34 -11
@@ -385,27 +385,70 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)
  	return aux && aux[subprog].linkage == BTF_FUNC_GLOBAL;
  }
 
-static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+static const struct btf_type *subprog_ret_type(struct bpf_verifier_env *env, int subprog)
  {
-	const struct btf_type *type, *func, *func_proto;
+	const struct btf_type *func, *func_proto;
  	const struct btf *btf = env->prog->aux->btf;
  	u32 btf_id;
 
+	if (!btf || !env->prog->aux->func_info)
+		return NULL;
+
  	btf_id = env->prog->aux->func_info[subprog].type_id;
 
+	/* Both already validated by prepare_btf_func() at prog load. */
  	func = btf_type_by_id(btf, btf_id);
-	if (verifier_bug_if(!func, env, "btf_id %u not found", btf_id))
-		return false;
-
  	func_proto = btf_type_by_id(btf, func->type);
-	if (!func_proto)
-		return false;
 
-	type = btf_type_skip_modifiers(btf, func_proto->type, NULL);
-	if (!type)
-		return false;
+	return btf_type_skip_modifiers(btf, func_proto->type, NULL);
+}
 
-	return btf_type_is_void(type);
+static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+{
+	const struct btf_type *type = subprog_ret_type(env, subprog);
+
+	return type && btf_type_is_void(type);
+}
+
+static u32 ret_regs_cnt(u32 size)
+{
+	return size > 8 && size <= 16 ? 2 : 1;
+}
+
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 441

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
+		type = subprog_ret_type(env, subprog);
+		/*
+		 * This runs before btf_validate_return_type(), so apply the same
+		 * type filter here.
+		 */
+		if (!type || !(btf_type_is_struct(type) || btf_type_is_scalar(type)))
+			continue;

Does the comment's claim about applying "the same type filter" hold?

btf_validate_return_type() accepts void, int, or enum types via:

kernel/bpf/btf.c:btf_validate_return_type() {
     /* We always accept void or scalars. */
     if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
         return 0;

     return -EOPNOTSUPP;
}

The new filter instead accepts struct/union or scalar (int or enum, but not ENUM64):

     if (!type || !(btf_type_is_struct(type) || btf_type_is_scalar(type)))

btf_type_is_scalar() is defined as btf_type_is_int() || btf_type_is_enum(), and btf_type_is_enum() excludes BTF_KIND_ENUM64.

The filters differ in several ways. btf_validate_return_type() accepts void and ENUM64 (via btf_is_any_enum()) but rejects struct/union at this commit. The new filter accepts struct/union but filters out void and ENUM64.

The divergence is harmless since every dropped type is at most 8 bytes and would need only one register anyway. But the comment implies an equivalence that does not exist, and specifically suggests structs are validated elsewhere when btf_validate_return_type() at this commit still rejects them with -EOPNOTSUPP.

Quoted 3 lines

Would describing what the filter actually needs be clearer? Something like "only struct/union and integer/enum returns can exceed 8 bytes and need sizing" rather than claiming parity with btf_validate_return_type()?

You are right. I missed enum64. I think the above 'if' condition should be

if (!type || !(btf_type_is_struct(type) || btf_type_is_int(type) || btf_is_any_enum(type)))

Quoted 20 lines

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 452

Quoted 12 lines
kernel/bpf/verifier.c +10 -0
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;
  }

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

maintainer
#5 lore raw

Re: [PATCH bpf-next v6 02/10] bpf: Add helpers to describe the R0:R2 return register pair

In reply to Yonghong Song

On Sun, 2026-08-16 at 21:21 -0700, Yonghong Song wrote:

...

Quoted 5 lines
kernel/bpf/verifier.c +0 -0
index b3c474ba7140..f1f1268d29c6 100644
@@ -385,27 +385,70 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)

...

Quoted 13 lines
kernel/bpf/verifier.c +13 -0
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 440

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
+		type = subprog_ret_type(env, subprog);
+		/*
+		 * This runs before btf_validate_return_type(), so apply the same
+		 * type filter here.
+		 */
+		if (!type || !(btf_type_is_struct(type) || btf_type_is_scalar(type)))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I still think that only !type check is necessary here:
- btf_resolve_size can handle any type
- BTF validation should guarantee that return type is not e.g. BTF_KIND_VAR that would trigger -EINVAL
- for local subprograms verifier checks if r0/r2 bear compatible types anyway
- for global subprograms BTF compliance is enforced.

Could you please explain why would you want to keep it here?

Quoted 14 lines
kernel/bpf/verifier.c +11 -0
+			continue;
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;

 }

 const char *bpf_subprog_name(const struct bpf_verifier_env *env, int subprog)

maintainer
#6 lore raw

Re: [PATCH bpf-next v6 02/10] bpf: Add helpers to describe the R0:R2 return register pair

Quoted 29 lines · Eduard Zingerman

On Sun, 2026-08-16 at 21:21 -0700, Yonghong Song wrote:

...

Quoted 5 lines
kernel/bpf/verifier.c +0 -0
index b3c474ba7140..f1f1268d29c6 100644
@@ -385,27 +385,70 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)

...

Quoted 17 lines
kernel/bpf/verifier.c +17 -0
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
Quoted 8 lines

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 440

Quoted 6 lines
kernel/bpf/verifier.c +6 -0
+		type = subprog_ret_type(env, subprog);
+		/*
+		 * This runs before btf_validate_return_type(), so apply the same
+		 * type filter here.
+		 */
+		if (!type || !(btf_type_is_struct(type) || btf_type_is_scalar(type)))

                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I still think that only !type check is necessary here:

Actually: if (!type || !btf_type_is_void(type)) ... since btf_resolve_size() does not like void.

Quoted 22 lines

- btf_resolve_size can handle any type
- BTF validation should guarantee that return type is not e.g. BTF_KIND_VAR that would trigger -EINVAL
- for local subprograms verifier checks if r0/r2 bear compatible types anyway
- for global subprograms BTF compliance is enforced.

Could you please explain why would you want to keep it here?

Quoted 14 lines
kernel/bpf/verifier.c +11 -0
+			continue;
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;

 }

 const char *bpf_subprog_name(const struct bpf_verifier_env *env, int subprog)

author reviewer
#7 lore raw

Re: [PATCH bpf-next v6 02/10] bpf: Add helpers to describe the R0:R2 return register pair

Quoted 37 lines · Eduard Zingerman
Quoted 28 lines · Eduard Zingerman

On Sun, 2026-08-16 at 21:21 -0700, Yonghong Song wrote:

...

Quoted 5 lines
kernel/bpf/verifier.c +0 -0
index b3c474ba7140..f1f1268d29c6 100644
@@ -385,27 +385,70 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)

...

Quoted 17 lines
kernel/bpf/verifier.c +17 -0
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
Quoted 8 lines

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 440

Quoted 6 lines
kernel/bpf/verifier.c +6 -0
+		type = subprog_ret_type(env, subprog);
+		/*
+		 * This runs before btf_validate_return_type(), so apply the same
+		 * type filter here.
+		 */
+		if (!type || !(btf_type_is_struct(type) || btf_type_is_scalar(type)))

                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I still think that only !type check is necessary here:

Quoted 2 lines

Actually: if (!type || !btf_type_is_void(type)) ... since btf_resolve_size() does not like void.

Thinking through. I think your above suggestion is correct.
In later patch "bpf: Enable aggregate return types up to 16 bytes", we have

btf_validate_return_type():

	...

if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
return 0;

+ if (btf_type_is_struct(t) && t->size <= 16) {
+ /*
+ * A global function's caller models the return as an opaque
+ * scalar pair, so it may only return scalars by value. A local
+ * function is verified inline, so a pointer field stays tracked
+ * and needs no such restriction.
+ */
+ bool local_func = subprog && !is_global;
+
+ if (local_func || btf_type_is_scalar_struct(env, btf, t, 0))
+ return 0;
+ }

	return -EOPNOTSUPP;

So only the above 'return 0' types are supported (void, int, enum, struct etc.). In bpf_compute_subprog_ret_regs(), we only need valid type and type won't be void. The 'void' type is for btf_resolve_size(). For other types, btf_validate_return_type() will reject earlier.

Quoted 23 lines
Quoted 23 lines

- btf_resolve_size can handle any type
- BTF validation should guarantee that return type is not e.g. BTF_KIND_VAR that would trigger -EINVAL
- for local subprograms verifier checks if r0/r2 bear compatible types anyway
- for global subprograms BTF compliance is enforced.

Could you please explain why would you want to keep it here?

Quoted 14 lines
kernel/bpf/verifier.c +11 -0
+			continue;
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;
  }
 
  const char *bpf_subprog_name(const struct bpf_verifier_env *env, int subprog)
author reviewer
#8 lore raw

PATCH v6 3/10 bpf: Wire up JIT support for 16-byte kfunc returns

In reply to Yonghong Song

LLVM 23 returns an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch added the shared helpers describing that convention; wire up the JIT side so that the second half of the return value actually lands in R2.

Placing the second return half into R2 is possible on any JIT, but it needs architecture-specific JIT work. Rather than requiring every JIT to implement it at once, add a bpf_jit_supports_kfunc_ret_reg_pair() capability, defaulting to false in the generic core; an architecture opts in once its JIT handles the R0:R2 pair, and the remaining ones are left for future work. Only the x86-64, arm64 and riscv64 JITs opt in so far.

On arm64 and riscv64 the native second return register is already BPF R2 (x1 in bpf2a64[] and a1 in regmap[] respectively), so the upper half needs no move at all, unlike x86-64's RDX->RSI. The lower half is covered by the move into BPF R0 that those JITs already emit after every call, from x0 into x8 and from a0 into a5. This has been tested on x86-64 and arm64. The riscv64 path is expected to work by the same register-mapping reasoning as arm64 but has not been tested.

bpf_add_kfunc_call() also rejects a kfunc that is marked KF_FASTCALL and returns more than 8 bytes. The bpf_fastcall contract implemented by mark_fastcall_pattern_for_call() assumes a call clobbers R0 plus the registers holding its arguments, so a return in the R0:R2 pair would clobber an R2 the caller expects the fastcall pattern to preserve. Such a kfunc is rejected with -EOPNOTSUPP as well.

---
 arch/arm64/net/bpf_jit_comp.c   |  5 +++++
 arch/riscv/net/bpf_jit_comp64.c |  5 +++++
 arch/x86/net/bpf_jit_comp.c     | 27 ++++++++++++++++++++-------
 include/linux/filter.h          |  1 +
 kernel/bpf/core.c               |  5 +++++
 kernel/bpf/verifier.c           | 12 ++++++++++++
 6 files changed, 48 insertions(+), 7 deletions(-)
arch/arm64/net/bpf_jit_comp.c +5 -0
index c18e005a41db..3aa3ea0bc30b 100644
@@ -2388,6 +2388,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_stack_args(void)
 {
 	return true;
arch/riscv/net/bpf_jit_comp64.c +5 -0
index 74efe4b138d2..47c7bf431ba8 100644
@@ -2121,6 +2121,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_ptr_xchg(void)
 {
 	return true;
arch/x86/net/bpf_jit_comp.c +20 -7
index 1a9fb530adc3..48429fae0641 100644
@@ -1689,17 +1689,12 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
  * arena NULL is offset 0. Return the number of emitted bytes.
  */
 static int emit_kfunc_arena_args(struct bpf_prog *bpf_prog,
-				 const struct bpf_insn *insn, u8 **pprog)
+				 const struct btf_func_model *fm, u8 **pprog)
 {
-	const struct btf_func_model *fm;
 	u8 *prog = *pprog;
 	u8 *start = prog;
 	int i;
 
-	fm = bpf_jit_find_kfunc_model(bpf_prog, insn);
-	if (!fm)
-		return -EINVAL;
-
 	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
 		u8 flags = fm->arg_flags[i];
 		u32 reg = BPF_REG_1 + i;
@@ -2644,6 +2639,8 @@ st:			insn_off = insn->off;
 
 			/* call */
 		case BPF_JMP | BPF_CALL: {
+			const struct btf_func_model *fm = NULL;
+
 			func = (u8 *) __bpf_call_base + imm32;
 			if (src_reg == BPF_PSEUDO_CALL && tail_call_reachable) {
 				LOAD_TAIL_CALL_CNT_PTR(stack_depth);
@@ -2652,7 +2649,10 @@ st:			insn_off = insn->off;
 			if (!imm32)
 				return -EINVAL;
 			if (src_reg == BPF_PSEUDO_KFUNC_CALL) {
-				err = emit_kfunc_arena_args(bpf_prog, insn, &prog);
+				fm = bpf_jit_find_kfunc_model(bpf_prog, insn);
+				if (!fm)
+					return -EINVAL;
+				err = emit_kfunc_arena_args(bpf_prog, fm, &prog);
 				if (err < 0)
 					return err;
 				ip += err;
@@ -2666,6 +2666,14 @@ st:			insn_off = insn->off;
 				return -EINVAL;
 			if (priv_frame_ptr)
 				pop_r9(&prog);
+			/*
+			 * A kfunc returning more than 8 bytes hands the second
+			 * half back in RDX (the native ABI's second return reg),
+			 * but BPF expects it in R0:R2. BPF R0 is RAX (no move
+			 * needed), while BPF R2 is RSI, so copy RDX into RSI.
+			 */
+			if (fm && fm->ret_size > 8)
+				emit_mov_reg(&prog, true, BPF_REG_2, BPF_REG_3);
 			break;
 		}
 
@@ -4156,6 +4164,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_stack_args(void)
 {
 	return true;
include/linux/filter.h +1 -0
index 4a9bc6a848f2..6e746b0a0930 100644
@@ -1237,6 +1237,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);
 bool bpf_jit_supports_subprog_tailcalls(void);
 bool bpf_jit_supports_percpu_insn(void);
 bool bpf_jit_supports_kfunc_call(void);
+bool bpf_jit_supports_kfunc_ret_reg_pair(void);
 bool bpf_jit_supports_stack_args(void);
 bool bpf_jit_supports_arena_args(void);
 bool bpf_jit_supports_far_kfunc_call(void);
kernel/bpf/core.c +5 -0
index d55e737ed75a..5db77d7915df 100644
@@ -3287,6 +3287,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)
 	return false;
 }
 
+bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return false;
+}
+
 bool __weak bpf_jit_supports_stack_args(void)
 {
 	return false;
kernel/bpf/verifier.c +12 -0
index f1f1268d29c6..54aca6c30506 100644
@@ -2920,6 +2920,18 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
 	err = btf_distill_func_proto(&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);
 	if (err)
 		return err;
+	if (func_model.ret_size > 8) {
+		if (kfunc.flags && (*kfunc.flags & KF_FASTCALL)) {
+			verbose(env, "kfunc %s with >8-byte return is not supported with KF_FASTCALL\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+		if (!bpf_jit_supports_kfunc_ret_reg_pair()) {
+			verbose(env, "kfunc %s with >8-byte return is not supported by JIT\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+	}
 
 	memset(&meta, 0, sizeof(meta));
 	meta.btf = kfunc.btf;
Signature · 1 line
2.53.0-Meta
#9 lore raw

Re: [PATCH bpf-next v6 03/10] bpf: Wire up JIT support for 16-byte kfunc returns

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The BPF verifier fails to initialize BPF_REG_2 for 16-byte scalar kfunc returns, breaking the…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The BPF verifier fails to initialize BPF_REG_2 for 16-byte scalar kfunc returns, breaking the feature. --

--- Patch [3]: [PATCH bpf-next v6 03/10] bpf: Wire up JIT support for 16-byte kfunc returns --- commit 70077e00d2d0c46ef0daa85eb2612ed7fbf545ef
Author: Yonghong Song <[email protected]>
bpf: Wire up JIT support for 16-byte kfunc returns

This commit wires up JIT support for 16-byte kfunc returns in BPF, such as large structs or 128-bit integers. It adds support to the x86-64, arm64, and riscv64 JITs to return the second half of the value in BPF_REG_2, and adds verifier checks to reject larger returns on unsupported architectures.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index f1f1268d29c66..54aca6c305061 100644

[ ... ]

Quoted 6 lines
kernel/bpf/verifier.c +2 -0
@@ -2920,6 +2920,18 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
 	err = btf_distill_func_proto(&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);
 	if (err)
 		return err;
+	if (func_model.ret_size > 8) {
+		if (kfunc.flags && (*kfunc.flags & KF_FASTCALL)) {

kernel/bpf/verifier.c int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) · line 2934

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+			verbose(env, "kfunc %s with >8-byte return is not supported with KF_FASTCALL\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+		if (!bpf_jit_supports_kfunc_ret_reg_pair()) {
+			verbose(env, "kfunc %s with >8-byte return is not supported by JIT\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+	}

[Severity: Medium]
While this patch adds JIT support and basic verifier checks for >8-byte returns, does the verifier correctly initialize BPF_REG_2 for 16-byte scalar returns?

Looking at check_kfunc_call() in kernel/bpf/verifier.c, it seems only BPF_REG_0 is marked as initialized for scalar returns:

    if (btf_type_is_scalar(t)) {
        mark_reg_unknown(env, regs, BPF_REG_0);
        if (meta.btf == btf_vmlinux &&
            (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] || ...

Since BPF caller-saved registers (including BPF_REG_2) are marked as NOT_INIT earlier in check_kfunc_call(), a BPF program attempting to access the upper 64 bits of a 16-byte scalar (like an __int128) in BPF_REG_2 would be rejected with a verifier read error.

Should check_kfunc_call() also initialize BPF_REG_2 when the return size is greater than 8 bytes?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
author reviewer
#10 lore raw

Re: [PATCH bpf-next v6 03/10] bpf: Wire up JIT support for 16-byte kfunc returns

Quoted 48 lines · On 8/16/26 9:37 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The BPF verifier fails to initialize BPF_REG_2 for 16-byte scalar kfunc returns, breaking the feature. --

--- Patch [3]: [PATCH bpf-next v6 03/10] bpf: Wire up JIT support for 16-byte kfunc returns --- commit 70077e00d2d0c46ef0daa85eb2612ed7fbf545ef
Author: Yonghong Song <[email protected]>
bpf: Wire up JIT support for 16-byte kfunc returns

This commit wires up JIT support for 16-byte kfunc returns in BPF, such as large structs or 128-bit integers. It adds support to the x86-64, arm64, and riscv64 JITs to return the second half of the value in BPF_REG_2, and adds verifier checks to reject larger returns on unsupported architectures.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index f1f1268d29c66..54aca6c305061 100644

[ ... ]

Quoted 6 lines
kernel/bpf/verifier.c +2 -0
@@ -2920,6 +2920,18 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
  	err = btf_distill_func_proto(&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);
  	if (err)
  		return err;
+	if (func_model.ret_size > 8) {
+		if (kfunc.flags && (*kfunc.flags & KF_FASTCALL)) {

kernel/bpf/verifier.c int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset) · line 2934

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+			verbose(env, "kfunc %s with >8-byte return is not supported with KF_FASTCALL\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+		if (!bpf_jit_supports_kfunc_ret_reg_pair()) {
+			verbose(env, "kfunc %s with >8-byte return is not supported by JIT\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+	}

[Severity: Medium]
While this patch adds JIT support and basic verifier checks for >8-byte returns, does the verifier correctly initialize BPF_REG_2 for 16-byte scalar returns?

Looking at check_kfunc_call() in kernel/bpf/verifier.c, it seems only BPF_REG_0 is marked as initialized for scalar returns:

     if (btf_type_is_scalar(t)) {
         mark_reg_unknown(env, regs, BPF_REG_0);
         if (meta.btf == btf_vmlinux &&
             (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] || ...
Quoted 7 lines

Since BPF caller-saved registers (including BPF_REG_2) are marked as NOT_INIT earlier in check_kfunc_call(), a BPF program attempting to access the upper 64 bits of a 16-byte scalar (like an __int128) in BPF_REG_2 would be rejected with a verifier read error.

Should check_kfunc_call() also initialize BPF_REG_2 when the return size is greater than 8 bytes?

We should be okay here. The actual BPF_REG_2 will be used in
commit "bpf: Add verifier support for 16-byte returns in R0:R2"

author reviewer
#11 lore raw

PATCH v6 4/10 bpf: Handle R2 as a return register in precision backtracking

In reply to Yonghong Song

Precision backtracking treats only R0 as a return register at a call/return boundary, so once the verifier starts modeling R2 that way, marking the second half of such a return precise would trip the "unexpected regs" checks in backtrack_insn() and reject a valid program.

Marking the upper half precise, for example by branching on it after a call to a static subprogram, walks backtracking into the callee and reaches its BPF_EXIT with R2 still set in the mask. Handle R2 like R0 in boundaries where a call defines the return registers.

R2 differs from R0 in that it is an argument register as well, so it is part of the BPF_REGMASK_ARGS check and has to be cleared before that check rather than next to R0. Clear it unconditionally, rather than only where the callee or the kfunc really does return a pair. That gives up the "unexpected regs" assertion for R2, and in exchange keeps backtracking free of any BTF lookup. Nothing is lost: a callee that does not return a pair leaves the caller's R2 uninitialized, so the main verification pass has already rejected any program that reads it, and backtracking is never asked for its precision.

At BPF_EXIT the return registers are sampled before the callback path clears R1-R5. That clear does not touch R0, but it does cover R2, and running it first would drop a pair return whenever the instruction following the call happens to be one that invokes a callback.

---
 kernel/bpf/backtrack.c | 46 +++++++++++++++++++++++++++---------------
 1 file changed, 30 insertions(+), 16 deletions(-)
kernel/bpf/backtrack.c +30 -16
index a2b18a9f1694..653db80bcc47 100644
@@ -423,6 +423,10 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 				 */
 				verifier_bug_if(idx + 1 != subseq_idx, env,
 						"extra insn from subprog");
+				/* global subprog always sets R0 */
+				bt_clear_reg(bt, BPF_REG_0);
+				/* and if it does not set R2, main pass would catch it */
+				bt_clear_reg(bt, BPF_REG_2);
 				/* r1-r5 are invalidated after subprog call,
 				 * so for global func call it shouldn't be set
 				 * anymore
@@ -432,8 +436,6 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 						     bt_reg_mask(bt));
 					return -EFAULT;
 				}
-				/* global subprog always sets R0 */
-				bt_clear_reg(bt, BPF_REG_0);
 				return 0;
 			} else {
 				/* static subprog call instruction, which
@@ -506,6 +508,8 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 				return -ENOTSUPP;
 			/* regular helper call sets R0 */
 			bt_clear_reg(bt, BPF_REG_0);
+			/* kfunc might also set R2 */
+			bt_clear_reg(bt, BPF_REG_2);
 			if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
 				/* if backtracking was looking for registers R1-R5
 				 * they should have been found already.
@@ -520,7 +524,25 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 					return -EFAULT;
 			}
 		} else if (opcode == BPF_EXIT) {
-			bool r0_precise;
+			bool from_subprog_call, r0_precise, r2_precise;
+
+			/* BPF_EXIT in subprog or callback always returns
+			 * right after the call instruction, so by checking
+			 * whether the instruction at subseq_idx-1 is subprog
+			 * call or not we can distinguish actual exit from
+			 * *subprog* from exit from *callback*. In the former
+			 * case, we need to propagate the precision of the
+			 * return registers, if necessary. In the latter we
+			 * never do that.
+			 */
+			from_subprog_call = subseq_idx - 1 >= 0 &&
+					    bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+
+			/* Sample the return registers before the callback
+			 * handling below clears R1-R5.
+			 */
+			r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
+			r2_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_2);
 
 			/* Backtracking to a nested function call, 'idx' is a part of
 			 * the inner frame 'subseq_idx' is a part of the outer frame.
@@ -533,30 +555,22 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 			if (subseq_idx >= 0 && bpf_calls_callback(env, subseq_idx))
 				for (i = BPF_REG_1; i <= BPF_REG_5; i++)
 					bt_clear_reg(bt, i);
+
+			bt_clear_reg(bt, BPF_REG_0);
+			bt_clear_reg(bt, BPF_REG_2);
 			if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
 				verifier_bug(env, "backtracking exit unexpected regs %x",
 					     bt_reg_mask(bt));
 				return -EFAULT;
 			}
 
-			/* BPF_EXIT in subprog or callback always returns
-			 * right after the call instruction, so by checking
-			 * whether the instruction at subseq_idx-1 is subprog
-			 * call or not we can distinguish actual exit from
-			 * *subprog* from exit from *callback*. In the former
-			 * case, we need to propagate r0 precision, if
-			 * necessary. In the former we never do that.
-			 */
-			r0_precise = subseq_idx - 1 >= 0 &&
-				     bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]) &&
-				     bt_is_reg_set(bt, BPF_REG_0);
-
-			bt_clear_reg(bt, BPF_REG_0);
 			if (bt_subprog_enter(bt))
 				return -EFAULT;
 
 			if (r0_precise)
 				bt_set_reg(bt, BPF_REG_0);
+			if (r2_precise)
+				bt_set_reg(bt, BPF_REG_2);
 			/* r6-r9 and stack slots will stay set in caller frame
 			 * bitmasks until we return back from callee(s)
 			 */
Signature · 1 line
2.53.0-Meta
#12 lore raw

Re: [PATCH bpf-next v6 04/10] bpf: Handle R2 as a return register in precision backtracking

This looks like a bug fix for 16-byte return support. Should this include:
Quoted 23 lines

bpf: Handle R2 as a return register in precision backtracking

Precision backtracking treats only R0 as a return register at a call/return boundary, so once the verifier starts modeling R2 that way, marking the second half of such a return precise would trip the "unexpected regs" checks in backtrack_insn() and reject a valid program.

Marking the upper half precise, for example by branching on it after a call to a static subprogram, walks backtracking into the callee and reaches its BPF_EXIT with R2 still set in the mask. Handle R2 like R0 in boundaries where a call defines the return registers.

R2 differs from R0 in that it is an argument register as well, so it is part of the BPF_REGMASK_ARGS check and has to be cleared before that check rather than next to R0. Clear it unconditionally, rather than only where the callee or the kfunc really does return a pair. That gives up the "unexpected regs" assertion for R2, and in exchange keeps backtracking free of any BTF lookup. Nothing is lost: a callee that does not return a pair leaves the caller's R2 uninitialized, so the main verification pass has already rejected any program that reads it, and backtracking is never asked for its precision.

Quoted 7 lines

At BPF_EXIT the return registers are sampled before the callback path clears R1-R5. That clear does not touch R0, but it does cover R2, and running it first would drop a pair return whenever the instruction following the call happens to be one that invokes a callback.

This looks like a bug fix for 16-byte return support. Should this include:

Fixes 162762023c59 ("bpf: Add helpers to describe the R0:R2 return register pair")
Quoted 4 lines
kernel/bpf/backtrack.c +0 -0
index a2b18a9f1694..653db80bcc47 100644

[ ... ]

Quoted 14 lines
kernel/bpf/backtrack.c +9 -1
@@ -520,7 +524,25 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 					return -EFAULT;
 			}
 		} else if (opcode == BPF_EXIT) {
-			bool r0_precise;
+			bool from_subprog_call, r0_precise, r2_precise;
+
+			/* BPF_EXIT in subprog or callback always returns
+			 * right after the call instruction, so by checking
+			 * whether the instruction at subseq_idx-1 is subprog
+			 * call or not we can distinguish actual exit from
+			 * *subprog* from exit from *callback*. In the former
+			 * case, we need to propagate the precision of the
+			 * return registers, if necessary. In the latter we

kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 545

Quoted 10 lines
kernel/bpf/backtrack.c +10 -0
+			 * never do that.
+			 */
+			from_subprog_call = subseq_idx - 1 >= 0 &&
+					    bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+
+			/* Sample the return registers before the callback
+			 * handling below clears R1-R5.
+			 */
+			r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
+			r2_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_2);

A subsystem pattern flags this as potentially concerning: the multi-line comment above starts its text on the opening /* line:

	/* Sample the return registers before the callback
	 * handling below clears R1-R5.
	 */

The BPF subsystem guide suggests putting /* on its own line for multi-line comments under kernel/bpf/:

	/*
	 * Sample the return registers before the callback
	 * handling below clears R1-R5.
	 */

Every existing multi-line comment in backtrack.c uses the older form, including the comment block that this same hunk moves, so matching the file's dominant style is a plausible reason to write it this way. Does the newer form matter here?

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

author reviewer
#13 lore raw

Re: [PATCH bpf-next v6 04/10] bpf: Handle R2 as a return register in precision backtracking

Quoted 28 lines · On 8/16/26 10:17 PM, [email protected] wrote:
Quoted 28 lines

bpf: Handle R2 as a return register in precision backtracking

Precision backtracking treats only R0 as a return register at a call/return boundary, so once the verifier starts modeling R2 that way, marking the second half of such a return precise would trip the "unexpected regs" checks in backtrack_insn() and reject a valid program.

Marking the upper half precise, for example by branching on it after a call to a static subprogram, walks backtracking into the callee and reaches its BPF_EXIT with R2 still set in the mask. Handle R2 like R0 in boundaries where a call defines the return registers.

R2 differs from R0 in that it is an argument register as well, so it is part of the BPF_REGMASK_ARGS check and has to be cleared before that check rather than next to R0. Clear it unconditionally, rather than only where the callee or the kfunc really does return a pair. That gives up the "unexpected regs" assertion for R2, and in exchange keeps backtracking free of any BTF lookup. Nothing is lost: a callee that does not return a pair leaves the caller's R2 uninitialized, so the main verification pass has already rejected any program that reads it, and backtracking is never asked for its precision.

At BPF_EXIT the return registers are sampled before the callback path clears R1-R5. That clear does not touch R0, but it does cover R2, and running it first would drop a pair return whenever the instruction following the call happens to be one that invokes a callback.

Quoted 6 lines
Quoted 2 lines

This looks like a bug fix for 16-byte return support. Should this include:

Fixes 162762023c59 ("bpf: Add helpers to describe the R0:R2 return register pair")

This Fixes tag is not needed, esp. it is in the same series.

Quoted 46 lines
Quoted 5 lines
kernel/bpf/backtrack.c +0 -0
index a2b18a9f1694..653db80bcc47 100644

[ ... ]

Quoted 14 lines
kernel/bpf/backtrack.c +9 -1
@@ -520,7 +524,25 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
  					return -EFAULT;
  			}
  		} else if (opcode == BPF_EXIT) {
-			bool r0_precise;
+			bool from_subprog_call, r0_precise, r2_precise;
+
+			/* BPF_EXIT in subprog or callback always returns
+			 * right after the call instruction, so by checking
+			 * whether the instruction at subseq_idx-1 is subprog
+			 * call or not we can distinguish actual exit from
+			 * *subprog* from exit from *callback*. In the former
+			 * case, we need to propagate the precision of the
+			 * return registers, if necessary. In the latter we

kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 545

Quoted 10 lines
kernel/bpf/backtrack.c +10 -0
+			 * never do that.
+			 */
+			from_subprog_call = subseq_idx - 1 >= 0 &&
+					    bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+
+			/* Sample the return registers before the callback
+			 * handling below clears R1-R5.
+			 */
+			r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
+			r2_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_2);

A subsystem pattern flags this as potentially concerning: the multi-line comment above starts its text on the opening /* line:

	/* Sample the return registers before the callback
	 * handling below clears R1-R5.
	 */

The BPF subsystem guide suggests putting /* on its own line for multi-line comments under kernel/bpf/:

	/*
	 * Sample the return registers before the callback
	 * handling below clears R1-R5.
	 */
Quoted 4 lines

Every existing multi-line comment in backtrack.c uses the older form, including the comment block that this same hunk moves, so matching the file's dominant style is a plausible reason to write it this way. Does the newer form matter here?

I think it is okay. The new comment format is from the old one.

Quoted 8 lines

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

maintainer Acked-by on v6 4/10
#14 lore raw

Re: [PATCH bpf-next v6 04/10] bpf: Handle R2 as a return register in precision backtracking

In reply to Yonghong Song
Quoted 21 lines · Yonghong Song

Precision backtracking treats only R0 as a return register at a call/return boundary, so once the verifier starts modeling R2 that way, marking the second half of such a return precise would trip the "unexpected regs" checks in backtrack_insn() and reject a valid program.

Marking the upper half precise, for example by branching on it after a call to a static subprogram, walks backtracking into the callee and reaches its BPF_EXIT with R2 still set in the mask. Handle R2 like R0 in boundaries where a call defines the return registers.

R2 differs from R0 in that it is an argument register as well, so it is part of the BPF_REGMASK_ARGS check and has to be cleared before that check rather than next to R0. Clear it unconditionally, rather than only where the callee or the kfunc really does return a pair. That gives up the "unexpected regs" assertion for R2, and in exchange keeps backtracking free of any BTF lookup. Nothing is lost: a callee that does not return a pair leaves the caller's R2 uninitialized, so the main verification pass has already rejected any program that reads it, and backtracking is never asked for its precision.

Quoted 8 lines

At BPF_EXIT the return registers are sampled before the callback path clears R1-R5. That clear does not touch R0, but it does cover R2, and running it first would drop a pair return whenever the instruction following the call happens to be one that invokes a callback.

---

...

author reviewer
#15 lore raw

PATCH v6 5/10 bpf: Account R2 of register-pair returns in live register analysis

In reply to Yonghong Song

A BPF_EXIT of a subprogram returning a value larger than 8 bytes (a struct/union or an __int128) reads R2 as well as R0, since the second half of the return value is passed back in R2. compute_insn_live_regs() only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.

Mark R2 as read at the BPF_EXIT of a subprogram that does return a register pair. bpf_compute_live_registers() now loops over the subprograms and, for each, over the [start, end) instruction range from env->subprog_info[], so the return convention is queried once per subprogram through bpf_ret_reg_pair() rather than once per instruction.

Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.

---
 kernel/bpf/liveness.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
kernel/bpf/liveness.c +14 -6
index 74fc4b3f80d6..71f998c6eb88 100644
@@ -2060,7 +2060,8 @@ static inline u16 mask_hi(u32 m) { return (u16)(m >> 16); }
 /* Compute info->{use,def} fields for the instruction */
 static void compute_insn_live_regs(struct bpf_verifier_env *env,
 				   struct bpf_insn *insn,
-				   struct insn_live_regs *info)
+				   struct insn_live_regs *info,
+				   bool ret_reg_pair)
 {
 	struct bpf_call_summary cs;
 	const u8 class = BPF_CLASS(insn->code);
@@ -2072,6 +2073,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 	const u32 src32 = mask_lo(src);
 	const u32 dst32 = mask_lo(dst);
 	const u32 r0  = reg64_mask(0);
+	const u32 r2  = reg64_mask(BPF_REG_2);
 	u32 def = 0;
 	u32 use = U32_MAX;
 
@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 			break;
 		case BPF_EXIT:
 			def = 0;
-			use = r0;
+			use = ret_reg_pair ? (r0 | r2) : r0;
 			break;
 		case BPF_CALL:
 			def = ALL_CALLER_SAVED_REGS;
@@ -2228,8 +2230,8 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 	struct insn_live_regs *state;
 	int insn_cnt = env->prog->len;
 	u64 pos, insn_pos;
-	int err = 0, i, j;
-	bool changed;
+	int err = 0, i, j, subprog, start, end;
+	bool changed, ret_reg_pair;
 
 	/* Use the following algorithm:
 	 * - define the following:
@@ -2256,8 +2258,14 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 		goto out;
 	}
 
-	for (i = 0; i < insn_cnt; ++i)
-		compute_insn_live_regs(env, &insns[i], &state[i]);
+	for (subprog = 0; subprog < env->subprog_cnt; subprog++) {
+		start = env->subprog_info[subprog].start;
+		end = env->subprog_info[subprog + 1].start;
+		ret_reg_pair = bpf_ret_reg_pair(env, subprog);
+
+		for (i = start; i < end; ++i)
+			compute_insn_live_regs(env, &insns[i], &state[i], ret_reg_pair);
+	}
 
 	/* Forward pass: resolve stack access through FP-derived pointers */
 	err = bpf_compute_subprog_arg_access(env);
Signature · 1 line
2.53.0-Meta
#16 lore raw

Re: [PATCH bpf-next v6 05/10] bpf: Account R2 of register-pair returns in live register analysis

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Critical] When a BPF program with 16-byte returns falls back to the interpreter (e.g., if JIT is…

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] When a BPF program with 16-byte returns falls back to the interpreter (e.g., if JIT is disabled), the interpreter silently discards the upper 8 bytes (R2) of the return value, causing runtime data corruption.
- [High] The verifier fails to propagate or initialize BPF_REG_2 back to the caller for subprograms and kfuncs returning 16-byte values, rendering the feature unusable.
- [High] The verifier fails to validate the safety of BPF_REG_2 at the exit of global subprograms returning 16-byte values, allowing potential pointer leaks or returning uninitialized memory. --

--- Patch [5]: [PATCH bpf-next v6 05/10] bpf: Account R2 of register-pair returns in live register analysis --- commit 1e84baf022beac38fd11fa1a392f1c5622154586
Author: Yonghong Song <[email protected]>

bpf: Account R2 of register-pair returns in live register analysis

This commit updates the live register analysis to account for subprograms returning a 16-byte register pair. It modifies compute_insn_live_regs() to mark BPF_REG_2 as read during BPF_EXIT when a subprogram returns a register pair.

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
index 74fc4b3f80d6e..71f998c6eb888 100644

[ ... ]

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2197

Quoted 7 lines
kernel/bpf/liveness.c +1 -1
@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 			break;
 		case BPF_EXIT:
 			def = 0;
-			use = r0;
+			use = ret_reg_pair ? (r0 | r2) : r0;
 			break;

[Severity: High]
How does the verifier propagate BPF_REG_2 back to the caller for subprograms and kfuncs returning 16-byte values?

In kernel/bpf/verifier.c:prepare_func_exit(), only BPF_REG_0 is copied from the callee's state to the caller's state:

kernel/bpf/verifier.c:prepare_func_exit() {
	...
	/* return to the caller whatever r0 had in the callee */
	bpf_diag_mod_begin(env, &caller->regs[BPF_REG_0], r0, BPF_DIAG_MOD_WRITE);
	caller->regs[BPF_REG_0] = *r0;
	bpf_diag_mod_end(env);
	...
}

Similarly, when returning from a non-void global subprogram or a kfunc, only BPF_REG_0 is initialized as a scalar. For instance, in check_func_call():

kernel/bpf/verifier.c:check_func_call() {
	...
	/* All non-void global functions return a 64-bit SCALAR_VALUE. */
	if (!returns_void) {
		mark_reg_unknown(env, caller->regs, BPF_REG_0);
		bpf_diag_mod_end(env);
	}
	...
}

And in check_kfunc_call():

kernel/bpf/verifier.c:check_kfunc_call() {
	...
	if (btf_type_is_scalar(t)) {
		mark_reg_unknown(env, regs, BPF_REG_0);
	...
}

Since clear_caller_saved_regs() marks R1-R5 as NOT_INIT prior to the call, would this leave the caller's BPF_REG_2 uninitialized, causing the verifier to reject legitimate reads of the upper 8 bytes of the return value?

[Severity: High]
Is the safety of BPF_REG_2 validated at the exit of global subprograms returning 16-byte values?

In kernel/bpf/verifier.c:check_global_subprog_return_code(), the return code validation is only performed for BPF_REG_0:

kernel/bpf/verifier.c:check_global_subprog_return_code() {
	...
	return check_global_ret_scalar_reg(env, BPF_REG_0);
}

Could this omit BPF_REG_2 for register-pair returns, potentially allowing a global subprogram to return an unreferenced pointer, a map value pointer, or uninitialized memory without being caught by the verifier?

[Severity: Critical]
What happens if a BPF program with 16-byte returns falls back to the interpreter?

If JIT is not requested, bpf_compute_subprog_ret_regs() exits early without setting env->prog->jit_required = 1:

kernel/bpf/verifier.c:bpf_compute_subprog_ret_regs() {
	...
	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
		return 0;
	...
}

This permits the program to fall back to the interpreter. However, the interpreter handles subprogram calls via JMP_CALL_ARGS, which only propagates BPF_R0:

kernel/bpf/core.c:___bpf_prog_run() {
	...
	JMP_CALL_ARGS:
		BPF_R0 = interpreters_args[insn->off](BPF_R1, BPF_R2, BPF_R3,
						      BPF_R4, BPF_R5,
						      insn + insn->imm + 1);
		CONT;
	...
}

Since the PROG_NAME_ARGS macro generates functions that return a single u64, does the interpreter silently discard the upper 8 bytes (R2) of the return value, causing data corruption at runtime?

Quoted 2 lines
kernel/bpf/liveness.c +0 -0
 		case BPF_CALL:
 			def = ALL_CALLER_SAVED_REGS;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
author reviewer
#17 lore raw

Re: [PATCH bpf-next v6 05/10] bpf: Account R2 of register-pair returns in live register analysis

Quoted 67 lines · On 8/16/26 9:43 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] When a BPF program with 16-byte returns falls back to the interpreter (e.g., if JIT is disabled), the interpreter silently discards the upper 8 bytes (R2) of the return value, causing runtime data corruption.
- [High] The verifier fails to propagate or initialize BPF_REG_2 back to the caller for subprograms and kfuncs returning 16-byte values, rendering the feature unusable.
- [High] The verifier fails to validate the safety of BPF_REG_2 at the exit of global subprograms returning 16-byte values, allowing potential pointer leaks or returning uninitialized memory. --

--- Patch [5]: [PATCH bpf-next v6 05/10] bpf: Account R2 of register-pair returns in live register analysis --- commit 1e84baf022beac38fd11fa1a392f1c5622154586
Author: Yonghong Song <[email protected]>

bpf: Account R2 of register-pair returns in live register analysis

This commit updates the live register analysis to account for subprograms returning a 16-byte register pair. It modifies compute_insn_live_regs() to mark BPF_REG_2 as read during BPF_EXIT when a subprogram returns a register pair.

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
index 74fc4b3f80d6e..71f998c6eb888 100644

[ ... ]

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2197

Quoted 7 lines
kernel/bpf/liveness.c +1 -1
@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
  			break;
  		case BPF_EXIT:
  			def = 0;
-			use = r0;
+			use = ret_reg_pair ? (r0 | r2) : r0;
  			break;

[Severity: High]
How does the verifier propagate BPF_REG_2 back to the caller for subprograms and kfuncs returning 16-byte values?

In kernel/bpf/verifier.c:prepare_func_exit(), only BPF_REG_0 is copied from the callee's state to the caller's state:

kernel/bpf/verifier.c:prepare_func_exit() {
	...
	/* return to the caller whatever r0 had in the callee */
	bpf_diag_mod_begin(env, &caller->regs[BPF_REG_0], r0, BPF_DIAG_MOD_WRITE);
	caller->regs[BPF_REG_0] = *r0;
	bpf_diag_mod_end(env);
	...
}

Similarly, when returning from a non-void global subprogram or a kfunc, only BPF_REG_0 is initialized as a scalar. For instance, in check_func_call():

kernel/bpf/verifier.c:check_func_call() {
	...
	/* All non-void global functions return a 64-bit SCALAR_VALUE. */
	if (!returns_void) {
		mark_reg_unknown(env, caller->regs, BPF_REG_0);
		bpf_diag_mod_end(env);
	}
	...
}

And in check_kfunc_call():

kernel/bpf/verifier.c:check_kfunc_call() {
	...
	if (btf_type_is_scalar(t)) {
		mark_reg_unknown(env, regs, BPF_REG_0);
	...
}
Quoted 3 lines

Since clear_caller_saved_regs() marks R1-R5 as NOT_INIT prior to the call, would this leave the caller's BPF_REG_2 uninitialized, causing the verifier to reject legitimate reads of the upper 8 bytes of the return value?

This should be okay. The BPF_REG_2 is introduced in later patch
bpf: Add verifier support for 16-byte returns in R0:R2

Quoted 8 lines

[Severity: High]
Is the safety of BPF_REG_2 validated at the exit of global subprograms returning 16-byte values?

In kernel/bpf/verifier.c:check_global_subprog_return_code(), the return code validation is only performed for BPF_REG_0:

Quoted 8 lines
kernel/bpf/verifier.c:check_global_subprog_return_code() {
	...
	return check_global_ret_scalar_reg(env, BPF_REG_0);
}

Could this omit BPF_REG_2 for register-pair returns, potentially allowing a global subprogram to return an unreferenced pointer, a map value pointer, or uninitialized memory without being caught by the verifier?

The same as above. The BPF_REG_2 is introduced in later patch
bpf: Add verifier support for 16-byte returns in R0:R2

Quoted 29 lines

[Severity: Critical]
What happens if a BPF program with 16-byte returns falls back to the interpreter?

If JIT is not requested, bpf_compute_subprog_ret_regs() exits early without setting env->prog->jit_required = 1:

kernel/bpf/verifier.c:bpf_compute_subprog_ret_regs() {
	...
	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
		return 0;
	...
}

This permits the program to fall back to the interpreter. However, the interpreter handles subprogram calls via JMP_CALL_ARGS, which only propagates BPF_R0:

kernel/bpf/core.c:___bpf_prog_run() {
	...
	JMP_CALL_ARGS:
		BPF_R0 = interpreters_args[insn->off](BPF_R1, BPF_R2, BPF_R3,
						      BPF_R4, BPF_R5,
						      insn + insn->imm + 1);
		CONT;
	...
}
Quoted 6 lines

Since the PROG_NAME_ARGS macro generates functions that return a single u64, does the interpreter silently discard the upper 8 bytes (R2) of the return value, causing data corruption at runtime?

Quoted 2 lines
kernel/bpf/liveness.c +0 -0
  		case BPF_CALL:
  			def = ALL_CALLER_SAVED_REGS;

In such case (jit not requested), verifier will fail. For example, suppose the subprog will return R0:R2, but in interpreter mode, only R0 is returned. R2 is not initialized. In caller, when trying to return R2, but R2 is not initialized, so verifier will fail.

author reviewer
#18 lore raw

PATCH v6 6/10 bpf: Add verifier support for 16-byte returns in R0:R2

In reply to Yonghong Song

LLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].

Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:

error: aggregate returns are not supported

and an __int128 return failed later in the backend with:

fatal error: error in backend: unable to allocate function return #1

Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.

Model that pair at calls to global and static BPF subprograms and at kfunc
calls: R2 is marked alongside R0 at the call, propagated out of a callee at its exit, and held to the same scalar-only and no-stack-pointer rules that already apply to R0. A struct returned by a kfunc must be composed of scalars, since its bytes reach the program as raw register contents and a pointer field would otherwise be laundered into a scalar.

An extension program is the one caller of the convention that cannot take part in it: only R0 is checked at its exit, since that is the program exit code, so it has no way to hand back an upper half. Replacing a function whose return value is larger than 8 bytes is therefore rejected in btf_check_func_type_match(). That function compares btf_type->info, which carries no size for an int, so an 8 byte long and a 16 byte __int128 compared equal; sizes up to 8 bytes stay interchangeable, as they always have been, so this only rejects what the R0:R2 convention newly makes incompatible.

[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876

---
 kernel/bpf/btf.c      |  6 ++++
 kernel/bpf/verifier.c | 68 ++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 67 insertions(+), 7 deletions(-)
kernel/bpf/btf.c +6 -0
index 5b9d767895c9..2ae7cb9b30f2 100644
@@ -7684,6 +7684,12 @@ static int btf_check_func_type_match(struct bpf_verifier_log *log,
 			btf_type_str(t2), fn2);
 		return -EINVAL;
 	}
+	if (btf_type_has_size(t1) && (t1->size > 8 || t2->size > 8)) {
+		bpf_log(log,
+			"Return type of %s() has size %u while %s() has size %u, and a size above 8 bytes cannot be replaced\n",
+			fn1, t1->size, fn2, t2->size);
+		return -EINVAL;
+	}
 
 	for (i = 0; i < nargs1; i++) {
 		t1 = btf_type_skip_modifiers(btf1, args1[i].type, NULL);
kernel/bpf/verifier.c +61 -7

129 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#19 lore raw

Re: [PATCH bpf-next v6 06/10] bpf: Add verifier support for 16-byte returns in R0:R2

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The newly added support for kfuncs returning structs by value is mathematically unreachable. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added support for kfuncs returning structs by value is mathematically unreachable. --

--- Patch [6]: [PATCH bpf-next v6 06/10] bpf: Add verifier support for 16-byte returns in R0:R2 --- commit 438f13b526c411cf77d7536a3436a0cc4625dbe4
Author: Yonghong Song <[email protected]>

bpf: Add verifier support for 16-byte returns in R0:R2

This patch adds BPF verifier support for handling 16-byte returns in the R0:R2 register pair, utilizing LLVM 23 capabilities. It models this register pair at calls to global/static subprograms and kfuncs, allowing kfuncs to return structs composed of scalars.

Quoted 3 lines
kernel/bpf/verifier.c +0 -0

[ ... ]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14004

Quoted 10 lines
kernel/bpf/verifier.c +2 -1
@@ -13967,10 +13997,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	}
 
 	if (btf_type_is_scalar(t)) {
-		mark_reg_unknown(env, regs, BPF_REG_0);
+		mark_kfunc_ret_regs(env, regs, t->size);
 		if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
 		    meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))
 			__mark_reg_const_zero(env, &regs[BPF_REG_0]);
+	} else if (btf_type_is_struct(t)) {

[Severity: Medium]
Is this newly added code for kfunc struct returns actually reachable during program load?

Looking at how kfuncs are processed earlier in the verifier, bpf_add_kfunc_call() calls btf_distill_func_proto(), which appears to still explicitly reject struct return types:

kernel/bpf/btf.c:btf_distill_func_proto() {
    ...
    if (ret < 0 || btf_type_is_struct(t)) {
        bpf_log(log,
            "The function %s return type %s is unsupported.\n",
            tname, btf_type_str(t));
        return -EINVAL;
    }
    ...
}

Since btf_distill_func_proto() is called during program load before check_kfunc_call() is ever reached, won't any BPF program calling a struct-returning kfunc be unconditionally rejected, making this new branch dead code?

Quoted 16 lines
kernel/bpf/verifier.c +14 -0
+		/*
+		 * The returned struct comes back as raw register bits modeled
+		 * as an unknown scalar, so it must contain only scalars:
+		 * otherwise a pointer field would be laundered into a scalar
+		 * and escape provenance and reference tracking.
+		 */
+		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+			verbose(env,
+				"kernel function %s returns %s %s that is not composed of scalars\n",
+				func_name, btf_type_str(t),
+				btf_name_by_offset(desc_btf, t->name_off));
+			return -EINVAL;
+		}
+		mark_kfunc_ret_regs(env, regs, t->size);
 	} else if (btf_type_is_ptr(t)) {
 		ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id);

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author reviewer
#20 lore raw

Re: [PATCH bpf-next v6 06/10] bpf: Add verifier support for 16-byte returns in R0:R2

Quoted 52 lines · On 8/16/26 9:42 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added support for kfuncs returning structs by value is mathematically unreachable. --

--- Patch [6]: [PATCH bpf-next v6 06/10] bpf: Add verifier support for 16-byte returns in R0:R2 --- commit 438f13b526c411cf77d7536a3436a0cc4625dbe4
Author: Yonghong Song <[email protected]>

bpf: Add verifier support for 16-byte returns in R0:R2

This patch adds BPF verifier support for handling 16-byte returns in the R0:R2 register pair, utilizing LLVM 23 capabilities. It models this register pair at calls to global/static subprograms and kfuncs, allowing kfuncs to return structs composed of scalars.

Quoted 3 lines
kernel/bpf/verifier.c +0 -0

[ ... ]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14004

Quoted 10 lines
kernel/bpf/verifier.c +2 -1
@@ -13967,10 +13997,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
  	}
 
  	if (btf_type_is_scalar(t)) {
-		mark_reg_unknown(env, regs, BPF_REG_0);
+		mark_kfunc_ret_regs(env, regs, t->size);
  		if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
  		    meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))
  			__mark_reg_const_zero(env, &regs[BPF_REG_0]);
+	} else if (btf_type_is_struct(t)) {

[Severity: Medium]
Is this newly added code for kfunc struct returns actually reachable during program load?

Looking at how kfuncs are processed earlier in the verifier, bpf_add_kfunc_call() calls btf_distill_func_proto(), which appears to still explicitly reject struct return types:

kernel/bpf/btf.c:btf_distill_func_proto() {
     ...
     if (ret < 0 || btf_type_is_struct(t)) {
         bpf_log(log,
             "The function %s return type %s is unsupported.\n",
             tname, btf_type_str(t));
         return -EINVAL;
     }
     ...
}
Quoted 4 lines

Since btf_distill_func_proto() is called during program load before check_kfunc_call() is ever reached, won't any BPF program calling a struct-returning kfunc be unconditionally rejected, making this new branch dead code?

The next patch "bpf: Enable aggregate return types up to 16 bytes" will enable to have successful prog load.

Quoted 19 lines
Quoted 17 lines
kernel/bpf/verifier.c +14 -0
+		/*
+		 * The returned struct comes back as raw register bits modeled
+		 * as an unknown scalar, so it must contain only scalars:
+		 * otherwise a pointer field would be laundered into a scalar
+		 * and escape provenance and reference tracking.
+		 */
+		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+			verbose(env,
+				"kernel function %s returns %s %s that is not composed of scalars\n",
+				func_name, btf_type_str(t),
+				btf_name_by_offset(desc_btf, t->name_off));
+			return -EINVAL;
+		}
+		mark_kfunc_ret_regs(env, regs, t->size);
  	} else if (btf_type_is_ptr(t)) {
  		ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id);

[ ... ]

#21 lore raw

Re: [PATCH bpf-next v6 06/10] bpf: Add verifier support for 16-byte returns in R0:R2

In reply to Yonghong Song
This isn't a bug, but since this check fires on the absolute size rather than on a mismatch, the message can end up reading "has size 16 while g() has size 16". Would it be clearer to state which…
Quoted 5 lines
kernel/bpf/btf.c +0 -0
index 5b9d767895c9..2ae7cb9b30f2 100644

kernel/bpf/btf.c static int btf_check_func_type_match(struct bpf_verifier_log *log, · line 7692

Quoted 9 lines
kernel/bpf/btf.c +6 -0
@@ -7684,6 +7684,12 @@ static int btf_check_func_type_match(struct bpf_verifier_log *log,
 			btf_type_str(t2), fn2);
 		return -EINVAL;
 	}
+	if (btf_type_has_size(t1) && (t1->size > 8 || t2->size > 8)) {
+		bpf_log(log,
+			"Return type of %s() has size %u while %s() has size %u, and a size above 8 bytes cannot be replaced\n",
+			fn1, t1->size, fn2, t2->size);
+		return -EINVAL;
+	}

This isn't a bug, but since this check fires on the absolute size rather than on a mismatch, the message can end up reading "has size 16 while g() has size 16". Would it be clearer to state which side exceeds 8 bytes instead of using the "X while Y" mismatch phrasing the neighbouring diagnostics use?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

author reviewer
#22 lore raw

Re: [PATCH bpf-next v6 06/10] bpf: Add verifier support for 16-byte returns in R0:R2

Quoted 11 lines · On 8/16/26 10:17 PM, [email protected] wrote:
Quoted 11 lines
kernel/bpf/btf.c +3 -0
index 5b9d767895c9..2ae7cb9b30f2 100644
@@ -7684,6 +7684,12 @@ static int btf_check_func_type_match(struct bpf_verifier_log *log,
  			btf_type_str(t2), fn2);
  		return -EINVAL;
  	}
+	if (btf_type_has_size(t1) && (t1->size > 8 || t2->size > 8)) {
+		bpf_log(log,
+			"Return type of %s() has size %u while %s() has size %u, and a size above 8 bytes cannot be replaced\n",
Quoted 8 lines

kernel/bpf/btf.c static int btf_check_func_type_match(struct bpf_verifier_log *log, · line 7692

Quoted 3 lines
kernel/bpf/btf.c +3 -0
+			fn1, t1->size, fn2, t2->size);
+		return -EINVAL;
+	}

This isn't a bug, but since this check fires on the absolute size rather than on a mismatch, the message can end up reading "has size 16 while g() has size 16". Would it be clearer to state which side exceeds 8 bytes instead of using the "X while Y" mismatch phrasing the neighbouring diagnostics use?

Okay, I see. Maybe
"Return type of %s() has size %u and of %s() size %u, and a size above 8 bytes cannot be replaced\n"
?
I guess the intention is to explicitly mention which function exceeds 8 bytes.
User should be able to know this by checking original func and freplace prog?

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

author reviewer
#23 lore raw

PATCH v6 7/10 bpf: Enable aggregate return types up to 16 bytes

In reply to Yonghong Song

Relax btf_distill_func_proto() to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry:

- a struct or union larger than 8 and up to 16 bytes, returned in the R0:R2 register pair, matching what LLVM emits for the BPF target;
- a struct or union up to 8 bytes, returned in R0 alone.

A >8 byte scalar (__int128) was already accepted and is unchanged. Everything else stays rejected: a return type larger than 16 bytes, and any type that __get_type_size() cannot return in registers at all (e.g. an array), which it already reports as ret < 0.

btf_validate_return_type() is relaxed as well, so that it accepts a by-value struct or union up to 16 bytes in addition to void and scalars.

With btf_distill_func_proto() and btf_validate_return_type() relaxed, the verifier, JIT, precision-backtracking and live-register support from the earlier patches becomes reachable: <=16 byte aggregate return values now work end to end.

---
 include/linux/bpf_verifier.h                  |  2 ++
 kernel/bpf/btf.c                              | 23 +++++++++++++++----
 kernel/bpf/verifier.c                         | 18 +++++++--------
 .../selftests/bpf/progs/exceptions_fail.c     |  2 +-
 4 files changed, 30 insertions(+), 15 deletions(-)
include/linux/bpf_verifier.h +2 -0
index f70d5878fbff..938c9a9eb9d2 100644
@@ -1467,6 +1467,8 @@ int bpf_jmp_offset(struct bpf_insn *insn);
 struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);
 void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);
 bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog);
+bool btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,
+			       const struct btf_type *t, int rec);
 
 int bpf_find_subprog(struct bpf_verifier_env *env, int off);
 bool bpf_is_throw_kfunc(struct bpf_insn *insn);
kernel/bpf/btf.c +18 -5
index 2ae7cb9b30f2..ba83fa1d52a3 100644
@@ -7591,7 +7591,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,
 		return -EINVAL;
 	}
 	ret = __get_type_size(btf, func->type, &t);
-	if (ret < 0 || btf_type_is_struct(t)) {
+	if (ret < 0 || ret > 16) {
 		bpf_log(log,
 			"The function %s return type %s is unsupported.\n",
 			tname, btf_type_str(t));
@@ -7970,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,
 
 /* Check whether the type is a valid return type. */
 static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btf,
-		const struct btf_type *t, int subprog)
+		const struct btf_type *t, int subprog, bool is_global)
 {
 	u32 tags = 0;
 	int err;
@@ -7993,6 +7993,19 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
 	if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
 		return 0;
 
+	if (btf_type_is_struct(t) && t->size <= 16) {
+		/*
+		 * A global function's caller models the return as an opaque
+		 * scalar pair, so it may only return scalars by value. A local
+		 * function is verified inline, so a pointer field stays tracked
+		 * and needs no such restriction.
+		 */
+		bool local_func = subprog && !is_global;
+
+		if (local_func || btf_type_is_scalar_struct(env, btf, t, 0))
+			return 0;
+	}
+
 	return -EOPNOTSUPP;
 }
 
@@ -8080,12 +8093,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 		return -EINVAL;
 	}
 
-	err = btf_validate_return_type(env, btf, t, subprog);
+	err = btf_validate_return_type(env, btf, t, subprog, is_global);
 	if (err) {
 		if (is_global) {
 			bpf_log(log,
-				"Global function %s() return value not void or scalar. "
-				"Only those are supported.\n",
+				"Global function %s() has unsupported return type. "
+				"Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\n",
 				tname);
 		}
 		return err;
kernel/bpf/verifier.c +9 -9
index e371b7e27ec9..e5c8d8f7d474 100644
@@ -11600,9 +11600,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_
 }
 
 /* Returns true if struct is composed of scalars, 4 levels of nesting allowed */
-static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
-					const struct btf *btf,
-					const struct btf_type *t, int rec)
+bool btf_type_is_scalar_struct(struct bpf_verifier_env *env,
+			       const struct btf *btf,
+			       const struct btf_type *t, int rec)
 {
 	const struct btf_type *member_type;
 	const struct btf_member *member;
@@ -11620,7 +11620,7 @@ static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
 				verbose(env, "max struct nesting depth exceeded\n");
 				return false;
 			}
-			if (!__btf_type_is_scalar_struct(env, btf, member_type, rec + 1))
+			if (!btf_type_is_scalar_struct(env, btf, member_type, rec + 1))
 				return false;
 			continue;
 		}
@@ -12019,7 +12019,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 		 (is_kfunc_arg_mem_size(meta->btf, &args[arg + 1]) ||
 		  is_kfunc_arg_const_mem_size(meta->btf, &args[arg + 1]))) {
 		if (!btf_type_is_void(ref_t) && !btf_type_is_scalar(ref_t) &&
-		    !__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
+		    !btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
 			verbose(env, "%s pointer type %s %s must point to void, scalar, or struct with scalar\n",
 				reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);
 			return -EINVAL;
@@ -12035,7 +12035,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 		 * scalars. The access size is derived from the pointed-to BTF type.
 		 */
 		if (!btf_type_is_scalar(ref_t) &&
-		    !__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
+		    !btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
 			verbose(env, "%s pointer type %s %s must point to scalar, or struct with scalar\n",
 				reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);
 			return -EINVAL;
@@ -13091,7 +13091,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
 				break;
 			}
 
-			if (!__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
+			if (!btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
 				enum bpf_reg_type reg2btf_type = lookup_reg2btf_ids(ref_id);
 				const char *expected_type;
 
@@ -13629,7 +13629,7 @@ static int check_special_kfunc(struct bpf_verifier_env *env, struct bpf_call_arg
 
 		struct_meta = btf_find_struct_meta(ret_btf, ret_btf_id);
 		if (is_bpf_percpu_obj_new_kfunc(meta->func_id)) {
-			if (!__btf_type_is_scalar_struct(env, ret_btf, ret_t, 0)) {
+			if (!btf_type_is_scalar_struct(env, ret_btf, ret_t, 0)) {
 				verbose(env, "bpf_percpu_obj_new type ID argument must be of a struct of scalars\n");
 				return -EINVAL;
 			}
@@ -14008,7 +14008,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		 * otherwise a pointer field would be laundered into a scalar
 		 * and escape provenance and reference tracking.
 		 */
-		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+		if (!btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
 			verbose(env,
 				"kernel function %s returns %s %s that is not composed of scalars\n",
 				func_name, btf_type_str(t),
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
index ac44d60e5066..9708efb93683 100644
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a)
 
 SEC("?tc")
 __exception_cb(exception_cb_bad_ret_type1)
-__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.")
+__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")
 int reject_exception_cb_type_1(struct __sk_buff *ctx)
 {
 	bpf_throw(0);
Signature · 1 line
2.53.0-Meta
#24 lore raw

Re: [PATCH bpf-next v6 07/10] bpf: Enable aggregate return types up to 16 bytes

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Attaching a BPF trampoline to a function returning a 16-byte struct silently corrupts the return value because the trampoline fails to preserve the second return register.

Pre-existing issues:
- [Medium] Kfuncs returning structs with padding or sizes that are not a multiple of 8 may leak uninitialized kernel memory to BPF programs. --

--- Patch [7]: [PATCH bpf-next v6 07/10] bpf: Enable aggregate return types up to 16 bytes --- commit 5c9b0fdbdfee0f07315dc190bc0c039af9bce8bd
Author: Yonghong Song <[email protected]>

bpf: Enable aggregate return types up to 16 bytes

This commit relaxes the verifier and BTF distillation to accept by-value struct or union return types up to 16 bytes. These are returned in the R0:R2 register pair.

kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 7594

Quoted 9 lines
kernel/bpf/btf.c +1 -1
@@ -7591,7 +7591,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,
 		return -EINVAL;
 	}
 	ret = __get_type_size(btf, func->type, &t);
-	if (ret < 0 || btf_type_is_struct(t)) {
+	if (ret < 0 || ret > 16) {

[Severity: High]
By relaxing this check to allow up to 16-byte returns, does this silently break fentry and freplace programs attached to such functions?

While bpf_check_attach_target() rejects programs if they explicitly use the trampoline retval:

bpf_check_attach_target() {
    if (tgt_info->fmodel.ret_size > 8 &&
        attach_uses_trampoline_retval(prog->expected_attach_type)) {
        ...
    }
}

It appears to still allow fentry and freplace programs to attach. When a BPF program attaches to a kernel function returning a 16-byte struct, the target function returns its value in RAX and RDX (on x86-64).

The BPF trampoline epilogue calls __bpf_tramp_exit(), which clobbers caller-saved registers like RDX.

If the trampoline only restores RAX from the stack before returning, won't RDX be left with garbage, corrupting the upper 8 bytes of the original function's return value?

Quoted 13 lines
kernel/bpf/btf.c +0 -0
 		bpf_log(log,
 			"The function %s return type %s is unsupported.\n",
 			tname, btf_type_str(t));
 
[ ... ]
kernel/bpf/verifier.c +0 -0
 
[ ... ]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14011

Quoted 5 lines
kernel/bpf/verifier.c +1 -1
@@ -14008,7 +14008,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		 * otherwise a pointer field would be laundered into a scalar
 		 * and escape provenance and reference tracking.
 		 */
-		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+		if (!btf_type_is_scalar_struct(env, desc_btf, t, 0)) {

[Severity: Medium]
This isn't a bug introduced by this patch, but since the scope of by-value returns is expanded from 8 bytes to 16 bytes, it might be worth verifying.

In check_kfunc_call(), the verifier immediately calls mark_kfunc_ret_regs() after this check:

check_kfunc_call() {
    if (!btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
        ...
        return -EINVAL;
    }
    mark_kfunc_ret_regs(env, regs, t->size);
}

If a kfunc returns a struct containing padding bytes, the C compiler might leave those padding bytes uninitialized. Since the verifier marks the entire return registers as fully initialized scalars, could this allow a BPF program to read uninitialized kernel memory via the padding bytes?

Quoted 3 lines
kernel/bpf/verifier.c +0 -0
 			verbose(env,
 				"kernel function %s returns %s %s that is not composed of scalars\n",
 				func_name, btf_type_str(t),
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
author reviewer
#25 lore raw

Re: [PATCH bpf-next v6 07/10] bpf: Enable aggregate return types up to 16 bytes

Quoted 46 lines · On 8/16/26 9:42 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Attaching a BPF trampoline to a function returning a 16-byte struct silently corrupts the return value because the trampoline fails to preserve the second return register.

Pre-existing issues:
- [Medium] Kfuncs returning structs with padding or sizes that are not a multiple of 8 may leak uninitialized kernel memory to BPF programs. --

--- Patch [7]: [PATCH bpf-next v6 07/10] bpf: Enable aggregate return types up to 16 bytes --- commit 5c9b0fdbdfee0f07315dc190bc0c039af9bce8bd
Author: Yonghong Song <[email protected]>

bpf: Enable aggregate return types up to 16 bytes

This commit relaxes the verifier and BTF distillation to accept by-value struct or union return types up to 16 bytes. These are returned in the R0:R2 register pair.

kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 7594

Quoted 9 lines
kernel/bpf/btf.c +1 -1
@@ -7591,7 +7591,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,
  		return -EINVAL;
  	}
  	ret = __get_type_size(btf, func->type, &t);
-	if (ret < 0 || btf_type_is_struct(t)) {
+	if (ret < 0 || ret > 16) {

[Severity: High]
By relaxing this check to allow up to 16-byte returns, does this silently break fentry and freplace programs attached to such functions?

While bpf_check_attach_target() rejects programs if they explicitly use the trampoline retval:

bpf_check_attach_target() {
     if (tgt_info->fmodel.ret_size > 8 &&
         attach_uses_trampoline_retval(prog->expected_attach_type)) {
         ...
     }
}

It appears to still allow fentry and freplace programs to attach. When a BPF program attaches to a kernel function returning a 16-byte struct, the target function returns its value in RAX and RDX (on x86-64).

Quoted 6 lines

The BPF trampoline epilogue calls __bpf_tramp_exit(), which clobbers caller-saved registers like RDX.

If the trampoline only restores RAX from the stack before returning, won't RDX be left with garbage, corrupting the upper 8 bytes of the original function's return value?

For fentry, we should be okay as there are no return values.
For freplace, in patch "bpf: Add verifier support for 16-byte returns in R0:R2", the return value cannot be more than 8 bytes.

So we should be okay here.

Quoted 30 lines
Quoted 10 lines
kernel/bpf/btf.c +0 -0
  		bpf_log(log,
  			"The function %s return type %s is unsupported.\n",
  			tname, btf_type_str(t));
[ ... ]
kernel/bpf/verifier.c +0 -0
[ ... ]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14011

Quoted 5 lines
kernel/bpf/verifier.c +1 -1
@@ -14008,7 +14008,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
  		 * otherwise a pointer field would be laundered into a scalar
  		 * and escape provenance and reference tracking.
  		 */
-		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+		if (!btf_type_is_scalar_struct(env, desc_btf, t, 0)) {

[Severity: Medium]
This isn't a bug introduced by this patch, but since the scope of by-value returns is expanded from 8 bytes to 16 bytes, it might be worth verifying.

In check_kfunc_call(), the verifier immediately calls mark_kfunc_ret_regs() after this check:

check_kfunc_call() {
     if (!btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
         ...
         return -EINVAL;
     }
     mark_kfunc_ret_regs(env, regs, t->size);
}
Quoted 4 lines

If a kfunc returns a struct containing padding bytes, the C compiler might leave those padding bytes uninitialized. Since the verifier marks the entire return registers as fully initialized scalars, could this allow a BPF program to read uninitialized kernel memory via the padding bytes?

This should not be a problem. For kfunc, the return value (if size is less than 8), the compiler will do proper narrowing.

Quoted 4 lines
Quoted 4 lines
kernel/bpf/verifier.c +0 -0
  			verbose(env,
  				"kernel function %s returns %s %s that is not composed of scalars\n",
  				func_name, btf_type_str(t),
author reviewer
#26 lore raw

PATCH v6 8/10 selftests/bpf: Add C tests for 16-byte returns in R0:R2

In reply to Yonghong Song

Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return.

The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a __clang_major__ guard. An older compiler builds the dummy test in the #else branch instead, which keeps the object non-empty and says in its description why nothing was exercised.

The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), which bpf_add_kfunc_call() requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_aggregate_ret.c        | 178 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  18 ++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |   9 +
 4 files changed, 207 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 64ac49ad67e6..f7f94ccebce2 100644
@@ -5,6 +5,7 @@
 #include "arena_kfunc.skel.h"
 #include "arena_kfunc_jit.skel.h"
 #include "cap_helpers.h"
+#include "verifier_aggregate_ret.skel.h"
 #include "verifier_align.skel.h"
 #include "verifier_and.skel.h"
 #include "verifier_arena.skel.h"
@@ -170,6 +171,7 @@ void test_arena_kfunc(void)                   { RUN_TESTS(arena_kfunc); }
 
 void test_arena_kfunc_jit(void)               { RUN_TESTS(arena_kfunc_jit); }
 
+void test_verifier_aggregate_ret(void)        { RUN_TESTS(verifier_aggregate_ret); }
 void test_verifier_align(void)                { RUN(verifier_align); }
 void test_verifier_and(void)                  { RUN(verifier_and); }
 void test_verifier_arena(void)                { RUN(verifier_arena); }
tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c new file +178 -0

178 lines not on this page · raw message

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +18 -0
index 9366a3c578f1..c4bc8e11c6e1 100644
@@ -954,6 +954,20 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)
 	return 0;
 }
 
+#if defined(__x86_64__) || defined(__aarch64__)
+__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b)
+{
+	return (__int128)(((unsigned __int128)(a + b) << 64) | (a - b));
+}
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b)
+{
+	struct prog_test_ret_pair r = { .lo = a + b, .hi = a - b };
+
+	return r;
+}
+#endif /* __x86_64__ || __aarch64__ */
+
 __bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,
 					 u64 e, u64 f, u64 g, u64 h,
 					 u64 i, u64 j)
@@ -1487,6 +1501,10 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test3)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test4)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test5)
+#if defined(__x86_64__) || defined(__aarch64__)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair)
+#endif
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +9 -0
index 7d81070eefe7..426000f9a14f 100644
@@ -55,6 +55,11 @@ struct prog_test_big_arg {
 	__u64 b;
 };
 
+struct prog_test_ret_pair {	/* 16 bytes: R0:R2 */
+	__u64 lo;
+	__u64 hi;
+};
+
 struct prog_test_fail1 {
 	void *p;
 	int x;
@@ -131,6 +136,10 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;
 struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;
 long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;
 int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
+#ifdef __SIZEOF_INT128__
+__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
+#endif
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
 __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,
 			       __u64 e, __u64 f, __u64 g, __u64 h,
 			       __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
AI review Acked-by Eduard Zingerman on v6 8/10
#27 lore raw

Re: [PATCH bpf-next v6 08/10] selftests/bpf: Add C tests for 16-byte returns in R0:R2

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The kfuncs `bpf_kfunc_call_test_i128` and `bpf_kfunc_call_test_ret_pair` are conditionally…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The kfuncs bpf_kfunc_call_test_i128 and bpf_kfunc_call_test_ret_pair are conditionally compiled out of bpf_testmod.ko on architectures other than x86_64 and arm64, but are unconditionally compiled and declared as strong __ksym externs in the selftest object verifier_aggregate_ret.c. This will cause libbpf to fail loading the entire object on unsupported architectures (when compiled with LLVM 23+), failing the entire test suite even for programs that don't use the kfuncs. --

--- Patch [8]: [PATCH bpf-next v6 08/10] selftests/bpf: Add C tests for 16-byte returns in R0:R2 --- commit 00f68ada7c423dbadb8514b3c5f6c309f446bc99
Author: Yonghong Song <[email protected]>

selftests/bpf: Add C tests for 16-byte returns in R0:R2

Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return.

The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a __clang_major__ guard. An older compiler builds the dummy test in the #else branch instead, which keeps the object non-empty and says in its description why nothing was exercised.

The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), which bpf_add_kfunc_call() requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.

Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +0 -0
index 7d81070eefe7f..426000f9a14fa 100644

[ ... ]

tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym; · line 142

Quoted 8 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +4 -0
@@ -131,6 +136,10 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;
 struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;
 long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;
 int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
+#ifdef __SIZEOF_INT128__
+__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
+#endif
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;

[Severity: Medium]
Should these kfuncs be declared with the __weak attribute?

While the test programs in verifier_aggregate_ret.c use the __arch_x86_64 and __arch_arm64 tags, they still unconditionally reference these kfuncs in the C code. Without the __weak attribute, Clang emits them as strong externs in the BPF object file.

When libbpf attempts to load the verifier_aggregate_ret object on unsupported architectures (when compiled with LLVM 23+), won't it fail to resolve these strong externs because they are conditionally compiled out of bpf_testmod.ko? This could cause the entire object load to fail, preventing even the non-kfunc tests in the same object from running.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
author reviewer
#28 lore raw

Re: [PATCH bpf-next v6 08/10] selftests/bpf: Add C tests for 16-byte returns in R0:R2

Quoted 50 lines · On 8/16/26 9:45 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The kfuncs bpf_kfunc_call_test_i128 and bpf_kfunc_call_test_ret_pair are conditionally compiled out of bpf_testmod.ko on architectures other than x86_64 and arm64, but are unconditionally compiled and declared as strong __ksym externs in the selftest object verifier_aggregate_ret.c. This will cause libbpf to fail loading the entire object on unsupported architectures (when compiled with LLVM 23+), failing the entire test suite even for programs that don't use the kfuncs. --

--- Patch [8]: [PATCH bpf-next v6 08/10] selftests/bpf: Add C tests for 16-byte returns in R0:R2 --- commit 00f68ada7c423dbadb8514b3c5f6c309f446bc99
Author: Yonghong Song <[email protected]>

selftests/bpf: Add C tests for 16-byte returns in R0:R2

Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return.

The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a __clang_major__ guard. An older compiler builds the dummy test in the #else branch instead, which keeps the object non-empty and says in its description why nothing was exercised.

The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), which bpf_add_kfunc_call() requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.

Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +0 -0
index 7d81070eefe7f..426000f9a14fa 100644

[ ... ]

tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym; · line 142

Quoted 8 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +4 -0
@@ -131,6 +136,10 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;
  struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;
  long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;
  int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
+#ifdef __SIZEOF_INT128__
+__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
+#endif
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;

[Severity: Medium]
Should these kfuncs be declared with the __weak attribute?

While the test programs in verifier_aggregate_ret.c use the __arch_x86_64 and __arch_arm64 tags, they still unconditionally reference these kfuncs in the C code. Without the __weak attribute, Clang emits them as strong externs in the BPF object file.

Quoted 5 lines

When libbpf attempts to load the verifier_aggregate_ret object on unsupported architectures (when compiled with LLVM 23+), won't it fail to resolve these strong externs because they are conditionally compiled out of bpf_testmod.ko? This could cause the entire object load to fail, preventing even the non-kfunc tests in the same object from running.

I think we are fine here. We can revisit this if newer architecture (e.g., riscv, s390x) needs __weak attribute in the future.

author reviewer
#29 lore raw

PATCH v6 9/10 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

In reply to Yonghong Song

Add inline-asm tests covering what the C tests cannot reach, since they need a callee that violates the convention on purpose.

The coverage includes BPF-to-BPF returns, kfunc calls, backtracking and liveness. In addition, a negative freplace test checks that an extension cannot replace a function returning R0:R2.

---
 .../selftests/bpf/prog_tests/aggregate_ret.c  |  11 +
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |  16 ++
 .../selftests/bpf/progs/aggregate_ret_func.c  | 235 ++++++++++++++++++
 .../selftests/bpf/progs/aggregate_ret_kfunc.c | 122 +++++++++
 .../bpf/progs/aggregate_ret_target.c          |  29 +++
 .../bpf/progs/compute_live_registers.c        |  30 +++
 .../selftests/bpf/progs/freplace_ret_pair.c   |  12 +
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  37 +++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  20 ++
 9 files changed, 512 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c
 create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +11 -0
new file mode 100644
index 000000000000..e0b94ed10f94
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include "aggregate_ret_func.skel.h"
+#include "aggregate_ret_kfunc.skel.h"
+
+void test_aggregate_ret(void)
+{
+	RUN_TESTS(aggregate_ret_func);
+	RUN_TESTS(aggregate_ret_kfunc);
+}
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +16 -0
index 2523c07a16c6..6c438df380bb 100644
@@ -441,6 +441,20 @@ static void test_func_replace_int_with_void(void)
 				     " doesn't match type INT of global_func2()");
 }
 
+static void test_func_replace_ret_pair(void)
+{
+	const char *msg = "Return type of new_agg_ret_target_func() has size 8 "
+			  "while agg_ret_target_func() has size 16";
+
+	/*
+	 * An extension cannot replace a function whose return value comes back
+	 * in the R0:R2 pair: only R0 is checked at the extension's exit, so it
+	 * would leave R2 stale for the target's callers.
+	 */
+	test_obj_load_failure_common("freplace_ret_pair.bpf.o",
+				     "./aggregate_ret_target.bpf.o", msg);
+}
+
 static int find_prog_btf_id(const char *name, __u32 attach_prog_fd)
 {
 	struct bpf_prog_info info = {};
@@ -660,6 +674,8 @@ void serial_test_fexit_bpf2bpf(void)
 		test_func_replace_progmap();
 	if (test__start_subtest("freplace_int_with_void"))
 		test_func_replace_int_with_void();
+	if (test__start_subtest("freplace_ret_pair"))
+		test_func_replace_ret_pair();
 	if (test__start_subtest("freplace_void"))
 		test_func_replace_void();
 	if (test__start_subtest("sleepable_fentry_to_xdp"))
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +235 -0

235 lines not on this page · raw message

tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +122 -0

122 lines not on this page · raw message

tools/testing/selftests/bpf/progs/aggregate_ret_target.c new file +29 -0
new file mode 100644
index 000000000000..cffd8d7d3241
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+/* freplace target: a global subprogram returning 16 bytes in R0:R2. */
+__naked unsigned __int128 agg_ret_target_func(void)
+{
+	asm volatile (
+	"r0 = 0x1234;"
+	"r2 = 0x5678;"
+	"exit;"
+	);
+}
+
+SEC("tc")
+__naked int agg_ret_target(void)
+{
+	asm volatile (
+	"call %[agg_ret_target_func];"
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm(agg_ret_target_func)
+	: __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/compute_live_registers.c +30 -0
index d055fc7b3b95..0be9441ec273 100644
@@ -431,6 +431,36 @@ __naked void subprog1(void)
 		::: __clobber_all);
 }
 
+static __used __naked unsigned __int128 aux2(void)
+{
+	asm volatile (
+		"r0 = 1;"
+		"r2 = 2;"
+		"exit;"
+		::: __clobber_all);
+}
+
+SEC("socket")
+/* A program observing the pair needs the JIT; see bpf_compute_subprog_ret_regs(). */
+__load_if_JITed()
+__log_level(2)
+__msg("0: .12345.... (85) call pc+2")
+__msg("1: ..2....... (bf) r0 = r2")
+/* R2 is not read at the exit of this program, which returns an int, ... */
+__msg("2: 0......... (95) exit")
+__msg("3: .......... (b7) r0 = 1")
+__msg("4: 0......... (b7) r2 = 2")
+/* ... but it is at the exit of aux2(), which returns a register pair. */
+__msg("5: 0.2....... (95) exit")
+__naked void subprog_ret_reg_pair(void)
+{
+	asm volatile (
+		"call aux2;"
+		"r0 = r2;"
+		"exit;"
+		::: __clobber_all);
+}
+
 #if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)
 
 SEC("socket")
tools/testing/selftests/bpf/progs/freplace_ret_pair.c new file +12 -0
new file mode 100644
index 000000000000..12c15d293bd7
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+SEC("freplace/agg_ret_target_func")
+__u64 new_agg_ret_target_func(void)
+{
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +37 -0
index c4bc8e11c6e1..20a9b9f20e96 100644
@@ -966,8 +966,41 @@ __bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b)
 
 	return r;
 }
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u64 b)
+{
+	struct prog_test_ret_pair r = { .lo = a + b, .hi = a - b };
+
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)
+{
+	struct prog_test_ret_ptr r = { .p = NULL, .tag = tag };
+
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b)
+{
+	struct prog_test_ret_ii r = { .a = a, .b = b };
+
+	return r;
+}
 #endif /* __x86_64__ || __aarch64__ */
 
+/*
+ * Takes no argument on purpose: with no arguments there is nothing for the sret
+ * pointer to displace, so this needs no architecture guard even though it
+ * returns 24 bytes. See the comment on bpf_kfunc_call_test_i128() above.
+ */
+__bpf_kfunc struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void)
+{
+	struct prog_test_ret_big r = { .a = 1, .b = 2, .c = 3 };
+
+	return r;
+}
+
 __bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,
 					 u64 e, u64 f, u64 g, u64 h,
 					 u64 i, u64 j)
@@ -1504,7 +1537,11 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test5)
 #if defined(__x86_64__) || defined(__aarch64__)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_fastcall, KF_FASTCALL)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ptr)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)
 #endif
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_big)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +20 -0
index 426000f9a14f..65e693ada736 100644
@@ -60,6 +60,22 @@ struct prog_test_ret_pair {	/* 16 bytes: R0:R2 */
 	__u64 hi;
 };
 
+struct prog_test_ret_ii {	/* 8 bytes: R0 only */
+	int a;
+	int b;
+};
+
+struct prog_test_ret_ptr {	/* 16 bytes: contains a pointer */
+	void *p;
+	__u64 tag;
+};
+
+struct prog_test_ret_big {	/* 24 bytes: too large for R0:R2 */
+	__u64 a;
+	__u64 b;
+	__u64 c;
+};
+
 struct prog_test_fail1 {
 	void *p;
 	int x;
@@ -140,6 +156,10 @@ int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
 __int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
 #endif
 struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
+struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym;
+struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void) __ksym;
 __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,
 			       __u64 e, __u64 f, __u64 g, __u64 h,
 			       __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
#30 lore raw

Re: [PATCH bpf-next v6 09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The test `aggregate_ret_kfunc_too_big_fail` is incorrectly restricted to only run on x86_64 and…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The test aggregate_ret_kfunc_too_big_fail is incorrectly restricted to only run on x86_64 and arm64 architectures. --

--- Patch [9]: [PATCH bpf-next v6 09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns --- commit b44fbeb59afa99f6be0f33037220d124e0fb4e12
Author: Yonghong Song <[email protected]>

selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

This commit adds BPF inline-assembly tests to cover BPF-to-BPF returns, kfunc calls, backtracking, and liveness for functions returning values in the R0:R2 register pair. It also includes an extension test to ensure a function returning R0:R2 cannot be replaced.

Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +0 -0
new file mode 100644
index 0000000000000..c23b4beb1773c

[ ... ]

tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c line 109

Quoted 7 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +7 -0
+/*
+ * A return value larger than 16 bytes does not fit in R0:R2 and is rejected by
+ * btf_distill_func_proto(), before the KF_FASTCALL and JIT-capability checks,
+ * so this behaves the same on every architecture.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64

[Severity: Medium]
Are these architecture restriction tags intended here?

The comment right above mentions that this test behaves the same on every architecture, but the __arch_x86_64 and __arch_arm64 tags artificially limit aggregate_ret_kfunc_too_big_fail() to only run on those two architectures, which prevents the test from running on others.

Quoted 11 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +11 -0
+__failure __msg("The function bpf_kfunc_call_test_ret_big return type STRUCT is unsupported")
+__naked int aggregate_ret_kfunc_too_big_fail(void)
+{
+	asm volatile (
+	"call %[bpf_kfunc_call_test_ret_big];"
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm(bpf_kfunc_call_test_ret_big)
+	: __clobber_all);
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
author reviewer
#31 lore raw

Re: [PATCH bpf-next v6 09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

Quoted 32 lines · On 8/16/26 9:42 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The test aggregate_ret_kfunc_too_big_fail is incorrectly restricted to only run on x86_64 and arm64 architectures. --

--- Patch [9]: [PATCH bpf-next v6 09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns --- commit b44fbeb59afa99f6be0f33037220d124e0fb4e12
Author: Yonghong Song <[email protected]>

selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

This commit adds BPF inline-assembly tests to cover BPF-to-BPF returns, kfunc calls, backtracking, and liveness for functions returning values in the R0:R2 register pair. It also includes an extension test to ensure a function returning R0:R2 cannot be replaced.

Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +0 -0
new file mode 100644
index 0000000000000..c23b4beb1773c

[ ... ]

tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c line 109

Quoted 7 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +7 -0
+/*
+ * A return value larger than 16 bytes does not fit in R0:R2 and is rejected by
+ * btf_distill_func_proto(), before the KF_FASTCALL and JIT-capability checks,
+ * so this behaves the same on every architecture.
+ */
+SEC("tc")
+__arch_x86_64 __arch_arm64

[Severity: Medium]
Are these architecture restriction tags intended here?

Quoted 4 lines

The comment right above mentions that this test behaves the same on every architecture, but the __arch_x86_64 and __arch_arm64 tags artificially limit aggregate_ret_kfunc_too_big_fail() to only run on those two architectures, which prevents the test from running on others.

Okay, I don't have 100% confidence about every architecture. So let us change to

/*
* A return value larger than 16 bytes does not fit in R0:R2 and is rejected by
* btf_distill_func_proto() before the KF_FASTCALL and JIT-capability checks. */

Quoted 12 lines
Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +11 -0
+__failure __msg("The function bpf_kfunc_call_test_ret_big return type STRUCT is unsupported")
+__naked int aggregate_ret_kfunc_too_big_fail(void)
+{
+	asm volatile (
+	"call %[bpf_kfunc_call_test_ret_big];"
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm(bpf_kfunc_call_test_ret_big)
+	: __clobber_all);
+}
#32 lore raw

Re: [PATCH bpf-next v6 09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

In reply to Yonghong Song
Does this test need __load_if_JITed()? Without it, the test runs on kernels where the JIT is disabled. When env->prog->jit_requested is false, bpf_compute_subprog_ret_regs() returns early without…
Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +0 -0
new file mode 100644
index 000000000000..e35baaa10fea

[ ... ]

tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 59

Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +12 -0
+SEC("tc")
+__failure __msg("At subprogram exit the register R2 is not a scalar value")
+__naked int aggregate_ret_global_ptr_fail(void)
+{
+	asm volatile (
+	"call %[global_agg_bad_ptr];"
+	"r0 = r2;"
+	"exit;"
+	:
+	: __imm(global_agg_bad_ptr)
+	: __clobber_all);
+}

Does this test need __load_if_JITed()? Without it, the test runs on kernels where the JIT is disabled. When env->prog->jit_requested is false, bpf_compute_subprog_ret_regs() returns early without marking any ret_reg_pair flag, so global_agg_bad_ptr is not treated as returning a register pair.

Main verification happens first, and with nregs = 1 at the call site, check_func_call() marks only R0 (the callee's R2 is never propagated to the caller). The next instruction "r0 = r2" then reads the caller's NOT_INIT R2 from clear_caller_saved_regs() and dies in __check_reg_arg() with "R2 !read_ok", never reaching do_check_subprogs() where the asserted message "At subprogram exit the register R2 is not a scalar value" would appear.

The program still fails to load, but with a different message than the test asserts. aggregate_ret_static_precise and aggregate_ret_global_precise both carry __load_if_JITed() to ensure the R0:R2 convention is active when they run. Would the same gate keep this test meaningful?

Quoted 7 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +7 -0
+/*
+ * R2 is a return register once the subprogram returns a pair, so a stack
+ * pointer left in it is rejected at the callee's exit exactly as one in R0
+ * is: the callee frame is gone by the time the caller could use it.
+ */
+SEC("tc")
+__failure __msg("cannot return stack pointer to the caller")

tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 86

Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +10 -0
+__naked int aggregate_ret_static_ptr_fail(void)
+{
+	asm volatile (
+	"call %[static_agg_bad_ptr];"
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm(static_agg_bad_ptr)
+	: __clobber_all);
+}

Does this test also need __load_if_JITed()? When the JIT is off, bpf_compute_subprog_ret_regs() bails out before setting ret_reg_pair for any subprogram, so bpf_ret_reg_pair() returns false for static_agg_bad_ptr and prepare_func_exit() computes nregs = 1.

The loop "for (i = 0; i < nregs; i++)" then only inspects R0. Since static_agg_bad_ptr leaves R0 as a scalar and the stack pointer only in R2, the loop finds nothing and the program verifies cleanly. The test body "r0 = 0; exit;" never reads R2, so the caller's R2 is already NOT_INIT from clear_caller_saved_regs() and stays that way.

The __failure tag would then cause the subtest to report failure, because the program loads successfully instead of failing with the asserted message.

[ ... ]

Quoted 4 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +0 -0
index d055fc7b3b95..0be9441ec273 100644

[ ... ]

tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void subprog1(void) · line 455

Quoted 13 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +13 -0
+SEC("socket")
+/* A program observing the pair needs the JIT; see bpf_compute_subprog_ret_regs(). */
+__load_if_JITed()
+__log_level(2)
+__msg("0: .12345.... (85) call pc+2")
+__msg("1: ..2....... (bf) r0 = r2")
+/* R2 is not read at the exit of this program, which returns an int, ... */
+__msg("2: 0......... (95) exit")
+__msg("3: .......... (b7) r0 = 1")
+__msg("4: 0......... (b7) r2 = 2")
+/* ... but it is at the exit of aux2(), which returns a register pair. */
+__msg("5: 0.2....... (95) exit")
+__naked void subprog_ret_reg_pair(void)

The comment says R2 is not live at the program's exit because "this program ... returns an int". The program is declared "__naked void subprog_ret_reg_pair(void)" - it does not return an int.

Is the reason actually that bpf_compute_subprog_ret_regs() skips the main program?

In kernel/bpf/verifier.c:437, the loop starts at subprog 1:

    /*
     * Skip the main program: its return value is the program's exit
     * code, read out of R0, so it never uses the register pair. ...
     */
    for (subprog = 1; subprog < env->subprog_cnt; subprog++) {

So R2 would not be live at the main program's exit even if subprog 0 were declared to return 16 bytes. The program's return type is not what decides this.

[ ... ]

Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +0 -0
index c4bc8e11c6e1..20a9b9f20e96 100644

[ ... ]

Quoted 11 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +11 -0
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u64 b)
+{
+	struct prog_test_ret_pair r = { .lo = a + b, .hi = a - b };
+
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)
+{
+	struct prog_test_ret_ptr r = { .p = NULL, .tag = tag };
+

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c __bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) · line 990

Quoted 10 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +9 -0
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b)
+{
+	struct prog_test_ret_ii r = { .a = a, .b = b };
+
+	return r;
+}
 #endif /* __x86_64__ || __aarch64__ */

Would riscv64 benefit from test coverage here? Commit 3bfe2cb81326 ("bpf: Wire up JIT support for 16-byte kfunc returns") added bpf_jit_supports_kfunc_ret_reg_pair() returning true for riscv64, so bpf_add_kfunc_call() accepts 16-byte kfunc returns there.

But the test infrastructure excludes riscv64:

- bpf_kfunc_call_test_ret_pair(), _ret_fastcall(), _ret_ptr(), and _ret_ii() are not compiled into bpf_testmod on riscv64 (arch guard)
- every test in progs/aggregate_ret_kfunc.c is tagged "__arch_x86_64 __arch_arm64", so test_loader.c skips them on riscv64

The riscv64 LP64D ABI returns a 16-byte two-word struct in a0:a1, matching the x86-64 rax:rdx and arm64 x0:x1 cases. bpf_misc.h already defines "__arch_riscv64", and test_loader.c maps ARCH_RISCV64.

Could the arch guard be extended to "defined(__riscv) && __riscv_xlen == 64" and the tests tagged with "__arch_riscv64" to cover the newly enabled JIT path?

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c __bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) · line 1002

Quoted 11 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +11 -0
+/*
+ * Takes no argument on purpose: with no arguments there is nothing for the sret
+ * pointer to displace, so this needs no architecture guard even though it
+ * returns 24 bytes. See the comment on bpf_kfunc_call_test_i128() above.
+ */
+__bpf_kfunc struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void)
+{
+	struct prog_test_ret_big r = { .a = 1, .b = 2, .c = 3 };
+
+	return r;
+}

The comment has two issues. First, it references "the comment on bpf_kfunc_call_test_i128() above", but there is no comment on bpf_kfunc_call_test_i128(). That function appears without explanation a few lines above.

Second, "with no arguments there is nothing for the sret pointer to displace" is not why this function is safe without an arch guard. On SysV x86-64, AAPCS64, and riscv64 LP64D, a 24-byte struct return is returned via memory through a hidden pointer passed in the first argument register.

With no declared arguments, the hidden sret pointer would still be taken from the first argument register (BPF R1), which for this kfunc is never set up. But that is not what makes this safe: bpf_add_kfunc_call() calls btf_distill_func_proto() which rejects any return larger than 16 bytes with "The function %s return type %s is unsupported."

The function is unreachable from BPF on every architecture, which is what aggregate_ret_kfunc_too_big_fail asserts. The test in progs/aggregate_ret_kfunc.c states this correct reason; the comment here states a different one.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

author reviewer
#33 lore raw

Re: [PATCH bpf-next v6 09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

Quoted 33 lines · On 8/16/26 10:17 PM, [email protected] wrote:
Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +0 -0
new file mode 100644
index 000000000000..e35baaa10fea

[ ... ]

tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 59

Quoted 12 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +12 -0
+SEC("tc")
+__failure __msg("At subprogram exit the register R2 is not a scalar value")
+__naked int aggregate_ret_global_ptr_fail(void)
+{
+	asm volatile (
+	"call %[global_agg_bad_ptr];"
+	"r0 = r2;"
+	"exit;"
+	:
+	: __imm(global_agg_bad_ptr)
+	: __clobber_all);
+}

Does this test need __load_if_JITed()? Without it, the test runs on kernels where the JIT is disabled. When env->prog->jit_requested is false, bpf_compute_subprog_ret_regs() returns early without marking any ret_reg_pair flag, so global_agg_bad_ptr is not treated as returning a register pair.

Main verification happens first, and with nregs = 1 at the call site, check_func_call() marks only R0 (the callee's R2 is never propagated to the caller). The next instruction "r0 = r2" then reads the caller's NOT_INIT R2 from clear_caller_saved_regs() and dies in __check_reg_arg() with "R2 !read_ok", never reaching do_check_subprogs() where the asserted message "At subprogram exit the register R2 is not a scalar value" would appear.

Quoted 5 lines

The program still fails to load, but with a different message than the test asserts. aggregate_ret_static_precise and aggregate_ret_global_precise both carry __load_if_JITed() to ensure the R0:R2 convention is active when they run. Would the same gate keep this test meaningful?

Right, we should have __load_if_JITed() in the asm test.

Quoted 29 lines
Quoted 8 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +7 -0
+/*
+ * R2 is a return register once the subprogram returns a pair, so a stack
+ * pointer left in it is rejected at the callee's exit exactly as one in R0
+ * is: the callee frame is gone by the time the caller could use it.
+ */
+SEC("tc")
+__failure __msg("cannot return stack pointer to the caller")

tools/testing/selftests/bpf/progs/aggregate_ret_func.c line 86

Quoted 10 lines
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +10 -0
+__naked int aggregate_ret_static_ptr_fail(void)
+{
+	asm volatile (
+	"call %[static_agg_bad_ptr];"
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm(static_agg_bad_ptr)
+	: __clobber_all);
+}

Does this test also need __load_if_JITed()? When the JIT is off, bpf_compute_subprog_ret_regs() bails out before setting ret_reg_pair for any subprogram, so bpf_ret_reg_pair() returns false for static_agg_bad_ptr and prepare_func_exit() computes nregs = 1.

The loop "for (i = 0; i < nregs; i++)" then only inspects R0. Since static_agg_bad_ptr leaves R0 as a scalar and the stack pointer only in R2, the loop finds nothing and the program verifies cleanly. The test body "r0 = 0; exit;" never reads R2, so the caller's R2 is already NOT_INIT from clear_caller_saved_regs() and stays that way.

Quoted 3 lines

The __failure tag would then cause the subtest to report failure, because the program loads successfully instead of failing with the asserted message.

The same as above, we should have __load_if_JITed() in the above asm.

Quoted 8 lines

[ ... ]

Quoted 4 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +0 -0
index d055fc7b3b95..0be9441ec273 100644

[ ... ]

Quoted 16 lines

tools/testing/selftests/bpf/progs/compute_live_registers.c __naked void subprog1(void) · line 455

Quoted 13 lines
tools/testing/selftests/bpf/progs/compute_live_registers.c +13 -0
+SEC("socket")
+/* A program observing the pair needs the JIT; see bpf_compute_subprog_ret_regs(). */
+__load_if_JITed()
+__log_level(2)
+__msg("0: .12345.... (85) call pc+2")
+__msg("1: ..2....... (bf) r0 = r2")
+/* R2 is not read at the exit of this program, which returns an int, ... */
+__msg("2: 0......... (95) exit")
+__msg("3: .......... (b7) r0 = 1")
+__msg("4: 0......... (b7) r2 = 2")
+/* ... but it is at the exit of aux2(), which returns a register pair. */
+__msg("5: 0.2....... (95) exit")
+__naked void subprog_ret_reg_pair(void)

The comment says R2 is not live at the program's exit because "this program ... returns an int". The program is declared "__naked void subprog_ret_reg_pair(void)" - it does not return an int.

You are right, the comment is not correct.

Quoted 12 lines

Is the reason actually that bpf_compute_subprog_ret_regs() skips the main program?

In kernel/bpf/verifier.c:437, the loop starts at subprog 1:

     /*
      * Skip the main program: its return value is the program's exit
      * code, read out of R0, so it never uses the register pair. ...
      */
     for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
Quoted 3 lines

So R2 would not be live at the main program's exit even if subprog 0 were declared to return 16 bytes. The program's return type is not what decides this.

We skip R2 for main program (subprog 0). It is user's resposibility to test return value for main program.

Quoted 42 lines

[ ... ]

Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +0 -0
index c4bc8e11c6e1..20a9b9f20e96 100644

[ ... ]

Quoted 11 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +11 -0
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u64 b)
+{
+	struct prog_test_ret_pair r = { .lo = a + b, .hi = a - b };
+
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)
+{
+	struct prog_test_ret_ptr r = { .p = NULL, .tag = tag };
+

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c __bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) · line 990

Quoted 10 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +9 -0
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b)
+{
+	struct prog_test_ret_ii r = { .a = a, .b = b };
+
+	return r;
+}
  #endif /* __x86_64__ || __aarch64__ */

Would riscv64 benefit from test coverage here? Commit 3bfe2cb81326 ("bpf: Wire up JIT support for 16-byte kfunc returns") added bpf_jit_supports_kfunc_ret_reg_pair() returning true for riscv64, so bpf_add_kfunc_call() accepts 16-byte kfunc returns there.

But the test infrastructure excludes riscv64:

- bpf_kfunc_call_test_ret_pair(), _ret_fastcall(), _ret_ptr(), and _ret_ii() are not compiled into bpf_testmod on riscv64 (arch guard)
- every test in progs/aggregate_ret_kfunc.c is tagged "__arch_x86_64 __arch_arm64", so test_loader.c skips them on riscv64

Quoted 7 lines

The riscv64 LP64D ABI returns a 16-byte two-word struct in a0:a1, matching the x86-64 rax:rdx and arm64 x0:x1 cases. bpf_misc.h already defines "__arch_riscv64", and test_loader.c maps ARCH_RISCV64.

Could the arch guard be extended to "defined(__riscv) && __riscv_xlen == 64" and the tests tagged with "__arch_riscv64" to cover the newly enabled JIT path?

I didn't test riscv64 as I don't have infrastructure for this. riscv64 can be implemented later.

Quoted 16 lines

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c __bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b) · line 1002

Quoted 12 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +11 -0
+/*
+ * Takes no argument on purpose: with no arguments there is nothing for the sret
+ * pointer to displace, so this needs no architecture guard even though it
+ * returns 24 bytes. See the comment on bpf_kfunc_call_test_i128() above.
+ */
+__bpf_kfunc struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void)
+{
+	struct prog_test_ret_big r = { .a = 1, .b = 2, .c = 3 };
+
+	return r;
+}

The comment has two issues. First, it references "the comment on bpf_kfunc_call_test_i128() above", but there is no comment on bpf_kfunc_call_test_i128(). That function appears without explanation a few lines above.

Okay, we can remove 'See the comment on bpf_kfunc_call_test_i128() above'.

Quoted 13 lines

Second, "with no arguments there is nothing for the sret pointer to displace" is not why this function is safe without an arch guard. On SysV x86-64, AAPCS64, and riscv64 LP64D, a 24-byte struct return is returned via memory through a hidden pointer passed in the first argument register.

With no declared arguments, the hidden sret pointer would still be taken from the first argument register (BPF R1), which for this kfunc is never set up. But that is not what makes this safe: bpf_add_kfunc_call() calls btf_distill_func_proto() which rejects any return larger than 16 bytes with "The function %s return type %s is unsupported."

Quoted 4 lines

The function is unreachable from BPF on every architecture, which is what aggregate_ret_kfunc_too_big_fail asserts. The test in progs/aggregate_ret_kfunc.c states this correct reason; the comment here states a different one.

Okay, I may just remove the above comments.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31995069373

maintainer Acked-by on v6 9/10
#34 lore raw

Re: [PATCH bpf-next v6 09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

In reply to Yonghong Song
Quoted 9 lines · Yonghong Song

Add inline-asm tests covering what the C tests cannot reach, since they need a callee that violates the convention on purpose.

The coverage includes BPF-to-BPF returns, kfunc calls, backtracking and liveness. In addition, a negative freplace test checks that an extension cannot replace a function returning R0:R2.

---

...

author reviewer
#35 lore raw

PATCH v6 10/10 Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2

In reply to Yonghong Song

kfuncs may now return a value larger than 8 bytes and up to 16 bytes (a scalar-only struct or union, or an __int128), passed back in the R0:R2 register pair. Add a kfunc return-value section documenting this, including that a struct or union up to 8 bytes is returned in R0 alone, which struct and union members are accepted, that the R0:R2 register pair requires JIT support (bpf_jit_supports_kfunc_ret_reg_pair()), and that a return value larger than 16 bytes is unsupported.

Also note that the same convention applies to BPF subprogram returns, and document the consequence for a global subprogram: it must assign both halves of a register-pair return, since an unassigned R2 may be left holding a pointer argument and is then rejected as a leak.

---
 Documentation/bpf/kfuncs.rst | 65 ++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)
Documentation/bpf/kfuncs.rst +65 -0
index 85f73e0bbd0f..89dea6b0b024 100644
@@ -575,6 +575,71 @@ is also covered by this recovery. A kfunc handed an arena pointer may
 therefore access up to ``GUARD_SZ / 2`` past it without bounds-checking
 against the arena. Larger accesses must verify the range explicitly.
 
+2.9 kfunc Return Values
+-----------------------
+
+A kfunc may return a scalar, a pointer, or a small struct or union by
+value. A scalar or pointer of up to 8 bytes is returned in R0, as usual.
+
+A struct or union returned by value must be composed only of scalars
+(recursively), where a scalar is an integer or an enum; arrays of scalars are
+allowed as members. Its bytes are handed back to the program as the raw
+contents of R0 (and R2), so a pointer field would be laundered into a scalar
+and escape the verifier's pointer provenance and reference tracking. A struct
+or union with a pointer member is therefore rejected at load time, and so is
+one with a floating-point member, which the ABI may not return in R0:R2 at
+all.
+
+A kfunc may also return a value larger than 8 bytes and up to 16 bytes -- a
+scalar-only struct or union, or an ``__int128``. Such a value is returned
+in the register pair R0:R2, matching the convention LLVM uses for the BPF
+target: the first 8 bytes in R0 and the second 8 bytes in R2. A struct or
+union of 8 bytes or less is returned in R0 alone.
+
+::
+
+        struct bpf_pair { __u64 a, b; };   /* 16 bytes */
+
+        __bpf_kfunc struct bpf_pair bpf_kfunc_get_pair(void)
+        {
+                struct bpf_pair p = { .a = 1, .b = 2 };
+
+                return p;      /* p.a in R0, p.b in R2 */
+        }
+
+Returning a value in the R0:R2 pair requires the JIT to place the second
+half of the return value into R2, which not every architecture supports
+right now. A kfunc with a return value larger than 8 bytes is therefore
+rejected at load time on a JIT that does not advertise this capability (see
+``bpf_jit_supports_kfunc_ret_reg_pair()``), and such a program is never run
+by the interpreter. A return value larger than 16 bytes is not supported.
+
+The same R0:R2 convention applies to a BPF subprogram, global or static, that
+returns an ``__int128`` or a struct or union larger than 8 bytes. It is only
+used when the program is JITed, since the interpreter propagates only R0 out of
+a subprogram: without a JIT the return value stays in R0 alone, and a caller
+reading R2 is rejected for reading an uninitialized register. A global
+subprogram is verified in isolation, so its by-value struct or union return is
+restricted to scalars just like a kfunc's; a static subprogram is verified
+inline and has no such restriction. The main program is not covered: its return
+value is the program's exit code, read out of R0 alone, so a declared upper
+half is never looked at.
+
+A global subprogram must leave a scalar in *every* register of the pair, so
+both halves of the returned value have to be assigned. Leaving the upper half
+uninitialized is not merely untidy: the compiler is then free to leave R2
+holding whatever it happened to hold, which for a subprogram taking a pointer
+argument is typically that pointer. Handing the caller an unknown scalar built
+from a pointer is a leak, so the verifier rejects it with::
+
+        At subprogram exit the register R2 is not a scalar value (...)
+
+Initialize the whole return value, for example ``struct pair p = {};``, to
+avoid this. A static subprogram is exempt from the scalar-only rule: it is
+verified inline, so an unassigned R2 is simply passed back to the caller as
+uninitialized and only a caller that reads it fails. A stack pointer left in
+R2 is still rejected there, just as one in R0 is.
+
 .. _BPF_kfunc_lifecycle_expectations:
 
 3. kfunc lifecycle expectations
Signature · 1 line
2.53.0-Meta