Skip to contentSkip to conversation

bpf: Support aggregate return values up to 16 bytes

PATCH Applied Yonghong Song 5 participants 32 messages Aug 19 – Aug 21

Subsystems (12)
Documentation x86 Kselftest BPF +8 more subsystems
lore
Share & tools
lore

Yonghong Song

author reviewer
#0 lore raw

v7 cover bpf: Support aggregate return values up to 16 bytes

LLVM 23 can return an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair [1][2]. Before that the BPF backend could not return such values at all: a by-value aggregate return was rejected at compile time with "aggregate returns are not supported", and an __int128 return failed in the backend with "unable to allocate function return #1".

This series teaches the kernel the same convention, so that BPF programs and kfuncs can return these values. The first 8 bytes of the value come back in R0 and the second 8 bytes in R2. It applies to kfunc returns and to BPF-to-BPF subprogram returns, both global and static. The main program is unchanged: its return value is the program's exit code, so a return larger than 8 bytes is still rejected at BPF_EXIT.

Patches 1-2 are preparation: patch 1 factors out the per-register check used by the global return path, and patch 2 adds the shared helpers that answer "does this subprogram return a register pair", so that the patches which follow can be ordered independently. Patch 3 wires up the JIT side. Patches 4-5 teach precision backtracking and live register analysis about R2 as a second return register, ahead of the patch that starts modeling it. Patch 6 adds the verifier support proper. Patch 7 relaxes btf_distill_func_proto() and btf_validate_return_type(), which is what makes the whole thing reachable. Patches 8-9 add selftests and patch 10 documents the convention.

Constraints worth calling out:

- A by-value struct or union returned by a kfunc or by a global subprogram must be composed only of scalars. The verifier models the returned register bits as an unknown scalar, so a pointer member would be laundered into one and escape provenance and reference tracking. A static subprogram is verified inline and is not restricted this way.

- Returning the pair from a kfunc needs the JIT to place the second half into R2, which is architecture-specific work. Architectures opt in through bpf_jit_supports_kfunc_ret_reg_pair(); x86_64, arm64 and riscv64 do so here, and elsewhere bpf_add_kfunc_call() rejects such a kfunc with -EOPNOTSUPP. A register-pair return from a BPF subprogram needs no such capability.

- The pair is modelled only when the JIT is enabled, since the interpreter propagates R0 alone out of a subprogram. Modelling it sets jit_required, so a JIT fallback to the interpreter becomes a load failure rather than a silent divergence from what was verified. With the JIT off, the pair is not modelled and a caller reading R2 fails verification.

- The compiler side requires LLVM 23 or newer. The selftests written in C sit behind a __clang_major__ guard; an older compiler builds a dummy test instead, whose description says why nothing was exercised. The inline-asm tests run everywhere, apart from the few whose callee prototype returns a struct or union by value, which are guarded the same way.

[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876

Changelog:
v6 -> v7:
- v6: https://lore.kernel.org/bpf/[email protected]/
- Simplify type checking in function bpf_compute_subprog_ret_regs().
- Add __load_if_JITed() for a few negative tests.
- Adjust a few comments and error message.
v5 -> v6:
- v5: https://lore.kernel.org/bpf/[email protected]/
- Returning R0 only if jit is not enabled. If jit is enabled, main prog will not force jit even if main prog may return R0:R2.
- Simplify precision backtracking.
- Check returning R0:R2 in btf_check_func_type_match().
- Remove some redundant tests.
v4 -> v5:
- v4: https://lore.kernel.org/bpf/[email protected]/
- Removed R0:R2 restriction on callback functions and its related tests.
- Simplify the code for backtracking.
- Reduce comments and verify R0/R2 together.
- Use RUN_TESTS for selftests.
v3 -> v4:
- v3: https://lore.kernel.org/bpf/[email protected]/
- Fix a few kernel comment format.
- Guard more kfunc's with x86_64/arm64 only to avoid s390x failure.
v2 -> v3:
- v2: https://lore.kernel.org/bpf/[email protected]/
- Add additional guard with __SIZEOF_INT128__ for 32bit kernel.
- Guard little endian to avoid arm64 big endian for selftests.
- Fix test failures for gcc15.
- Rebase to avoid conflict with latest master branch.
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- Split the R0:R2 helpers out of the verifier patch into their own preparation patch, and reordered the series so the core verifier patch comes after the infrastructure it depends on.
- New patch rejecting callbacks that return more than 8 bytes, both helper/kfunc callbacks and exception callbacks, with selftests.
- New patch rejecting a register-pair return once btf_check_subprog_call() has marked the subprogram's BTF unreliable, rather than silently mistracking R2.
- Folded "bpf: Force JIT for programs using the R0:R2 register pair" into the verifier patch.
- Dropped "bpf: Reject >8 byte return values on return-reading trampoline paths" and its selftests; that went in separately as commit c48796aa6c39.
- Described the register mapping as the first and second 8 bytes rather than the low and high 64 bits, which is only correct on little-endian, and reworded "16-byte" to "up to 16 bytes" where the range 9..16 was meant.

Yonghong Song (10):
bpf: Factor check_global_ret_scalar_reg() out of the global return
check
bpf: Add helpers to describe the R0:R2 return register pair
bpf: Wire up JIT support for 16-byte kfunc returns
bpf: Handle R2 as a return register in precision backtracking
bpf: Account R2 of register-pair returns in live register analysis
bpf: Add verifier support for 16-byte returns in R0:R2
bpf: Enable aggregate return types up to 16 bytes selftests/bpf: Add C tests for 16-byte returns in R0:R2 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2

 Documentation/bpf/kfuncs.rst                  |  65 +++++
 arch/arm64/net/bpf_jit_comp.c                 |   5 +
 arch/riscv/net/bpf_jit_comp64.c               |   5 +
 arch/x86/net/bpf_jit_comp.c                   |  27 +-
 include/linux/bpf_verifier.h                  |   9 +
 include/linux/filter.h                        |   1 +
 kernel/bpf/backtrack.c                        |  46 ++--
 kernel/bpf/btf.c                              |  29 ++-
 kernel/bpf/core.c                             |   5 +
 kernel/bpf/liveness.c                         |  20 +-
 kernel/bpf/verifier.c                         | 192 +++++++++++---
 .../selftests/bpf/prog_tests/aggregate_ret.c  |  11 +
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |  16 ++
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/aggregate_ret_func.c  | 237 ++++++++++++++++++
 .../selftests/bpf/progs/aggregate_ret_kfunc.c | 121 +++++++++
 .../bpf/progs/aggregate_ret_target.c          |  29 +++
 .../bpf/progs/compute_live_registers.c        |  28 +++
 .../selftests/bpf/progs/exceptions_fail.c     |   2 +-
 .../selftests/bpf/progs/freplace_ret_pair.c   |  12 +
 .../bpf/progs/verifier_aggregate_ret.c        | 178 +++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  55 ++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  29 +++
 23 files changed, 1052 insertions(+), 72 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c
 create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c
Signature · 1 line
2.53.0-Meta
author reviewer
#1 lore raw

PATCH v7 1/10 bpf: Factor check_global_ret_scalar_reg() out of the global return check

check_global_subprog_return_code() verifies that a global subprogram returns void, an arena pointer, or register R0 holding a scalar value. Later patches in this series add 16-byte aggregate return support, whose second half is returned in R2 and needs the same validation.

Factor the per-register check into check_global_ret_scalar_reg(env, regno) so that it can be reused for R2. No functional change.

---
 kernel/bpf/verifier.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)
kernel/bpf/verifier.c +20 -12
index e421ea2b80c3..d8bf897460ea 100644
@@ -17453,37 +17453,45 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
 	return 0;
 }
 
-static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+static int check_global_ret_scalar_reg(struct bpf_verifier_env *env, u32 regno)
 {
-	struct bpf_reg_state *reg = reg_state(env, BPF_REG_0);
-	struct bpf_func_state *cur_frame = cur_func(env);
+	struct bpf_reg_state *reg;
 	int err;
 
-	if (subprog_returns_void(env, cur_frame->subprogno))
-		return 0;
-
-	err = check_reg_arg(env, BPF_REG_0, SRC_OP);
+	err = check_reg_arg(env, regno, SRC_OP);
 	if (err)
 		return err;
 
 	/* Pointers to arena are safe to pass between subprograms. */
-	if (is_arena_reg(env, BPF_REG_0))
+	if (is_arena_reg(env, regno))
 		return 0;
 
-	if (is_pointer_value(env, BPF_REG_0)) {
-		verbose(env, "R%d leaks addr as return value\n", BPF_REG_0);
+	if (is_pointer_value(env, regno)) {
+		verbose(env, "R%d leaks addr as return value\n", regno);
 		return -EACCES;
 	}
 
+	reg = reg_state(env, regno);
 	if (reg->type != SCALAR_VALUE) {
-		verbose(env, "At subprogram exit the register R0 is not a scalar value (%s)\n",
-			reg_type_str(env, reg->type));
+		verbose(env, "At subprogram exit the register R%d is not a scalar value (%s)\n",
+			regno, reg_type_str(env, reg->type));
 		return -EINVAL;
 	}
 
 	return 0;
 }
 
+static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+{
+	struct bpf_func_state *cur_frame = cur_func(env);
+	u32 subprog = cur_frame->subprogno;
+
+	if (subprog_returns_void(env, subprog))
+		return 0;
+
+	return check_global_ret_scalar_reg(env, BPF_REG_0);
+}
+
 /* Bitmask with 1s for all caller saved registers */
 #define ALL_CALLER_SAVED_REGS ((1u << CALLER_SAVED_REGS) - 1)
 
Signature · 1 line
2.53.0-Meta
author reviewer
#2 lore raw

PATCH v7 2/10 bpf: Add helpers to describe the R0:R2 return register pair

In reply to Yonghong Song

LLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes: such a value comes back in the R0:R2 register pair, with R2 holding the upper half. See LLVM patches [1] and [2].

Later patches teach the JIT, live register analysis and the verifier itself about that convention. They need to answer the same question: does this subprogram return its value in a register pair? Add bpf_ret_reg_pair() up front so it can be used in subsequent patches. It is answered from a per-subprogram flag that bpf_compute_subprog_ret_regs() derives once from the BTF prototype.

jit_requested only says that the JIT is enabled, not that it succeeded:
bpf_fixup_call_args() falls back to the interpreter when bpf_jit_subprogs() fails with anything other than -EFAULT. So jit_required is still set once the pair is modelled, which turns that fallback into a load failure rather than a silent divergence from what was verified.

[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876

---
 include/linux/bpf_verifier.h |  7 ++++
 kernel/bpf/verifier.c        | 66 ++++++++++++++++++++++++++++++------
 2 files changed, 62 insertions(+), 11 deletions(-)
include/linux/bpf_verifier.h +7 -0
index 5fad59fdab0d..953d8191563a 100644
@@ -819,6 +819,8 @@ struct bpf_subprog_info {
 	bool is_async_cb: 1;
 	bool is_exception_cb: 1;
 	bool args_cached: 1;
+	/* true if the return value is passed in the R0:R2 register pair */
+	bool ret_reg_pair: 1;
 	/* true if bpf_fastcall stack region is used by functions that can't be inlined */
 	bool keep_fastcall_stack: 1;
 	bool changes_pkt_data: 1;
@@ -1055,6 +1057,11 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env
 	return &env->subprog_info[subprog];
 }
 
+static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
+{
+	return subprog_info(env, subprog)->ret_reg_pair;
+}
+
 struct bpf_call_summary {
 	u8 num_params;
 	bool is_void;
kernel/bpf/verifier.c +55 -11

88 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#3 lore raw

Re: [PATCH bpf-next v7 02/10] bpf: Add helpers to describe the R0:R2 return register pair

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Extension programs (BPF_PROG_TYPE_EXT) are mishandled because `bpf_compute_subprog_ret_regs` skips…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Extension programs (BPF_PROG_TYPE_EXT) are mishandled because bpf_compute_subprog_ret_regs skips subprog 0, failing to set up ret_reg_pair. --

--- Patch [2]: [PATCH bpf-next v7 02/10] bpf: Add helpers to describe the R0:R2 return register pair --- commit f6c1f4f14c6e8bda9545fe1b82f42648b4f3c475
Author: Yonghong Song <[email protected]>

bpf: Add helpers to describe the R0:R2 return register pair

This commit adds bpf_ret_reg_pair() to determine if a BPF subprogram returns a value in the R0:R2 register pair. It implements bpf_compute_subprog_ret_regs() to compute this flag based on the BTF prototype of each subprogram, intentionally skipping the main program.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index d8bf897460eae..2b5e88fdda07e 100644

[ ... ]

Quoted 7 lines
kernel/bpf/verifier.c +7 -0
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 434

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {

[Severity: High]
Does this code mishandle extension programs (BPF_PROG_TYPE_EXT)?

The comment mentions that bpf_check_attach_target() refuses to replace a function returning more than 8 bytes. However, for extension programs, prog->expected_attach_type is 0, so attach_uses_trampoline_retval(0) evaluates to false, bypassing the size check.

If an extension program successfully attaches and replaces a subprogram returning an __int128 (16 bytes), skipping subprog 0 here means ret_reg_pair won't be set. Could this result in the verifier failing to enforce the initialization of R2, potentially leaking uninitialized register contents to the caller?

Quoted 14 lines
kernel/bpf/verifier.c +14 -0
+		type = subprog_ret_type(env, subprog);
+		if (!type || btf_type_is_void(type))
+			continue;
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
author reviewer
#4 lore raw

Re: [PATCH bpf-next v7 02/10] bpf: Add helpers to describe the R0:R2 return register pair

Quoted 49 lines · On 8/18/26 11:17 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Extension programs (BPF_PROG_TYPE_EXT) are mishandled because bpf_compute_subprog_ret_regs skips subprog 0, failing to set up ret_reg_pair. --

--- Patch [2]: [PATCH bpf-next v7 02/10] bpf: Add helpers to describe the R0:R2 return register pair --- commit f6c1f4f14c6e8bda9545fe1b82f42648b4f3c475
Author: Yonghong Song <[email protected]>

bpf: Add helpers to describe the R0:R2 return register pair

This commit adds bpf_ret_reg_pair() to determine if a BPF subprogram returns a value in the R0:R2 register pair. It implements bpf_compute_subprog_ret_regs() to compute this flag based on the BTF prototype of each subprogram, intentionally skipping the main program.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index d8bf897460eae..2b5e88fdda07e 100644

[ ... ]

Quoted 7 lines
kernel/bpf/verifier.c +7 -0
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 434

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {

[Severity: High]
Does this code mishandle extension programs (BPF_PROG_TYPE_EXT)?

The comment mentions that bpf_check_attach_target() refuses to replace a function returning more than 8 bytes. However, for extension programs, prog->expected_attach_type is 0, so attach_uses_trampoline_retval(0) evaluates to false, bypassing the size check.

Quoted 5 lines

If an extension program successfully attaches and replaces a subprogram returning an __int128 (16 bytes), skipping subprog 0 here means ret_reg_pair won't be set. Could this result in the verifier failing to enforce the initialization of R2, potentially leaking uninitialized register contents to the caller?

We should be okay. For freplace program, in later commit
bpf: Add verifier support for 16-byte returns in R0:R2
freplace will get rejected if kernel function returns R0:R2 values. If the freplace is for original subprog 0, the R2 (if to-be-returned in freplace) may cause incorrect result but it is user's problem.

Quoted 15 lines
Quoted 15 lines
kernel/bpf/verifier.c +14 -0
+		type = subprog_ret_type(env, subprog);
+		if (!type || btf_type_is_void(type))
+			continue;
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;
+}
author reviewer
#5 lore raw

Re: [PATCH bpf-next v7 02/10] bpf: Add helpers to describe the R0:R2 return register pair

In reply to [email protected]
Quoted 49 lines · On 8/18/26 11:17 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Extension programs (BPF_PROG_TYPE_EXT) are mishandled because bpf_compute_subprog_ret_regs skips subprog 0, failing to set up ret_reg_pair. --

--- Patch [2]: [PATCH bpf-next v7 02/10] bpf: Add helpers to describe the R0:R2 return register pair --- commit f6c1f4f14c6e8bda9545fe1b82f42648b4f3c475
Author: Yonghong Song <[email protected]>

bpf: Add helpers to describe the R0:R2 return register pair

This commit adds bpf_ret_reg_pair() to determine if a BPF subprogram returns a value in the R0:R2 register pair. It implements bpf_compute_subprog_ret_regs() to compute this flag based on the BTF prototype of each subprogram, intentionally skipping the main program.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index d8bf897460eae..2b5e88fdda07e 100644

[ ... ]

Quoted 7 lines
kernel/bpf/verifier.c +7 -0
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 434

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {

[Severity: High]
Does this code mishandle extension programs (BPF_PROG_TYPE_EXT)?

The comment mentions that bpf_check_attach_target() refuses to replace a function returning more than 8 bytes. However, for extension programs, prog->expected_attach_type is 0, so attach_uses_trampoline_retval(0) evaluates to false, bypassing the size check.

Quoted 5 lines

If an extension program successfully attaches and replaces a subprogram returning an __int128 (16 bytes), skipping subprog 0 here means ret_reg_pair won't be set. Could this result in the verifier failing to enforce the initialization of R2, potentially leaking uninitialized register contents to the caller?

But the comment needs update for this version.
bpf_check_attach_target() -> btf_check_func_type_match()

Quoted 15 lines
Quoted 15 lines
kernel/bpf/verifier.c +14 -0
+		type = subprog_ret_type(env, subprog);
+		if (!type || btf_type_is_void(type))
+			continue;
+		if (verifier_bug_if(IS_ERR(btf_resolve_size(btf, type, &size)), env,
+				    "cannot size return type of subprog %d", subprog))
+			return -EFAULT;
+		if (ret_regs_cnt(size) > 1) {
+			subprog_info(env, subprog)->ret_reg_pair = true;
+			env->prog->jit_required = 1;
+		}
+	}
+
+	return 0;
+}
maintainer
#6 lore raw

Re: [PATCH bpf-next v7 02/10] bpf: Add helpers to describe the R0:R2 return register pair

On Wed, 2026-08-19 at 08:50 -0700, Yonghong Song wrote:

...

Quoted 25 lines
Quoted 25 lines
Quoted 7 lines
kernel/bpf/verifier.c +7 -0
+static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+	const struct btf *btf = env->prog->aux->btf;
+	const struct btf_type *type;
+	int subprog;
+	u32 size;
+

kernel/bpf/verifier.c bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog) · line 434

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	if (!env->prog->jit_requested || bpf_prog_is_offloaded(env->prog->aux))
+		return 0;
+
+	/*
+	 * Skip the main program: its return value is the program's exit code,
+	 * read out of R0, so it never uses the register pair. An extension does
+	 * have a real prototype for subprog 0, but bpf_check_attach_target()
+	 * refuses to replace a function returning more than 8 bytes.
+	 */
+	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {

[Severity: High]
Does this code mishandle extension programs (BPF_PROG_TYPE_EXT)?

The comment mentions that bpf_check_attach_target() refuses to replace a function returning more than 8 bytes. However, for extension programs, prog->expected_attach_type is 0, so attach_uses_trampoline_retval(0) evaluates to false, bypassing the size check.

Quoted 8 lines
Quoted 5 lines

If an extension program successfully attaches and replaces a subprogram returning an __int128 (16 bytes), skipping subprog 0 here means ret_reg_pair won't be set. Could this result in the verifier failing to enforce the initialization of R2, potentially leaking uninitialized register contents to the caller?

But the comment needs update for this version. bpf_check_attach_target() -> btf_check_func_type_match()

Fixed while applying.

author reviewer
#7 lore raw

PATCH v7 3/10 bpf: Wire up JIT support for 16-byte kfunc returns

In reply to Yonghong Song

LLVM 23 returns an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch added the shared helpers describing that convention; wire up the JIT side so that the second half of the return value actually lands in R2.

Placing the second return half into R2 is possible on any JIT, but it needs architecture-specific JIT work. Rather than requiring every JIT to implement it at once, add a bpf_jit_supports_kfunc_ret_reg_pair() capability, defaulting to false in the generic core; an architecture opts in once its JIT handles the R0:R2 pair, and the remaining ones are left for future work. Only the x86-64, arm64 and riscv64 JITs opt in so far.

On arm64 and riscv64 the native second return register is already BPF R2 (x1 in bpf2a64[] and a1 in regmap[] respectively), so the upper half needs no move at all, unlike x86-64's RDX->RSI. The lower half is covered by the move into BPF R0 that those JITs already emit after every call, from x0 into x8 and from a0 into a5. This has been tested on x86-64 and arm64. The riscv64 path is expected to work by the same register-mapping reasoning as arm64 but has not been tested.

bpf_add_kfunc_call() also rejects a kfunc that is marked KF_FASTCALL and returns more than 8 bytes. The bpf_fastcall contract implemented by mark_fastcall_pattern_for_call() assumes a call clobbers R0 plus the registers holding its arguments, so a return in the R0:R2 pair would clobber an R2 the caller expects the fastcall pattern to preserve. Such a kfunc is rejected with -EOPNOTSUPP as well.

---
 arch/arm64/net/bpf_jit_comp.c   |  5 +++++
 arch/riscv/net/bpf_jit_comp64.c |  5 +++++
 arch/x86/net/bpf_jit_comp.c     | 27 ++++++++++++++++++++-------
 include/linux/filter.h          |  1 +
 kernel/bpf/core.c               |  5 +++++
 kernel/bpf/verifier.c           | 12 ++++++++++++
 6 files changed, 48 insertions(+), 7 deletions(-)
arch/arm64/net/bpf_jit_comp.c +5 -0
index c18e005a41db..3aa3ea0bc30b 100644
@@ -2388,6 +2388,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_stack_args(void)
 {
 	return true;
arch/riscv/net/bpf_jit_comp64.c +5 -0
index 74efe4b138d2..47c7bf431ba8 100644
@@ -2121,6 +2121,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_ptr_xchg(void)
 {
 	return true;
arch/x86/net/bpf_jit_comp.c +20 -7
index 1a9fb530adc3..48429fae0641 100644
@@ -1689,17 +1689,12 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
  * arena NULL is offset 0. Return the number of emitted bytes.
  */
 static int emit_kfunc_arena_args(struct bpf_prog *bpf_prog,
-				 const struct bpf_insn *insn, u8 **pprog)
+				 const struct btf_func_model *fm, u8 **pprog)
 {
-	const struct btf_func_model *fm;
 	u8 *prog = *pprog;
 	u8 *start = prog;
 	int i;
 
-	fm = bpf_jit_find_kfunc_model(bpf_prog, insn);
-	if (!fm)
-		return -EINVAL;
-
 	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
 		u8 flags = fm->arg_flags[i];
 		u32 reg = BPF_REG_1 + i;
@@ -2644,6 +2639,8 @@ st:			insn_off = insn->off;
 
 			/* call */
 		case BPF_JMP | BPF_CALL: {
+			const struct btf_func_model *fm = NULL;
+
 			func = (u8 *) __bpf_call_base + imm32;
 			if (src_reg == BPF_PSEUDO_CALL && tail_call_reachable) {
 				LOAD_TAIL_CALL_CNT_PTR(stack_depth);
@@ -2652,7 +2649,10 @@ st:			insn_off = insn->off;
 			if (!imm32)
 				return -EINVAL;
 			if (src_reg == BPF_PSEUDO_KFUNC_CALL) {
-				err = emit_kfunc_arena_args(bpf_prog, insn, &prog);
+				fm = bpf_jit_find_kfunc_model(bpf_prog, insn);
+				if (!fm)
+					return -EINVAL;
+				err = emit_kfunc_arena_args(bpf_prog, fm, &prog);
 				if (err < 0)
 					return err;
 				ip += err;
@@ -2666,6 +2666,14 @@ st:			insn_off = insn->off;
 				return -EINVAL;
 			if (priv_frame_ptr)
 				pop_r9(&prog);
+			/*
+			 * A kfunc returning more than 8 bytes hands the second
+			 * half back in RDX (the native ABI's second return reg),
+			 * but BPF expects it in R0:R2. BPF R0 is RAX (no move
+			 * needed), while BPF R2 is RSI, so copy RDX into RSI.
+			 */
+			if (fm && fm->ret_size > 8)
+				emit_mov_reg(&prog, true, BPF_REG_2, BPF_REG_3);
 			break;
 		}
 
@@ -4156,6 +4164,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_stack_args(void)
 {
 	return true;
include/linux/filter.h +1 -0
index 4a9bc6a848f2..6e746b0a0930 100644
@@ -1237,6 +1237,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);
 bool bpf_jit_supports_subprog_tailcalls(void);
 bool bpf_jit_supports_percpu_insn(void);
 bool bpf_jit_supports_kfunc_call(void);
+bool bpf_jit_supports_kfunc_ret_reg_pair(void);
 bool bpf_jit_supports_stack_args(void);
 bool bpf_jit_supports_arena_args(void);
 bool bpf_jit_supports_far_kfunc_call(void);
kernel/bpf/core.c +5 -0
index d55e737ed75a..5db77d7915df 100644
@@ -3287,6 +3287,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)
 	return false;
 }
 
+bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+	return false;
+}
+
 bool __weak bpf_jit_supports_stack_args(void)
 {
 	return false;
kernel/bpf/verifier.c +12 -0
index 2b5e88fdda07..991782b402f2 100644
@@ -2916,6 +2916,18 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
 	err = btf_distill_func_proto(&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);
 	if (err)
 		return err;
+	if (func_model.ret_size > 8) {
+		if (kfunc.flags && (*kfunc.flags & KF_FASTCALL)) {
+			verbose(env, "kfunc %s with >8-byte return is not supported with KF_FASTCALL\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+		if (!bpf_jit_supports_kfunc_ret_reg_pair()) {
+			verbose(env, "kfunc %s with >8-byte return is not supported by JIT\n",
+				kfunc.name);
+			return -EOPNOTSUPP;
+		}
+	}
 
 	memset(&meta, 0, sizeof(meta));
 	meta.btf = kfunc.btf;
Signature · 1 line
2.53.0-Meta
#8 lore raw

Re: [PATCH bpf-next v7 03/10] bpf: Wire up JIT support for 16-byte kfunc returns

This isn't a bug, but could the closing sentence be folded into the opening one? The paragraph opens by saying bpf_add_kfunc_call() rejects a KF_FASTCALL kfunc that returns more than 8 bytes, and…
Quoted 22 lines

bpf: Wire up JIT support for 16-byte kfunc returns

LLVM 23 returns an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch added the shared helpers describing that convention; wire up the JIT side so that the second half of the return value actually lands in R2.

Placing the second return half into R2 is possible on any JIT, but it needs architecture-specific JIT work. Rather than requiring every JIT to implement it at once, add a bpf_jit_supports_kfunc_ret_reg_pair() capability, defaulting to false in the generic core; an architecture opts in once its JIT handles the R0:R2 pair, and the remaining ones are left for future work. Only the x86-64, arm64 and riscv64 JITs opt in so far.

On arm64 and riscv64 the native second return register is already BPF R2 (x1 in bpf2a64[] and a1 in regmap[] respectively), so the upper half needs no move at all, unlike x86-64's RDX->RSI. The lower half is covered by the move into BPF R0 that those JITs already emit after every call, from x0 into x8 and from a0 into a5. This has been tested on x86-64 and arm64. The riscv64 path is expected to work by the same register-mapping reasoning as arm64 but has not been tested.

Quoted 6 lines

bpf_add_kfunc_call() also rejects a kfunc that is marked KF_FASTCALL and returns more than 8 bytes. The bpf_fastcall contract implemented by mark_fastcall_pattern_for_call() assumes a call clobbers R0 plus the registers holding its arguments, so a return in the R0:R2 pair would clobber an R2 the caller expects the fastcall pattern to preserve. Such a kfunc is rejected with -EOPNOTSUPP as well.

This isn't a bug, but could the closing sentence be folded into the opening one? The paragraph opens by saying bpf_add_kfunc_call() rejects a KF_FASTCALL kfunc that returns more than 8 bytes, and closes with "Such a kfunc is rejected with -EOPNOTSUPP as well", which restates the same fact. The middle sentence carries the rationale (the fastcall clobber mask covers only R0 plus the argument registers, so R2 is assumed preserved), while the closing sentence adds only the errno. Something like "... is rejected with -EOPNOTSUPP" in the opening sentence would let the paragraph state the rejection once and spend its length on the fastcall rationale.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#9 lore raw

Re: [PATCH bpf-next v7 03/10] bpf: Wire up JIT support for 16-byte kfunc returns

Quoted 28 lines · On 8/18/26 11:49 PM, [email protected] wrote:
Quoted 22 lines

bpf: Wire up JIT support for 16-byte kfunc returns

LLVM 23 returns an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair. The previous patch added the shared helpers describing that convention; wire up the JIT side so that the second half of the return value actually lands in R2.

Placing the second return half into R2 is possible on any JIT, but it needs architecture-specific JIT work. Rather than requiring every JIT to implement it at once, add a bpf_jit_supports_kfunc_ret_reg_pair() capability, defaulting to false in the generic core; an architecture opts in once its JIT handles the R0:R2 pair, and the remaining ones are left for future work. Only the x86-64, arm64 and riscv64 JITs opt in so far.

On arm64 and riscv64 the native second return register is already BPF R2 (x1 in bpf2a64[] and a1 in regmap[] respectively), so the upper half needs no move at all, unlike x86-64's RDX->RSI. The lower half is covered by the move into BPF R0 that those JITs already emit after every call, from x0 into x8 and from a0 into a5. This has been tested on x86-64 and arm64. The riscv64 path is expected to work by the same register-mapping reasoning as arm64 but has not been tested.

Quoted 6 lines

bpf_add_kfunc_call() also rejects a kfunc that is marked KF_FASTCALL and returns more than 8 bytes. The bpf_fastcall contract implemented by mark_fastcall_pattern_for_call() assumes a call clobbers R0 plus the registers holding its arguments, so a return in the R0:R2 pair would clobber an R2 the caller expects the fastcall pattern to preserve. Such a kfunc is rejected with -EOPNOTSUPP as well.

Quoted 9 lines

This isn't a bug, but could the closing sentence be folded into the opening one? The paragraph opens by saying bpf_add_kfunc_call() rejects a KF_FASTCALL kfunc that returns more than 8 bytes, and closes with "Such a kfunc is rejected with -EOPNOTSUPP as well", which restates the same fact. The middle sentence carries the rationale (the fastcall clobber mask covers only R0 plus the argument registers, so R2 is assumed preserved), while the closing sentence adds only the errno. Something like "... is rejected with -EOPNOTSUPP" in the opening sentence would let the paragraph state the rejection once and spend its length on the fastcall rationale.

We should be okay here.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#10 lore raw

PATCH v7 4/10 bpf: Handle R2 as a return register in precision backtracking

In reply to Yonghong Song

Precision backtracking treats only R0 as a return register at a call/return boundary, so once the verifier starts modeling R2 that way, marking the second half of such a return precise would trip the "unexpected regs" checks in backtrack_insn() and reject a valid program.

Marking the upper half precise, for example by branching on it after a call to a static subprogram, walks backtracking into the callee and reaches its BPF_EXIT with R2 still set in the mask. Handle R2 like R0 in boundaries where a call defines the return registers.

R2 differs from R0 in that it is an argument register as well, so it is part of the BPF_REGMASK_ARGS check and has to be cleared before that check rather than next to R0. Clear it unconditionally, rather than only where the callee or the kfunc really does return a pair. That gives up the "unexpected regs" assertion for R2, and in exchange keeps backtracking free of any BTF lookup. Nothing is lost: a callee that does not return a pair leaves the caller's R2 uninitialized, so the main verification pass has already rejected any program that reads it, and backtracking is never asked for its precision.

At BPF_EXIT the return registers are sampled before the callback path clears R1-R5. That clear does not touch R0, but it does cover R2, and running it first would drop a pair return whenever the instruction following the call happens to be one that invokes a callback.

---
 kernel/bpf/backtrack.c | 46 +++++++++++++++++++++++++++---------------
 1 file changed, 30 insertions(+), 16 deletions(-)
kernel/bpf/backtrack.c +30 -16
index a2b18a9f1694..653db80bcc47 100644
@@ -423,6 +423,10 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 				 */
 				verifier_bug_if(idx + 1 != subseq_idx, env,
 						"extra insn from subprog");
+				/* global subprog always sets R0 */
+				bt_clear_reg(bt, BPF_REG_0);
+				/* and if it does not set R2, main pass would catch it */
+				bt_clear_reg(bt, BPF_REG_2);
 				/* r1-r5 are invalidated after subprog call,
 				 * so for global func call it shouldn't be set
 				 * anymore
@@ -432,8 +436,6 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 						     bt_reg_mask(bt));
 					return -EFAULT;
 				}
-				/* global subprog always sets R0 */
-				bt_clear_reg(bt, BPF_REG_0);
 				return 0;
 			} else {
 				/* static subprog call instruction, which
@@ -506,6 +508,8 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 				return -ENOTSUPP;
 			/* regular helper call sets R0 */
 			bt_clear_reg(bt, BPF_REG_0);
+			/* kfunc might also set R2 */
+			bt_clear_reg(bt, BPF_REG_2);
 			if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
 				/* if backtracking was looking for registers R1-R5
 				 * they should have been found already.
@@ -520,7 +524,25 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 					return -EFAULT;
 			}
 		} else if (opcode == BPF_EXIT) {
-			bool r0_precise;
+			bool from_subprog_call, r0_precise, r2_precise;
+
+			/* BPF_EXIT in subprog or callback always returns
+			 * right after the call instruction, so by checking
+			 * whether the instruction at subseq_idx-1 is subprog
+			 * call or not we can distinguish actual exit from
+			 * *subprog* from exit from *callback*. In the former
+			 * case, we need to propagate the precision of the
+			 * return registers, if necessary. In the latter we
+			 * never do that.
+			 */
+			from_subprog_call = subseq_idx - 1 >= 0 &&
+					    bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+
+			/* Sample the return registers before the callback
+			 * handling below clears R1-R5.
+			 */
+			r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
+			r2_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_2);
 
 			/* Backtracking to a nested function call, 'idx' is a part of
 			 * the inner frame 'subseq_idx' is a part of the outer frame.
@@ -533,30 +555,22 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 			if (subseq_idx >= 0 && bpf_calls_callback(env, subseq_idx))
 				for (i = BPF_REG_1; i <= BPF_REG_5; i++)
 					bt_clear_reg(bt, i);
+
+			bt_clear_reg(bt, BPF_REG_0);
+			bt_clear_reg(bt, BPF_REG_2);
 			if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
 				verifier_bug(env, "backtracking exit unexpected regs %x",
 					     bt_reg_mask(bt));
 				return -EFAULT;
 			}
 
-			/* BPF_EXIT in subprog or callback always returns
-			 * right after the call instruction, so by checking
-			 * whether the instruction at subseq_idx-1 is subprog
-			 * call or not we can distinguish actual exit from
-			 * *subprog* from exit from *callback*. In the former
-			 * case, we need to propagate r0 precision, if
-			 * necessary. In the former we never do that.
-			 */
-			r0_precise = subseq_idx - 1 >= 0 &&
-				     bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]) &&
-				     bt_is_reg_set(bt, BPF_REG_0);
-
-			bt_clear_reg(bt, BPF_REG_0);
 			if (bt_subprog_enter(bt))
 				return -EFAULT;
 
 			if (r0_precise)
 				bt_set_reg(bt, BPF_REG_0);
+			if (r2_precise)
+				bt_set_reg(bt, BPF_REG_2);
 			/* r6-r9 and stack slots will stay set in caller frame
 			 * bitmasks until we return back from callee(s)
 			 */
Signature · 1 line
2.53.0-Meta
#11 lore raw

Re: [PATCH bpf-next v7 04/10] bpf: Handle R2 as a return register in precision backtracking

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Multi-line comments do not follow the required BPF subsystem style. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Multi-line comments do not follow the required BPF subsystem style. --

--- Patch [4]: [PATCH bpf-next v7 04/10] bpf: Handle R2 as a return register in precision backtracking --- commit 827448f59c95b01086f128d7ec2894a154de5092
Author: Yonghong Song <[email protected]>

bpf: Handle R2 as a return register in precision backtracking

This commit updates precision backtracking to treat R2 as a return register alongside R0 at call/return boundaries. It clears R2 unconditionally when evaluating subprograms or kfuncs, and samples it before callback paths clear R1-R5.

Quoted 4 lines
kernel/bpf/backtrack.c +0 -0
index a2b18a9f1694c..653db80bcc47b 100644

[ ... ]

kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 529

Quoted 8 lines
kernel/bpf/backtrack.c +3 -1
@@ -520,7 +524,25 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 					return -EFAULT;
 			}
 		} else if (opcode == BPF_EXIT) {
-			bool r0_precise;
+			bool from_subprog_call, r0_precise, r2_precise;
+
+			/* BPF_EXIT in subprog or callback always returns

[Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred BPF subsystem style? The opening /* is usually required to be on its own line.

kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 541

Quoted 12 lines
kernel/bpf/backtrack.c +12 -0
+			 * right after the call instruction, so by checking
+			 * whether the instruction at subseq_idx-1 is subprog
+			 * call or not we can distinguish actual exit from
+			 * *subprog* from exit from *callback*. In the former
+			 * case, we need to propagate the precision of the
+			 * return registers, if necessary. In the latter we
+			 * never do that.
+			 */
+			from_subprog_call = subseq_idx - 1 >= 0 &&
+					    bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+
+			/* Sample the return registers before the callback

[Severity: Low]
This isn't a bug either, but the same formatting question applies here. Should the opening /* be on its own line for multi-line comments?

Quoted 4 lines
kernel/bpf/backtrack.c +4 -0
+			 * handling below clears R1-R5.
+			 */
+			r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
+			r2_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_2);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author reviewer
#12 lore raw

Re: [PATCH bpf-next v7 04/10] bpf: Handle R2 as a return register in precision backtracking

Quoted 33 lines · On 8/18/26 11:03 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Multi-line comments do not follow the required BPF subsystem style. --

--- Patch [4]: [PATCH bpf-next v7 04/10] bpf: Handle R2 as a return register in precision backtracking --- commit 827448f59c95b01086f128d7ec2894a154de5092
Author: Yonghong Song <[email protected]>

bpf: Handle R2 as a return register in precision backtracking

This commit updates precision backtracking to treat R2 as a return register alongside R0 at call/return boundaries. It clears R2 unconditionally when evaluating subprograms or kfuncs, and samples it before callback paths clear R1-R5.

Quoted 4 lines
kernel/bpf/backtrack.c +0 -0
index a2b18a9f1694c..653db80bcc47b 100644

[ ... ]

kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 529

Quoted 8 lines
kernel/bpf/backtrack.c +3 -1
@@ -520,7 +524,25 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
  					return -EFAULT;
  			}
  		} else if (opcode == BPF_EXIT) {
-			bool r0_precise;
+			bool from_subprog_call, r0_precise, r2_precise;
+
+			/* BPF_EXIT in subprog or callback always returns

[Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred BPF subsystem style? The opening /* is usually required to be on its own line.

Quoted 15 lines

kernel/bpf/backtrack.c static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx, · line 541

Quoted 12 lines
kernel/bpf/backtrack.c +12 -0
+			 * right after the call instruction, so by checking
+			 * whether the instruction at subseq_idx-1 is subprog
+			 * call or not we can distinguish actual exit from
+			 * *subprog* from exit from *callback*. In the former
+			 * case, we need to propagate the precision of the
+			 * return registers, if necessary. In the latter we
+			 * never do that.
+			 */
+			from_subprog_call = subseq_idx - 1 >= 0 &&
+					    bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+
+			/* Sample the return registers before the callback

[Severity: Low]
This isn't a bug either, but the same formatting question applies here. Should the opening /* be on its own line for multi-line comments?

We should be okay here.

Quoted 5 lines
Quoted 5 lines
kernel/bpf/backtrack.c +4 -0
+			 * handling below clears R1-R5.
+			 */
+			r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
+			r2_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_2);
author reviewer
#13 lore raw

PATCH v7 5/10 bpf: Account R2 of register-pair returns in live register analysis

In reply to Yonghong Song

A BPF_EXIT of a subprogram returning a value larger than 8 bytes (a struct/union or an __int128) reads R2 as well as R0, since the second half of the return value is passed back in R2. compute_insn_live_regs() only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.

Mark R2 as read at the BPF_EXIT of a subprogram that does return a register pair. bpf_compute_live_registers() now loops over the subprograms and, for each, over the [start, end) instruction range from env->subprog_info[], so the return convention is queried once per subprogram through bpf_ret_reg_pair() rather than once per instruction.

Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.

---
 kernel/bpf/liveness.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
kernel/bpf/liveness.c +14 -6
index 74fc4b3f80d6..71f998c6eb88 100644
@@ -2060,7 +2060,8 @@ static inline u16 mask_hi(u32 m) { return (u16)(m >> 16); }
 /* Compute info->{use,def} fields for the instruction */
 static void compute_insn_live_regs(struct bpf_verifier_env *env,
 				   struct bpf_insn *insn,
-				   struct insn_live_regs *info)
+				   struct insn_live_regs *info,
+				   bool ret_reg_pair)
 {
 	struct bpf_call_summary cs;
 	const u8 class = BPF_CLASS(insn->code);
@@ -2072,6 +2073,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 	const u32 src32 = mask_lo(src);
 	const u32 dst32 = mask_lo(dst);
 	const u32 r0  = reg64_mask(0);
+	const u32 r2  = reg64_mask(BPF_REG_2);
 	u32 def = 0;
 	u32 use = U32_MAX;
 
@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 			break;
 		case BPF_EXIT:
 			def = 0;
-			use = r0;
+			use = ret_reg_pair ? (r0 | r2) : r0;
 			break;
 		case BPF_CALL:
 			def = ALL_CALLER_SAVED_REGS;
@@ -2228,8 +2230,8 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 	struct insn_live_regs *state;
 	int insn_cnt = env->prog->len;
 	u64 pos, insn_pos;
-	int err = 0, i, j;
-	bool changed;
+	int err = 0, i, j, subprog, start, end;
+	bool changed, ret_reg_pair;
 
 	/* Use the following algorithm:
 	 * - define the following:
@@ -2256,8 +2258,14 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 		goto out;
 	}
 
-	for (i = 0; i < insn_cnt; ++i)
-		compute_insn_live_regs(env, &insns[i], &state[i]);
+	for (subprog = 0; subprog < env->subprog_cnt; subprog++) {
+		start = env->subprog_info[subprog].start;
+		end = env->subprog_info[subprog + 1].start;
+		ret_reg_pair = bpf_ret_reg_pair(env, subprog);
+
+		for (i = start; i < end; ++i)
+			compute_insn_live_regs(env, &insns[i], &state[i], ret_reg_pair);
+	}
 
 	/* Forward pass: resolve stack access through FP-derived pointers */
 	err = bpf_compute_subprog_arg_access(env);
Signature · 1 line
2.53.0-Meta
#14 lore raw

Re: [PATCH bpf-next v7 05/10] bpf: Account R2 of register-pair returns in live register analysis

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The verifier fails to properly model execution for BPF subprograms and kfuncs returning >8 bytes…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The verifier fails to properly model execution for BPF subprograms and kfuncs returning >8 bytes (register pairs), specifically by not restoring R2 to the caller and not verifying R2 initialization. --

--- Patch [5]: [PATCH bpf-next v7 05/10] bpf: Account R2 of register-pair returns in live register analysis --- commit 1cab80ee79c6b836ae7862851a98688dd786f8f2
Author: Yonghong Song <[email protected]>

bpf: Account R2 of register-pair returns in live register analysis

This commit modifies live register analysis to mark R2 as read at BPF_EXIT when a subprogram returns a register pair, such as an __int128. This prevents a callee's R2 from being considered dead and incorrectly pruned from checkpointed states.

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
index 74fc4b3f80d6e..71f998c6eb888 100644

[ ... ]

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2196

Quoted 6 lines
kernel/bpf/liveness.c +1 -1
@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 			break;
 		case BPF_EXIT:
 			def = 0;
-			use = r0;
+			use = ret_reg_pair ? (r0 | r2) : r0;

[Severity: High]
Does the verifier also need to be updated to properly model execution for BPF subprograms and kfuncs returning > 8 bytes?

While this patch handles the liveness side, it looks like kernel/bpf/verifier.c only copies or initializes BPF_REG_0 (and not BPF_REG_2) to the caller's state in several places.

In prepare_func_exit():

		/* return to the caller whatever r0 had in the callee */
		bpf_diag_mod_begin(env, &caller->regs[BPF_REG_0], r0, BPF_DIAG_MOD_WRITE);
		caller->regs[BPF_REG_0] = *r0;

In check_func_call():

		/* All non-void global functions return a 64-bit SCALAR_VALUE. */
		if (!returns_void) {
			mark_reg_unknown(env, caller->regs, BPF_REG_0);
			bpf_diag_mod_end(env);
		}

In check_kfunc_call():

	if (btf_type_is_scalar(t)) {
		mark_reg_unknown(env, regs, BPF_REG_0);

And in check_global_subprog_return_code():

	if (subprog_returns_void(env, subprog))
		return 0;

	return check_global_ret_scalar_reg(env, BPF_REG_0);

If a program attempts to read the upper half (R2) of a 16-byte return value, will it be rejected by the verifier with an R2 !read_ok error?

Additionally, for global subprograms returning 16-byte types, could the lack of R2 initialization checks allow uninitialized memory to be passed back to the kernel?

Quoted 3 lines
kernel/bpf/liveness.c +0 -0
 			break;
 		case BPF_CALL:
 			def = ALL_CALLER_SAVED_REGS;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
author reviewer
#15 lore raw

Re: [PATCH bpf-next v7 05/10] bpf: Account R2 of register-pair returns in live register analysis

Quoted 61 lines · On 8/18/26 11:20 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The verifier fails to properly model execution for BPF subprograms and kfuncs returning >8 bytes (register pairs), specifically by not restoring R2 to the caller and not verifying R2 initialization. --

--- Patch [5]: [PATCH bpf-next v7 05/10] bpf: Account R2 of register-pair returns in live register analysis --- commit 1cab80ee79c6b836ae7862851a98688dd786f8f2
Author: Yonghong Song <[email protected]>

bpf: Account R2 of register-pair returns in live register analysis

This commit modifies live register analysis to mark R2 as read at BPF_EXIT when a subprogram returns a register pair, such as an __int128. This prevents a callee's R2 from being considered dead and incorrectly pruned from checkpointed states.

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
index 74fc4b3f80d6e..71f998c6eb888 100644

[ ... ]

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2196

Quoted 6 lines
kernel/bpf/liveness.c +1 -1
@@ -2191,7 +2193,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
  			break;
  		case BPF_EXIT:
  			def = 0;
-			use = r0;
+			use = ret_reg_pair ? (r0 | r2) : r0;

[Severity: High]
Does the verifier also need to be updated to properly model execution for BPF subprograms and kfuncs returning > 8 bytes?

While this patch handles the liveness side, it looks like kernel/bpf/verifier.c only copies or initializes BPF_REG_0 (and not BPF_REG_2) to the caller's state in several places.

In prepare_func_exit():

		/* return to the caller whatever r0 had in the callee */
		bpf_diag_mod_begin(env, &caller->regs[BPF_REG_0], r0, BPF_DIAG_MOD_WRITE);
		caller->regs[BPF_REG_0] = *r0;

In check_func_call():

		/* All non-void global functions return a 64-bit SCALAR_VALUE. */
		if (!returns_void) {
			mark_reg_unknown(env, caller->regs, BPF_REG_0);
			bpf_diag_mod_end(env);
		}

In check_kfunc_call():

	if (btf_type_is_scalar(t)) {
		mark_reg_unknown(env, regs, BPF_REG_0);

And in check_global_subprog_return_code():

	if (subprog_returns_void(env, subprog))
		return 0;

	return check_global_ret_scalar_reg(env, BPF_REG_0);
Quoted 6 lines

If a program attempts to read the upper half (R2) of a 16-byte return value, will it be rejected by the verifier with an R2 !read_ok error?

Additionally, for global subprograms returning 16-byte types, could the lack of R2 initialization checks allow uninitialized memory to be passed back to the kernel?

We should be okay, later patch "bpf: Add verifier support for 16-byte returns in R0:R2" will add R2 support properly.

Quoted 4 lines
Quoted 4 lines
kernel/bpf/liveness.c +0 -0
  			break;
  		case BPF_CALL:
  			def = ALL_CALLER_SAVED_REGS;
#16 lore raw

Re: [PATCH bpf-next v7 05/10] bpf: Account R2 of register-pair returns in live register analysis

In reply to Yonghong Song
This commit fixes a bug where R2 was not marked as live at BPF_EXIT for subprograms returning register pairs, which could allow unsound state pruning. The bug was introduced when register-pair…
Quoted 14 lines

bpf: Account R2 of register-pair returns in live register analysis

A BPF_EXIT of a subprogram returning a value larger than 8 bytes (a struct/union or an __int128) reads R2 as well as R0, since the second half of the return value is passed back in R2. compute_insn_live_regs() only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.

Mark R2 as read at the BPF_EXIT of a subprogram that does return a register pair. bpf_compute_live_registers() now loops over the subprograms and, for each, over the [start, end) instruction range from env->subprog_info[], so the return convention is queried once per subprogram through bpf_ret_reg_pair() rather than once per instruction.

Quoted 6 lines

Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.

This commit fixes a bug where R2 was not marked as live at BPF_EXIT for subprograms returning register pairs, which could allow unsound state pruning. The bug was introduced when register-pair return support was added. Should this include:

Fixes b176d222c14a ("bpf: Add helpers to describe the R0:R2 return register pair")

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#17 lore raw

Re: [PATCH bpf-next v7 05/10] bpf: Account R2 of register-pair returns in live register analysis

Quoted 18 lines · On 8/18/26 11:32 PM, [email protected] wrote:
Quoted 18 lines

bpf: Account R2 of register-pair returns in live register analysis

A BPF_EXIT of a subprogram returning a value larger than 8 bytes (a struct/union or an __int128) reads R2 as well as R0, since the second half of the return value is passed back in R2. compute_insn_live_regs() only marked R0 used at exit, so a callee's R2 could be considered dead and cleaned from checkpointed states, which would allow unsound state pruning.

Mark R2 as read at the BPF_EXIT of a subprogram that does return a register pair. bpf_compute_live_registers() now loops over the subprograms and, for each, over the [start, end) instruction range from env->subprog_info[], so the return convention is queried once per subprogram through bpf_ret_reg_pair() rather than once per instruction.

Marking R2 at every exit instead would be simpler, but R2 would then stay live backwards across any call that is not followed by a write to R2, which is nearly every program, and would needlessly hurt state pruning.

Quoted 8 lines
Quoted 2 lines

This commit fixes a bug where R2 was not marked as live at BPF_EXIT for subprograms returning register pairs, which could allow unsound state pruning. The bug was introduced when register-pair return support was added. Should this include:

Fixes b176d222c14a ("bpf: Add helpers to describe the R0:R2 return register pair")

No need to have above Fixes in the same series.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#18 lore raw

PATCH v7 6/10 bpf: Add verifier support for 16-byte returns in R0:R2

In reply to Yonghong Song

LLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].

Before LLVM 23 the BPF backend could not return these values at all. A by-value struct or union return (of any size) was rejected at compile time with:

error: aggregate returns are not supported

and an __int128 return failed later in the backend with:

fatal error: error in backend: unable to allocate function return #1

Both are resolved in LLVM 23, which lowers such returns into the R0:R2 register pair.

Model that pair at calls to global and static BPF subprograms and at kfunc
calls: R2 is marked alongside R0 at the call, propagated out of a callee at its exit, and held to the same scalar-only and no-stack-pointer rules that already apply to R0. A struct returned by a kfunc must be composed of scalars, since its bytes reach the program as raw register contents and a pointer field would otherwise be laundered into a scalar.

An extension program is the one caller of the convention that cannot take part in it: only R0 is checked at its exit, since that is the program exit code, so it has no way to hand back an upper half. Replacing a function whose return value is larger than 8 bytes is therefore rejected in btf_check_func_type_match(). That function compares btf_type->info, which carries no size for an int, so an 8 byte long and a 16 byte __int128 compared equal; sizes up to 8 bytes stay interchangeable, as they always have been, so this only rejects what the R0:R2 convention newly makes incompatible.

[1] https://github.com/llvm/llvm-project/pull/190894
[2] https://github.com/llvm/llvm-project/pull/206876

---
 kernel/bpf/btf.c      |  6 ++++
 kernel/bpf/verifier.c | 68 ++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 67 insertions(+), 7 deletions(-)
kernel/bpf/btf.c +6 -0
index 6967d48bba49..5e9f6a487524 100644
@@ -7684,6 +7684,12 @@ static int btf_check_func_type_match(struct bpf_verifier_log *log,
 			btf_type_str(t2), fn2);
 		return -EINVAL;
 	}
+	if (btf_type_has_size(t1) && (t1->size > 8 || t2->size > 8)) {
+		bpf_log(log,
+			"Return type of %s() has size %u and of %s() size %u, and a size above 8 bytes cannot be replaced\n",
+			fn1, t1->size, fn2, t2->size);
+		return -EINVAL;
+	}
 
 	for (i = 0; i < nargs1; i++) {
 		t1 = btf_type_skip_modifiers(btf1, args1[i].type, NULL);
kernel/bpf/verifier.c +61 -7

129 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#19 lore raw

Re: [PATCH bpf-next v7 06/10] bpf: Add verifier support for 16-byte returns in R0:R2

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The newly added support for kfuncs returning structs is completely unreachable due to an…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added support for kfuncs returning structs is completely unreachable due to an overlooked check in btf_distill_func_proto, and hides a critical missing size limit. --

--- Patch [6]: [PATCH bpf-next v7 06/10] bpf: Add verifier support for 16-byte returns in R0:R2 --- commit c9e349733faae615b87f8da3417558d2ede26348
Author: Yonghong Song <[email protected]>

bpf: Add verifier support for 16-byte returns in R0:R2

LLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].

Before LLVM 23 the BPF backend could not return these values at all.

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index 991782b402f20..d79f0c069795c 100644

[ ... ]

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -13965,10 +13995,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	}
 
 	if (btf_type_is_scalar(t)) {
-		mark_reg_unknown(env, regs, BPF_REG_0);
+		mark_kfunc_ret_regs(env, regs, t->size);

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14008

Quoted 10 lines
kernel/bpf/verifier.c +7 -0
 		if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
 		    meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))
 			__mark_reg_const_zero(env, &regs[BPF_REG_0]);
+	} else if (btf_type_is_struct(t)) {
+		/*
+		 * The returned struct comes back as raw register bits modeled
+		 * as an unknown scalar, so it must contain only scalars:
+		 * otherwise a pointer field would be laundered into a scalar
+		 * and escape provenance and reference tracking.
+		 */

[Severity: Medium]
Is this new struct return handling reachable?

Looking at early program loading, btf_distill_func_proto() unconditionally rejects struct return types:

kernel/bpf/btf.c:btf_distill_func_proto() {
    ...
	if (ret < 0 || btf_type_is_struct(t)) {
		bpf_log(log,
			"The function %s return type %s is unsupported.\n",
			tname, btf_type_str(t));
		return -EINVAL;
	}
    ...
}

As a result, wouldn't programs be rejected before check_kfunc_call() is executed?

Additionally, if this code were reachable, does it lack a bounds check on t->size?

Because kfuncs use the C ABI, returning a large struct requires a hidden 'sret' pointer. Without a size limit, returning a struct larger than 16 bytes might bypass the BPF return pair logic, as the BPF JIT would pass the attacker-controlled BPF R1 as this pointer.

Could this potentially allow a kfunc to overwrite arbitrary kernel memory?

Quoted 11 lines
kernel/bpf/verifier.c +8 -0
+		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+			verbose(env,
+				"kernel function %s returns %s %s that is not composed of scalars\n",
+				func_name, btf_type_str(t),
+				btf_name_by_offset(desc_btf, t->name_off));
+			return -EINVAL;
+		}
+		mark_kfunc_ret_regs(env, regs, t->size);
 	} else if (btf_type_is_ptr(t)) {
 		ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id);
 		err = check_special_kfunc(env, &meta, regs, insn_aux, ptr_type, desc_btf);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author reviewer
#20 lore raw

Re: [PATCH bpf-next v7 06/10] bpf: Add verifier support for 16-byte returns in R0:R2

Quoted 63 lines · On 8/18/26 11:14 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added support for kfuncs returning structs is completely unreachable due to an overlooked check in btf_distill_func_proto, and hides a critical missing size limit. --

--- Patch [6]: [PATCH bpf-next v7 06/10] bpf: Add verifier support for 16-byte returns in R0:R2 --- commit c9e349733faae615b87f8da3417558d2ede26348
Author: Yonghong Song <[email protected]>

bpf: Add verifier support for 16-byte returns in R0:R2

LLVM 23 added support for returning a value in two registers for an __int128, or a struct/union whose size is greater than 8 but not more than 16 bytes. See LLVM patches [1] and [2].

Before LLVM 23 the BPF backend could not return these values at all.

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index 991782b402f20..d79f0c069795c 100644

[ ... ]

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -13965,10 +13995,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
  	}
 
  	if (btf_type_is_scalar(t)) {
-		mark_reg_unknown(env, regs, BPF_REG_0);
+		mark_kfunc_ret_regs(env, regs, t->size);

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14008

Quoted 10 lines
kernel/bpf/verifier.c +7 -0
  		if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
  		    meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))
  			__mark_reg_const_zero(env, &regs[BPF_REG_0]);
+	} else if (btf_type_is_struct(t)) {
+		/*
+		 * The returned struct comes back as raw register bits modeled
+		 * as an unknown scalar, so it must contain only scalars:
+		 * otherwise a pointer field would be laundered into a scalar
+		 * and escape provenance and reference tracking.
+		 */

[Severity: Medium]
Is this new struct return handling reachable?

Looking at early program loading, btf_distill_func_proto() unconditionally rejects struct return types:

kernel/bpf/btf.c:btf_distill_func_proto() {
     ...
	if (ret < 0 || btf_type_is_struct(t)) {
		bpf_log(log,
			"The function %s return type %s is unsupported.\n",
			tname, btf_type_str(t));
		return -EINVAL;
	}
     ...
}

As a result, wouldn't programs be rejected before check_kfunc_call() is executed?

Additionally, if this code were reachable, does it lack a bounds check on t->size?

Quoted 6 lines

Because kfuncs use the C ABI, returning a large struct requires a hidden 'sret' pointer. Without a size limit, returning a struct larger than 16 bytes might bypass the BPF return pair logic, as the BPF JIT would pass the attacker-controlled BPF R1 as this pointer.

Could this potentially allow a kfunc to overwrite arbitrary kernel memory?

Patch 7 "bpf: Enable aggregate return types up to 16 bytes" will have proper guard (e.g. return type must be <= 16 bytes). So we should be fine here.

Quoted 12 lines
Quoted 12 lines
kernel/bpf/verifier.c +8 -0
+		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+			verbose(env,
+				"kernel function %s returns %s %s that is not composed of scalars\n",
+				func_name, btf_type_str(t),
+				btf_name_by_offset(desc_btf, t->name_off));
+			return -EINVAL;
+		}
+		mark_kfunc_ret_regs(env, regs, t->size);
  	} else if (btf_type_is_ptr(t)) {
  		ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id);
  		err = check_special_kfunc(env, &meta, regs, insn_aux, ptr_type, desc_btf);
author reviewer
#21 lore raw

PATCH v7 7/10 bpf: Enable aggregate return types up to 16 bytes

In reply to Yonghong Song

Relax btf_distill_func_proto() to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry:

- a struct or union larger than 8 and up to 16 bytes, returned in the R0:R2 register pair, matching what LLVM emits for the BPF target;
- a struct or union up to 8 bytes, returned in R0 alone.

A >8 byte scalar (__int128) was already accepted and is unchanged. Everything else stays rejected: a return type larger than 16 bytes, and any type that __get_type_size() cannot return in registers at all (e.g. an array), which it already reports as ret < 0.

btf_validate_return_type() is relaxed as well, so that it accepts a by-value struct or union up to 16 bytes in addition to void and scalars.

With btf_distill_func_proto() and btf_validate_return_type() relaxed, the verifier, JIT, precision-backtracking and live-register support from the earlier patches becomes reachable: <=16 byte aggregate return values now work end to end.

---
 include/linux/bpf_verifier.h                  |  2 ++
 kernel/bpf/btf.c                              | 23 +++++++++++++++----
 kernel/bpf/verifier.c                         | 18 +++++++--------
 .../selftests/bpf/progs/exceptions_fail.c     |  2 +-
 4 files changed, 30 insertions(+), 15 deletions(-)
include/linux/bpf_verifier.h +2 -0
index 953d8191563a..004b06785521 100644
@@ -1488,6 +1488,8 @@ int bpf_jmp_offset(struct bpf_insn *insn);
 struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);
 void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);
 bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog);
+bool btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,
+			       const struct btf_type *t, int rec);
 
 int bpf_find_subprog(struct bpf_verifier_env *env, int off);
 bool bpf_is_throw_kfunc(struct bpf_insn *insn);
kernel/bpf/btf.c +18 -5
index 5e9f6a487524..58b81c6f5288 100644
@@ -7591,7 +7591,7 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,
 		return -EINVAL;
 	}
 	ret = __get_type_size(btf, func->type, &t);
-	if (ret < 0 || btf_type_is_struct(t)) {
+	if (ret < 0 || ret > 16) {
 		bpf_log(log,
 			"The function %s return type %s is unsupported.\n",
 			tname, btf_type_str(t));
@@ -7970,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,
 
 /* Check whether the type is a valid return type. */
 static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btf,
-		const struct btf_type *t, int subprog)
+		const struct btf_type *t, int subprog, bool is_global)
 {
 	u32 tags = 0;
 	int err;
@@ -7993,6 +7993,19 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
 	if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
 		return 0;
 
+	if (btf_type_is_struct(t) && t->size <= 16) {
+		/*
+		 * A global function's caller models the return as an opaque
+		 * scalar pair, so it may only return scalars by value. A local
+		 * function is verified inline, so a pointer field stays tracked
+		 * and needs no such restriction.
+		 */
+		bool local_func = subprog && !is_global;
+
+		if (local_func || btf_type_is_scalar_struct(env, btf, t, 0))
+			return 0;
+	}
+
 	return -EOPNOTSUPP;
 }
 
@@ -8080,12 +8093,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 		return -EINVAL;
 	}
 
-	err = btf_validate_return_type(env, btf, t, subprog);
+	err = btf_validate_return_type(env, btf, t, subprog, is_global);
 	if (err) {
 		if (is_global) {
 			bpf_log(log,
-				"Global function %s() return value not void or scalar. "
-				"Only those are supported.\n",
+				"Global function %s() has unsupported return type. "
+				"Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\n",
 				tname);
 		}
 		return err;
kernel/bpf/verifier.c +9 -9
index d79f0c069795..7a939dfa3797 100644
@@ -11594,9 +11594,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_
 }
 
 /* Returns true if struct is composed of scalars, 4 levels of nesting allowed */
-static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
-					const struct btf *btf,
-					const struct btf_type *t, int rec)
+bool btf_type_is_scalar_struct(struct bpf_verifier_env *env,
+			       const struct btf *btf,
+			       const struct btf_type *t, int rec)
 {
 	const struct btf_type *member_type;
 	const struct btf_member *member;
@@ -11614,7 +11614,7 @@ static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
 				verbose(env, "max struct nesting depth exceeded\n");
 				return false;
 			}
-			if (!__btf_type_is_scalar_struct(env, btf, member_type, rec + 1))
+			if (!btf_type_is_scalar_struct(env, btf, member_type, rec + 1))
 				return false;
 			continue;
 		}
@@ -12013,7 +12013,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 		 (is_kfunc_arg_mem_size(meta->btf, &args[arg + 1]) ||
 		  is_kfunc_arg_const_mem_size(meta->btf, &args[arg + 1]))) {
 		if (!btf_type_is_void(ref_t) && !btf_type_is_scalar(ref_t) &&
-		    !__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
+		    !btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
 			verbose(env, "%s pointer type %s %s must point to void, scalar, or struct with scalar\n",
 				reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);
 			return -EINVAL;
@@ -12029,7 +12029,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 		 * scalars. The access size is derived from the pointed-to BTF type.
 		 */
 		if (!btf_type_is_scalar(ref_t) &&
-		    !__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
+		    !btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
 			verbose(env, "%s pointer type %s %s must point to scalar, or struct with scalar\n",
 				reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);
 			return -EINVAL;
@@ -13085,7 +13085,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
 				break;
 			}
 
-			if (!__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
+			if (!btf_type_is_scalar_struct(env, meta->btf, ref_t, 0)) {
 				enum bpf_reg_type reg2btf_type = lookup_reg2btf_ids(ref_id);
 				const char *expected_type;
 
@@ -13627,7 +13627,7 @@ static int check_special_kfunc(struct bpf_verifier_env *env, struct bpf_call_arg
 
 		struct_meta = btf_find_struct_meta(ret_btf, ret_btf_id);
 		if (is_bpf_percpu_obj_new_kfunc(meta->func_id)) {
-			if (!__btf_type_is_scalar_struct(env, ret_btf, ret_t, 0)) {
+			if (!btf_type_is_scalar_struct(env, ret_btf, ret_t, 0)) {
 				verbose(env, "bpf_percpu_obj_new type ID argument must be of a struct of scalars\n");
 				return -EINVAL;
 			}
@@ -14006,7 +14006,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		 * otherwise a pointer field would be laundered into a scalar
 		 * and escape provenance and reference tracking.
 		 */
-		if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+		if (!btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
 			verbose(env,
 				"kernel function %s returns %s %s that is not composed of scalars\n",
 				func_name, btf_type_str(t),
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
index ac44d60e5066..9708efb93683 100644
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a)
 
 SEC("?tc")
 __exception_cb(exception_cb_bad_ret_type1)
-__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.")
+__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")
 int reject_exception_cb_type_1(struct __sk_buff *ctx)
 {
 	bpf_throw(0);
Signature · 1 line
2.53.0-Meta
#22 lore raw

Re: [PATCH bpf-next v7 07/10] bpf: Enable aggregate return types up to 16 bytes

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] BPF extension programs (BPF_PROG_TYPE_EXT) bypass return value verification for BPF_REG_2 when…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] BPF extension programs (BPF_PROG_TYPE_EXT) bypass return value verification for BPF_REG_2 when replacing a function that returns a 16-byte struct, allowing uninitialized memory or kernel pointers to leak. --

--- Patch [7]: [PATCH bpf-next v7 07/10] bpf: Enable aggregate return types up to 16 bytes --- commit 19d85b11c1fe37fb0f420056e54cf29ffa04a0bb
Author: Yonghong Song <[email protected]>

bpf: Enable aggregate return types up to 16 bytes

Relax btf_distill_func_proto() to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry.

Quoted 4 lines
kernel/bpf/btf.c +0 -0
index 5e9f6a4875244..58b81c6f5288c 100644

[ ... ]

Quoted 9 lines
kernel/bpf/btf.c +5 -0
@@ -7993,6 +7993,19 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
 	if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
 		return 0;
 
+	if (btf_type_is_struct(t) && t->size <= 16) {
+		/*
+		 * A global function's caller models the return as an opaque
+		 * scalar pair, so it may only return scalars by value. A local
+		 * function is verified inline, so a pointer field stays tracked

kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8010

Quoted 10 lines
kernel/bpf/btf.c +8 -0
+		 * and needs no such restriction.
+		 */
+		bool local_func = subprog && !is_global;
+
+		if (local_func || btf_type_is_scalar_struct(env, btf, t, 0))
+			return 0;
+	}
+
 	return -EOPNOTSUPP;
 }

[Severity: High]
Does this allow BPF_PROG_TYPE_EXT extension programs to bypass return value verification for BPF_REG_2?

Since this patch enables global functions to return structs up to 16 bytes, a BPF_PROG_TYPE_EXT program can now replace such a function. This is allowed by bpf_check_attach_target() because attach_uses_trampoline_retval() is false for extension programs.

However, during verification of the extension program, bpf_compute_subprog_ret_regs() explicitly skips subprog 0 (the extension program itself), which leaves ret_reg_pair as false:

kernel/bpf/verifier.c:bpf_compute_subprog_ret_regs() {
	/*
	 * Skip the main program: its return value is the program's exit code,
	 * read out of R0, so it never uses the register pair. An extension does
	 * have a real prototype for subprog 0, but bpf_check_attach_target()
	 * refuses to replace a function returning more than 8 bytes.
	 */
	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
        ...
}

Because ret_reg_pair is false, process_bpf_exit_full() later invokes check_return_code(..., BPF_REG_0) for subprog 0 instead of checking the full register pair. Could this fail to validate R2 on exit, potentially allowing an uninitialized or sensitive kernel pointer to leak to the caller?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
author reviewer
#23 lore raw

Re: [PATCH bpf-next v7 07/10] bpf: Enable aggregate return types up to 16 bytes

Quoted 61 lines · On 8/18/26 11:13 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] BPF extension programs (BPF_PROG_TYPE_EXT) bypass return value verification for BPF_REG_2 when replacing a function that returns a 16-byte struct, allowing uninitialized memory or kernel pointers to leak. --

--- Patch [7]: [PATCH bpf-next v7 07/10] bpf: Enable aggregate return types up to 16 bytes --- commit 19d85b11c1fe37fb0f420056e54cf29ffa04a0bb
Author: Yonghong Song <[email protected]>

bpf: Enable aggregate return types up to 16 bytes

Relax btf_distill_func_proto() to accept a by-value struct or union that the R0:R2 convention added in earlier patches can carry.

Quoted 4 lines
kernel/bpf/btf.c +0 -0
index 5e9f6a4875244..58b81c6f5288c 100644

[ ... ]

Quoted 9 lines
kernel/bpf/btf.c +5 -0
@@ -7993,6 +7993,19 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
  	if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
  		return 0;
 
+	if (btf_type_is_struct(t) && t->size <= 16) {
+		/*
+		 * A global function's caller models the return as an opaque
+		 * scalar pair, so it may only return scalars by value. A local
+		 * function is verified inline, so a pointer field stays tracked

kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8010

Quoted 10 lines
kernel/bpf/btf.c +8 -0
+		 * and needs no such restriction.
+		 */
+		bool local_func = subprog && !is_global;
+
+		if (local_func || btf_type_is_scalar_struct(env, btf, t, 0))
+			return 0;
+	}
+
  	return -EOPNOTSUPP;
  }

[Severity: High]
Does this allow BPF_PROG_TYPE_EXT extension programs to bypass return value verification for BPF_REG_2?

Since this patch enables global functions to return structs up to 16 bytes, a BPF_PROG_TYPE_EXT program can now replace such a function. This is allowed by bpf_check_attach_target() because attach_uses_trampoline_retval() is false for extension programs.

However, during verification of the extension program, bpf_compute_subprog_ret_regs() explicitly skips subprog 0 (the extension program itself), which leaves ret_reg_pair as false:

kernel/bpf/verifier.c:bpf_compute_subprog_ret_regs() {
	/*
	 * Skip the main program: its return value is the program's exit code,
	 * read out of R0, so it never uses the register pair. An extension does
	 * have a real prototype for subprog 0, but bpf_check_attach_target()
	 * refuses to replace a function returning more than 8 bytes.
	 */
	for (subprog = 1; subprog < env->subprog_cnt; subprog++) {
         ...
}
Quoted 4 lines

Because ret_reg_pair is false, process_bpf_exit_full() later invokes check_return_code(..., BPF_REG_0) for subprog 0 instead of checking the full register pair. Could this fail to validate R2 on exit, potentially allowing an uninitialized or sensitive kernel pointer to leak to the caller?

We should be okay here. If freplace replacing the original subprog 0, it is up to users to check the result if the freplace returning R0:R2.

Quoted 1 line
#24 lore raw

Re: [PATCH bpf-next v7 07/10] bpf: Enable aggregate return types up to 16 bytes

In reply to Yonghong Song
Does this code reject enum64-valued fields in scalar structs?
Quoted 13 lines
include/linux/bpf_verifier.h +2 -0
index 953d8191563a..004b06785521 100644
@@ -1488,6 +1488,8 @@ int bpf_jmp_offset(struct bpf_insn *insn);
 struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);
 void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);
 bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog);
+bool btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,
+			       const struct btf_type *t, int rec);
 
 int bpf_find_subprog(struct bpf_verifier_env *env, int off);
 bool bpf_is_throw_kfunc(struct bpf_insn *insn);

[ ... ]

Quoted 4 lines
kernel/bpf/btf.c +0 -0
index 5e9f6a487524..58b81c6f5288 100644

[ ... ]

Quoted 18 lines
kernel/bpf/btf.c +6 -1
@@ -7970,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,
 
 /* Check whether the type is a valid return type. */
 static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btf,
-		const struct btf_type *t, int subprog)
+		const struct btf_type *t, int subprog, bool is_global)
 {
 	u32 tags = 0;
 	int err;
@@ -7993,6 +7993,19 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
 	if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
 		return 0;
 
+	if (btf_type_is_struct(t) && t->size <= 16) {
+		/*
+		 * A global function's caller models the return as an opaque
+		 * scalar pair, so it may only return scalars by value. A local
+		 * function is verified inline, so a pointer field stays tracked

kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8010

Quoted 10 lines
kernel/bpf/btf.c +8 -0
+		 * and needs no such restriction.
+		 */
+		bool local_func = subprog && !is_global;
+
+		if (local_func || btf_type_is_scalar_struct(env, btf, t, 0))
+			return 0;
+	}
+
 	return -EOPNOTSUPP;
 }

Does this code reject enum64-valued fields in scalar structs?

Nine lines above, btf_is_any_enum(t) accepts a bare BTF_KIND_ENUM64 return value as a scalar. However, btf_type_is_scalar_struct() calls btf_type_is_scalar() (include/linux/btf.h:361), which only matches btf_type_is_int() or btf_type_is_enum(). The latter (include/linux/btf.h:286) covers BTF_KIND_ENUM only, while BTF_KIND_ENUM64 requires btf_is_any_enum() (include/linux/btf.h:291).

This means a bare 'return enum64_t' passes the bare-scalar branch above, but 'return struct { enum64_t e; }' - a struct of one scalar - fails the scalar-struct test and gets rejected with -EOPNOTSUPP.

The commit message describes 'a struct or union up to 8 bytes, returned in R0 alone', and the series documentation (7ba16e29a69f) says the aggregate 'must be composed only of scalars (recursively), where a scalar is an integer or an enum', so enum64 members appear to be intended.

The same gap affects kfuncs: check_kfunc_call() gates struct returns on btf_type_is_scalar_struct() as well and logs 'returns %s %s that is not composed of scalars'.

Could btf_type_is_scalar() be updated to include btf_is_any_enum(), or should these two call sites use a different helper?

Quoted 17 lines
kernel/bpf/btf.c +3 -3
@@ -8080,12 +8093,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 		return -EINVAL;
 	}
 
-	err = btf_validate_return_type(env, btf, t, subprog);
+	err = btf_validate_return_type(env, btf, t, subprog, is_global);
 	if (err) {
 		if (is_global) {
 			bpf_log(log,
-				"Global function %s() return value not void or scalar. "
-				"Only those are supported.\n",
+				"Global function %s() has unsupported return type. "
+				"Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\n",
 				tname);
 		}
 		return err;

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index d79f0c069795..7a939dfa3797 100644

[ ... ]

tools/testing/selftests/bpf/progs/exceptions_fail.c __noinline int exception_cb_ok_arg_small(int a) · line 66

Quoted 13 lines
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
index ac44d60e5066..9708efb93683 100644
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a)
 
 SEC("?tc")
 __exception_cb(exception_cb_bad_ret_type1)
-__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.")
+__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")
 int reject_exception_cb_type_1(struct __sk_buff *ctx)
 {
 	bpf_throw(0);

Does this updated expectation still pin the function being tested?

The old assertion string included both the function name and the specific
reason: 'Global function exception_cb_bad_ret_type1() return value not void or scalar.' The replacement now matches only the generic second sentence of the message, which contains no function name.

This means the test now passes if the verifier logs an unsupported return type error for any global function in the object, not specifically exception_cb_bad_ret_type1().

The new bpf_log() message at btf.c:8099-8102 still interpolates the name ('Global function %s() has unsupported return type'), so a more specific assertion form was available:

__msg("Global function exception_cb_bad_ret_type1() has unsupported return type")

The later selftest commit in this series (8b8a84ddcc3e, selftests/bpf: Add tests for aggregate return values) does keep the function name in its equivalent expectations.

Could this test retain the function name in the assertion to avoid matching unrelated failures?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#25 lore raw

Re: [PATCH bpf-next v7 07/10] bpf: Enable aggregate return types up to 16 bytes

Quoted 67 lines · On 8/18/26 11:50 PM, [email protected] wrote:
Quoted 13 lines
include/linux/bpf_verifier.h +2 -0
index 953d8191563a..004b06785521 100644
@@ -1488,6 +1488,8 @@ int bpf_jmp_offset(struct bpf_insn *insn);
  struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);
  void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);
  bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog);
+bool btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,
+			       const struct btf_type *t, int rec);
 
  int bpf_find_subprog(struct bpf_verifier_env *env, int off);
  bool bpf_is_throw_kfunc(struct bpf_insn *insn);

[ ... ]

Quoted 4 lines
kernel/bpf/btf.c +0 -0
index 5e9f6a487524..58b81c6f5288 100644

[ ... ]

Quoted 18 lines
kernel/bpf/btf.c +6 -1
@@ -7970,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,
 
  /* Check whether the type is a valid return type. */
  static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btf,
-		const struct btf_type *t, int subprog)
+		const struct btf_type *t, int subprog, bool is_global)
  {
  	u32 tags = 0;
  	int err;
@@ -7993,6 +7993,19 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
  	if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
  		return 0;
 
+	if (btf_type_is_struct(t) && t->size <= 16) {
+		/*
+		 * A global function's caller models the return as an opaque
+		 * scalar pair, so it may only return scalars by value. A local
+		 * function is verified inline, so a pointer field stays tracked

kernel/bpf/btf.c static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt · line 8010

Quoted 10 lines
kernel/bpf/btf.c +8 -0
+		 * and needs no such restriction.
+		 */
+		bool local_func = subprog && !is_global;
+
+		if (local_func || btf_type_is_scalar_struct(env, btf, t, 0))
+			return 0;
+	}
+
  	return -EOPNOTSUPP;
  }

Does this code reject enum64-valued fields in scalar structs?

Nine lines above, btf_is_any_enum(t) accepts a bare BTF_KIND_ENUM64 return value as a scalar. However, btf_type_is_scalar_struct() calls btf_type_is_scalar() (include/linux/btf.h:361), which only matches btf_type_is_int() or btf_type_is_enum(). The latter (include/linux/btf.h:286) covers BTF_KIND_ENUM only, while BTF_KIND_ENUM64 requires btf_is_any_enum() (include/linux/btf.h:291).

This means a bare 'return enum64_t' passes the bare-scalar branch above, but 'return struct { enum64_t e; }' - a struct of one scalar - fails the scalar-struct test and gets rejected with -EOPNOTSUPP.

The commit message describes 'a struct or union up to 8 bytes, returned in R0 alone', and the series documentation (7ba16e29a69f) says the aggregate 'must be composed only of scalars (recursively), where a scalar is an integer or an enum', so enum64 members appear to be intended.

Quoted 6 lines

The same gap affects kfuncs: check_kfunc_call() gates struct returns on btf_type_is_scalar_struct() as well and logs 'returns %s %s that is not composed of scalars'.

Could btf_type_is_scalar() be updated to include btf_is_any_enum(), or should these two call sites use a different helper?

It is rare that we have enum64 in a struct, so I would like keep this way.

Quoted 53 lines
Quoted 17 lines
kernel/bpf/btf.c +3 -3
@@ -8080,12 +8093,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
  		return -EINVAL;
  	}
 
-	err = btf_validate_return_type(env, btf, t, subprog);
+	err = btf_validate_return_type(env, btf, t, subprog, is_global);
  	if (err) {
  		if (is_global) {
  			bpf_log(log,
-				"Global function %s() return value not void or scalar. "
-				"Only those are supported.\n",
+				"Global function %s() has unsupported return type. "
+				"Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\n",
  				tname);
  		}
  		return err;

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index d79f0c069795..7a939dfa3797 100644

[ ... ]

tools/testing/selftests/bpf/progs/exceptions_fail.c __noinline int exception_cb_ok_arg_small(int a) · line 66

Quoted 13 lines
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
index ac44d60e5066..9708efb93683 100644
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a)
 
  SEC("?tc")
  __exception_cb(exception_cb_bad_ret_type1)
-__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.")
+__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")
  int reject_exception_cb_type_1(struct __sk_buff *ctx)
  {
  	bpf_throw(0);

Does this updated expectation still pin the function being tested?

The old assertion string included both the function name and the specific
reason: 'Global function exception_cb_bad_ret_type1() return value not void or scalar.' The replacement now matches only the generic second sentence of the message, which contains no function name.

This means the test now passes if the verifier logs an unsupported return type error for any global function in the object, not specifically exception_cb_bad_ret_type1().

The new bpf_log() message at btf.c:8099-8102 still interpolates the name ('Global function %s() has unsupported return type'), so a more specific assertion form was available:

Quoted 8 lines

__msg("Global function exception_cb_bad_ret_type1() has unsupported return type")

The later selftest commit in this series (8b8a84ddcc3e, selftests/bpf: Add tests for aggregate return values) does keep the function name in its equivalent expectations.

Could this test retain the function name in the assertion to avoid matching unrelated failures?

I think it is okay.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#26 lore raw

PATCH v7 8/10 selftests/bpf: Add C tests for 16-byte returns in R0:R2

In reply to Yonghong Song

Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return.

The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a __clang_major__ guard. An older compiler builds the dummy test in the #else branch instead, which keeps the object non-empty and says in its description why nothing was exercised.

The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), which bpf_add_kfunc_call() requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_aggregate_ret.c        | 178 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  18 ++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |   9 +
 4 files changed, 207 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 64ac49ad67e6..f7f94ccebce2 100644
@@ -5,6 +5,7 @@
 #include "arena_kfunc.skel.h"
 #include "arena_kfunc_jit.skel.h"
 #include "cap_helpers.h"
+#include "verifier_aggregate_ret.skel.h"
 #include "verifier_align.skel.h"
 #include "verifier_and.skel.h"
 #include "verifier_arena.skel.h"
@@ -170,6 +171,7 @@ void test_arena_kfunc(void)                   { RUN_TESTS(arena_kfunc); }
 
 void test_arena_kfunc_jit(void)               { RUN_TESTS(arena_kfunc_jit); }
 
+void test_verifier_aggregate_ret(void)        { RUN_TESTS(verifier_aggregate_ret); }
 void test_verifier_align(void)                { RUN(verifier_align); }
 void test_verifier_and(void)                  { RUN(verifier_and); }
 void test_verifier_arena(void)                { RUN(verifier_arena); }
tools/testing/selftests/bpf/progs/verifier_aggregate_ret.c new file +178 -0

178 lines not on this page · raw message

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +18 -0
index 9366a3c578f1..c4bc8e11c6e1 100644
@@ -954,6 +954,20 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)
 	return 0;
 }
 
+#if defined(__x86_64__) || defined(__aarch64__)
+__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b)
+{
+	return (__int128)(((unsigned __int128)(a + b) << 64) | (a - b));
+}
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b)
+{
+	struct prog_test_ret_pair r = { .lo = a + b, .hi = a - b };
+
+	return r;
+}
+#endif /* __x86_64__ || __aarch64__ */
+
 __bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,
 					 u64 e, u64 f, u64 g, u64 h,
 					 u64 i, u64 j)
@@ -1487,6 +1501,10 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test3)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test4)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test5)
+#if defined(__x86_64__) || defined(__aarch64__)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair)
+#endif
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +9 -0
index 7d81070eefe7..426000f9a14f 100644
@@ -55,6 +55,11 @@ struct prog_test_big_arg {
 	__u64 b;
 };
 
+struct prog_test_ret_pair {	/* 16 bytes: R0:R2 */
+	__u64 lo;
+	__u64 hi;
+};
+
 struct prog_test_fail1 {
 	void *p;
 	int x;
@@ -131,6 +136,10 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;
 struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;
 long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;
 int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
+#ifdef __SIZEOF_INT128__
+__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
+#endif
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
 __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,
 			       __u64 e, __u64 f, __u64 g, __u64 h,
 			       __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
#27 lore raw

Re: [PATCH bpf-next v7 08/10] selftests/bpf: Add C tests for 16-byte returns in R0:R2

The changelog states that x86_64 and arm64 are "the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair()". Is this accurate?
Quoted 10 lines

Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return.

The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a __clang_major__ guard. An older compiler builds the dummy test in the #else branch instead, which keeps the object non-empty and says in its description why nothing was exercised.

Quoted 5 lines

The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), which bpf_add_kfunc_call() requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.

The changelog states that x86_64 and arm64 are "the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair()". Is this accurate?

Looking at commit 4b4530101f35 ("bpf: Wire up JIT support for 16-byte kfunc returns"), the first patch of this same series, it adds bpf_jit_supports_kfunc_ret_reg_pair() returning true to three JITs, not two:

  arch/x86/net/bpf_jit_comp.c       bool bpf_jit_supports_kfunc_ret_reg_pair(void) { return true; }
  arch/arm64/net/bpf_jit_comp.c     bool bpf_jit_supports_kfunc_ret_reg_pair(void) { return true; }
  arch/riscv/net/bpf_jit_comp64.c   bool bpf_jit_supports_kfunc_ret_reg_pair(void) { return true; }

and its own changelog says so explicitly: "Only the x86-64, arm64 and riscv64 JITs opt in so far." kernel/bpf/core.c keeps the __weak default at false for everything else.

Stating the JIT capability as the reason for the architecture tagging misleads a reader into believing riscv64 has no R0:R2 JIT support, which is the opposite of what the series does.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#28 lore raw

Re: [PATCH bpf-next v7 08/10] selftests/bpf: Add C tests for 16-byte returns in R0:R2

Quoted 27 lines · On 8/18/26 11:49 PM, [email protected] wrote:
Quoted 10 lines

Add selftests that exercise a 16-byte return value passed in the R0:R2 register pair, written in C so that they depend on the compiler lowering the register-pair return.

The R0:R2 convention is only emitted by LLVM 23 and newer, and a by-value aggregate return does not compile at all before that, so the programs sit behind a __clang_major__ guard. An older compiler builds the dummy test in the #else branch instead, which keeps the object non-empty and says in its description why nothing was exercised.

Quoted 5 lines

The kfunc tests are tagged __arch_x86_64/__arch_arm64 and skip elsewhere. Those are the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), which bpf_add_kfunc_call() requires before it accepts a kfunc returning more than 8 bytes, and they are also the only ones building the kfuncs.

The changelog states that x86_64 and arm64 are "the architectures whose JIT advertises bpf_jit_supports_kfunc_ret_reg_pair()". Is this accurate?

Looking at commit 4b4530101f35 ("bpf: Wire up JIT support for 16-byte kfunc returns"), the first patch of this same series, it adds bpf_jit_supports_kfunc_ret_reg_pair() returning true to three JITs, not two:

   arch/x86/net/bpf_jit_comp.c       bool bpf_jit_supports_kfunc_ret_reg_pair(void) { return true; }
   arch/arm64/net/bpf_jit_comp.c     bool bpf_jit_supports_kfunc_ret_reg_pair(void) { return true; }
   arch/riscv/net/bpf_jit_comp64.c   bool bpf_jit_supports_kfunc_ret_reg_pair(void) { return true; }
Quoted 7 lines

and its own changelog says so explicitly: "Only the x86-64, arm64 and riscv64 JITs opt in so far." kernel/bpf/core.c keeps the __weak default at false for everything else.

Stating the JIT capability as the reason for the architecture tagging misleads a reader into believing riscv64 has no R0:R2 JIT support, which is the opposite of what the series does.

We are fine. I didn't add riscv64 selftest. As mentioned in
bpf: Wire up JIT support for 16-byte kfunc returns
riscv64 is not tested.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32221968339

author reviewer
#29 lore raw

PATCH v7 9/10 selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns

In reply to Yonghong Song

Add inline-asm tests covering what the C tests cannot reach, since they need a callee that violates the convention on purpose.

The coverage includes BPF-to-BPF returns, kfunc calls, backtracking and liveness. In addition, a negative freplace test checks that an extension cannot replace a function returning R0:R2.

---
 .../selftests/bpf/prog_tests/aggregate_ret.c  |  11 +
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |  16 ++
 .../selftests/bpf/progs/aggregate_ret_func.c  | 237 ++++++++++++++++++
 .../selftests/bpf/progs/aggregate_ret_kfunc.c | 121 +++++++++
 .../bpf/progs/aggregate_ret_target.c          |  29 +++
 .../bpf/progs/compute_live_registers.c        |  28 +++
 .../selftests/bpf/progs/freplace_ret_pair.c   |  12 +
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  37 +++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  20 ++
 9 files changed, 511 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_func.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_ret_target.c
 create mode 100644 tools/testing/selftests/bpf/progs/freplace_ret_pair.c
tools/testing/selftests/bpf/prog_tests/aggregate_ret.c new file +11 -0
new file mode 100644
index 000000000000..e0b94ed10f94
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include "aggregate_ret_func.skel.h"
+#include "aggregate_ret_kfunc.skel.h"
+
+void test_aggregate_ret(void)
+{
+	RUN_TESTS(aggregate_ret_func);
+	RUN_TESTS(aggregate_ret_kfunc);
+}
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +16 -0
index 2523c07a16c6..6262c81cdaa9 100644
@@ -441,6 +441,20 @@ static void test_func_replace_int_with_void(void)
 				     " doesn't match type INT of global_func2()");
 }
 
+static void test_func_replace_ret_pair(void)
+{
+	const char *msg = "Return type of new_agg_ret_target_func() has size 8 "
+			  "and of agg_ret_target_func() size 16";
+
+	/*
+	 * An extension cannot replace a function whose return value comes back
+	 * in the R0:R2 pair: only R0 is checked at the extension's exit, so it
+	 * would leave R2 stale for the target's callers.
+	 */
+	test_obj_load_failure_common("freplace_ret_pair.bpf.o",
+				     "./aggregate_ret_target.bpf.o", msg);
+}
+
 static int find_prog_btf_id(const char *name, __u32 attach_prog_fd)
 {
 	struct bpf_prog_info info = {};
@@ -660,6 +674,8 @@ void serial_test_fexit_bpf2bpf(void)
 		test_func_replace_progmap();
 	if (test__start_subtest("freplace_int_with_void"))
 		test_func_replace_int_with_void();
+	if (test__start_subtest("freplace_ret_pair"))
+		test_func_replace_ret_pair();
 	if (test__start_subtest("freplace_void"))
 		test_func_replace_void();
 	if (test__start_subtest("sleepable_fentry_to_xdp"))
tools/testing/selftests/bpf/progs/aggregate_ret_func.c new file +237 -0

237 lines not on this page · raw message

tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c new file +121 -0

121 lines not on this page · raw message

tools/testing/selftests/bpf/progs/aggregate_ret_target.c new file +29 -0
new file mode 100644
index 000000000000..cffd8d7d3241
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+/* freplace target: a global subprogram returning 16 bytes in R0:R2. */
+__naked unsigned __int128 agg_ret_target_func(void)
+{
+	asm volatile (
+	"r0 = 0x1234;"
+	"r2 = 0x5678;"
+	"exit;"
+	);
+}
+
+SEC("tc")
+__naked int agg_ret_target(void)
+{
+	asm volatile (
+	"call %[agg_ret_target_func];"
+	"r0 = 0;"
+	"exit;"
+	:
+	: __imm(agg_ret_target_func)
+	: __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/compute_live_registers.c +28 -0
index d055fc7b3b95..338e67cd8856 100644
@@ -431,6 +431,34 @@ __naked void subprog1(void)
 		::: __clobber_all);
 }
 
+static __used __naked unsigned __int128 aux2(void)
+{
+	asm volatile (
+		"r0 = 1;"
+		"r2 = 2;"
+		"exit;"
+		::: __clobber_all);
+}
+
+SEC("socket")
+/* A program observing the pair needs the JIT; see bpf_compute_subprog_ret_regs(). */
+__load_if_JITed()
+__log_level(2)
+__msg("0: .12345.... (85) call pc+2")
+__msg("1: ..2....... (bf) r0 = r2")
+__msg("2: 0......... (95) exit")
+__msg("3: .......... (b7) r0 = 1")
+__msg("4: 0......... (b7) r2 = 2")
+__msg("5: 0.2....... (95) exit")
+__naked void subprog_ret_reg_pair(void)
+{
+	asm volatile (
+		"call aux2;"
+		"r0 = r2;"
+		"exit;"
+		::: __clobber_all);
+}
+
 #if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)
 
 SEC("socket")
tools/testing/selftests/bpf/progs/freplace_ret_pair.c new file +12 -0
new file mode 100644
index 000000000000..12c15d293bd7
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+SEC("freplace/agg_ret_target_func")
+__u64 new_agg_ret_target_func(void)
+{
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +37 -0
index c4bc8e11c6e1..850cf4f830c4 100644
@@ -966,8 +966,41 @@ __bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b)
 
 	return r;
 }
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u64 b)
+{
+	struct prog_test_ret_pair r = { .lo = a + b, .hi = a - b };
+
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)
+{
+	struct prog_test_ret_ptr r = { .p = NULL, .tag = tag };
+
+	return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b)
+{
+	struct prog_test_ret_ii r = { .a = a, .b = b };
+
+	return r;
+}
 #endif /* __x86_64__ || __aarch64__ */
 
+/*
+ * Takes no argument on purpose: with no arguments there is nothing for the sret
+ * pointer to displace, so this needs no architecture guard even though it
+ * returns 24 bytes.
+ */
+__bpf_kfunc struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void)
+{
+	struct prog_test_ret_big r = { .a = 1, .b = 2, .c = 3 };
+
+	return r;
+}
+
 __bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,
 					 u64 e, u64 f, u64 g, u64 h,
 					 u64 i, u64 j)
@@ -1504,7 +1537,11 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test5)
 #if defined(__x86_64__) || defined(__aarch64__)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_fastcall, KF_FASTCALL)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ptr)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)
 #endif
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_big)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)
 BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +20 -0
index 426000f9a14f..65e693ada736 100644
@@ -60,6 +60,22 @@ struct prog_test_ret_pair {	/* 16 bytes: R0:R2 */
 	__u64 hi;
 };
 
+struct prog_test_ret_ii {	/* 8 bytes: R0 only */
+	int a;
+	int b;
+};
+
+struct prog_test_ret_ptr {	/* 16 bytes: contains a pointer */
+	void *p;
+	__u64 tag;
+};
+
+struct prog_test_ret_big {	/* 24 bytes: too large for R0:R2 */
+	__u64 a;
+	__u64 b;
+	__u64 c;
+};
+
 struct prog_test_fail1 {
 	void *p;
 	int x;
@@ -140,6 +156,10 @@ int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
 __int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
 #endif
 struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
+struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym;
+struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void) __ksym;
 __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,
 			       __u64 e, __u64 f, __u64 g, __u64 h,
 			       __u64 i, __u64 j) __ksym;
Signature · 1 line
2.53.0-Meta
author reviewer
#30 lore raw

PATCH v7 10/10 Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2

In reply to Yonghong Song

kfuncs may now return a value larger than 8 bytes and up to 16 bytes (a scalar-only struct or union, or an __int128), passed back in the R0:R2 register pair. Add a kfunc return-value section documenting this, including that a struct or union up to 8 bytes is returned in R0 alone, which struct and union members are accepted, that the R0:R2 register pair requires JIT support (bpf_jit_supports_kfunc_ret_reg_pair()), and that a return value larger than 16 bytes is unsupported.

Also note that the same convention applies to BPF subprogram returns, and document the consequence for a global subprogram: it must assign both halves of a register-pair return, since an unassigned R2 may be left holding a pointer argument and is then rejected as a leak.

---
 Documentation/bpf/kfuncs.rst | 65 ++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)
Documentation/bpf/kfuncs.rst +65 -0
index 85f73e0bbd0f..89dea6b0b024 100644
@@ -575,6 +575,71 @@ is also covered by this recovery. A kfunc handed an arena pointer may
 therefore access up to ``GUARD_SZ / 2`` past it without bounds-checking
 against the arena. Larger accesses must verify the range explicitly.
 
+2.9 kfunc Return Values
+-----------------------
+
+A kfunc may return a scalar, a pointer, or a small struct or union by
+value. A scalar or pointer of up to 8 bytes is returned in R0, as usual.
+
+A struct or union returned by value must be composed only of scalars
+(recursively), where a scalar is an integer or an enum; arrays of scalars are
+allowed as members. Its bytes are handed back to the program as the raw
+contents of R0 (and R2), so a pointer field would be laundered into a scalar
+and escape the verifier's pointer provenance and reference tracking. A struct
+or union with a pointer member is therefore rejected at load time, and so is
+one with a floating-point member, which the ABI may not return in R0:R2 at
+all.
+
+A kfunc may also return a value larger than 8 bytes and up to 16 bytes -- a
+scalar-only struct or union, or an ``__int128``. Such a value is returned
+in the register pair R0:R2, matching the convention LLVM uses for the BPF
+target: the first 8 bytes in R0 and the second 8 bytes in R2. A struct or
+union of 8 bytes or less is returned in R0 alone.
+
+::
+
+        struct bpf_pair { __u64 a, b; };   /* 16 bytes */
+
+        __bpf_kfunc struct bpf_pair bpf_kfunc_get_pair(void)
+        {
+                struct bpf_pair p = { .a = 1, .b = 2 };
+
+                return p;      /* p.a in R0, p.b in R2 */
+        }
+
+Returning a value in the R0:R2 pair requires the JIT to place the second
+half of the return value into R2, which not every architecture supports
+right now. A kfunc with a return value larger than 8 bytes is therefore
+rejected at load time on a JIT that does not advertise this capability (see
+``bpf_jit_supports_kfunc_ret_reg_pair()``), and such a program is never run
+by the interpreter. A return value larger than 16 bytes is not supported.
+
+The same R0:R2 convention applies to a BPF subprogram, global or static, that
+returns an ``__int128`` or a struct or union larger than 8 bytes. It is only
+used when the program is JITed, since the interpreter propagates only R0 out of
+a subprogram: without a JIT the return value stays in R0 alone, and a caller
+reading R2 is rejected for reading an uninitialized register. A global
+subprogram is verified in isolation, so its by-value struct or union return is
+restricted to scalars just like a kfunc's; a static subprogram is verified
+inline and has no such restriction. The main program is not covered: its return
+value is the program's exit code, read out of R0 alone, so a declared upper
+half is never looked at.
+
+A global subprogram must leave a scalar in *every* register of the pair, so
+both halves of the returned value have to be assigned. Leaving the upper half
+uninitialized is not merely untidy: the compiler is then free to leave R2
+holding whatever it happened to hold, which for a subprogram taking a pointer
+argument is typically that pointer. Handing the caller an unknown scalar built
+from a pointer is a leak, so the verifier rejects it with::
+
+        At subprogram exit the register R2 is not a scalar value (...)
+
+Initialize the whole return value, for example ``struct pair p = {};``, to
+avoid this. A static subprogram is exempt from the scalar-only rule: it is
+verified inline, so an unassigned R2 is simply passed back to the caller as
+uninitialized and only a caller that reads it fails. A stack pointer left in
+R2 is still rejected there, just as one in R0 is.
+
 .. _BPF_kfunc_lifecycle_expectations:
 
 3. kfunc lifecycle expectations
Signature · 1 line
2.53.0-Meta
#31 lore raw
In reply to Yonghong Song
This series was applied to bpf/bpf-next.git (master) by Eduard Zingerman <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Eduard Zingerman <[email protected]>:

Quoted 8 lines · On Tue, 18 Aug 2026 22:52:39 -0700 you wrote:

LLVM 23 can return an __int128, or a struct/union larger than 8 bytes and no larger than 16 bytes, in the BPF R0:R2 register pair [1][2]. Before that the BPF backend could not return such values at all: a by-value aggregate return was rejected at compile time with "aggregate returns are not supported", and an __int128 return failed in the backend with "unable to allocate function return #1".

[...]

Here is the summary with links:
- [bpf-next,v7,01/10] bpf: Factor check_global_ret_scalar_reg() out of the global return check

    https://git.kernel.org/bpf/bpf-next/c/da13c047cda6
  - [bpf-next,v7,02/10] bpf: Add helpers to describe the R0:R2 return register pair
    https://git.kernel.org/bpf/bpf-next/c/cf5b96d95548
  - [bpf-next,v7,03/10] bpf: Wire up JIT support for 16-byte kfunc returns
    https://git.kernel.org/bpf/bpf-next/c/34c3503cb29a
  - [bpf-next,v7,04/10] bpf: Handle R2 as a return register in precision backtracking
    https://git.kernel.org/bpf/bpf-next/c/5bbb242c0859
  - [bpf-next,v7,05/10] bpf: Account R2 of register-pair returns in live register analysis
    https://git.kernel.org/bpf/bpf-next/c/991fb7823106
  - [bpf-next,v7,06/10] bpf: Add verifier support for 16-byte returns in R0:R2
    https://git.kernel.org/bpf/bpf-next/c/0630ad00d96d
  - [bpf-next,v7,07/10] bpf: Enable aggregate return types up to 16 bytes
    https://git.kernel.org/bpf/bpf-next/c/a2e98481a639
  - [bpf-next,v7,08/10] selftests/bpf: Add C tests for 16-byte returns in R0:R2
    https://git.kernel.org/bpf/bpf-next/c/7b7b511e0cf5
  - [bpf-next,v7,09/10] selftests/bpf: Add inline-asm and subprog tests for R0:R2 returns
    https://git.kernel.org/bpf/bpf-next/c/0b9fc795f83f
  - [bpf-next,v7,10/10] Documentation/bpf: Document up to 16-byte kfunc return values in R0:R2
    https://git.kernel.org/bpf/bpf-next/c/d50d9a06c86a

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html