Skip to contentSkip to conversation

bpf: Support by-value struct and __int128 arguments

PATCH Superseded Yonghong Song 3 participants 31 messages Sep 9 – Sep 11

Subsystems (9)
x86 Kselftest BPF arm64 +5 more subsystems

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Yonghong Song

author reviewer
#0 lore raw

v2 cover bpf: Support by-value struct and __int128 arguments

A global function or a kfunc taking a struct or union by value is rejected today:

Arg#1 type STRUCT in tar() is not supported yet. Unrecognized R2 type STRUCT

and an __int128 argument is accepted but mis-counted: the compiler passes it in two registers while the verifier gives it one, so every argument after it is checked against the wrong register and the program is rejected for a register its source never names.

The compiler passes such a value in one argument register per eightbyte, so a value of at most 16 bytes arrives in one or two consecutive registers. This series teaches the verifier to describe that: sub->args[] gains one entry per argument slot rather than one per BTF parameter, and the kfunc argument walk counts slots the same way. A value may only be composed of scalars, as a pointer would reach the callee as an opaque scalar, losing the provenance and reference tracking that make it safe to use. A global function still has no stack arguments, so all of its slots have to fit in the argument registers.

For a kfunc the two calling conventions have to agree, and they do not always. Where the BPF convention splits a value between the last argument register and the stack, x86_64 moves the whole of it to the stack and gives the register to the argument that follows, while arm64 rounds the register number up to an even one for a 16-byte aligned value. For stack arguments, both x86_64 and arm64 will have 16-byte alignment for those 16-byte aligned parameters. Otherwise the alignment will be 8-byte. This is implemented in x86_64 and arm64 jit.

Patch 1 records the __int128 failure as it stands today, which patch 4 turns into a success. Patches 2 to 4 index the arguments of a global function by slot and accept a by-value struct and an __int128. Patches 6 and 7 have the same for a kfunc call. Patches 8 and 9 place the arguments per the kernel calling convention in the x86-64 and arm64 JITs. Patches 10 to 12 are the tests.

Tested on x86-64 and arm64 with test_progs. The by-value struct tests are built with clang, as GCC passes an aggregate by invisible reference, and some of them need __BPF_FEATURE_STACK_ARGUMENT.

Changelog:
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- Fix a few issues due to potential out of bound access.
- Fix a few missed cases for kfunc with '> 8 byte' arguments.
- Add jit support for x86_64 and arm64. Previously certain functions are rejected, and now these functions can succeed.

Yonghong Song (12):
selftests/bpf: Add a test for an __int128 by-value argument
bpf: Index global function arguments by argument slot
bpf: Support by-value struct arguments up to 16 bytes
bpf: Support __int128 as a by-value function argument
bpf: Rename bpf_call_summary::num_params to arg_slot_cnt
bpf: Recognize by-value struct and __int128 kfunc arguments
bpf: Prepare kfunc arguments for the JIT from an ABI description bpf, x86: Move kfunc arguments into the x86-64 calling convention bpf, arm64: Move kfunc arguments into the arm64 calling convention selftests/bpf: Add C tests for by-value arguments up to 16 bytes selftests/bpf: Add inline-asm tests for by-value arguments selftests/bpf: Add tests for by-value kfunc arguments

 arch/arm64/net/bpf_jit_comp.c                 |  77 +++++-
 arch/x86/net/bpf_jit_comp.c                   |  73 +++++-
 include/linux/bpf.h                           |   8 +
 include/linux/bpf_verifier.h                  |   3 +-
 include/linux/filter.h                        |  32 +++
 kernel/bpf/btf.c                              |  94 +++++--
 kernel/bpf/core.c                             |  94 +++++++
 kernel/bpf/liveness.c                         |  10 +-
 kernel/bpf/verifier.c                         | 239 +++++++++++++++---
 .../selftests/bpf/prog_tests/aggregate_arg.c  |  11 +
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/aggregate_arg_func.c  | 155 ++++++++++++
 .../selftests/bpf/progs/aggregate_arg_kfunc.c | 221 ++++++++++++++++
 .../testing/selftests/bpf/progs/arena_kfunc.c |  16 ++
 .../selftests/bpf/progs/stack_arg_fail.c      |  10 -
 .../selftests/bpf/progs/verifier_int128_arg.c | 196 ++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  77 ++++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  32 +++
 18 files changed, 1271 insertions(+), 79 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_arg_func.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_int128_arg.c
Signature · 1 line
2.53.0-Meta
author reviewer
#1 lore raw

PATCH v2 1/12 selftests/bpf: Add a test for an __int128 by-value argument

A 128-bit integer is passed in two consecutive argument registers, but the verifier counts one argument register per parameter whatever its size. For

__u64 take_i128_global(int a, u128 v, int c)

the compiler passes a in R1, v in R2:R3 and c in R4, while the verifier marks only R1 through R3 at the entry of the global function. The callee then reads its own third parameter out of a register the verifier considers uninitialized, and the program is rejected for a register the source never names:

Validating take_i128_global() func#1...
20: R1=scalar() R2=scalar() R3=scalar() R10=fp0
; __noinline __u64 take_i128_global(int a, u128 v, int c) @ verifier_int128_arg.c:12
20: (bf) r0 = r2 ; R0=scalar(id=4) R2=scalar(id=4)
; return (__u64)a + (__u64)(v >> 64) + (__u64)v + c; @ verifier_int128_arg.c:14

  21: (bc) w1 = w1                      ; R1=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))
  22: (67) r1 <<= 32                    ; R1=scalar(smax=0x7fffffff00000000,smin32=0,smax32=umax32=0,var_off=(0x0; 0xffffffff00000000))
  23: (c7) r1 s>>= 32                   ; R1=scalar(smin=0xffffffff80000000,smax=0x7fffffff)
  24: (0f) r0 += r1                     ; R0=scalar() R1=scalar(smin=0xffffffff80000000,smax=0x7fffffff)
  25: (0f) r0 += r3                     ; R0=scalar() R3=scalar()

26: (bc) w1 = w4
R4 !read_ok

The log is from clang 23. LLVM 21/22 place the argument in the same registers.

Add the test with the failure it produces now. A later patch will fix this issue so this test should succeed.

---
 .../selftests/bpf/prog_tests/verifier.c       |  2 ++
 .../selftests/bpf/progs/verifier_int128_arg.c | 36 +++++++++++++++++++
 2 files changed, 38 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_int128_arg.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index f7f94ccebce2..0d68b92d6692 100644
@@ -50,6 +50,7 @@
 #include "verifier_helper_packet_access.skel.h"
 #include "verifier_helper_restricted.skel.h"
 #include "verifier_helper_value_access.skel.h"
+#include "verifier_int128_arg.skel.h"
 #include "verifier_int_ptr.skel.h"
 #include "verifier_iterating_callbacks.skel.h"
 #include "verifier_jeq_infer_not_null.skel.h"
@@ -214,6 +215,7 @@ void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_
 void test_verifier_helper_packet_access(void) { RUN(verifier_helper_packet_access); }
 void test_verifier_helper_restricted(void)    { RUN(verifier_helper_restricted); }
 void test_verifier_helper_value_access(void)  { RUN(verifier_helper_value_access); }
+void test_verifier_int128_arg(void)           { RUN_TESTS(verifier_int128_arg); }
 void test_verifier_int_ptr(void)              { RUN(verifier_int_ptr); }
 void test_verifier_iterating_callbacks(void)  { RUN(verifier_iterating_callbacks); }
 void test_verifier_jeq_infer_not_null(void)   { RUN(verifier_jeq_infer_not_null); }
tools/testing/selftests/bpf/progs/verifier_int128_arg.c new file +36 -0
new file mode 100644
index 000000000000..419851f2d8d0
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+#define MIX_A	0xdeadbeefcafef00dULL
+#define MIX_B	0x0123456789abcdefULL
+
+typedef unsigned __int128 u128;
+
+__noinline __u64 take_i128_global(int a, u128 v, int c)
+{
+	return (__u64)a + (__u64)(v >> 64) + (__u64)v + c;
+}
+
+SEC("tc")
+/*
+ * The verifier counts one argument register for the __int128 and marks only
+ * R1 through R3 at the entry of take_i128_global(), while the compiler passed
+ * a in R1, v in R2:R3 and c in R4.
+ */
+__failure __msg("R4 !read_ok")
+int aggregate_arg_int128_c_test(struct __sk_buff *skb)
+{
+	__u64 a = skb->len ^ MIX_A;
+	__u64 b = skb->len ^ MIX_B;
+	u128 v = ((u128)a << 64) | b;
+
+	if (take_i128_global(1, v, 2) != a + b + 3)
+		return 1;
+
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
Signature · 1 line
2.53.0-Meta
#2 lore raw

Re: [PATCH bpf-next v2 01/12] selftests/bpf: Add a test for an __int128 by-value argument

This isn't a bug, but the name verifier_int128_arg describes only the one case progs/verifier_int128_arg.c holds at this commit. Two patches later in the same series, f4da9830f5a9 adds by-value…
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/verifier.c +1 -0
index f7f94ccebce2..0d68b92d6692 100644
@@ -50,6 +50,7 @@
 #include "verifier_helper_packet_access.skel.h"
 #include "verifier_helper_restricted.skel.h"
 #include "verifier_helper_value_access.skel.h"
+#include "verifier_int128_arg.skel.h"

[ ... ]

tools/testing/selftests/bpf/prog_tests/verifier.c void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_ · line 219

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/verifier.c +1 -0
@@ -214,6 +215,7 @@ void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_
 void test_verifier_helper_packet_access(void) { RUN(verifier_helper_packet_access); }
 void test_verifier_helper_restricted(void)    { RUN(verifier_helper_restricted); }
 void test_verifier_helper_value_access(void)  { RUN(verifier_helper_value_access); }
+void test_verifier_int128_arg(void)           { RUN_TESTS(verifier_int128_arg); }
 void test_verifier_int_ptr(void)              { RUN(verifier_int_ptr); }

This isn't a bug, but the name verifier_int128_arg describes only the one case progs/verifier_int128_arg.c holds at this commit. Two patches later in the same series, f4da9830f5a9 adds by-value struct argument tests (struct pair, struct small, struct with_ptr) to the same file, none of which are __int128. The sibling entry two lines up is test_verifier_aggregate_ret for the aggregate-return family.

Since progs/verifier_int128_arg.c later grows to cover by-value struct arguments too, would a name in the same family as verifier_aggregate_ret -- say verifier_aggregate_arg -- fit the file better and save a rename later in the series?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#3 lore raw

Re: [PATCH bpf-next v2 01/12] selftests/bpf: Add a test for an __int128 by-value argument

Quoted 24 lines · On 9/9/26 12:13 AM, [email protected] wrote:
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/verifier.c +1 -0
index f7f94ccebce2..0d68b92d6692 100644
@@ -50,6 +50,7 @@
  #include "verifier_helper_packet_access.skel.h"
  #include "verifier_helper_restricted.skel.h"
  #include "verifier_helper_value_access.skel.h"
+#include "verifier_int128_arg.skel.h"

[ ... ]

tools/testing/selftests/bpf/prog_tests/verifier.c void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_ · line 219

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/verifier.c +1 -0
@@ -214,6 +215,7 @@ void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_
  void test_verifier_helper_packet_access(void) { RUN(verifier_helper_packet_access); }
  void test_verifier_helper_restricted(void)    { RUN(verifier_helper_restricted); }
  void test_verifier_helper_value_access(void)  { RUN(verifier_helper_value_access); }
+void test_verifier_int128_arg(void)           { RUN_TESTS(verifier_int128_arg); }
  void test_verifier_int_ptr(void)              { RUN(verifier_int_ptr); }

This isn't a bug, but the name verifier_int128_arg describes only the one case progs/verifier_int128_arg.c holds at this commit. Two patches later in the same series, f4da9830f5a9 adds by-value struct argument tests (struct pair, struct small, struct with_ptr) to the same file, none of which are __int128. The sibling entry two lines up is test_verifier_aggregate_ret for the aggregate-return family.

Quoted 4 lines

Since progs/verifier_int128_arg.c later grows to cover by-value struct arguments too, would a name in the same family as verifier_aggregate_ret -- say verifier_aggregate_arg -- fit the file better and save a rename later in the series?

Correct, will use verifier_aggregate_arg.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#4 lore raw

PATCH v2 2/12 bpf: Index global function arguments by argument slot

In reply to Yonghong Song

btf_prepare_func_args() indexes sub->args[] by BTF parameter, so a parameter can only ever stand for a single argument register. Give the loop a second index: 'i' keeps walking the BTF parameters while 'slots_used' walks the argument slots, and sub->arg_cnt becomes the number of slots, so that a later patch can give a parameter two of them.

No functional change: every parameter still takes exactly one slot.

---
 kernel/bpf/btf.c | 40 ++++++++++++++++++++++++----------------
 1 file changed, 24 insertions(+), 16 deletions(-)
kernel/bpf/btf.c +24 -16
index 31057c8f3a7c..104a91f5ffd5 100644
@@ -8037,7 +8037,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 	const struct btf_param *args;
 	const struct btf_type *t, *ref_t, *fn_t;
 	int err;
-	u32 i, nargs, btf_id;
+	u32 i, slots_used, nargs, btf_id;
 	const char *tname;
 
 	if (sub->args_cached)
@@ -8122,8 +8122,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 	/* Convert BTF function arguments into verifier types.
 	 * Only PTR_TO_CTX and SCALAR are supported atm.
 	 */
-	for (i = 0; i < nargs; i++) {
+	for (i = 0, slots_used = 0; i < nargs; i++) {
 		u32 tags = 0;
+
 		err = btf_scan_decl_tags(env, btf, fn_t, i, is_global, &tags);
 		if (err)
 			return err;
@@ -8145,7 +8146,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 			    btf_validate_prog_ctx_type(log, btf, t, i, prog_type,
 						       prog->expected_attach_type))
 				return -EINVAL;
-			sub->args[i].arg_type = ARG_PTR_TO_CTX;
+			sub->args[slots_used++].arg_type = ARG_PTR_TO_CTX;
 			continue;
 		}
 		if (btf_is_dynptr_ptr(btf, t)) {
@@ -8153,7 +8154,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 				bpf_log(log, "arg#%d has invalid combination of tags\n", i);
 				return -EINVAL;
 			}
-			sub->args[i].arg_type = ARG_PTR_TO_DYNPTR;
+			sub->args[slots_used++].arg_type = ARG_PTR_TO_DYNPTR;
 			continue;
 		}
 		if (tags & ARG_TAG_TRUSTED) {
@@ -8168,10 +8169,11 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 			if (kern_type_id < 0)
 				return kern_type_id;
 
-			sub->args[i].arg_type = ARG_PTR_TO_BTF_ID | PTR_TRUSTED;
+			sub->args[slots_used].arg_type = ARG_PTR_TO_BTF_ID | PTR_TRUSTED;
 			if (tags & ARG_TAG_NULLABLE)
-				sub->args[i].arg_type |= PTR_MAYBE_NULL;
-			sub->args[i].btf_id = kern_type_id;
+				sub->args[slots_used].arg_type |= PTR_MAYBE_NULL;
+			sub->args[slots_used].btf_id = kern_type_id;
+			slots_used++;
 			continue;
 		}
 		if (tags & ARG_TAG_UNTRUSTED) {
@@ -8185,8 +8187,10 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 
 			ref_t = btf_type_skip_modifiers(btf, t->type, NULL);
 			if (btf_type_is_void(ref_t) || btf_type_is_primitive(ref_t)) {
-				sub->args[i].arg_type = ARG_PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED;
-				sub->args[i].mem_size = 0;
+				sub->args[slots_used].arg_type = ARG_PTR_TO_MEM | MEM_RDONLY |
+								 PTR_UNTRUSTED;
+				sub->args[slots_used].mem_size = 0;
+				slots_used++;
 				continue;
 			}
 
@@ -8202,8 +8206,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 					i, btf_type_str(ref_t), tname);
 				return -EINVAL;
 			}
-			sub->args[i].arg_type = ARG_PTR_TO_BTF_ID | PTR_UNTRUSTED;
-			sub->args[i].btf_id = kern_type_id;
+			sub->args[slots_used].arg_type = ARG_PTR_TO_BTF_ID | PTR_UNTRUSTED;
+			sub->args[slots_used].btf_id = kern_type_id;
+			slots_used++;
 			continue;
 		}
 		if (tags & ARG_TAG_ARENA) {
@@ -8211,7 +8216,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 				bpf_log(log, "arg#%d arena cannot be combined with any other tags\n", i);
 				return -EINVAL;
 			}
-			sub->args[i].arg_type = ARG_PTR_TO_ARENA;
+			sub->args[slots_used++].arg_type = ARG_PTR_TO_ARENA;
 			continue;
 		}
 		if (is_global) { /* generic user data pointer */
@@ -8231,10 +8236,11 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 				return -EINVAL;
 			}
 
-			sub->args[i].arg_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL;
+			sub->args[slots_used].arg_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL;
 			if (tags & ARG_TAG_NONNULL)
-				sub->args[i].arg_type &= ~PTR_MAYBE_NULL;
-			sub->args[i].mem_size = mem_size;
+				sub->args[slots_used].arg_type &= ~PTR_MAYBE_NULL;
+			sub->args[slots_used].mem_size = mem_size;
+			slots_used++;
 			continue;
 		}
 
@@ -8244,7 +8250,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 			return -EINVAL;
 		}
 		if (btf_type_is_int(t) || btf_is_any_enum(t)) {
-			sub->args[i].arg_type = ARG_ANYTHING;
+			sub->args[slots_used++].arg_type = ARG_ANYTHING;
 			continue;
 		}
 		if (!is_global)
@@ -8254,6 +8260,8 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 		return -EINVAL;
 	}
 
+	sub->arg_cnt = slots_used;
+
 	sub->args_cached = true;
 
 	return 0;
Signature · 1 line
2.53.0-Meta
#5 lore raw

Re: [PATCH bpf-next v2 02/12] bpf: Index global function arguments by argument slot

This isn't a bug, but would a short comment on the early 'sub->arg_cnt = nargs;' noting that it exists only so the error path in btf_check_func_arg_match() has a stack-arg count, and that the…
Quoted 11 lines
kernel/bpf/btf.c +1 -1
index 31057c8f3a7c..104a91f5ffd5 100644
@@ -8037,7 +8037,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 	const struct btf_param *args;
 	const struct btf_type *t, *ref_t, *fn_t;
 	int err;
-	u32 i, nargs, btf_id;
+	u32 i, slots_used, nargs, btf_id;
 	const char *tname;

[ ... ]

Quoted 3 lines
kernel/bpf/btf.c +0 -0
 	args = (const struct btf_param *)(t + 1);
 	nargs = btf_type_vlen(t);
 	sub->arg_cnt = nargs;

[ ... ]

Quoted 8 lines
kernel/bpf/btf.c +2 -1
@@ -8122,8 +8122,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 	/* Convert BTF function arguments into verifier types.
 	 * Only PTR_TO_CTX and SCALAR are supported atm.
 	 */
-	for (i = 0; i < nargs; i++) {
+	for (i = 0, slots_used = 0; i < nargs; i++) {
 		u32 tags = 0;
+

[ ... ]

kernel/bpf/btf.c int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog) · line 8264

Quoted 6 lines
kernel/bpf/btf.c +2 -0
@@ -8254,6 +8260,8 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 		return -EINVAL;
 	}
 
+	sub->arg_cnt = slots_used;
+

This isn't a bug, but would a short comment on the early 'sub->arg_cnt = nargs;' noting that it exists only so the error path in btf_check_func_arg_match() has a stack-arg count, and that the authoritative slot count is set after the loop, make the two meanings of arg_cnt easier to keep straight in the later patches?

kernel/bpf/btf.c int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog) · line 8267

Quoted 3 lines
kernel/bpf/btf.c +0 -0
 	sub->args_cached = true;
 
 	return 0;

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#6 lore raw

Re: [PATCH bpf-next v2 02/12] bpf: Index global function arguments by argument slot

Quoted 32 lines · On 9/9/26 12:13 AM, [email protected] wrote:
Quoted 11 lines
kernel/bpf/btf.c +1 -1
index 31057c8f3a7c..104a91f5ffd5 100644
@@ -8037,7 +8037,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
  	const struct btf_param *args;
  	const struct btf_type *t, *ref_t, *fn_t;
  	int err;
-	u32 i, nargs, btf_id;
+	u32 i, slots_used, nargs, btf_id;
  	const char *tname;

[ ... ]

Quoted 3 lines
  	args = (const struct btf_param *)(t + 1);
  	nargs = btf_type_vlen(t);
  	sub->arg_cnt = nargs;

[ ... ]

Quoted 8 lines
kernel/bpf/btf.c +2 -1
@@ -8122,8 +8122,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
  	/* Convert BTF function arguments into verifier types.
  	 * Only PTR_TO_CTX and SCALAR are supported atm.
  	 */
-	for (i = 0; i < nargs; i++) {
+	for (i = 0, slots_used = 0; i < nargs; i++) {
  		u32 tags = 0;
+

[ ... ]

Quoted 4 lines
kernel/bpf/btf.c +0 -0
@@ -8254,6 +8260,8 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
  		return -EINVAL;
  	}
 
Quoted 7 lines

kernel/bpf/btf.c int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog) · line 8264

Quoted 2 lines
kernel/bpf/btf.c +2 -0
+	sub->arg_cnt = slots_used;
+

This isn't a bug, but would a short comment on the early 'sub->arg_cnt = nargs;' noting that it exists only so the error path in btf_check_func_arg_match() has a stack-arg count, and that the authoritative slot count is set after the loop, make the two meanings of arg_cnt easier to keep straight in the later patches?

I will rename arg_cnt to arg_slot_cnt to make it clear.

Quoted 10 lines

kernel/bpf/btf.c int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog) · line 8267

Quoted 4 lines
kernel/bpf/btf.c +0 -0
  	sub->args_cached = true;
 
  	return 0;

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#7 lore raw

PATCH v2 3/12 bpf: Support by-value struct arguments up to 16 bytes

In reply to Yonghong Song

A global function taking a struct or union by value is rejected today:

Arg#1 type STRUCT in tar() is not supported yet.

Accept one of at most 16 bytes, which arrives in one or two consecutive argument registers. Only structs composed entirely of scalars are taken for now; btf_struct_is_composed_of() enforces that.

The slots of a value are independent of each other, so the compiler may split one across the last argument register and the stack, as in

static void f(int a, int b, int c, int d, struct pair p);

or place it wholly past the registers, and the verifier describes either the same way. What has to follow the slots is stack_arg_cnt, recomputed from the slots consumed, together with the "no stack args in global functions" and JIT support checks, and the MAX_BPF_FUNC_ARGS bound, which now has to account for a parameter that takes two slots at once.

The previous patch made sub->arg_cnt a count of argument slots; this is the first patch where that count can exceed the number of BTF parameters, so a slot index no longer indexes the parameter array. Have btf_check_func_arg_match() hand the parameters to check_outgoing_stack_args(), which uses them only to name a stack argument in its diagnostic, just when the two counts still agree.

---
 kernel/bpf/btf.c      | 52 +++++++++++++++++++++++++++++++++++++++++++
 kernel/bpf/verifier.c |  7 ++++++
 2 files changed, 59 insertions(+)
kernel/bpf/btf.c +52 -0
index 104a91f5ffd5..4502b888b668 100644
@@ -8125,6 +8125,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 	for (i = 0, slots_used = 0; i < nargs; i++) {
 		u32 tags = 0;
 
+		if (slots_used >= MAX_BPF_FUNC_ARGS)
+			goto too_many_slots;
+
 		err = btf_scan_decl_tags(env, btf, fn_t, i, is_global, &tags);
 		if (err)
 			return err;
@@ -8253,6 +8256,33 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 			sub->args[slots_used++].arg_type = ARG_ANYTHING;
 			continue;
 		}
+		if (btf_type_is_struct(t)) {
+			u32 nslots;
+
+			if (!t->size || t->size > 2 * BPF_REG_SIZE) {
+				if (!is_global)
+					return -EINVAL;
+				bpf_log(log,
+					"Arg#%d type %s in %s() has size %u, only 1 to %d bytes "
+					"can be passed by value\n",
+					i, btf_type_str(t), tname, t->size, 2 * BPF_REG_SIZE);
+				return -EINVAL;
+			}
+			if (!btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
+				if (!is_global)
+					return -EINVAL;
+				bpf_log(log, "Arg#%d type %s in %s() is not composed of scalars\n",
+					i, btf_type_str(t), tname);
+				return -EINVAL;
+			}
+
+			nslots = (t->size + BPF_REG_SIZE - 1) / BPF_REG_SIZE;
+			if (slots_used + nslots > MAX_BPF_FUNC_ARGS)
+				goto too_many_slots;
+			while (nslots--)
+				sub->args[slots_used++].arg_type = ARG_ANYTHING;
+			continue;
+		}
 		if (!is_global)
 			return -EINVAL;
 		bpf_log(log, "Arg#%d type %s in %s() is not supported yet.\n",
@@ -8260,11 +8290,33 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 		return -EINVAL;
 	}
 
+	if (slots_used > MAX_BPF_FUNC_REG_ARGS) {
+		if (is_global) {
+			bpf_log(log,
+				"global function %s() needs %d > %d argument slots, "
+				"stack args not supported\n",
+				tname, slots_used, MAX_BPF_FUNC_REG_ARGS);
+			return -EINVAL;
+		}
+		if (!bpf_jit_supports_stack_args()) {
+			bpf_log(log, "JIT does not support function %s() with %d argument slots\n",
+				tname, slots_used);
+			return -EFAULT;
+		}
+		sub->stack_arg_cnt = slots_used - MAX_BPF_FUNC_REG_ARGS;
+	}
 	sub->arg_cnt = slots_used;
 
 	sub->args_cached = true;
 
 	return 0;
+
+too_many_slots:
+	if (!is_global)
+		return -EINVAL;
+	bpf_log(log, "Arguments of %s() need more than %d argument slots\n",
+		tname, MAX_BPF_FUNC_ARGS);
+	return -EINVAL;
 }
 
 static void btf_type_show(const struct btf *btf, u32 type_id, void *obj,
kernel/bpf/verifier.c +7 -0
index 9e79750e2480..a9243066a114 100644
@@ -9784,6 +9784,13 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
 	func = btf_type_by_id(btf, env->prog->aux->func_info[subprog].type_id);
 	func_proto = btf_type_by_id(btf, func->type);
 	args = btf_params(func_proto);
+	/*
+	 * An argument slot index can run past the end of the parameter array,
+	 * so only pass the parameters, for the names of the stack arguments,
+	 * if the two match.
+	 */
+	if (sub->arg_cnt != btf_type_vlen(func_proto))
+		args = NULL;
 	ret = check_outgoing_stack_args(env, caller, sub->arg_cnt,
 					bpf_subprog_name(env, subprog), btf, args);
 	if (ret)
Signature · 1 line
2.53.0-Meta
#8 lore raw

Re: [PATCH bpf-next v2 03/12] bpf: Support by-value struct arguments up to 16 bytes

This isn't a bug, but now that slots_used is always >= nargs, is the pre-loop nargs-based block still needed? Looking earlier in the function:
Quoted 14 lines
kernel/bpf/btf.c +3 -0
index 104a91f5ffd5..4502b888b668 100644
@@ -8125,6 +8125,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 	for (i = 0, slots_used = 0; i < nargs; i++) {
 		u32 tags = 0;
 
+		if (slots_used >= MAX_BPF_FUNC_ARGS)
+			goto too_many_slots;
+
 		err = btf_scan_decl_tags(env, btf, fn_t, i, is_global, &tags);
 		if (err)
 			return err;

[ ... ]

Quoted 44 lines
kernel/bpf/btf.c +33 -0
@@ -8253,6 +8256,33 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 			sub->args[slots_used++].arg_type = ARG_ANYTHING;
 			continue;
 		}
+		if (btf_type_is_struct(t)) {
+			u32 nslots;
+
+			if (!t->size || t->size > 2 * BPF_REG_SIZE) {
+				if (!is_global)
+					return -EINVAL;
+				bpf_log(log,
+					"Arg#%d type %s in %s() has size %u, only 1 to %d bytes "
+					"can be passed by value\n",
+					i, btf_type_str(t), tname, t->size, 2 * BPF_REG_SIZE);
+				return -EINVAL;
+			}
+			if (!btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
+				if (!is_global)
+					return -EINVAL;
+				bpf_log(log, "Arg#%d type %s in %s() is not composed of scalars\n",
+					i, btf_type_str(t), tname);
+				return -EINVAL;
+			}
+
+			nslots = (t->size + BPF_REG_SIZE - 1) / BPF_REG_SIZE;
+			if (slots_used + nslots > MAX_BPF_FUNC_ARGS)
+				goto too_many_slots;
+			while (nslots--)
+				sub->args[slots_used++].arg_type = ARG_ANYTHING;
+			continue;
+		}
 		if (!is_global)
 			return -EINVAL;
 		bpf_log(log, "Arg#%d type %s in %s() is not supported yet.\n",
@@ -8260,11 +8290,33 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 		return -EINVAL;
 	}
 
+	if (slots_used > MAX_BPF_FUNC_REG_ARGS) {
+		if (is_global) {
+			bpf_log(log,
+				"global function %s() needs %d > %d argument slots, "
+				"stack args not supported\n",
+				tname, slots_used, MAX_BPF_FUNC_REG_ARGS);

kernel/bpf/btf.c int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog) · line 8308

Quoted 10 lines
kernel/bpf/btf.c +9 -0
+			return -EINVAL;
+		}
+		if (!bpf_jit_supports_stack_args()) {
+			bpf_log(log, "JIT does not support function %s() with %d argument slots\n",
+				tname, slots_used);
+			return -EFAULT;
+		}
+		sub->stack_arg_cnt = slots_used - MAX_BPF_FUNC_REG_ARGS;
+	}
 	sub->arg_cnt = slots_used;

This isn't a bug, but now that slots_used is always >= nargs, is the pre-loop nargs-based block still needed? Looking earlier in the function:

	if (nargs > MAX_BPF_FUNC_REG_ARGS) {
		if (!bpf_jit_supports_stack_args()) {
			bpf_log(log, "JIT does not support function %s() with %d args\n",
				tname, nargs);
			return -EFAULT;
		}
		sub->stack_arg_cnt = nargs - MAX_BPF_FUNC_REG_ARGS;
	}

	if (is_global && nargs > MAX_BPF_FUNC_REG_ARGS) {
		bpf_log(log, "global function %s has %d > %d args, stack args not supported\n",
			tname, nargs, MAX_BPF_FUNC_REG_ARGS);
		return -EINVAL;
	}

The new block above recomputes stack_arg_cnt from slots_used. Since every parameter consumes at least one slot, whenever nargs > MAX_BPF_FUNC_REG_ARGS, slots_used > MAX_BPF_FUNC_REG_ARGS as well, so the assignment sub->stack_arg_cnt = nargs - MAX_BPF_FUNC_REG_ARGS in the earlier block is always overwritten by the new sub->stack_arg_cnt = slots_used - MAX_BPF_FUNC_REG_ARGS assignment.

Also, the two blocks order their tests differently. The earlier block tests bpf_jit_supports_stack_args() before is_global, while the new block tests is_global first. On an architecture without stack-arg JIT support, a global function needing stack args therefore fails with -EFAULT and "JIT does not support function f() with 6 args" when the sixth slot comes from a sixth parameter, but with -EINVAL and "global function f() needs 6 > 5 argument slots, stack args not supported" when it comes from a two-slot struct.

Could this new block be the only place that sets stack_arg_cnt and applies the global/JIT checks, or does the earlier nargs-based block serve a purpose I'm missing?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#9 lore raw

Re: [PATCH bpf-next v2 03/12] bpf: Support by-value struct arguments up to 16 bytes

Quoted 103 lines · On 9/9/26 12:13 AM, [email protected] wrote:
Quoted 14 lines
kernel/bpf/btf.c +3 -0
index 104a91f5ffd5..4502b888b668 100644
@@ -8125,6 +8125,9 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
  	for (i = 0, slots_used = 0; i < nargs; i++) {
  		u32 tags = 0;
 
+		if (slots_used >= MAX_BPF_FUNC_ARGS)
+			goto too_many_slots;
+
  		err = btf_scan_decl_tags(env, btf, fn_t, i, is_global, &tags);
  		if (err)
  			return err;

[ ... ]

Quoted 44 lines
kernel/bpf/btf.c +33 -0
@@ -8253,6 +8256,33 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
  			sub->args[slots_used++].arg_type = ARG_ANYTHING;
  			continue;
  		}
+		if (btf_type_is_struct(t)) {
+			u32 nslots;
+
+			if (!t->size || t->size > 2 * BPF_REG_SIZE) {
+				if (!is_global)
+					return -EINVAL;
+				bpf_log(log,
+					"Arg#%d type %s in %s() has size %u, only 1 to %d bytes "
+					"can be passed by value\n",
+					i, btf_type_str(t), tname, t->size, 2 * BPF_REG_SIZE);
+				return -EINVAL;
+			}
+			if (!btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
+				if (!is_global)
+					return -EINVAL;
+				bpf_log(log, "Arg#%d type %s in %s() is not composed of scalars\n",
+					i, btf_type_str(t), tname);
+				return -EINVAL;
+			}
+
+			nslots = (t->size + BPF_REG_SIZE - 1) / BPF_REG_SIZE;
+			if (slots_used + nslots > MAX_BPF_FUNC_ARGS)
+				goto too_many_slots;
+			while (nslots--)
+				sub->args[slots_used++].arg_type = ARG_ANYTHING;
+			continue;
+		}
  		if (!is_global)
  			return -EINVAL;
  		bpf_log(log, "Arg#%d type %s in %s() is not supported yet.\n",
@@ -8260,11 +8290,33 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
  		return -EINVAL;
  	}
 
+	if (slots_used > MAX_BPF_FUNC_REG_ARGS) {
+		if (is_global) {
+			bpf_log(log,
+				"global function %s() needs %d > %d argument slots, "
+				"stack args not supported\n",
+				tname, slots_used, MAX_BPF_FUNC_REG_ARGS);

kernel/bpf/btf.c int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog) · line 8308

Quoted 10 lines
kernel/bpf/btf.c +9 -0
+			return -EINVAL;
+		}
+		if (!bpf_jit_supports_stack_args()) {
+			bpf_log(log, "JIT does not support function %s() with %d argument slots\n",
+				tname, slots_used);
+			return -EFAULT;
+		}
+		sub->stack_arg_cnt = slots_used - MAX_BPF_FUNC_REG_ARGS;
+	}
  	sub->arg_cnt = slots_used;

This isn't a bug, but now that slots_used is always >= nargs, is the pre-loop nargs-based block still needed? Looking earlier in the function:

	if (nargs > MAX_BPF_FUNC_REG_ARGS) {
		if (!bpf_jit_supports_stack_args()) {
			bpf_log(log, "JIT does not support function %s() with %d args\n",
				tname, nargs);
			return -EFAULT;
		}
		sub->stack_arg_cnt = nargs - MAX_BPF_FUNC_REG_ARGS;
	}

	if (is_global && nargs > MAX_BPF_FUNC_REG_ARGS) {
		bpf_log(log, "global function %s has %d > %d args, stack args not supported\n",
			tname, nargs, MAX_BPF_FUNC_REG_ARGS);
		return -EINVAL;
	}

The new block above recomputes stack_arg_cnt from slots_used. Since every parameter consumes at least one slot, whenever nargs > MAX_BPF_FUNC_REG_ARGS, slots_used > MAX_BPF_FUNC_REG_ARGS as well, so the assignment sub->stack_arg_cnt = nargs - MAX_BPF_FUNC_REG_ARGS in the earlier block is always overwritten by the new sub->stack_arg_cnt = slots_used - MAX_BPF_FUNC_REG_ARGS assignment.

Also, the two blocks order their tests differently. The earlier block tests bpf_jit_supports_stack_args() before is_global, while the new block tests is_global first. On an architecture without stack-arg JIT support, a global function needing stack args therefore fails with -EFAULT and "JIT does not support function f() with 6 args" when the sixth slot comes from a sixth parameter, but with -EINVAL and "global function f() needs 6 > 5 argument slots, stack args not supported" when it comes from a two-slot struct.

Quoted 3 lines

Could this new block be the only place that sets stack_arg_cnt and applies the global/JIT checks, or does the earlier nargs-based block serve a purpose I'm missing?

As you mentioned, actually we do have some issues. The above nargs and is_global chcking is before going through all arguments. So they works. But we should check again after all arguments processing since some arguments may have two registers.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#10 lore raw

PATCH v2 4/12 bpf: Support __int128 as a by-value function argument

In reply to Yonghong Song

A 128-bit integer follows the same calling convention as a 16-byte by-value struct: LLVM emits it as a 16-byte BTF_KIND_INT and passes it in two consecutive argument registers. So a __int128 gets its two slots.

The test added at the start of the series, which recorded the wrong register map as an R4 !read_ok rejection, now passes and is flipped to __success.

---
 kernel/bpf/btf.c                                        | 9 +++------
 tools/testing/selftests/bpf/progs/verifier_int128_arg.c | 7 +------
 2 files changed, 4 insertions(+), 12 deletions(-)
kernel/bpf/btf.c +3 -6
index 4502b888b668..baa370f3f331 100644
@@ -8252,11 +8252,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 			bpf_log(log, "arg#%d has pointer tag, but is not a pointer type\n", i);
 			return -EINVAL;
 		}
-		if (btf_type_is_int(t) || btf_is_any_enum(t)) {
-			sub->args[slots_used++].arg_type = ARG_ANYTHING;
-			continue;
-		}
-		if (btf_type_is_struct(t)) {
+		if (btf_type_is_int(t) || btf_is_any_enum(t) || btf_type_is_struct(t)) {
 			u32 nslots;
 
 			if (!t->size || t->size > 2 * BPF_REG_SIZE) {
@@ -8268,7 +8264,8 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
 					i, btf_type_str(t), tname, t->size, 2 * BPF_REG_SIZE);
 				return -EINVAL;
 			}
-			if (!btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
+			if (btf_type_is_struct(t) &&
+			    !btf_struct_is_composed_of(env, btf, t, BTF_MEMBER_SCALAR)) {
 				if (!is_global)
 					return -EINVAL;
 				bpf_log(log, "Arg#%d type %s in %s() is not composed of scalars\n",
tools/testing/selftests/bpf/progs/verifier_int128_arg.c +1 -6
index 419851f2d8d0..044fc4a80055 100644
@@ -15,12 +15,7 @@ __noinline __u64 take_i128_global(int a, u128 v, int c)
 }
 
 SEC("tc")
-/*
- * The verifier counts one argument register for the __int128 and marks only
- * R1 through R3 at the entry of take_i128_global(), while the compiler passed
- * a in R1, v in R2:R3 and c in R4.
- */
-__failure __msg("R4 !read_ok")
+__success __retval(0)
 int aggregate_arg_int128_c_test(struct __sk_buff *skb)
 {
 	__u64 a = skb->len ^ MIX_A;
Signature · 1 line
2.53.0-Meta
author reviewer
#11 lore raw

PATCH v2 5/12 bpf: Rename bpf_call_summary::num_params to arg_slot_cnt

In reply to Yonghong Song

The field 'num_params' counts the argument registers and outgoing stack slots a helper or kfunc call takes. The next patch gives a 16-byte parameter two slots, so the name stops describing what the field holds.

Rename 'num_params' to 'arg_slot_cnt'. No functional change.

---
 include/linux/bpf_verifier.h |  2 +-
 kernel/bpf/liveness.c        | 10 +++++-----
 kernel/bpf/verifier.c        |  8 ++++----
 3 files changed, 10 insertions(+), 10 deletions(-)
include/linux/bpf_verifier.h +1 -1
index 9727df5af83a..0a857793c134 100644
@@ -1066,7 +1066,7 @@ static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
 }
 
 struct bpf_call_summary {
-	u8 num_params;
+	u8 arg_slot_cnt;
 	bool is_void;
 	bool fastcall;
 };
kernel/bpf/liveness.c +5 -5
index 7165ea325961..44ecdc5b4ec2 100644
@@ -1434,21 +1434,21 @@ static int record_call_access(struct bpf_verifier_env *env,
 {
 	struct bpf_insn *insn = &env->prog->insnsi[insn_idx];
 	struct bpf_call_summary cs;
-	int r, err, num_params = 5;
+	int r, err, arg_slot_cnt = 5;
 
 	if (bpf_pseudo_call(insn))
 		return 0;
 
 	if (bpf_get_call_summary(env, insn, &cs))
-		num_params = cs.num_params;
+		arg_slot_cnt = cs.arg_slot_cnt;
 
-	for (r = BPF_REG_1; r < BPF_REG_1 + min(num_params, MAX_BPF_FUNC_REG_ARGS); r++) {
+	for (r = BPF_REG_1; r < BPF_REG_1 + min(arg_slot_cnt, MAX_BPF_FUNC_REG_ARGS); r++) {
 		err = record_arg_access(env, instance, insn, &at[r], r - 1, insn_idx);
 		if (err)
 			return err;
 	}
 
-	for (r = 0; r < MAX_STACK_ARG_SLOTS && r < num_params - MAX_BPF_FUNC_REG_ARGS; r++) {
+	for (r = 0; r < MAX_STACK_ARG_SLOTS && r < arg_slot_cnt - MAX_BPF_FUNC_REG_ARGS; r++) {
 		err = record_arg_access(env, instance, insn, &at[MAX_BPF_REG + r],
 					r + MAX_BPF_FUNC_REG_ARGS, insn_idx);
 		if (err)
@@ -2199,7 +2199,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 			def = ALL_CALLER_SAVED_REGS;
 			use = def & ~BIT(BPF_REG_0);
 			if (bpf_get_call_summary(env, insn, &cs))
-				use = GENMASK(min_t(u8, cs.num_params, MAX_BPF_FUNC_REG_ARGS), 1);
+				use = GENMASK(min_t(u8, cs.arg_slot_cnt, MAX_BPF_FUNC_REG_ARGS), 1);
 			def = mask_widen(def);
 			use = mask_widen(use);
 			break;
kernel/bpf/verifier.c +4 -4
index a9243066a114..cf526f28f3e5 100644
@@ -17977,11 +17977,11 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
 			       (bpf_verifier_inlines_helper_call(env, call->imm) ||
 				bpf_jit_inlines_helper_call(call->imm));
 		cs->is_void = fn->ret_type == RET_VOID;
-		cs->num_params = 0;
+		cs->arg_slot_cnt = 0;
 		for (i = 0; i < ARRAY_SIZE(fn->arg_type); ++i) {
 			if (fn->arg_type[i] == ARG_DONTCARE)
 				break;
-			cs->num_params++;
+			cs->arg_slot_cnt++;
 		}
 		return true;
 	}
@@ -17993,7 +17993,7 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
 		if (err < 0)
 			/* error would be reported later */
 			return false;
-		cs->num_params = btf_type_vlen(meta.func_proto);
+		cs->arg_slot_cnt = btf_type_vlen(meta.func_proto);
 		cs->fastcall = meta.kfunc_flags & KF_FASTCALL;
 		cs->is_void = btf_type_is_void(btf_type_by_id(meta.btf, meta.func_proto->type));
 		return true;
@@ -18102,7 +18102,7 @@ static void mark_fastcall_pattern_for_call(struct bpf_verifier_env *env,
 	 * - includes R1-R5 if corresponding parameter has is described
 	 *   in the function prototype.
 	 */
-	clobbered_regs_mask = GENMASK(cs.num_params, cs.is_void ? 1 : 0);
+	clobbered_regs_mask = GENMASK(cs.arg_slot_cnt, cs.is_void ? 1 : 0);
 	/* e.g. if helper call clobbers r{0,1}, expect r{2,3,4,5} in the pattern */
 	expected_regs_mask = ~clobbered_regs_mask & ALL_CALLER_SAVED_REGS;
 
Signature · 1 line
2.53.0-Meta
author reviewer
#12 lore raw

PATCH v2 6/12 bpf: Recognize by-value struct and __int128 kfunc arguments

In reply to Yonghong Song

A kfunc taking a struct or union by value is rejected today, and one taking an __int128 is accepted but mis-counted:

Unrecognized R2 type STRUCT

The kfunc arguments walk the same slot as a BPF-to-BPF call:
one argument register per eightbyte, and a 16-byte value takes two. The outgoing stack argument count at the call site follows the slots for the same reason. Similar to BPF-to-BPF aggregate handling, a kfunc aggregate argument is only supported when it is composed of scalars. Everything that maps a kfunc argument to a register has to follow the slots too.

An argument of a single eightbyte lands in the same place under every calling convention, so those are taken. The conventions the JIT has to reconcile do not agree on where a larger one goes, so refuse it for now with

Function f arg#1 type INT cannot be passed at argument slot 1 on this architecture

which the next patch turns into an answer from the JIT. The paths that handle a two-slot argument are therefore unreachable until then.

---
 kernel/bpf/verifier.c | 148 ++++++++++++++++++++++++++++++++++--------
 1 file changed, 121 insertions(+), 27 deletions(-)
kernel/bpf/verifier.c +121 -27
index cf526f28f3e5..94c359351bb6 100644
@@ -12154,12 +12154,30 @@ bool bpf_is_kfunc_pkt_changing(struct bpf_call_arg_meta *meta)
 	return meta->func_id == special_kfunc_list[KF_bpf_xdp_pull_data];
 }
 
+static u32 kfunc_arg_slots(const struct btf_type *t)
+{
+	if (btf_type_is_int(t) || btf_type_is_struct(t))
+		return (t->size + BPF_REG_SIZE - 1) / BPF_REG_SIZE;
+	return 1;
+}
+
+static u32 kfunc_proto_slots(const struct btf *btf, const struct btf_type *func_proto)
+{
+	const struct btf_param *args = btf_params(func_proto);
+	u32 i, nargs = btf_type_vlen(func_proto), slots_used = 0;
+
+	for (i = 0; i < nargs; i++)
+		slots_used += kfunc_arg_slots(btf_type_skip_modifiers(btf, args[i].type, NULL));
+
+	return slots_used;
+}
+
 static int
 get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
-		   const struct btf_param *args, int arg, int nargs)
+		   const struct btf_param *args, int arg, int nargs, u32 slot)
 {
 	const struct btf_type *t, *ref_t = NULL;
-	argno_t argno = argno_from_arg(arg + 1);
+	argno_t argno = argno_from_arg(slot + 1);
 	const char *ref_tname = NULL;
 	int arg_type;
 
@@ -12179,6 +12197,23 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 		return KF_ARG_ANYTHING;
 	}
 
+	if (btf_type_is_struct(t)) {
+		if (!t->size || t->size > 2 * BPF_REG_SIZE) {
+			verbose(env,
+				"%s type %s has size %u, only 1 to %d bytes "
+				"can be passed by value\n",
+				reg_arg_name(env, argno), btf_type_str(t), t->size,
+				2 * BPF_REG_SIZE);
+			return -EINVAL;
+		}
+		if (!btf_type_is_scalar_struct(env, meta->btf, t)) {
+			verbose(env, "%s type %s is not composed of scalars\n",
+				reg_arg_name(env, argno), btf_type_str(t));
+			return -EINVAL;
+		}
+		return KF_ARG_ANYTHING;
+	}
+
 	if (!btf_type_is_ptr(t)) {
 		verbose(env, "Unrecognized %s type %s\n",
 			reg_arg_name(env, argno), btf_type_str(t));
@@ -12294,7 +12329,7 @@ static int gen_kfunc_arg_proto(struct bpf_verifier_env *env, struct bpf_call_arg
 {
 	const struct btf *btf = meta->btf;
 	const struct btf_param *args;
-	u32 i, nargs;
+	u32 i, nargs, slots_used;
 	int arg_type;
 
 	args = (const struct btf_param *)(meta->func_proto + 1);
@@ -12310,19 +12345,44 @@ static int gen_kfunc_arg_proto(struct bpf_verifier_env *env, struct bpf_call_arg
 		return -ENOTSUPP;
 	}
 
-	for (i = 0; i < nargs; i++) {
+	for (i = 0, slots_used = 0; i < nargs; i++) {
+		const struct btf_type *t;
+		u32 nslots;
+
+		t = btf_type_skip_modifiers(btf, args[i].type, NULL);
+		nslots = kfunc_arg_slots(t);
+		/*
+		 * The calling conventions the JIT has to reconcile do not
+		 * agree on where an argument of more than one eightbyte goes,
+		 * so refuse one until the JIT can say where this arch puts it.
+		 */
+		if (nslots > 1) {
+			verbose(env,
+				"Function %s arg#%d type %s cannot be passed at "
+				"argument slot %d on this architecture\n",
+				meta->func_name, i, btf_type_str(t), slots_used);
+			return -EINVAL;
+		}
+		slots_used += nslots;
+
 		if (is_kfunc_arg_prog_aux(btf, &args[i]) ||
 		    is_kfunc_arg_ignore(btf, &args[i]) ||
 		    is_kfunc_arg_implicit(meta, i))
 			continue;
 
-		arg_type = get_kfunc_arg_type(env, meta, args, i, nargs);
+		arg_type = get_kfunc_arg_type(env, meta, args, i, nargs, slots_used - nslots);
 		if (arg_type < 0)
 			return arg_type;
 
 		proto->arg_type[i] = arg_type;
 	}
 
+	if (slots_used > MAX_BPF_FUNC_REG_ARGS && !bpf_jit_supports_stack_args()) {
+		verbose(env, "JIT does not support kfunc %s() with %d argument slots\n",
+			meta->func_name, slots_used);
+		return -ENOTSUPP;
+	}
+
 	return 0;
 }
 
@@ -12897,29 +12957,35 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
 	const struct btf *btf = meta->btf;
 	const struct btf_param *args;
 	struct btf_record *rec;
-	u32 i, nargs;
+	u32 i, k, nargs, proto_slots, slots_used, prev_slot = 0, nslots = 0;
 	int ret;
 
 	args = (const struct btf_param *)(meta->func_proto + 1);
 	nargs = btf_type_vlen(meta->func_proto);
+	proto_slots = kfunc_proto_slots(btf, meta->func_proto);
 
-	ret = check_outgoing_stack_args(env, caller, nargs, func_name, btf, args);
+	ret = check_outgoing_stack_args(env, caller, proto_slots, func_name, btf,
+					proto_slots == nargs ? args : NULL);
 	if (ret)
 		return ret;
 
 	/* Check that BTF function arguments match actual types that the
 	 * verifier sees.
 	 */
-	for (i = 0; i < nargs; i++) {
-		struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, i);
+	for (i = 0, slots_used = 0; i < nargs;
+	     i++, prev_slot = slots_used, slots_used += nslots) {
+		struct bpf_reg_state *reg = get_func_arg_reg(caller, regs, slots_used);
 		const struct btf_type *t, *ref_t, *resolve_ret;
 		enum bpf_arg_type arg_type = ARG_DONTCARE;
-		argno_t argno = argno_from_arg(i + 1);
+		argno_t argno = argno_from_arg(slots_used + 1);
 		int regno = reg_from_argno(argno);
 		bool btf_id_fixed_off_ok = true;
 		u32 ref_id = args[i].type, type_size;
 		int kf_arg_type = meta->fn->arg_type[i];
 
+		t = btf_type_skip_modifiers(btf, args[i].type, NULL);
+		nslots = kfunc_arg_slots(t);
+
 		if (is_kfunc_arg_prog_aux(btf, &args[i])) {
 			/* Reject repeated use bpf_prog_aux */
 			if (meta->arg_prog) {
@@ -12939,8 +13005,6 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
 		if (is_kfunc_arg_ignore(btf, &args[i]) || is_kfunc_arg_implicit(meta, i))
 			continue;
 
-		t = btf_type_skip_modifiers(btf, args[i].type, NULL);
-
 		if (btf_type_is_ptr(t)) {
 			ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id);
 			ref_tname = btf_name_by_offset(btf, ref_t->name_off);
@@ -12991,6 +13055,27 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
 			ref_tname = btf_name_by_offset(btf, ref_t->name_off);
 		}
 
+		/*
+		 * The first register is checked in below KF_ARG_ANYTHING.
+		 * The rest of it has to be a scalar.
+		 */
+		for (k = 1; k < nslots; k++) {
+			argno_t hi_argno = argno_from_arg(slots_used + k + 1);
+			struct bpf_reg_state *hi = get_func_arg_reg(caller, regs, slots_used + k);
+
+			if (hi->type != SCALAR_VALUE) {
+				verbose(env, "%s is not a scalar\n", reg_arg_name(env, hi_argno));
+				bpf_diag_call_arg_fmt(env, insn_idx, hi_argno, func_name,
+						      "Pass an integer scalar value for this "
+						      "argument, not a pointer or resource object.",
+						      "the kfunc expects an integer scalar, "
+						      "but %s is %s",
+						      reg_arg_name(env, hi_argno),
+						      bpf_diag_reg_type_plain(env, hi->type));
+				return -EINVAL;
+			}
+		}
+
 		switch (base_type(kf_arg_type)) {
 		case KF_ARG_CONST:
 		case KF_ARG_CONST_MEM_SIZE:
@@ -13408,9 +13493,9 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_call_arg_me
 			fallthrough;
 		case KF_ARG_MEM_SIZE:
 		{
-			struct bpf_reg_state *buff_reg = get_func_arg_reg(caller, regs, i - 1);
+			struct bpf_reg_state *buff_reg = get_func_arg_reg(caller, regs, prev_slot);
 			struct bpf_reg_state *size_reg = reg;
-			argno_t buff_argno = argno_from_arg(i);
+			argno_t buff_argno = argno_from_arg(prev_slot + 1);
 			enum bpf_mem_size_failure failure;
 
 			if (reg->type != SCALAR_VALUE) {
@@ -13756,7 +13841,7 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
 	const struct btf_param *args;
 	const struct btf_type *t, *ref_t;
 	const struct btf *btf;
-	u32 nargs, type_size;
+	u32 i, slot, nargs, type_size;
 	s64 size;
 
 	if (bpf_fetch_kfunc_arg_meta(env, insn->imm, insn->off, &meta) < 0)
@@ -13765,23 +13850,32 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
 	btf = meta.btf;
 	args = btf_params(meta.func_proto);
 	nargs = btf_type_vlen(meta.func_proto);
-	if (arg >= nargs)
+
+	/*
+	 * @arg is an argument slot and a 16-byte parameter takes two of them,
+	 * so walk the parameters to find the one that starts at this slot. A
+	 * slot holding the upper eightbyte of such a parameter belongs to no
+	 * pointer, and neither does a slot past the last parameter.
+	 */
+	for (i = 0, slot = 0; i < nargs && slot < arg; i++)
+		slot += kfunc_arg_slots(btf_type_skip_modifiers(btf, args[i].type, NULL));
+	if (i >= nargs || slot != arg)
 		return 0;
 
-	t = btf_type_skip_modifiers(btf, args[arg].type, NULL);
+	t = btf_type_skip_modifiers(btf, args[i].type, NULL);
 	if (!btf_type_is_ptr(t))
 		return 0;
 
 	/* dynptr: fixed 16-byte on-stack representation */
-	if (is_kfunc_arg_dynptr(btf, &args[arg])) {
+	if (is_kfunc_arg_dynptr(btf, &args[i])) {
 		size = BPF_DYNPTR_SIZE;
 		goto out;
 	}
 
 	/* ptr + __sz/__szk pair: size is in the next register */
-	if (arg + 1 < nargs &&
-	    (btf_param_match_suffix(btf, &args[arg + 1], "__sz") ||
-	     btf_param_match_suffix(btf, &args[arg + 1], "__szk"))) {
+	if (i + 1 < nargs &&
+	    (btf_param_match_suffix(btf, &args[i + 1], "__sz") ||
+	     btf_param_match_suffix(btf, &args[i + 1], "__szk"))) {
 		int size_reg = BPF_REG_1 + arg + 1;
 
 		if (aux->const_reg_mask & BIT(size_reg)) {
@@ -13803,7 +13897,7 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
 	/* KF_ITER_NEW kfuncs initialize the iterator state at arg 0 */
 	if (arg == 0 && meta.kfunc_flags & KF_ITER_NEW)
 		return -size;
-	if (is_kfunc_arg_uninit(btf, &args[arg]))
+	if (is_kfunc_arg_uninit(btf, &args[i]))
 		return -size;
 	return size;
 }
@@ -13996,7 +14090,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	struct bpf_insn_aux_data *insn_aux;
 	const char *operation;
 	int err, insn_idx = *insn_idx_p;
-	u32 i, nargs, ptr_type_id, ret_nregs = 1;
+	u32 i, proto_slots, ptr_type_id, ret_nregs = 1;
 	struct bpf_kfunc_desc *desc;
 	struct btf *desc_btf;
 	int id;
@@ -14422,11 +14516,11 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	if (bpf_is_kfunc_pkt_changing(&meta))
 		clear_all_pkt_pointers(env);
 
-	nargs = btf_type_vlen(meta.func_proto);
-	if (nargs > MAX_BPF_FUNC_REG_ARGS) {
+	proto_slots = kfunc_proto_slots(desc_btf, meta.func_proto);
+	if (proto_slots > MAX_BPF_FUNC_REG_ARGS) {
 		struct bpf_func_state *caller = cur_func(env);
 		struct bpf_subprog_info *caller_info = &env->subprog_info[caller->subprogno];
-		u16 out_stack_arg_cnt = nargs - MAX_BPF_FUNC_REG_ARGS;
+		u16 out_stack_arg_cnt = proto_slots - MAX_BPF_FUNC_REG_ARGS;
 		u16 stack_arg_cnt = bpf_in_stack_arg_cnt(caller_info) + out_stack_arg_cnt;
 
 		if (stack_arg_cnt > caller_info->stack_arg_cnt)
@@ -17993,7 +18087,7 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
 		if (err < 0)
 			/* error would be reported later */
 			return false;
-		cs->arg_slot_cnt = btf_type_vlen(meta.func_proto);
+		cs->arg_slot_cnt = kfunc_proto_slots(meta.btf, meta.func_proto);
 		cs->fastcall = meta.kfunc_flags & KF_FASTCALL;
 		cs->is_void = btf_type_is_void(btf_type_by_id(meta.btf, meta.func_proto->type));
 		return true;
Signature · 1 line
2.53.0-Meta
#13 lore raw

Re: [PATCH bpf-next v2 06/12] bpf: Recognize by-value struct and __int128 kfunc arguments

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] bpf_kfunc_stack_access_bytes incorrectly maps stack slots to BPF_REG_6 through BPF_REG_9 (or higher), causing an out-of-bounds read and bypassing uninitialized memory checks. --

commit 7f4b25d0dad6ba6fed374b3111fcc89d67434d87
Author: Yonghong Song <[email protected]>

bpf: Recognize by-value struct and __int128 kfunc arguments

This commit modifies the BPF verifier to support kfuncs taking structs or int128 by value. It updates argument tracking to walk slots rather than just registers, rejecting multi-slot arguments until JIT support is fully added.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index cf526f28f3e5b..94c359351bb69 100644

[ ... ]

Quoted 39 lines
kernel/bpf/verifier.c +15 -6
@@ -13765,23 +13850,32 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
 	btf = meta.btf;
 	args = btf_params(meta.func_proto);
 	nargs = btf_type_vlen(meta.func_proto);
-	if (arg >= nargs)
+
+	/*
+	 * @arg is an argument slot and a 16-byte parameter takes two of them,
+	 * so walk the parameters to find the one that starts at this slot. A
+	 * slot holding the upper eightbyte of such a parameter belongs to no
+	 * pointer, and neither does a slot past the last parameter.
+	 */
+	for (i = 0, slot = 0; i < nargs && slot < arg; i++)
+		slot += kfunc_arg_slots(btf_type_skip_modifiers(btf, args[i].type, NULL));
+	if (i >= nargs || slot != arg)
 		return 0;
 
-	t = btf_type_skip_modifiers(btf, args[arg].type, NULL);
+	t = btf_type_skip_modifiers(btf, args[i].type, NULL);
 	if (!btf_type_is_ptr(t))
 		return 0;
 
 	/* dynptr: fixed 16-byte on-stack representation */
-	if (is_kfunc_arg_dynptr(btf, &args[arg])) {
+	if (is_kfunc_arg_dynptr(btf, &args[i])) {
 		size = BPF_DYNPTR_SIZE;
 		goto out;
 	}
 
 	/* ptr + __sz/__szk pair: size is in the next register */
-	if (arg + 1 < nargs &&
-	    (btf_param_match_suffix(btf, &args[arg + 1], "__sz") ||
-	     btf_param_match_suffix(btf, &args[arg + 1], "__szk"))) {
+	if (i + 1 < nargs &&
+	    (btf_param_match_suffix(btf, &args[i + 1], "__sz") ||
+	     btf_param_match_suffix(btf, &args[i + 1], "__szk"))) {
 		int size_reg = BPF_REG_1 + arg + 1;
 
 		if (aux->const_reg_mask & BIT(size_reg)) {
Quoted 1 line

size = (s64)aux->const_reg_vals[size_reg];

[Severity: High]
This is a pre-existing issue, but does this code lead to an out-of-bounds read or bypass uninitialized memory checks when the pointer argument is passed on the stack?

The size_reg calculation inherently assumes the size argument is always passed in a register. If the pointer argument slot is 4 or higher (which might be more common now that large by-value structs consume multiple slots), the size argument lands on the stack.

In that scenario, size_reg maps to BPF_REG_6 or higher, which are callee-saved registers. A BPF program could place a fake constant in one of these registers to trick the verifier into thinking 0 bytes are read, while the runtime stack argument is non-zero. This could bypass stack initialization checks and potentially leak uninitialized kernel memory.

Furthermore, if the pointer argument slot is 8 or higher, size_reg becomes greater than or equal to 10. Checking BIT(size_reg) and reading from aux->const_reg_vals[size_reg] would result in an out-of-bounds array read, since const_reg_vals is sized for 10 registers. Should this logic be updated to correctly query the BPF stack when the size argument is passed there?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author reviewer
#14 lore raw

Re: [PATCH bpf-next v2 06/12] bpf: Recognize by-value struct and __int128 kfunc arguments

Quoted 77 lines · On 9/8/26 11:46 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] bpf_kfunc_stack_access_bytes incorrectly maps stack slots to BPF_REG_6 through BPF_REG_9 (or higher), causing an out-of-bounds read and bypassing uninitialized memory checks. --

commit 7f4b25d0dad6ba6fed374b3111fcc89d67434d87
Author: Yonghong Song <[email protected]>

bpf: Recognize by-value struct and __int128 kfunc arguments

This commit modifies the BPF verifier to support kfuncs taking structs or int128 by value. It updates argument tracking to walk slots rather than just registers, rejecting multi-slot arguments until JIT support is fully added.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index cf526f28f3e5b..94c359351bb69 100644

[ ... ]

Quoted 39 lines
kernel/bpf/verifier.c +15 -6
@@ -13765,23 +13850,32 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
  	btf = meta.btf;
  	args = btf_params(meta.func_proto);
  	nargs = btf_type_vlen(meta.func_proto);
-	if (arg >= nargs)
+
+	/*
+	 * @arg is an argument slot and a 16-byte parameter takes two of them,
+	 * so walk the parameters to find the one that starts at this slot. A
+	 * slot holding the upper eightbyte of such a parameter belongs to no
+	 * pointer, and neither does a slot past the last parameter.
+	 */
+	for (i = 0, slot = 0; i < nargs && slot < arg; i++)
+		slot += kfunc_arg_slots(btf_type_skip_modifiers(btf, args[i].type, NULL));
+	if (i >= nargs || slot != arg)
  		return 0;
 
-	t = btf_type_skip_modifiers(btf, args[arg].type, NULL);
+	t = btf_type_skip_modifiers(btf, args[i].type, NULL);
  	if (!btf_type_is_ptr(t))
  		return 0;
 
  	/* dynptr: fixed 16-byte on-stack representation */
-	if (is_kfunc_arg_dynptr(btf, &args[arg])) {
+	if (is_kfunc_arg_dynptr(btf, &args[i])) {
  		size = BPF_DYNPTR_SIZE;
  		goto out;
  	}
 
  	/* ptr + __sz/__szk pair: size is in the next register */
-	if (arg + 1 < nargs &&
-	    (btf_param_match_suffix(btf, &args[arg + 1], "__sz") ||
-	     btf_param_match_suffix(btf, &args[arg + 1], "__szk"))) {
+	if (i + 1 < nargs &&
+	    (btf_param_match_suffix(btf, &args[i + 1], "__sz") ||
+	     btf_param_match_suffix(btf, &args[i + 1], "__szk"))) {
  		int size_reg = BPF_REG_1 + arg + 1;
 
  		if (aux->const_reg_mask & BIT(size_reg)) {
Quoted 1 line
  			size = (s64)aux->const_reg_vals[size_reg];

[Severity: High]
This is a pre-existing issue, but does this code lead to an out-of-bounds read or bypass uninitialized memory checks when the pointer argument is passed on the stack?

The size_reg calculation inherently assumes the size argument is always passed in a register. If the pointer argument slot is 4 or higher (which might be more common now that large by-value structs consume multiple slots), the size argument lands on the stack.

In that scenario, size_reg maps to BPF_REG_6 or higher, which are callee-saved registers. A BPF program could place a fake constant in one of these registers to trick the verifier into thinking 0 bytes are read, while the runtime stack argument is non-zero. This could bypass stack initialization checks and potentially leak uninitialized kernel memory.

Quoted 5 lines

Furthermore, if the pointer argument slot is 8 or higher, size_reg becomes greater than or equal to 10. Checking BIT(size_reg) and reading from aux->const_reg_vals[size_reg] would result in an out-of-bounds array read, since const_reg_vals is sized for 10 registers. Should this logic be updated to correctly query the BPF stack when the size argument is passed there?

You are right. Will fix.

Quoted 1 line
author reviewer
#15 lore raw

PATCH v2 7/12 bpf: Prepare kfunc arguments for the JIT from an ABI description

In reply to Yonghong Song

The previous patch refuses a kfunc argument of more than one eightbyte. This patch allows up to 16 byte kfunc arguments.

But different architectures have different ways to map bpf calling convention (no gap, no backfill) to native convention. Rather than have each arch open-code where it wants an argument, describe the following common parameters where each architecture can set their specific items:

  struct bpf_jit_arg_abi {
        u8   nr_arg_regs;
        bool even_reg_align;
        bool even_stack_align;
        bool split_at_boundary;
        bool backfill_after_stack;
  };

The above four booleans cover x86-64, arm64, RISC-V LP64, PowerPC64 ELFv2 etc. The bpf_jit_place_args() and bpf_jit_plan_arg_moves() utilizes the above information to do proper work to be used in JIT later on.

---
 include/linux/bpf.h          |  8 +++
 include/linux/bpf_verifier.h |  1 +
 include/linux/filter.h       | 32 ++++++++++++
 kernel/bpf/btf.c             |  3 ++
 kernel/bpf/core.c            | 94 ++++++++++++++++++++++++++++++++++
 kernel/bpf/verifier.c        | 98 +++++++++++++++++++++++++++++++-----
 6 files changed, 223 insertions(+), 13 deletions(-)
include/linux/bpf.h +8 -0
index e80963971f68..6736a95cc854 100644
@@ -1194,6 +1194,9 @@ struct bpf_prog_offload {
 	u32			jited_len;
 };
 
+/* The argument is aligned to 16 bytes. */
+#define BTF_FMODEL_ALIGN16_ARG		BIT(0)
+
 /* The argument is signed. */
 #define BTF_FMODEL_SIGNED_ARG		BIT(1)
 
@@ -1211,6 +1214,11 @@ struct btf_func_model {
 	u8 arg_flags[MAX_BPF_FUNC_ARGS];
 };
 
+static inline u32 btf_func_model_arg_slots(const struct btf_func_model *m, u32 arg)
+{
+	return (m->arg_size[arg] + sizeof(u64) - 1) / sizeof(u64);
+}
+
 /* Restore arguments before returning from trampoline to let original function
  * continue executing. This flag is used for fentry progs when there are no
  * fexit progs.
include/linux/bpf_verifier.h +1 -0
index 0a857793c134..06d082d94630 100644
@@ -1512,6 +1512,7 @@ enum btf_member_kind {
 
 bool btf_struct_is_composed_of(struct bpf_verifier_env *env, const struct btf *btf,
 			       const struct btf_type *t, u32 member_kinds);
+u32 btf_func_arg_align(const struct btf *btf, const struct btf_type *t);
 
 int bpf_find_subprog(struct bpf_verifier_env *env, int off);
 bool bpf_is_throw_kfunc(struct bpf_insn *insn);
include/linux/filter.h +32 -0
index 00ad8b63aa47..01b52d0259d1 100644
@@ -1248,6 +1248,38 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena);
 bool bpf_jit_supports_private_stack(void);
 bool bpf_jit_supports_timed_may_goto(void);
 bool bpf_jit_supports_fsession(void);
+
+struct bpf_jit_arg_abi {
+	/* Argument registers of the kernel convention. */
+	u8 nr_arg_regs;
+	/* Round the register number up to an even one for 16-byte alignment. */
+	bool even_reg_align;
+	/* Round the stack slot up to an even one for 16-byte alignment. */
+	bool even_stack_align;
+	/* An argument may straddle the last register and the stack. */
+	bool split_at_boundary;
+	/* A later argument may reuse a register a stack-passed one skipped. */
+	bool backfill_after_stack;
+};
+
+const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void);
+u32 bpf_jit_place_args(const struct bpf_jit_arg_abi *abi,
+		       const struct btf_func_model *fm, u8 *pos_of_slot);
+
+/* The JIT's scratch register, in place of an argument slot. */
+#define BPF_JIT_ARG_TMP		0xff
+
+/* Every argument slot moves at most once, and the scratch goes out and back. */
+#define BPF_JIT_MAX_ARG_MOVES	(MAX_BPF_FUNC_ARGS + 2)
+
+struct bpf_jit_arg_move {
+	u8 dst;
+	u8 src;
+};
+
+u32 bpf_jit_plan_arg_moves(const struct bpf_jit_arg_abi *abi,
+			   const struct btf_func_model *fm,
+			   struct bpf_jit_arg_move *moves);
 u64 bpf_arch_uaddress_limit(void);
 void arch_bpf_stack_walk(bool (*consume_fn)(void *cookie, u64 ip, u64 sp, u64 bp), void *cookie);
 u64 arch_bpf_timed_may_goto(void);
kernel/bpf/btf.c +3 -0
index baa370f3f331..239a9eb2dcc9 100644
@@ -7579,6 +7579,9 @@ static u8 __get_arg_fmodel_flags(const struct btf *btf,
 {
 	u8 flags = __get_type_fmodel_flags(t);
 
+	if (btf_func_arg_align(btf, t) > sizeof(u64))
+		flags |= BTF_FMODEL_ALIGN16_ARG;
+
 	if (btf_param_match_suffix(btf, arg, "__arena__nullable"))
 		flags |= BTF_FMODEL_ARENA_ARG | BTF_FMODEL_NULLABLE_ARG;
 	else if (btf_param_match_suffix(btf, arg, "__arena"))
kernel/bpf/core.c +94 -0

100 lines not on this page · raw message

kernel/bpf/verifier.c +85 -13

139 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#16 lore raw

Re: [PATCH bpf-next v2 07/12] bpf: Prepare kfunc arguments for the JIT from an ABI description

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Stack buffer overflow in `bpf_jit_plan_arg_moves` and `kfunc_abi_slots` due to undersized arrays…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Stack buffer overflow in bpf_jit_plan_arg_moves and kfunc_abi_slots due to undersized arrays based on maximum arguments rather than maximum slots. --

commit 180655f49c406c35435ba5ebc05f2900a3c56656
Author: Yonghong Song <[email protected]>

bpf: Prepare kfunc arguments for the JIT from an ABI description

The previous patch refuses a kfunc argument of more than one eightbyte. This patch allows up to 16 byte kfunc arguments.

Quoted 4 lines
include/linux/filter.h +0 -0
index 00ad8b63aa477..01b52d0259d18 100644

[ ... ]

include/linux/filter.h bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena); · line 1253

Quoted 7 lines
include/linux/filter.h +6 -0
@@ -1248,6 +1248,38 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena);
+
+/* The JIT's scratch register, in place of an argument slot. */
+#define BPF_JIT_ARG_TMP		0xff
+
+/* Every argument slot moves at most once, and the scratch goes out and back. */
+#define BPF_JIT_MAX_ARG_MOVES	(MAX_BPF_FUNC_ARGS + 2)

[Severity: Medium]
Since arguments can now take up to 2 slots (for 16-byte arguments), could the maximum number of slot moves actually be up to 26 (24 slots + 2) instead of 14?

Quoted 5 lines
include/linux/filter.h +5 -0
+
+struct bpf_jit_arg_move {
+	u8 dst;
+	u8 src;
+};

[ ... ]

Quoted 4 lines
kernel/bpf/core.c +0 -0
index c673b02d55a65..d4bd2ba9aadee 100644

[ ... ]

Quoted 13 lines
kernel/bpf/core.c +12 -0
@@ -3287,6 +3287,100 @@ bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
+
+u32 bpf_jit_place_args(const struct bpf_jit_arg_abi *abi,
+		       const struct btf_func_model *fm, u8 *pos_of_slot)
+{
+	u32 i, k, nslots, slot = 0, nregs_used = 0, stack_off = 0;
+	bool on_stack = false;
+
+	for (i = 0; i < fm->nr_args; i++) {
+		bool align16 = fm->arg_flags[i] & BTF_FMODEL_ALIGN16_ARG;
+		u32 pos;
+
+		nslots = btf_func_model_arg_slots(fm, i);

[ ... ]

kernel/bpf/core.c bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void) · line 3336

Quoted 2 lines
kernel/bpf/core.c +2 -0
+		for (k = 0; k < nslots; k++)
+			pos_of_slot[slot + k] = pos + k;

[Severity: Medium]
Will this overflow the pos_of_slot array if an architecture adopts this API for BPF trampolines?

The callers currently size the array based on MAX_BPF_FUNC_ARGS (12), but slot + k can reach up to 23 because some arguments can require multiple slots.

kernel/bpf/core.c bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void) · line 3347

Quoted 11 lines
kernel/bpf/core.c +11 -0
+		slot += nslots;
+	}
+
+	return slot;
+}
+
+u32 bpf_jit_plan_arg_moves(const struct bpf_jit_arg_abi *abi,
+			   const struct btf_func_model *fm,
+			   struct bpf_jit_arg_move *moves)
+{
+	u8 pos_of_slot[MAX_BPF_FUNC_ARGS];

[Severity: Medium]
Does this stack array need to be sized for the maximum number of slots (e.g., MAX_BPF_FUNC_ARGS * 2) instead of the maximum number of arguments?

Quoted 3 lines
kernel/bpf/core.c +3 -0
+	u32 nslots, n = 0, s, back;
+
+	nslots = bpf_jit_place_args(abi, fm, pos_of_slot);

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index 94c359351bb69..2b8df8c7f098c 100644

[ ... ]

kernel/bpf/verifier.c static u32 kfunc_proto_slots(const struct btf *btf, const struct btf_type *func_ · line 12178

Quoted 8 lines
kernel/bpf/verifier.c +4 -0
@@ -12172,6 +12172,58 @@ static u32 kfunc_proto_slots(const struct btf *btf, const struct btf_type *func_
 	return slots_used;
 }
 
+static u32 kfunc_abi_slots(const struct btf_func_model *fm)
+{
+	const struct bpf_jit_arg_abi *abi = bpf_jit_arg_abi();
+	u8 pos_of_slot[MAX_BPF_FUNC_ARGS];

[Severity: Medium]
Similarly, does this stack array also need to be sized to accommodate up to 24 slots rather than 12 arguments to prevent an out-of-bounds write in bpf_jit_place_args()?

Quoted 9 lines
kernel/bpf/verifier.c +9 -0
+	u32 i, nslots, slots = 0;
+
+	for (i = 0; i < fm->nr_args; i++)
+		slots += btf_func_model_arg_slots(fm, i);
+
+	if (!abi)
+		return slots;
+
+	nslots = bpf_jit_place_args(abi, fm, pos_of_slot);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
author reviewer
#17 lore raw

Re: [PATCH bpf-next v2 07/12] bpf: Prepare kfunc arguments for the JIT from an ABI description

Quoted 17 lines · On 9/8/26 11:46 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Stack buffer overflow in bpf_jit_plan_arg_moves and kfunc_abi_slots due to undersized arrays based on maximum arguments rather than maximum slots. --

commit 180655f49c406c35435ba5ebc05f2900a3c56656
Author: Yonghong Song <[email protected]>

bpf: Prepare kfunc arguments for the JIT from an ABI description

The previous patch refuses a kfunc argument of more than one eightbyte. This patch allows up to 16 byte kfunc arguments.

Quoted 4 lines
include/linux/filter.h +0 -0
index 00ad8b63aa477..01b52d0259d18 100644

[ ... ]

Quoted 10 lines

include/linux/filter.h bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena); · line 1253

Quoted 7 lines
include/linux/filter.h +6 -0
@@ -1248,6 +1248,38 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena);
+
+/* The JIT's scratch register, in place of an argument slot. */
+#define BPF_JIT_ARG_TMP		0xff
+
+/* Every argument slot moves at most once, and the scratch goes out and back. */
+#define BPF_JIT_MAX_ARG_MOVES	(MAX_BPF_FUNC_ARGS + 2)

[Severity: Medium]
Since arguments can now take up to 2 slots (for 16-byte arguments), could the maximum number of slot moves actually be up to 26 (24 slots + 2) instead of 14?

In jit, for x86_64, we have

        /* x86-64 supports up to MAX_BPF_FUNC_ARGS arguments. 1-6
         * are passed through regs, the remains are through stack.
         */
        if (nr_regs > MAX_BPF_FUNC_ARGS)
                return -ENOTSUPP;

So the maximum number of regs is 12, so we are okay for x86_64.

But arm64 is different. It has

static int calc_arg_aux(const struct btf_func_model *m,
struct arg_aux *a)

{
        int stack_slots, nregs, slots, i;

        /* verifier ensures m->nr_args <= MAX_BPF_FUNC_ARGS */
        for (i = 0, nregs = 0; i < m->nr_args; i++) {
                slots = (m->arg_size[i] + 7) / 8;
                if (nregs + slots <= 8) /* passed through register ? */
                        nregs += slots;
                else
                        break;
        }

	...
}

So the maximum number of reg arguments can be 24. I think I will add a patch similar to x86_64 above.

Quoted 26 lines
Quoted 6 lines
include/linux/filter.h +5 -0
+
+struct bpf_jit_arg_move {
+	u8 dst;
+	u8 src;
+};

[ ... ]

Quoted 4 lines
kernel/bpf/core.c +0 -0
index c673b02d55a65..d4bd2ba9aadee 100644

[ ... ]

Quoted 13 lines
kernel/bpf/core.c +12 -0
@@ -3287,6 +3287,100 @@ bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
+
+u32 bpf_jit_place_args(const struct bpf_jit_arg_abi *abi,
+		       const struct btf_func_model *fm, u8 *pos_of_slot)
+{
+	u32 i, k, nslots, slot = 0, nregs_used = 0, stack_off = 0;
+	bool on_stack = false;
+
+	for (i = 0; i < fm->nr_args; i++) {
+		bool align16 = fm->arg_flags[i] & BTF_FMODEL_ALIGN16_ARG;
+		u32 pos;
+
+		nslots = btf_func_model_arg_slots(fm, i);

[ ... ]

Quoted 8 lines

kernel/bpf/core.c bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void) · line 3336

Quoted 2 lines
kernel/bpf/core.c +2 -0
+		for (k = 0; k < nslots; k++)
+			pos_of_slot[slot + k] = pos + k;

[Severity: Medium]
Will this overflow the pos_of_slot array if an architecture adopts this API for BPF trampolines?

The callers currently size the array based on MAX_BPF_FUNC_ARGS (12), but slot + k can reach up to 23 because some arguments can require multiple slots.

I will add a check to ensure maximum reg arguments is 12.

Quoted 15 lines

kernel/bpf/core.c bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void) · line 3347

Quoted 12 lines
kernel/bpf/core.c +11 -0
+		slot += nslots;
+	}
+
+	return slot;
+}
+
+u32 bpf_jit_plan_arg_moves(const struct bpf_jit_arg_abi *abi,
+			   const struct btf_func_model *fm,
+			   struct bpf_jit_arg_move *moves)
+{
+	u8 pos_of_slot[MAX_BPF_FUNC_ARGS];

[Severity: Medium]
Does this stack array need to be sized for the maximum number of slots (e.g., MAX_BPF_FUNC_ARGS * 2) instead of the maximum number of arguments?

This is for kfunc. I will check whether we need to do
MAX_BPF_FUNC_ARGS * 2
or not.

Quoted 14 lines
Quoted 4 lines
kernel/bpf/core.c +3 -0
+	u32 nslots, n = 0, s, back;
+
+	nslots = bpf_jit_place_args(abi, fm, pos_of_slot);

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index 94c359351bb69..2b8df8c7f098c 100644

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
@@ -12172,6 +12172,58 @@ static u32 kfunc_proto_slots(const struct btf *btf, const struct btf_type *func_
  	return slots_used;
  }
 
Quoted 8 lines

kernel/bpf/verifier.c static u32 kfunc_proto_slots(const struct btf *btf, const struct btf_type *func_ · line 12178

Quoted 4 lines
kernel/bpf/verifier.c +4 -0
+static u32 kfunc_abi_slots(const struct btf_func_model *fm)
+{
+	const struct bpf_jit_arg_abi *abi = bpf_jit_arg_abi();
+	u8 pos_of_slot[MAX_BPF_FUNC_ARGS];

[Severity: Medium]
Similarly, does this stack array also need to be sized to accommodate up to 24 slots rather than 12 arguments to prevent an out-of-bounds write in bpf_jit_place_args()?

Yes, I will check whether we need to have 24 slots or not.

Quoted 10 lines
Quoted 10 lines
kernel/bpf/verifier.c +9 -0
+	u32 i, nslots, slots = 0;
+
+	for (i = 0; i < fm->nr_args; i++)
+		slots += btf_func_model_arg_slots(fm, i);
+
+	if (!abi)
+		return slots;
+
+	nslots = bpf_jit_place_args(abi, fm, pos_of_slot);
author reviewer
#18 lore raw

PATCH v2 8/12 bpf, x86: Move kfunc arguments into the x86-64 calling convention

In reply to Yonghong Song

Do proper move from bpf calling convention to x86_64 calling convention to satisfy native requirement.

In addition, the arena argument walk counts eightbytes rather than parameters, as an argument may take two registers.

---
 arch/x86/net/bpf_jit_comp.c | 73 +++++++++++++++++++++++++++++++++++--
 1 file changed, 70 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +70 -3
index bba351944202..0496607a7003 100644
@@ -1839,6 +1839,60 @@ static int emit_spectre_bhb_barrier(u8 **pprog, u8 *ip,
 	return 0;
 }
 
+static const struct bpf_jit_arg_abi x86_arg_abi = {
+	.nr_arg_regs		= 6,
+	.backfill_after_stack	= true,
+	.even_stack_align	= true,
+};
+
+static const u8 x86_arg_reg[] = {
+	BPF_REG_1, BPF_REG_2, BPF_REG_3, BPF_REG_4, BPF_REG_5, X86_REG_R9,
+};
+
+/*
+ * Move the arguments the x86-64 ABI places somewhere other than the argument
+ * slot the BPF calling convention gave them. @stack_base addresses the
+ * outgoing stack argument area from RBP. Return the number of emitted bytes.
+ */
+static int emit_kfunc_arg_moves(const struct btf_func_model *fm, s32 stack_base, u8 **pprog)
+{
+	struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
+	const u8 nreg = x86_arg_abi.nr_arg_regs;
+	u8 *prog = *pprog, *start = prog;
+	u32 i, n;
+
+	n = bpf_jit_plan_arg_moves(&x86_arg_abi, fm, moves);
+
+	for (i = 0; i < n; i++) {
+		u8 dst = moves[i].dst, src = moves[i].src, reg;
+		bool dst_mem = dst != BPF_JIT_ARG_TMP && dst >= nreg;
+		bool src_mem = src != BPF_JIT_ARG_TMP && src >= nreg;
+
+		/* Take the value into a register. */
+		if (src == BPF_JIT_ARG_TMP) {
+			reg = AUX_REG;
+		} else if (src_mem) {
+			reg = dst_mem || dst == BPF_JIT_ARG_TMP ? BPF_REG_AX : x86_arg_reg[dst];
+			emit_ldx(&prog, BPF_DW, reg, BPF_REG_FP,
+				 stack_base + (src - nreg) * 8);
+		} else {
+			reg = x86_arg_reg[src];
+		}
+
+		/* And leave it where the argument belongs. */
+		if (dst == BPF_JIT_ARG_TMP)
+			emit_mov_reg(&prog, true, AUX_REG, reg);
+		else if (dst_mem)
+			emit_stx(&prog, BPF_DW, BPF_REG_FP, reg,
+				 stack_base + (dst - nreg) * 8);
+		else if (reg != x86_arg_reg[dst])
+			emit_mov_reg(&prog, true, x86_arg_reg[dst], reg);
+	}
+
+	*pprog = prog;
+	return prog - start;
+}
+
 /*
  * Rebase the __arena args of a kfunc call to arena kernel addresses,
  * rN = kern_vm_start + (u32)rN, with R12 holding kern_vm_start. A nullable
@@ -1850,11 +1904,17 @@ static int emit_kfunc_arena_args(struct bpf_prog *bpf_prog,
 {
 	u8 *prog = *pprog;
 	u8 *start = prog;
-	int i;
+	int i, slot;
 
-	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
+	for (i = 0, slot = 0; i < fm->nr_args; i++) {
+		u32 arg_regs = (fm->arg_size[i] + 7) / 8;
 		u8 flags = fm->arg_flags[i];
-		u32 reg = BPF_REG_1 + i;
+		u32 reg;
+
+		if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
+			break;
+		reg = BPF_REG_1 + slot;
+		slot += arg_regs;
 
 		if (!(flags & BTF_FMODEL_ARENA_ARG))
 			continue;
@@ -2837,6 +2897,8 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 				if (err < 0)
 					return err;
 				ip += err;
+				ip += emit_kfunc_arg_moves(fm, outgoing_arg_base -
+							   outgoing_rsp, &prog);
 			}
 			if (priv_frame_ptr) {
 				push_r9(&prog);
@@ -4351,6 +4413,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return true;
 }
 
+const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
+{
+	return &x86_arg_abi;
+}
+
 bool bpf_jit_supports_stack_args(void)
 {
 	return true;
Signature · 1 line
2.53.0-Meta
#19 lore raw

Re: [PATCH bpf-next v2 08/12] bpf, x86: Move kfunc arguments into the x86-64 calling convention

This looks like a bug fix for multi-slot kfunc argument handling. The fix corrects emit_kfunc_arena_args() to count eightbytes (register slots) instead of parameter indices when walking arena…
Quoted 6 lines

commit 7d2a9d78dc93fad535592157230f508d5bc54a7c

Author: Yonghong Song <[email protected]>
Date:   Tue Sep 8 23:26:04 2026 -0700

bpf, x86: Move kfunc arguments into the x86-64 calling convention

Quoted 7 lines

Do proper move from bpf calling convention to x86_64 calling convention to satisfy native requirement.

In addition, the arena argument walk counts eightbytes rather than parameters, as an argument may take two registers.

This looks like a bug fix for multi-slot kfunc argument handling. The fix corrects emit_kfunc_arena_args() to count eightbytes (register slots) instead of parameter indices when walking arena arguments, which would cause incorrect register assignment when kfunc arguments take multiple registers (such as __int128 or large structs).

Should this include a Fixes: tag for the commit that introduced emit_kfunc_arena_args() with the parameter-index bug?

Fixes 34c3503cb29a ("bpf: Wire up JIT support for 16-byte kfunc returns")

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#20 lore raw

Re: [PATCH bpf-next v2 08/12] bpf, x86: Move kfunc arguments into the x86-64 calling convention

Quoted 19 lines · On 9/9/26 12:29 AM, [email protected] wrote:
Quoted 6 lines

commit 7d2a9d78dc93fad535592157230f508d5bc54a7c

Author: Yonghong Song <[email protected]>
Date:   Tue Sep 8 23:26:04 2026 -0700

bpf, x86: Move kfunc arguments into the x86-64 calling convention

Quoted 7 lines

Do proper move from bpf calling convention to x86_64 calling convention to satisfy native requirement.

In addition, the arena argument walk counts eightbytes rather than parameters, as an argument may take two registers.

This looks like a bug fix for multi-slot kfunc argument handling. The fix corrects emit_kfunc_arena_args() to count eightbytes (register slots) instead of parameter indices when walking arena arguments, which would cause incorrect register assignment when kfunc arguments take multiple registers (such as __int128 or large structs).

Quoted 4 lines

Should this include a Fixes: tag for the commit that introduced emit_kfunc_arena_args() with the parameter-index bug?

Fixes 34c3503cb29a ("bpf: Wire up JIT support for 16-byte kfunc returns")

There is no need to have this Fixes tag as it is in the same series.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#21 lore raw

PATCH v2 9/12 bpf, arm64: Move kfunc arguments into the arm64 calling convention

In reply to Yonghong Song

Do proper move from bpf calling convention to arm64 calling convention to satisfy native requirement. AAPCS64 only ever moves an argument to a higher slot, so the moves need one scratch register to carry an eightbyte from one stack slot to another, and never the one a convention moving an argument down would need.

In addition, the arena argument walk counts eightbytes rather than parameters, as an argument may take two registers. The walk takes the func model from the caller now, as the moves need it too, and runs first so that they carry the rebased value.

---
 arch/arm64/net/bpf_jit_comp.c | 77 ++++++++++++++++++++++++++++++-----
 1 file changed, 67 insertions(+), 10 deletions(-)
arch/arm64/net/bpf_jit_comp.c +67 -10
index 3aa3ea0bc30b..bdac930dbdec 100644
@@ -1220,6 +1220,12 @@ static int add_exception_handler(const struct bpf_insn *insn,
 	return 0;
 }
 
+static const struct bpf_jit_arg_abi arm64_arg_abi = {
+	.nr_arg_regs		= 8,
+	.even_reg_align		= true,
+	.even_stack_align	= true,
+};
+
 static const u8 stack_arg_reg[] = { A64_R(5), A64_R(6), A64_R(7) };
 
 #define NR_STACK_ARG_REGS	ARRAY_SIZE(stack_arg_reg)
@@ -1262,19 +1268,20 @@ static void emit_stack_arg_store_imm(s32 imm, s16 bpf_off, const u8 tmp, struct
  * kern_vm_start. A nullable arg preserves NULL by skipping the add, tested
  * on the truncated value as arena NULL is offset 0.
  */
-static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *insn)
+static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct btf_func_model *fm)
 {
 	const u8 arena_vm_base = bpf2a64[ARENA_VM_START];
-	const struct btf_func_model *fm;
-	int i;
-
-	fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
-	if (!fm)
-		return -EINVAL;
+	int i, slot;
 
-	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
-		const u8 reg = bpf2a64[BPF_REG_1 + i];
+	for (i = 0, slot = 0; i < fm->nr_args; i++) {
+		u32 arg_regs = (fm->arg_size[i] + 7) / 8;
 		u8 flags = fm->arg_flags[i];
+		u8 reg;
+
+		if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
+			break;
+		reg = bpf2a64[BPF_REG_1 + slot];
+		slot += arg_regs;
 
 		if (!(flags & BTF_FMODEL_ARENA_ARG))
 			continue;
@@ -1293,6 +1300,45 @@ static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins
 	return 0;
 }
 
+static bool a64_arg_on_stack(u8 slot)
+{
+	return slot >= arm64_arg_abi.nr_arg_regs;
+}
+
+static s32 a64_arg_stack_off(u8 slot)
+{
+	return (slot - arm64_arg_abi.nr_arg_regs) * sizeof(u64);
+}
+
+/*
+ * AAPCS64 only ever moves an argument to a higher slot, so the planner asks
+ * for the scratch only to carry an eightbyte from one stack slot to another.
+ */
+static void emit_kfunc_arg_moves(struct jit_ctx *ctx, const struct btf_func_model *fm)
+{
+	struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
+	const u8 tmp = bpf2a64[TMP_REG_1];
+	u32 i, n;
+
+	n = bpf_jit_plan_arg_moves(&arm64_arg_abi, fm, moves);
+
+	for (i = 0; i < n; i++) {
+		u8 dst = moves[i].dst, src = moves[i].src, reg;
+
+		if (a64_arg_on_stack(src)) {
+			reg = a64_arg_on_stack(dst) ? tmp : dst;
+			emit(A64_LDR64I(reg, A64_SP, a64_arg_stack_off(src)), ctx);
+		} else {
+			reg = src;
+		}
+
+		if (a64_arg_on_stack(dst))
+			emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);
+		else if (reg != dst)
+			emit(A64_MOV(1, dst, reg), ctx);
+	}
+}
+
 /* JITs an eBPF instruction.
  * Returns:
  * 0  - successfully JITed an 8-byte eBPF instruction.
@@ -1716,9 +1762,15 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 		if (ret < 0)
 			return ret;
 		if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) {
-			ret = emit_kfunc_arena_args(ctx, insn);
+			const struct btf_func_model *fm;
+
+			fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
+			if (!fm)
+				return -EINVAL;
+			ret = emit_kfunc_arena_args(ctx, fm);
 			if (ret < 0)
 				return ret;
+			emit_kfunc_arg_moves(ctx, fm);
 		}
 		emit_call(func_addr, ctx);
 		/*
@@ -2393,6 +2445,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return true;
 }
 
+const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
+{
+	return &arm64_arg_abi;
+}
+
 bool bpf_jit_supports_stack_args(void)
 {
 	return true;
Signature · 1 line
2.53.0-Meta
#22 lore raw

Re: [PATCH bpf-next v2 09/12] bpf, arm64: Move kfunc arguments into the arm64 calling convention

This looks like a bug fix for multi-slot kfunc argument support. Should this include:
Quoted 11 lines

commit 7c4f41d1b9c4dfdbada8f6d1b1097849ccb20de8
Author: Yonghong Song <[email protected]>

bpf, arm64: Move kfunc arguments into the arm64 calling convention

Do proper move from bpf calling convention to arm64 calling convention to satisfy native requirement. AAPCS64 only ever moves an argument to a higher slot, so the moves need one scratch register to carry an eightbyte from one stack slot to another, and never the one a convention moving an argument down would need.

Quoted 6 lines

In addition, the arena argument walk counts eightbytes rather than parameters, as an argument may take two registers. The walk takes the func model from the caller now, as the moves need it too, and runs first so that they carry the rebased value.

This looks like a bug fix for multi-slot kfunc argument support. Should this include:

Fixes 2092c3247e45 ("bpf: Prepare kfunc arguments for the JIT from an ABI description")

In v1 of this series, Alexei Starovoitov raised a design concern about the overall approach. He questioned why the BPF calling convention should diverge from x86-64/arm64, noting that both JIT patches doing argument shuffling is a sign that the approach may be wrong on the BPF side. He suggested adjusting the BPF calling convention to match arm64 (the stricter of the two architectures) rather than forcing all JITs to adapt:

"Why cannot we adjust bpf calling convention to match arm64/x86 the best? Since arm64 is stricter, I'd pick that style. There are no kfuncs that use int128 or 16+ byte args, so it's a matter of bpf subprogs calling bpf subprogs. Seems cleaner to adjust what llvm emits instead of forcing all jits to adapt."

The v2 took the opposite approach - keeping the BPF calling convention and implementing JIT argument shuffling in both x86 and arm64. Was this design decision explicitly discussed and agreed upon?

Quoted 4 lines
arch/arm64/net/bpf_jit_comp.c +0 -0
index 3aa3ea0bc30b..bdac930dbdec 100644

[ ... ]

Quoted 30 lines
arch/arm64/net/bpf_jit_comp.c +10 -9
@@ -1262,19 +1268,20 @@ static void emit_stack_arg_store_imm(s32 imm, s16 bpf_off, const u8 tmp, struct
  * kern_vm_start. A nullable arg preserves NULL by skipping the add, tested
  * on the truncated value as arena NULL is offset 0.
  */
-static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *insn)
+static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct btf_func_model *fm)
 {
 	const u8 arena_vm_base = bpf2a64[ARENA_VM_START];
-	const struct btf_func_model *fm;
-	int i;
-
-	fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
-	if (!fm)
-		return -EINVAL;
+	int i, slot;
 
-	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
-		const u8 reg = bpf2a64[BPF_REG_1 + i];
+	for (i = 0, slot = 0; i < fm->nr_args; i++) {
+		u32 arg_regs = (fm->arg_size[i] + 7) / 8;
 		u8 flags = fm->arg_flags[i];
+		u8 reg;
+
+		if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
+			break;
+		reg = bpf2a64[BPF_REG_1 + slot];
+		slot += arg_regs;
 
 		if (!(flags & BTF_FMODEL_ARENA_ARG))
 			continue;

[ ... ]

Quoted 32 lines
arch/arm64/net/bpf_jit_comp.c +28 -0
@@ -1293,6 +1300,45 @@ static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins
 	return 0;
 }
 
+static bool a64_arg_on_stack(u8 slot)
+{
+	return slot >= arm64_arg_abi.nr_arg_regs;
+}
+
+static s32 a64_arg_stack_off(u8 slot)
+{
+	return (slot - arm64_arg_abi.nr_arg_regs) * sizeof(u64);
+}
+
+/*
+ * AAPCS64 only ever moves an argument to a higher slot, so the planner asks
+ * for the scratch only to carry an eightbyte from one stack slot to another.
+ */
+static void emit_kfunc_arg_moves(struct jit_ctx *ctx, const struct btf_func_model *fm)
+{
+	struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
+	const u8 tmp = bpf2a64[TMP_REG_1];
+	u32 i, n;
+
+	n = bpf_jit_plan_arg_moves(&arm64_arg_abi, fm, moves);
+
+	for (i = 0; i < n; i++) {
+		u8 dst = moves[i].dst, src = moves[i].src, reg;
+
+		if (a64_arg_on_stack(src)) {
+			reg = a64_arg_on_stack(dst) ? tmp : dst;
+			emit(A64_LDR64I(reg, A64_SP, a64_arg_stack_off(src)), ctx);

arch/arm64/net/bpf_jit_comp.c static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins · line 1340

Quoted 10 lines
arch/arm64/net/bpf_jit_comp.c +10 -0
+		} else {
+			reg = src;
+		}
+
+		if (a64_arg_on_stack(dst))
+			emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);
+		else if (reg != dst)
+			emit(A64_MOV(1, dst, reg), ctx);
+	}
+}

This isn't a bug, but would it be clearer to say the plan never needs a scratch on AAPCS64 and that TMP_REG_1 is only this helper's stack-to-stack carrier? As written, the comment reads as though moves[] can contain BPF_JIT_ARG_TMP (which the loop does not handle - unlike the x86 version at arch/x86/net/bpf_jit_comp.c which does), and the ': dst' arm in the conditional looks unreachable for the same reason (src on the stack implies dst on the stack, since dst >= src >= nr_arg_regs).

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#23 lore raw

Re: [PATCH bpf-next v2 09/12] bpf, arm64: Move kfunc arguments into the arm64 calling convention

Quoted 35 lines · On 9/9/26 12:30 AM, [email protected] wrote:
Quoted 11 lines

commit 7c4f41d1b9c4dfdbada8f6d1b1097849ccb20de8
Author: Yonghong Song <[email protected]>

bpf, arm64: Move kfunc arguments into the arm64 calling convention

Do proper move from bpf calling convention to arm64 calling convention to satisfy native requirement. AAPCS64 only ever moves an argument to a higher slot, so the moves need one scratch register to carry an eightbyte from one stack slot to another, and never the one a convention moving an argument down would need.

Quoted 6 lines

In addition, the arena argument walk counts eightbytes rather than parameters, as an argument may take two registers. The walk takes the func model from the caller now, as the moves need it too, and runs first so that they carry the rebased value.

This looks like a bug fix for multi-slot kfunc argument support. Should this include:

Fixes 2092c3247e45 ("bpf: Prepare kfunc arguments for the JIT from an ABI description")

In v1 of this series, Alexei Starovoitov raised a design concern about the overall approach. He questioned why the BPF calling convention should diverge from x86-64/arm64, noting that both JIT patches doing argument shuffling is a sign that the approach may be wrong on the BPF side. He suggested adjusting the BPF calling convention to match arm64 (the stricter of the two architectures) rather than forcing all JITs to adapt:

"Why cannot we adjust bpf calling convention to match arm64/x86 the best? Since arm64 is stricter, I'd pick that style. There are no kfuncs that use int128 or 16+ byte args, so it's a matter of bpf subprogs calling bpf subprogs. Seems cleaner to adjust what llvm emits instead of forcing all jits to adapt."

Quoted 3 lines

The v2 took the opposite approach - keeping the BPF calling convention and implementing JIT argument shuffling in both x86 and arm64. Was this design decision explicitly discussed and agreed upon?

Yes, we will keep original bpf calling convention and then jit will do proper adjustment.

Quoted 80 lines
Quoted 5 lines
arch/arm64/net/bpf_jit_comp.c +0 -0
index 3aa3ea0bc30b..bdac930dbdec 100644

[ ... ]

Quoted 30 lines
arch/arm64/net/bpf_jit_comp.c +10 -9
@@ -1262,19 +1268,20 @@ static void emit_stack_arg_store_imm(s32 imm, s16 bpf_off, const u8 tmp, struct
   * kern_vm_start. A nullable arg preserves NULL by skipping the add, tested
   * on the truncated value as arena NULL is offset 0.
   */
-static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *insn)
+static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct btf_func_model *fm)
  {
  	const u8 arena_vm_base = bpf2a64[ARENA_VM_START];
-	const struct btf_func_model *fm;
-	int i;
-
-	fm = bpf_jit_find_kfunc_model(ctx->prog, insn);
-	if (!fm)
-		return -EINVAL;
+	int i, slot;
 
-	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
-		const u8 reg = bpf2a64[BPF_REG_1 + i];
+	for (i = 0, slot = 0; i < fm->nr_args; i++) {
+		u32 arg_regs = (fm->arg_size[i] + 7) / 8;
  		u8 flags = fm->arg_flags[i];
+		u8 reg;
+
+		if (slot + arg_regs > MAX_BPF_FUNC_REG_ARGS)
+			break;
+		reg = bpf2a64[BPF_REG_1 + slot];
+		slot += arg_regs;
 
  		if (!(flags & BTF_FMODEL_ARENA_ARG))
  			continue;

[ ... ]

Quoted 41 lines
arch/arm64/net/bpf_jit_comp.c +37 -0
@@ -1293,6 +1300,45 @@ static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins
  	return 0;
  }
 
+static bool a64_arg_on_stack(u8 slot)
+{
+	return slot >= arm64_arg_abi.nr_arg_regs;
+}
+
+static s32 a64_arg_stack_off(u8 slot)
+{
+	return (slot - arm64_arg_abi.nr_arg_regs) * sizeof(u64);
+}
+
+/*
+ * AAPCS64 only ever moves an argument to a higher slot, so the planner asks
+ * for the scratch only to carry an eightbyte from one stack slot to another.
+ */
+static void emit_kfunc_arg_moves(struct jit_ctx *ctx, const struct btf_func_model *fm)
+{
+	struct bpf_jit_arg_move moves[BPF_JIT_MAX_ARG_MOVES];
+	const u8 tmp = bpf2a64[TMP_REG_1];
+	u32 i, n;
+
+	n = bpf_jit_plan_arg_moves(&arm64_arg_abi, fm, moves);
+
+	for (i = 0; i < n; i++) {
+		u8 dst = moves[i].dst, src = moves[i].src, reg;
+
+		if (a64_arg_on_stack(src)) {
+			reg = a64_arg_on_stack(dst) ? tmp : dst;
+			emit(A64_LDR64I(reg, A64_SP, a64_arg_stack_off(src)), ctx);
+		} else {
+			reg = src;
+		}
+
+		if (a64_arg_on_stack(dst))
+			emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);
+		else if (reg != dst)
+			emit(A64_MOV(1, dst, reg), ctx);
+	}
Quoted 8 lines

arch/arm64/net/bpf_jit_comp.c static int emit_kfunc_arena_args(struct jit_ctx *ctx, const struct bpf_insn *ins · line 1340

Quoted 1 line
arch/arm64/net/bpf_jit_comp.c +1 -0
+}

This isn't a bug, but would it be clearer to say the plan never needs a scratch on AAPCS64 and that TMP_REG_1 is only this helper's stack-to-stack carrier? As written, the comment reads as though moves[] can contain BPF_JIT_ARG_TMP (which the loop does not handle - unlike the x86 version at arch/x86/net/bpf_jit_comp.c which does), and the ': dst' arm in the conditional looks unreachable for the same reason (src on the stack implies dst on the stack, since dst >= src >= nr_arg_regs).

Okay, will fix.

Quoted 9 lines

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#24 lore raw

PATCH v2 10/12 selftests/bpf: Add C tests for by-value arguments up to 16 bytes

In reply to Yonghong Song

Extend the by-value argument test with the aggregate cases, written in C so that they depend on the compiler lowering the argument into a pair of argument registers rather than on a hand-written register layout.

The programs cover a struct and a union that fill two registers, a smaller struct that fills one, and two struct arguments in a row, alongside the __int128 already there. Each has an int argument around it so that a wrong slot count shows up as a wrong value in the parameters beside it; two pairs leave room for only one, which follows them. A global function taking a struct with a pointer member is rejected: the callee would receive the pointer as an opaque scalar.

A struct the argument registers cannot hold reaches the callee partly on the stack, which the interpreter does not implement, so that case is loaded only when the JIT is on.

---
 .../selftests/bpf/progs/verifier_int128_arg.c | 165 ++++++++++++++++++
 1 file changed, 165 insertions(+)
tools/testing/selftests/bpf/progs/verifier_int128_arg.c +165 -0
index 044fc4a80055..f4610030e662 100644
@@ -7,6 +7,131 @@
 #define MIX_A	0xdeadbeefcafef00dULL
 #define MIX_B	0x0123456789abcdefULL
 
+struct pair {
+	__u64 lo;
+	__u64 hi;
+};
+
+struct small {
+	__u32 a;
+	__u32 b;
+};
+
+union upair {
+	__u64 halves[2];
+	struct {
+		__u64 lo;
+		__u64 hi;
+	} parts;
+};
+
+struct with_ptr {
+	void *p;
+	__u64 x;
+};
+
+static __noinline __u64 take_pair(int a, struct pair p, int c)
+{
+	return (__u64)a + p.lo + p.hi + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_static_struct_c_test(struct __sk_buff *skb)
+{
+	__u64 a = skb->len ^ MIX_A;
+	__u64 b = skb->len ^ MIX_B;
+	struct pair p = { .lo = a, .hi = b };
+
+	if (take_pair(1, p, 2) != a + b + 3)
+		return 1;
+
+	return 0;
+}
+
+#if defined(__clang__)
+
+__noinline __u64 take_pair_global(int a, struct pair p, int c)
+{
+	return (__u64)a + p.lo + p.hi + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_global_struct_c_test(struct __sk_buff *skb)
+{
+	__u64 a = skb->len ^ MIX_A;
+	__u64 b = skb->len ^ MIX_B;
+	struct pair p = { .lo = a, .hi = b };
+
+	if (take_pair_global(1, p, 2) != a + b + 3)
+		return 1;
+
+	return 0;
+}
+
+__noinline __u64 take_two_pairs_global(struct pair p, struct pair q, int d)
+{
+	return p.lo + p.hi + q.lo + q.hi + d;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_two_structs_c_test(struct __sk_buff *skb)
+{
+	__u64 a = skb->len ^ MIX_A;
+	__u64 b = skb->len ^ MIX_B;
+	struct pair p = { .lo = a, .hi = b };
+	struct pair q = { .lo = a + 1, .hi = b + 2 };
+
+	if (take_two_pairs_global(p, q, 3) != 2 * a + 2 * b + 6)
+		return 1;
+
+	return 0;
+}
+
+__noinline __u64 take_small_global(int a, struct small s, int c)
+{
+	return (__u64)a + s.a + s.b + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_small_struct_c_test(struct __sk_buff *skb)
+{
+	__u32 a = skb->len ^ (__u32)MIX_A;
+	__u32 b = skb->len ^ (__u32)MIX_B;
+	struct small s = { .a = a, .b = b };
+
+	if (take_small_global(1, s, 2) != (__u64)a + b + 3)
+		return 1;
+
+	return 0;
+}
+
+__noinline __u64 take_upair_global(int a, union upair u, int c)
+{
+	return (__u64)a + u.parts.lo + u.parts.hi + c;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_arg_union_c_test(struct __sk_buff *skb)
+{
+	__u64 a = skb->len ^ MIX_A;
+	__u64 b = skb->len ^ MIX_B;
+	union upair u;
+
+	u.halves[0] = a;
+	u.halves[1] = b;
+	if (take_upair_global(1, u, 2) != a + b + 3)
+		return 1;
+
+	return 0;
+}
+
+#endif
+
 typedef unsigned __int128 u128;
 
 __noinline __u64 take_i128_global(int a, u128 v, int c)
@@ -28,4 +153,44 @@ int aggregate_arg_int128_c_test(struct __sk_buff *skb)
 	return 0;
 }
 
+#if defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+static __noinline __u64 take_spilled_pair(int a, int b, int c, int d, struct pair p)
+{
+	return (__u64)a + b + c + d + p.lo + p.hi;
+}
+
+SEC("tc")
+__arch_x86_64 __arch_arm64
+__load_if_JITed()
+__success __retval(0)
+int aggregate_arg_spilled_struct_c_test(struct __sk_buff *skb)
+{
+	__u64 a = skb->len ^ MIX_A;
+	__u64 b = skb->len ^ MIX_B;
+	struct pair p = { .lo = a, .hi = b };
+	int n = skb->len;
+
+	if (take_spilled_pair(n, n + 1, n + 2, n + 3, p) != a + b + 4 * n + 6)
+		return 1;
+
+	return 0;
+}
+
+#endif
+
+__noinline __u64 take_with_ptr_global(struct with_ptr s)
+{
+	return s.x;
+}
+
+SEC("tc")
+__failure __msg("type STRUCT in take_with_ptr_global() is not composed of scalars")
+int aggregate_arg_ptr_member_fail(struct __sk_buff *skb)
+{
+	struct with_ptr s = { .p = skb, .x = skb->len };
+
+	return take_with_ptr_global(s);
+}
+
 char _license[] SEC("license") = "GPL";
Signature · 1 line
2.53.0-Meta
author reviewer
#25 lore raw

PATCH v2 11/12 selftests/bpf: Add inline-asm tests for by-value arguments

In reply to Yonghong Song

The tests cover a struct passed in a register pair, a pointer in one half of it refused at the call site, a struct too large to pass by value, and three placements a global function cannot have: a struct split between the last argument register and the stack, one wholly past the registers, and an __int128 whose two slots push the last parameter out.

GCC passes an aggregate by invisible reference, so a callee it compiles expects a pointer where BTF says the halves of the struct are, and those tests are left to clang.

---
 .../selftests/bpf/prog_tests/aggregate_arg.c  |   9 +
 .../selftests/bpf/progs/aggregate_arg_func.c  | 155 ++++++++++++++++++
 2 files changed, 164 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/aggregate_arg.c
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_arg_func.c
tools/testing/selftests/bpf/prog_tests/aggregate_arg.c new file +9 -0
new file mode 100644
index 000000000000..b230f3bd3b2a
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include "aggregate_arg_func.skel.h"
+
+void test_aggregate_arg(void)
+{
+	RUN_TESTS(aggregate_arg_func);
+}
tools/testing/selftests/bpf/progs/aggregate_arg_func.c new file +155 -0

155 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#26 lore raw

Re: [PATCH bpf-next v2 11/12] selftests/bpf: Add inline-asm tests for by-value arguments

This isn't a bug, but would it help a later reader to carry the changelog's gcc-invisible-reference rationale into a short comment above the #if defined(__clang__), and to note why the last three…
Quoted 1 line
tools/testing/selftests/bpf/prog_tests/aggregate_arg.c +0 -0

[ ... ]

Quoted 31 lines
tools/testing/selftests/bpf/progs/aggregate_arg_func.c new file +25 -0
new file mode 100644
index 000000000000..61dd5f86c3a4
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+typedef unsigned __int128 u128;
+
+struct pair {
+	__u64 lo;
+	__u64 hi;
+};
+
+struct too_big {
+	__u64 a;
+	__u64 b;
+	__u64 c;
+};
+
+#if defined(__clang__)
+
+__noinline __u64 global_arg_pair(int a, struct pair p, int c)
+{
+	return (__u64)a + p.lo + p.hi + c;
+}

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/progs/aggregate_arg_func.c new file +6 -0
+#endif
+
+__noinline __u64 global_arg_too_big(struct too_big s)
+{
+	return s.a + s.b + s.c;
+}

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/progs/aggregate_arg_func.c new file +6 -0
+#if defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+__noinline __u64 global_arg_split(int a, int b, int c, int d, struct pair p)
+{
+	return (__u64)a + b + c + d + p.lo + p.hi;
+}

[ ... ]

Quoted 1 line

+#endif

This isn't a bug, but would it help a later reader to carry the changelog's gcc-invisible-reference rationale into a short comment above the #if defined(__clang__), and to note why the last three cases need __BPF_FEATURE_STACK_ARGUMENT (and maybe label the two #endif closures)?

The sibling file aggregate_ret_func.c documents its equivalent gate:

/*
* gcc returns a by-value struct through a hidden pointer, and emits the
* 'r0 = r1' returning it after the __naked body's exit, leaving the
* subprogram falling through. Build these with clang only. */
#if defined(__clang__)

and other progs files label the closing directive, for example verifier_ldsx.c uses #endif /* __BPF_FEATURE_ADDR_SPACE_CAST */.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#27 lore raw

Re: [PATCH bpf-next v2 11/12] selftests/bpf: Add inline-asm tests for by-value arguments

Quoted 52 lines · On 9/9/26 12:30 AM, [email protected] wrote:
Quoted 1 line
tools/testing/selftests/bpf/prog_tests/aggregate_arg.c +0 -0

[ ... ]

Quoted 31 lines
tools/testing/selftests/bpf/progs/aggregate_arg_func.c new file +25 -0
new file mode 100644
index 000000000000..61dd5f86c3a4
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+typedef unsigned __int128 u128;
+
+struct pair {
+	__u64 lo;
+	__u64 hi;
+};
+
+struct too_big {
+	__u64 a;
+	__u64 b;
+	__u64 c;
+};
+
+#if defined(__clang__)
+
+__noinline __u64 global_arg_pair(int a, struct pair p, int c)
+{
+	return (__u64)a + p.lo + p.hi + c;
+}

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/progs/aggregate_arg_func.c new file +6 -0
+#endif
+
+__noinline __u64 global_arg_too_big(struct too_big s)
+{
+	return s.a + s.b + s.c;
+}

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/progs/aggregate_arg_func.c new file +6 -0
+#if defined(__BPF_FEATURE_STACK_ARGUMENT)
+
+__noinline __u64 global_arg_split(int a, int b, int c, int d, struct pair p)
+{
+	return (__u64)a + b + c + d + p.lo + p.hi;
+}

[ ... ]

Quoted 5 lines
Quoted 1 line

+#endif

This isn't a bug, but would it help a later reader to carry the changelog's gcc-invisible-reference rationale into a short comment above the #if defined(__clang__), and to note why the last three cases need __BPF_FEATURE_STACK_ARGUMENT (and maybe label the two #endif closures)?

stack argument is implemented in llvm23, and that is why we want this feature gating.

Quoted 10 lines

The sibling file aggregate_ret_func.c documents its equivalent gate:

/*
* gcc returns a by-value struct through a hidden pointer, and emits the
* 'r0 = r1' returning it after the __naked body's exit, leaving the
* subprogram falling through. Build these with clang only. */
#if defined(__clang__)

Quoted 2 lines

and other progs files label the closing directive, for example verifier_ldsx.c uses #endif /* __BPF_FEATURE_ADDR_SPACE_CAST */.

This is fine. We do not want to have this comment in many places. The commit message should explain this.

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#28 lore raw

PATCH v2 12/12 selftests/bpf: Add tests for by-value kfunc arguments

In reply to Yonghong Song

Add kfuncs taking a 16-byte struct and an __int128 by value, with combinations of <= 8 byte arguments and '> 8 && <= 16' byte arguments, and tests that call them. Each checks the value the kfunc returns, so a misplaced argument shows up rather than passing quietly.

Between them the cases cover an argument the two conventions place alike, one arm64 moves between registers, one an ABI moves to or from the stack, and one both ABIs pad the stack for. All of them run on both arches; what differs is how many instructions the JIT emits to get there. Two more cover the rejections that hold everywhere, an aggregate holding a pointer and one too large to pass by value.

test_stack_arg_big() in stack_arg_fail.c passed a 16-byte struct as the sixth argument and asserted the unrecognized stack argument type it used to be reported as. The JIT places that argument now, so the test is removed and the same call is covered by the cases above.

---
 .../selftests/bpf/prog_tests/aggregate_arg.c  |   2 +
 .../selftests/bpf/progs/aggregate_arg_kfunc.c | 221 ++++++++++++++++++
 .../testing/selftests/bpf/progs/arena_kfunc.c |  16 ++
 .../selftests/bpf/progs/stack_arg_fail.c      |  10 -
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  77 ++++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  32 +++
 6 files changed, 348 insertions(+), 10 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c
tools/testing/selftests/bpf/prog_tests/aggregate_arg.c +2 -0
index b230f3bd3b2a..aa7562f48737 100644
@@ -2,8 +2,10 @@
 /* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
 #include <test_progs.h>
 #include "aggregate_arg_func.skel.h"
+#include "aggregate_arg_kfunc.skel.h"
 
 void test_aggregate_arg(void)
 {
 	RUN_TESTS(aggregate_arg_func);
+	RUN_TESTS(aggregate_arg_kfunc);
 }
tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c new file +221 -0

221 lines not on this page · raw message

tools/testing/selftests/bpf/progs/arena_kfunc.c +16 -0
index 50609f3b0564..8d1ce360a309 100644
@@ -228,6 +228,22 @@ int arena_arg_stack(void *ctx)
 	bpf_kfunc_arena_stack_arg_test(1, 2, 3, 4, 5, (u64 *)1);
 	return 0;
 }
+
+#if defined(__clang__)
+/* The struct takes two slots, so the arena pointer is the sixth. */
+SEC("syscall")
+__arch_x86_64
+__arch_arm64
+__failure __msg("arena pointer cannot be a stack argument")
+int arena_arg_stack_after_pair(void *ctx)
+{
+	struct prog_test_pair_arg s = { .lo = 1, .hi = 2 };
+
+	bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+	bpf_kfunc_call_test_pair_arena_arg(1, 2, 3, s, (u64 *)1);
+	return 0;
+}
+#endif
 #else
 SEC("syscall")
 __arch_x86_64
tools/testing/selftests/bpf/progs/stack_arg_fail.c +0 -10
index eed97d582515..fff2e947ea33 100644
@@ -3,20 +3,10 @@
 
 #include <vmlinux.h>
 #include <bpf/bpf_helpers.h>
-#include "../test_kmods/bpf_testmod_kfunc.h"
 #include "bpf_misc.h"
 
 #if defined(__BPF_FEATURE_STACK_ARGUMENT)
 
-SEC("tc")
-__failure __msg("Unrecognized *(R11-8) type STRUCT")
-int test_stack_arg_big(struct __sk_buff *skb)
-{
-	struct prog_test_big_arg s = { .a = 1, .b = 2 };
-
-	return bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s);
-}
-
 SEC("socket")
 __description("r11 in ALU instruction")
 __failure __msg("R11 is invalid")
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +77 -0

89 lines not on this page · raw message

tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +32 -0
index b213ef14848b..195ec37d5bbc 100644
@@ -61,6 +61,16 @@ struct prog_test_big_arg {
 	__u64 b;
 };
 
+struct prog_test_pair_arg {	/* 16 bytes: two argument registers */
+	__u64 lo;
+	__u64 hi;
+};
+
+struct prog_test_ptr_arg {	/* 16 bytes, but holds a pointer */
+	void *p;
+	__u64 x;
+};
+
 struct prog_test_ret_pair {	/* 16 bytes: R0:R2 */
 	__u64 lo;
 	__u64 hi;
@@ -221,6 +231,28 @@ __int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
 struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
 struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym;
 struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
+__u64 bpf_kfunc_call_test_pair_arg(__u64 a, struct prog_test_pair_arg s, __u64 b) __ksym;
+#ifdef __SIZEOF_INT128__
+__u64 bpf_kfunc_call_test_i128_arg(__u64 a, __u64 b, __int128 v) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_odd(__u64 a, __int128 v, __u64 b) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_shift(__u64 a, __int128 v, __u64 b, __u64 c,
+					 __u64 d) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_ovf(__u64 a, __int128 v, __u64 b, __u64 c,
+				       __u64 d, __u64 e, __u64 f) __ksym;
+__u64 bpf_kfunc_call_test_i128_arg_pad(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				       __u64 f, __u64 g, __int128 v) __ksym;
+#endif
+__u64 bpf_kfunc_call_test_pair_arg_nofit(__u64 a, __u64 b, __u64 c, __u64 d,
+					 struct prog_test_pair_arg s) __ksym;
+__u64 bpf_kfunc_call_test_pair_arg_tail(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+					struct prog_test_pair_arg s, __u64 f) __ksym;
+__u64 bpf_kfunc_call_test_pair_arg_split8(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+					  __u64 f, __u64 g,
+					  struct prog_test_pair_arg s) __ksym;
+__u64 bpf_kfunc_call_test_ptr_arg(struct prog_test_ptr_arg s) __ksym;
+__u64 bpf_kfunc_call_test_pair_arena_arg(__u64 a, __u64 b, __u64 c,
+					 struct prog_test_pair_arg s,
+					 __u64 *f__arena) __ksym;
 struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym;
 struct prog_test_ret_nested bpf_kfunc_call_test_ret_nested(__u64 tag) __ksym;
 struct prog_test_ret_ptr_arr bpf_kfunc_call_test_ret_ptr_arr(void) __ksym;
Signature · 1 line
2.53.0-Meta
#29 lore raw

Re: [PATCH bpf-next v2 12/12] selftests/bpf: Add tests for by-value kfunc arguments

The changelog describes two new rejection ('__failure') cases, but only one of the two is added by this commit, and a third rejection case that IS added is never mentioned.
Quoted 11 lines

Add kfuncs taking a 16-byte struct and an __int128 by value, with combinations of <= 8 byte arguments and '> 8 && <= 16' byte arguments, and tests that call them. Each checks the value the kfunc returns, so a misplaced argument shows up rather than passing quietly.

Between them the cases cover an argument the two conventions place alike, one arm64 moves between registers, one an ABI moves to or from the stack, and one both ABIs pad the stack for. All of them run on both arches; what differs is how many instructions the JIT emits to get there. Two more cover the rejections that hold everywhere, an aggregate holding a pointer and one too large to pass by value.

The changelog describes two new rejection ('__failure') cases, but only one of the two is added by this commit, and a third rejection case that IS added is never mentioned.

Looking at the added tests:

1. 'an aggregate holding a pointer' -> present:

progs/aggregate_arg_kfunc.c adds:
SEC("tc") __arch_x86_64 __arch_arm64
__failure __msg("R1 type STRUCT is not composed of scalars") int aggregate_arg_kfunc_ptr_member(struct __sk_buff *skb)

2. 'one too large to pass by value' -> NOT added here:

progs/aggregate_arg_kfunc.c contains exactly one __failure program (the ptr_member one above), and the commit declares no kfunc taking a by-value aggregate larger than 16 bytes in test_kmods/bpf_testmod_kfunc.h, so such a case cannot exist in this commit.

The only test matching that description is aggregate_arg_too_big_fail() in progs/aggregate_arg_func.c with:
__failure __msg("... has size 24, only 1 to 16 bytes can be passed by value")

which was added by the earlier commit b3f328dbfb8a ("selftests/bpf: Add inline-asm tests for by-value arguments") and is untouched here.

3. This commit adds a rejection case the changelog never mentions:

progs/arena_kfunc.c gains 16 lines adding:
/* The struct takes two slots, so the arena pointer is the sixth. */ SEC("syscall") __arch_x86_64 __arch_arm64
__failure __msg("arena pointer cannot be a stack argument") int arena_arg_stack_after_pair(void *ctx)

together with the new bpf_kfunc_call_test_pair_arena_arg() kfunc and its __ksym declaration. arena_kfunc.c is one of the six files in the diffstat but is not described anywhere in the message.

Could the message be updated to say that one new rejection case is added for an aggregate holding a pointer (with the too-large rejection being already covered by aggregate_arg_func.c), and add a sentence for the arena stack-argument rejection in arena_kfunc.c?

Quoted 5 lines
tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c new file +0 -0
new file mode 100644
index 000000000000..c8a088e2f5e3

[ ... ]

Quoted 5 lines

+/*
+ * One argument further and the hole pushes the last one off x7 and onto the
+ * arm64 stack, which the JIT does not shift into. The x86-64 ABI packs the
+ * eightbytes, so its last two are on the stack where BPF put them.
+ */

Does the comment accurately reflect what the arm64 JIT does? Looking at bpf_jit_place_args() with arm64_arg_abi:

For bpf_kfunc_call_test_i128_arg_ovf(u64 a, __int128 v, u64 b, u64 c, u64 d, u64 e, u64 f), the arm64 calling convention computes:
slot0(a)->pos0, slot1/2(v)->pos2/3, slot3(b)->4, slot4(c)->5, slot5(d)->6, slot6(e)->7, slot7(f)->pos8

Position 8 is the first arm64 stack slot (a64_arg_on_stack(8) is true), so bpf_jit_plan_arg_moves() emits the move {dst=8, src=7}.

arch/arm64/net/bpf_jit_comp.c:emit_kfunc_arg_moves() handles this with:

    if (a64_arg_on_stack(dst))
            emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);

The space is reserved by kfunc_abi_slots() returning 9 for this kfunc, giving out_stack_arg_cnt = 4, nr_on_stack = 1, and ctx->stack_arg_size becomes 16 bytes.

This test is annotated '__success __retval(0)', which depends on the JIT correctly placing argument f into the arm64 stack. Does the comment need updating to reflect that the JIT does shift into the arm64 stack?

Quoted 4 lines
tools/testing/selftests/bpf/progs/stack_arg_fail.c +0 -0
index 1cd3eee07802..80b8cbf15cdb 100644

[ ... ]

Quoted 8 lines
tools/testing/selftests/bpf/progs/stack_arg_fail.c +0 -1
@@ -3,20 +3,10 @@
 
 #include <vmlinux.h>
 #include <bpf/bpf_helpers.h>
-#include "../test_kmods/bpf_testmod_kfunc.h"
 #include "bpf_misc.h"
 
 #if defined(__BPF_FEATURE_STACK_ARGUMENT)

tools/testing/selftests/bpf/progs/stack_arg_fail.c line 19

Quoted 10 lines
tools/testing/selftests/bpf/progs/stack_arg_fail.c +0 -9
 
-SEC("socket")
-__failure __msg("Unrecognized *(R11-8) type STRUCT")
-int test_stack_arg_big(struct __sk_buff *ctx)
-{
-	struct prog_test_big_arg s = {1, 2};
-
-	return bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s);
-}
-

The changelog says "the test is removed and the same call is covered by the cases above." That holds only on x86_64 and arm64.

The replacement, aggregate_arg_kfunc_straddle() in progs/aggregate_arg_kfunc.c:97-107, makes the byte-identical call bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s) and correctly checks the return value, but it is tagged:

SEC("tc")
__arch_x86_64 __arch_arm64
__load_if_JITed()
__success __retval(0)

test_loader.c:1339 skips a test whose arch_mask excludes the host arch. The deleted test was gated only on __BPF_FEATURE_STACK_ARGUMENT and ran everywhere.

On riscv64, s390x and loongarch this call is no longer exercised at all. Only x86 and arm64 define bpf_jit_arg_abi(); elsewhere gen_kfunc_arg_proto() takes the '!abi && nslots > 1' branch at verifier.c:12428 and rejects the argument with:

"Function %s arg#%d type %s cannot be passed at argument slot %d on this architecture"

That message is new in this series and has no selftest coverage.

So on the non-ABI architectures the series swapped one rejection message for another and this commit removed the only test that exercised the call there. Should a __failure __msg("cannot be passed at argument slot") case be added to restore coverage on those architectures?

Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +0 -0
index 1ae88a12d0c4..1d71ba0bc6ad 100644

[ ... ]

Quoted 14 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +14 -0
+__bpf_kfunc u64 bpf_kfunc_call_test_pair_arg(u64 a, struct prog_test_pair_arg s, u64 b)
+{
+	return a + s.lo + s.hi + b;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg(u64 a, u64 b, __int128 v)
+{
+	return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_odd(u64 a, __int128 v, u64 b)
+{
+	return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}

[ ... ]

Quoted 9 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +9 -0
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_shift(u64 a, __int128 v, u64 b, u64 c, u64 d)
+{
+	return a + b + c + d + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_ovf(u64 a, __int128 v, u64 b, u64 c, u64 d, u64 e, u64 f)
+{
+	return a + b + c + d + e + f + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}

[ ... ]

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c

Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +4 -0
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_pad(u64 a, u64 b, u64 c, u64 d, u64 e, u64 f, u64 g, __int128 v)
+{
+	return a + b + c + d + e + f + g + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}

Three of the new kfuncs place the __int128 after a single-register
scalar: bpf_kfunc_call_test_i128_arg_odd(), i128_arg_shift(), and i128_arg_ovf(), all of the shape 'u64 a, __int128 v, ...'.

This is exactly the shape a comment already in this same file warns against. bpf_testmod.c:174-186 says, above bpf_testmod_test_int128_arg():

/*
* The __int128 'a' is the first argument on purpose. On arm64 a 16-byte
* argument must start in an even-numbered register pair, so placing it
* after a single-register scalar would leave a padding register (x1)
* unused. pahole maps parameters to registers positionally and would then
* see the following argument in an "unexpected" register and skip BTF
* encoding of the whole function, making it unattachable. [...] */

On arm64 AAPCS64 rounds NGRN up to an even number for a 16-byte-aligned type, so for (u64 a, __int128 v, u64 b) the actual placement is a=x0, x1 padding, v=x2:x3, b=x4, while a positional mapping expects v=x1:x2 and b=x3.

Per the in-file comment that makes pahole drop the BTF FUNC record for the whole function on arm64. For a kfunc that is worse than for an fentry
target: the name is registered with BTF_ID_FLAGS(), so resolve_btfids would leave the ID at 0 and emit:

'WARN: resolve_btfids: unresolved symbol bpf_kfunc_call_test_i128_arg_odd'

which is fatal when CONFIG_WERROR selects --fatal_warnings. Even when only a warning, libbpf cannot resolve the ksym against the module BTF, so aggregate_arg_kfunc_int128_odd, aggregate_arg_kfunc_int128_shift and aggregate_arg_kfunc_int128_ovf fail to load on arm64 instead of testing the JIT.

bpf_kfunc_call_test_i128_arg_pad() has the same hazard in the milder stack-vs-register form (a..g fill x0..x6, then v is pushed to the stack where x7 would be expected positionally).

The pre-existing bpf_kfunc_call_test_i128() and bpf_testmod_test_int128_arg() both deliberately keep the __int128 in the first position, and only bpf_kfunc_call_test_i128_arg() (u64 a, u64 b, __int128 v) among the new ones happens to land on an even register with no padding.

Is the placement worth confirming against the pahole version used for CI before this lands, given it's the one thing in the change that the in-tree documentation says should not work on arm64?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441

author reviewer
#30 lore raw

Re: [PATCH bpf-next v2 12/12] selftests/bpf: Add tests for by-value kfunc arguments

Quoted 46 lines · On 9/9/26 12:30 AM, [email protected] wrote:
Quoted 11 lines

Add kfuncs taking a 16-byte struct and an __int128 by value, with combinations of <= 8 byte arguments and '> 8 && <= 16' byte arguments, and tests that call them. Each checks the value the kfunc returns, so a misplaced argument shows up rather than passing quietly.

Between them the cases cover an argument the two conventions place alike, one arm64 moves between registers, one an ABI moves to or from the stack, and one both ABIs pad the stack for. All of them run on both arches; what differs is how many instructions the JIT emits to get there. Two more cover the rejections that hold everywhere, an aggregate holding a pointer and one too large to pass by value.

The changelog describes two new rejection ('__failure') cases, but only one of the two is added by this commit, and a third rejection case that IS added is never mentioned.

Looking at the added tests:

1. 'an aggregate holding a pointer' -> present:

progs/aggregate_arg_kfunc.c adds:
SEC("tc") __arch_x86_64 __arch_arm64
__failure __msg("R1 type STRUCT is not composed of scalars") int aggregate_arg_kfunc_ptr_member(struct __sk_buff *skb)

2. 'one too large to pass by value' -> NOT added here:

progs/aggregate_arg_kfunc.c contains exactly one __failure program (the ptr_member one above), and the commit declares no kfunc taking a by-value aggregate larger than 16 bytes in test_kmods/bpf_testmod_kfunc.h, so such a case cannot exist in this commit.

The only test matching that description is aggregate_arg_too_big_fail() in progs/aggregate_arg_func.c with:
__failure __msg("... has size 24, only 1 to 16 bytes can be passed by value")

which was added by the earlier commit b3f328dbfb8a ("selftests/bpf: Add inline-asm tests for by-value arguments") and is untouched here.

3. This commit adds a rejection case the changelog never mentions:

progs/arena_kfunc.c gains 16 lines adding:
/* The struct takes two slots, so the arena pointer is the sixth. */ SEC("syscall") __arch_x86_64 __arch_arm64
__failure __msg("arena pointer cannot be a stack argument") int arena_arg_stack_after_pair(void *ctx)

Quoted 8 lines

together with the new bpf_kfunc_call_test_pair_arena_arg() kfunc and its __ksym declaration. arena_kfunc.c is one of the six files in the diffstat but is not described anywhere in the message.

Could the message be updated to say that one new rejection case is added for an aggregate holding a pointer (with the too-large rejection being already covered by aggregate_arg_func.c), and add a sentence for the arena stack-argument rejection in arena_kfunc.c?

Yes, I can do this.

Quoted 29 lines
Quoted 6 lines
tools/testing/selftests/bpf/progs/aggregate_arg_kfunc.c new file +0 -0
new file mode 100644
index 000000000000..c8a088e2f5e3

[ ... ]

Quoted 5 lines

+/*
+ * One argument further and the hole pushes the last one off x7 and onto the
+ * arm64 stack, which the JIT does not shift into. The x86-64 ABI packs the
+ * eightbytes, so its last two are on the stack where BPF put them.
+ */

Does the comment accurately reflect what the arm64 JIT does? Looking at bpf_jit_place_args() with arm64_arg_abi:

For bpf_kfunc_call_test_i128_arg_ovf(u64 a, __int128 v, u64 b, u64 c, u64 d, u64 e, u64 f), the arm64 calling convention computes:

   slot0(a)->pos0, slot1/2(v)->pos2/3, slot3(b)->4, slot4(c)->5,
   slot5(d)->6, slot6(e)->7, slot7(f)->pos8

Position 8 is the first arm64 stack slot (a64_arg_on_stack(8) is true), so bpf_jit_plan_arg_moves() emits the move {dst=8, src=7}.

arch/arm64/net/bpf_jit_comp.c:emit_kfunc_arg_moves() handles this with:

     if (a64_arg_on_stack(dst))
             emit(A64_STR64I(reg, A64_SP, a64_arg_stack_off(dst)), ctx);
Quoted 7 lines

The space is reserved by kfunc_abi_slots() returning 9 for this kfunc, giving out_stack_arg_cnt = 4, nr_on_stack = 1, and ctx->stack_arg_size becomes 16 bytes.

This test is annotated '__success __retval(0)', which depends on the JIT correctly placing argument f into the arm64 stack. Does the comment need updating to reflect that the JIT does shift into the arm64 stack?

The prog should already mentioned this. Not all of them but a couple of cases to illustrate.

Quoted 47 lines
Quoted 5 lines
tools/testing/selftests/bpf/progs/stack_arg_fail.c +0 -0
index 1cd3eee07802..80b8cbf15cdb 100644

[ ... ]

Quoted 8 lines
tools/testing/selftests/bpf/progs/stack_arg_fail.c +0 -1
@@ -3,20 +3,10 @@
 
  #include <vmlinux.h>
  #include <bpf/bpf_helpers.h>
-#include "../test_kmods/bpf_testmod_kfunc.h"
  #include "bpf_misc.h"
 
  #if defined(__BPF_FEATURE_STACK_ARGUMENT)

tools/testing/selftests/bpf/progs/stack_arg_fail.c line 19

Quoted 10 lines
tools/testing/selftests/bpf/progs/stack_arg_fail.c +0 -9
 
-SEC("socket")
-__failure __msg("Unrecognized *(R11-8) type STRUCT")
-int test_stack_arg_big(struct __sk_buff *ctx)
-{
-	struct prog_test_big_arg s = {1, 2};
-
-	return bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s);
-}
-

The changelog says "the test is removed and the same call is covered by the cases above." That holds only on x86_64 and arm64.

The replacement, aggregate_arg_kfunc_straddle() in progs/aggregate_arg_kfunc.c:97-107, makes the byte-identical call bpf_kfunc_call_stack_arg_big(1, 2, 3, 4, 5, s) and correctly checks the return value, but it is tagged:

   SEC("tc")
   __arch_x86_64 __arch_arm64
   __load_if_JITed()
   __success __retval(0)

test_loader.c:1339 skips a test whose arch_mask excludes the host arch. The deleted test was gated only on __BPF_FEATURE_STACK_ARGUMENT and ran everywhere.

On riscv64, s390x and loongarch this call is no longer exercised at all. Only x86 and arm64 define bpf_jit_arg_abi(); elsewhere gen_kfunc_arg_proto() takes the '!abi && nslots > 1' branch at verifier.c:12428 and rejects the argument with:

Quoted 8 lines

"Function %s arg#%d type %s cannot be passed at argument slot %d on this architecture"

That message is new in this series and has no selftest coverage.

So on the non-ABI architectures the series swapped one rejection message for another and this commit removed the only test that exercised the call there. Should a __failure __msg("cannot be passed at argument slot") case be added to restore coverage on those architectures?

I didn't test on riscv64, s390x and loongarch. That is why I skipped them.

Quoted 82 lines
Quoted 5 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +0 -0
index 1ae88a12d0c4..1d71ba0bc6ad 100644

[ ... ]

Quoted 14 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +14 -0
+__bpf_kfunc u64 bpf_kfunc_call_test_pair_arg(u64 a, struct prog_test_pair_arg s, u64 b)
+{
+	return a + s.lo + s.hi + b;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg(u64 a, u64 b, __int128 v)
+{
+	return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_odd(u64 a, __int128 v, u64 b)
+{
+	return a + b + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}

[ ... ]

Quoted 9 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +9 -0
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_shift(u64 a, __int128 v, u64 b, u64 c, u64 d)
+{
+	return a + b + c + d + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}
+
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_ovf(u64 a, __int128 v, u64 b, u64 c, u64 d, u64 e, u64 f)
+{
+	return a + b + c + d + e + f + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}

[ ... ]

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c

Quoted 4 lines
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +4 -0
+__bpf_kfunc u64 bpf_kfunc_call_test_i128_arg_pad(u64 a, u64 b, u64 c, u64 d, u64 e, u64 f, u64 g, __int128 v)
+{
+	return a + b + c + d + e + f + g + (u64)((unsigned __int128)v >> 64) + (u64)v;
+}

Three of the new kfuncs place the __int128 after a single-register
scalar: bpf_kfunc_call_test_i128_arg_odd(), i128_arg_shift(), and i128_arg_ovf(), all of the shape 'u64 a, __int128 v, ...'.

This is exactly the shape a comment already in this same file warns against. bpf_testmod.c:174-186 says, above bpf_testmod_test_int128_arg():

/*
* The __int128 'a' is the first argument on purpose. On arm64 a 16-byte
* argument must start in an even-numbered register pair, so placing it
* after a single-register scalar would leave a padding register (x1)
* unused. pahole maps parameters to registers positionally and would then
* see the following argument in an "unexpected" register and skip BTF
* encoding of the whole function, making it unattachable. [...] */

On arm64 AAPCS64 rounds NGRN up to an even number for a 16-byte-aligned type, so for (u64 a, __int128 v, u64 b) the actual placement is a=x0, x1 padding, v=x2:x3, b=x4, while a positional mapping expects v=x1:x2 and b=x3.

Per the in-file comment that makes pahole drop the BTF FUNC record for the whole function on arm64. For a kfunc that is worse than for an fentry
target: the name is registered with BTF_ID_FLAGS(), so resolve_btfids would leave the ID at 0 and emit:

'WARN: resolve_btfids: unresolved symbol bpf_kfunc_call_test_i128_arg_odd'

which is fatal when CONFIG_WERROR selects --fatal_warnings. Even when only a warning, libbpf cannot resolve the ksym against the module BTF, so aggregate_arg_kfunc_int128_odd, aggregate_arg_kfunc_int128_shift and aggregate_arg_kfunc_int128_ovf fail to load on arm64 instead of testing the JIT.

bpf_kfunc_call_test_i128_arg_pad() has the same hazard in the milder stack-vs-register form (a..g fill x0..x6, then v is pushed to the stack where x7 would be expected positionally).

The pre-existing bpf_kfunc_call_test_i128() and bpf_testmod_test_int128_arg() both deliberately keep the __int128 in the first position, and only bpf_kfunc_call_test_i128_arg() (u64 a, u64 b, __int128 v) among the new ones happens to land on an even register with no padding.

Quoted 3 lines

Is the placement worth confirming against the pahole version used for CI before this lands, given it's the one thing in the change that the in-tree documentation says should not work on arm64?

this pahole patch should fix the issue:
https://lore.kernel.org/bpf/[email protected]/

Quoted 7 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34320399441