Skip to contentSkip to conversation

Retire KF_ARENA_ARG kfunc flags

PATCH In review Kumar Kartikeya Dwivedi 7 participants 45 messages Aug 21 – Aug 30

Subsystems (17)
Documentation x86 PowerPC Kselftest +13 more subsystems
lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

cover Retire KF_ARENA_ARG kfunc flags

KF_ARENA_ARG1 and KF_ARENA_ARG2 duplicate the BTF argument-name suffix mechanism and only cover the first two kfunc parameters. Retire the flags from the kernel and resolve_btfids, and describe the arena allocation kfunc arguments with __arena suffixes instead.

Suffix conversion makes the JIT responsible for translating the 32-bit arena offset passed by BPF into a kernel address before the kfunc call. Translate the address back inside the allocation kfuncs so the existing arena helpers continue to receive lower-32-bit user addresses. Keep free and reserve non-nullable: an arena starting at the 4 GiB boundary has a valid first page whose lower 32 bits are zero.

Split the JIT capability hook because kfunc calls and struct_ops callbacks perform opposite conversions at different boundaries. Add kfunc argument rebasing to RISC-V, s390, LoongArch, and PowerPC64, and add the reverse struct_ops conversion to RISC-V, s390, and LoongArch. RISC-V and LoongArch also need independent fixes for stack-passed arguments in indirect trampolines.

Extend the shared arena tests to every JIT advertising each capability and update the conversion-specific expectations and large-arena coverage.

The resulting support now with these changes for __arena suffixes is:

kfunc arguments: x86-64, arm64, RISC-V, s390, LoongArch, PowerPC64
struct_ops arguments: x86-64, arm64, RISC-V, s390, LoongArch

NOTE: The individual JIT patches are not tested by me, and need the acks of individual JIT maintainers before they can be landed. However, to truly retire this flag, these changes are necessary so that arena kfuncs for page allocation and freeing remain functional on those architectures.

Kumar Kartikeya Dwivedi (14):
bpf: Split arena kfunc and struct_ops JIT capabilities bpf, riscv: Fix stack-passed arguments for indirect trampolines bpf, riscv: JIT arena kfunc argument rebasing
bpf, riscv: Convert struct_ops arena arguments in the trampoline bpf, s390: JIT arena kfunc argument rebasing
bpf, s390: Convert struct_ops arena arguments
bpf, loongarch: Fix stack arguments for indirect trampolines bpf, loongarch: JIT arena kfunc argument rebasing
bpf, loongarch: Convert struct_ops arena arguments in trampolines bpf, powerpc: JIT arena kfunc argument rebasing
bpf: Replace arena kfunc argument flags with suffixes
resolve_btfids: Drop KF_ARENA_ARG flag support selftests/bpf: Exercise arena arguments on every capable JIT docs/bpf: Document split arena argument JIT capabilities

 Documentation/bpf/kfuncs.rst                  |  15 +-
 arch/arm64/net/bpf_jit_comp.c                 |   7 +-
 arch/loongarch/net/bpf_jit.c                  |  98 ++++++++++--
 arch/powerpc/net/bpf_jit_comp.c               |   5 +
 arch/powerpc/net/bpf_jit_comp64.c             |  18 ++-
 arch/riscv/net/bpf_jit_comp64.c               | 139 ++++++++++++++++--
 arch/s390/net/bpf_jit_comp.c                  | 118 ++++++++++++++-
 arch/x86/net/bpf_jit_comp.c                   |   7 +-
 include/linux/btf.h                           |   2 -
 include/linux/filter.h                        |   3 +-
 kernel/bpf/arena.c                            |  40 +++--
 kernel/bpf/core.c                             |   7 +-
 kernel/bpf/verifier.c                         |   4 +-
 tools/bpf/resolve_btfids/main.c               |  22 +--
 .../selftests/bpf/prog_tests/resolve_btfids.c |  22 +--
 .../bpf/prog_tests/test_struct_ops_arena.c    |  16 +-
 .../testing/selftests/bpf/progs/arena_kfunc.c |  36 +++++
 .../selftests/bpf/progs/arena_kfunc_jit.c     |  16 +-
 tools/testing/selftests/bpf/progs/bpf_misc.h  |   1 +
 .../selftests/bpf/progs/verifier_arena.c      |   6 +
 .../bpf/progs/verifier_arena_large.c          |   4 +-
 tools/testing/selftests/bpf/test_loader.c     |   5 +
 22 files changed, 487 insertions(+), 104 deletions(-)

base-commit: 669e4fa766000ae4137bb02eb22ce75ba78ad32d

Signature · 1 line
2.53.0
author maintainer
#1 lore raw

PATCH 1/14 bpf: Split arena kfunc and struct_ops JIT capabilities

Arena pointer kfunc calls and struct_ops callbacks need different JIT support. The former rebases BPF arena offsets before a kfunc call, while the latter converts kernel pointers when an indirect trampoline builds a callback context.

A single bpf_jit_supports_arena_args() hook forces an architecture to implement both paths at once. That ties bpf_arena_alloc_pages() conversion to struct_ops trampoline support and prevents the paths from being enabled and reviewed independently.

Replace it with separate kfunc and struct_ops capability hooks. Make the verifier query the hook for the path it is checking, and have x86-64 and arm64 advertise both capabilities to preserve their current behavior.

---
 arch/arm64/net/bpf_jit_comp.c | 7 ++++++-
 arch/x86/net/bpf_jit_comp.c   | 7 ++++++-
 include/linux/filter.h        | 3 ++-
 kernel/bpf/core.c             | 7 ++++++-
 kernel/bpf/verifier.c         | 4 ++--
 5 files changed, 22 insertions(+), 6 deletions(-)
arch/arm64/net/bpf_jit_comp.c +6 -1
index 3aa3ea0bc30b..eecaa0027a95 100644
@@ -2398,7 +2398,12 @@ bool bpf_jit_supports_stack_args(void)
 	return true;
 }
 
-bool bpf_jit_supports_arena_args(void)
+bool bpf_jit_supports_arena_kfunc_args(void)
+{
+	return true;
+}
+
+bool bpf_jit_supports_arena_struct_ops_args(void)
 {
 	return true;
 }
arch/x86/net/bpf_jit_comp.c +6 -1
index 48429fae0641..6e89f1c8738b 100644
@@ -4174,7 +4174,12 @@ bool bpf_jit_supports_stack_args(void)
 	return true;
 }
 
-bool bpf_jit_supports_arena_args(void)
+bool bpf_jit_supports_arena_kfunc_args(void)
+{
+	return true;
+}
+
+bool bpf_jit_supports_arena_struct_ops_args(void)
 {
 	return true;
 }
include/linux/filter.h +2 -1
index 6e746b0a0930..907d774fd355 100644
@@ -1239,7 +1239,8 @@ bool bpf_jit_supports_percpu_insn(void);
 bool bpf_jit_supports_kfunc_call(void);
 bool bpf_jit_supports_kfunc_ret_reg_pair(void);
 bool bpf_jit_supports_stack_args(void);
-bool bpf_jit_supports_arena_args(void);
+bool bpf_jit_supports_arena_kfunc_args(void);
+bool bpf_jit_supports_arena_struct_ops_args(void);
 bool bpf_jit_supports_far_kfunc_call(void);
 bool bpf_jit_supports_exceptions(void);
 bool bpf_jit_supports_ptr_xchg(void);
kernel/bpf/core.c +6 -1
index 5db77d7915df..92b0a3bf27be 100644
@@ -3297,7 +3297,12 @@ bool __weak bpf_jit_supports_stack_args(void)
 	return false;
 }
 
-bool __weak bpf_jit_supports_arena_args(void)
+bool __weak bpf_jit_supports_arena_kfunc_args(void)
+{
+	return false;
+}
+
+bool __weak bpf_jit_supports_arena_struct_ops_args(void)
 {
 	return false;
 }
kernel/bpf/verifier.c +2 -2
index e036ae20bf6b..6402e94c2097 100644
@@ -12013,7 +12013,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 	else if (is_kfunc_arg_callback(env, meta->btf, &args[arg]))
 		arg_type = KF_ARG_PTR_TO_CALLBACK;
 	else if (is_kfunc_arg_arena(meta->btf, &args[arg])) {
-		if (!bpf_jit_supports_arena_args()) {
+		if (!bpf_jit_supports_arena_kfunc_args()) {
 			verbose(env, "JIT does not support kfunc %s() with arena pointer arguments\n",
 				meta->func_name);
 			return -ENOTSUPP;
@@ -19778,7 +19778,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 		if (info->refcounted)
 			has_refcounted_arg = true;
 		if (base_type(info->reg_type) == PTR_TO_ARENA) {
-			if (!bpf_jit_supports_arena_args()) {
+			if (!bpf_jit_supports_arena_struct_ops_args()) {
 				verbose(env, "JIT does not support arena arguments\n");
 				return -ENOTSUPP;
 			}
Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH bpf-next v1 01/14] bpf: Split arena kfunc and struct_ops JIT capabilities

This isn't a bug, but since bpf_arena_alloc_pages() still takes addr__ign at this point in the series and only picks up the __arena__nullable suffix in a later patch, would it read more clearly to…
Quoted 12 lines

bpf: Split arena kfunc and struct_ops JIT capabilities

Arena pointer kfunc calls and struct_ops callbacks need different JIT support. The former rebases BPF arena offsets before a kfunc call, while the latter converts kernel pointers when an indirect trampoline builds a callback context.

A single bpf_jit_supports_arena_args() hook forces an architecture to implement both paths at once. That ties bpf_arena_alloc_pages() conversion to struct_ops trampoline support and prevents the paths from being enabled and reviewed independently.

Quoted 5 lines

Replace it with separate kfunc and struct_ops capability hooks. Make the verifier query the hook for the path it is checking, and have x86-64 and arm64 advertise both capabilities to preserve their current behavior.

This isn't a bug, but since bpf_arena_alloc_pages() still takes addr__ign at this point in the series and only picks up the __arena__nullable suffix in a later patch, would it read more clearly to phrase this as what the split enables for the following patches, or to point at an argument that is already suffixed today?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32538351576

maintainer
#3 lore raw

Re: [PATCH bpf-next v1 01/14] bpf: Split arena kfunc and struct_ops JIT capabilities

In reply to Kumar Kartikeya Dwivedi
Quoted 10 lines · Kumar Kartikeya Dwivedi

Arena pointer kfunc calls and struct_ops callbacks need different JIT support. The former rebases BPF arena offsets before a kfunc call, while the latter converts kernel pointers when an indirect trampoline builds a callback context.

A single bpf_jit_supports_arena_args() hook forces an architecture to implement both paths at once. That ties bpf_arena_alloc_pages() conversion to struct_ops trampoline support and prevents the paths from being enabled and reviewed independently.

Quoted 6 lines

Replace it with separate kfunc and struct_ops capability hooks. Make the verifier query the hook for the path it is checking, and have x86-64 and arm64 advertise both capabilities to preserve their current behavior.

---

I see no particular reason for this split.
Don't see any guards in the generic or powerpc related code preventing struct_ops programs from loading. Hence, I presume these are supported by the arch. Why excluding the arch, given that all changes are vibe-coded?

author maintainer
#4 lore raw

Re: [PATCH bpf-next v1 01/14] bpf: Split arena kfunc and struct_ops JIT capabilities

Quoted 16 lines · Eduard Zingerman
Quoted 14 lines · Kumar Kartikeya Dwivedi

Arena pointer kfunc calls and struct_ops callbacks need different JIT support. The former rebases BPF arena offsets before a kfunc call, while the latter converts kernel pointers when an indirect trampoline builds a callback context.

A single bpf_jit_supports_arena_args() hook forces an architecture to implement both paths at once. That ties bpf_arena_alloc_pages() conversion to struct_ops trampoline support and prevents the paths from being enabled and reviewed independently.

Replace it with separate kfunc and struct_ops capability hooks. Make the verifier query the hook for the path it is checking, and have x86-64 and arm64 advertise both capabilities to preserve their current behavior.

Quoted 8 lines
Quoted 2 lines

---

I see no particular reason for this split.
Don't see any guards in the generic or powerpc related code preventing struct_ops programs from loading. Hence, I presume these are supported by the arch. Why excluding the arch, given that all changes are vibe-coded?

I don't think it has support for struct_ops trampoline if I'm not mistaken, but yeah, we can probably let powerpc folks finish the support instead of making the split (at that point split will be unnecessary anyway).

reviewer
#5 lore raw

Re: [PATCH bpf-next v1 01/14] bpf: Split arena kfunc and struct_ops JIT capabilities

In reply to Eduard Zingerman
Quoted 11 lines · Eduard Zingerman
Quoted 10 lines · Kumar Kartikeya Dwivedi

Arena pointer kfunc calls and struct_ops callbacks need different JIT support. The former rebases BPF arena offsets before a kfunc call, while the latter converts kernel pointers when an indirect trampoline builds a callback context.

A single bpf_jit_supports_arena_args() hook forces an architecture to implement both paths at once. That ties bpf_arena_alloc_pages() conversion to struct_ops trampoline support and prevents the paths from being enabled and reviewed independently.

Quoted 8 lines
Quoted 6 lines

Replace it with separate kfunc and struct_ops capability hooks. Make the verifier query the hook for the path it is checking, and have x86-64 and arm64 advertise both capabilities to preserve their current behavior.

---

I see no particular reason for this split.

+1

I don't think the split makes sense in general, independent of the powerpc point.

The arch either supports the arena pointers everywhere, or the support is incomplete, which IMO means it doesn't support them.

Quoted 4 lines

Don't see any guards in the generic or powerpc related code preventing struct_ops programs from loading. Hence, I presume these are supported by the arch. Why excluding the arch, given that all changes are vibe-coded?

author maintainer
#6 lore raw

PATCH 2/14 bpf, riscv: Fix stack-passed arguments for indirect trampolines

In reply to Kumar Kartikeya Dwivedi

store_args() reads stack-passed arguments relative to FP assuming the trampoline was entered through the fentry call from a traced function. In that path, the trampoline pushes the parent frame before establishing its final FP, so the incoming stack arguments start at FP + 16.

An indirect trampoline for a struct_ops callback is called through a function pointer. Its prologue allocates only the trampoline frame and sets FP to the incoming SP. The RISC-V ABI places the first stack argument at that incoming SP, so the arguments start at FP, not FP + 16. Every stack-passed argument of a callback with more than eight argument slots is therefore read two slots late.

Pass the prologue-dependent offset to store_args(), using zero for a direct struct_ops trampoline and 16 for the fentry path.

Fixes 6801b0aef79d ("riscv, bpf: Add 12-argument support for RV64 bpf trampoline")
Cc Björn Töpel <[email protected]>
Cc Pu Lehui <[email protected]>
---
 arch/riscv/net/bpf_jit_comp64.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +10 -4
index 47c7bf431ba8..b1084f30f3ce 100644
@@ -857,7 +857,8 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
 	return ret;
 }
 
-static void store_args(int nr_arg_slots, int args_off, struct rv_jit_context *ctx)
+static void store_args(int nr_arg_slots, int args_off, int stack_args_off,
+		       struct rv_jit_context *ctx)
 {
 	int i;
 
@@ -865,8 +866,8 @@ static void store_args(int nr_arg_slots, int args_off, struct rv_jit_context *ct
 		if (i < RV_MAX_REG_ARGS) {
 			emit_sd(RV_REG_FP, -args_off, RV_REG_A0 + i, ctx);
 		} else {
-			/* skip slots for T0 and FP of traced function */
-			emit_ld(RV_REG_T1, 16 + (i - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
+			emit_ld(RV_REG_T1, stack_args_off +
+				(i - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
 			emit_sd(RV_REG_FP, -args_off, RV_REG_T1, ctx);
 		}
 		args_off -= 8;
@@ -1152,7 +1153,12 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 	func_meta = nr_arg_slots;
 	emit_store_stack_imm64(RV_REG_T1, -func_meta_off, func_meta, ctx);
 
-	store_args(nr_arg_slots, args_off, ctx);
+	/*
+	 * A direct struct_ops call has its first stack argument at the incoming
+	 * SP, which the trampoline keeps as FP. The fentry path pushes the
+	 * parent frame first, so its incoming stack arguments start at FP + 16.
+	 */
+	store_args(nr_arg_slots, args_off, is_struct_ops ? 0 : 16, ctx);
 
 	if (bpf_fsession_cnt(tnodes)) {
 		/* clear all session cookies' value */
Signature · 1 line
2.53.0
maintainer Reviewed-by on v1 2/14
#7 lore raw

Re: [PATCH bpf-next v1 02/14] bpf, riscv: Fix stack-passed arguments for indirect trampolines

Quoted 52 lines · Kumar Kartikeya Dwivedi

store_args() reads stack-passed arguments relative to FP assuming the trampoline was entered through the fentry call from a traced function. In that path, the trampoline pushes the parent frame before establishing its final FP, so the incoming stack arguments start at FP + 16.

An indirect trampoline for a struct_ops callback is called through a function pointer. Its prologue allocates only the trampoline frame and sets FP to the incoming SP. The RISC-V ABI places the first stack argument at that incoming SP, so the arguments start at FP, not FP + 16. Every stack-passed argument of a callback with more than eight argument slots is therefore read two slots late.

Pass the prologue-dependent offset to store_args(), using zero for a direct struct_ops trampoline and 16 for the fentry path.

Fixes 6801b0aef79d ("riscv, bpf: Add 12-argument support for RV64 bpf trampoline")
Cc Björn Töpel <[email protected]>
Cc Pu Lehui <[email protected]>
---
  arch/riscv/net/bpf_jit_comp64.c | 14 ++++++++++----
  1 file changed, 10 insertions(+), 4 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +4 -3
index 47c7bf431ba8..b1084f30f3ce 100644
@@ -857,7 +857,8 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
  	return ret;
  }
 
-static void store_args(int nr_arg_slots, int args_off, struct rv_jit_context *ctx)
+static void store_args(int nr_arg_slots, int args_off, int stack_args_off,
+		       struct rv_jit_context *ctx)
  {
  	int i;
 
@@ -865,8 +866,8 @@ static void store_args(int nr_arg_slots, int args_off, struct rv_jit_context *ct
  		if (i < RV_MAX_REG_ARGS) {
  			emit_sd(RV_REG_FP, -args_off, RV_REG_A0 + i, ctx);
  		} else {
-			/* skip slots for T0 and FP of traced function */
-			emit_ld(RV_REG_T1, 16 + (i - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
+			emit_ld(RV_REG_T1, stack_args_off +
+				(i - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
  			emit_sd(RV_REG_FP, -args_off, RV_REG_T1, ctx);
  		}
  		args_off -= 8;
@@ -1152,7 +1153,12 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
  	func_meta = nr_arg_slots;
  	emit_store_stack_imm64(RV_REG_T1, -func_meta_off, func_meta, ctx);
 

arch/riscv/net/bpf_jit_comp64.c static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, · line 1164

Quoted 10 lines
arch/riscv/net/bpf_jit_comp64.c +6 -1
-	store_args(nr_arg_slots, args_off, ctx);
+	/*
+	 * A direct struct_ops call has its first stack argument at the incoming
+	 * SP, which the trampoline keeps as FP. The fentry path pushes the
+	 * parent frame first, so its incoming stack arguments start at FP + 16.
+	 */
+	store_args(nr_arg_slots, args_off, is_struct_ops ? 0 : 16, ctx);
 
  	if (bpf_fsession_cnt(tnodes)) {
  		/* clear all session cookies' value */
author maintainer
#8 lore raw

PATCH 3/14 bpf, riscv: JIT arena kfunc argument rebasing

In reply to Kumar Kartikeya Dwivedi

A BPF arena pointer is represented as a zero-extended 32-bit offset while a kfunc receives a kernel address. Kfuncs whose BTF argument names carry the __arena suffix therefore need the JIT to rebase those offsets immediately before the native call.

RV_REG_ARENA already holds kern_vm_start whenever the program uses an arena. Zero-extend each tagged argument and add that base:

zext.w aN, aN
add aN, s7, aN

For an __arena__nullable argument, branch over the fixed-width add when the truncated offset is zero so that NULL remains NULL. An unconditionally tagged zero is intentionally converted to the arena base.

The sequence is emitted as native code after the BPF instruction stream has been blinded, and its size depends only on the function model and enabled ISA extensions. Advertise the kfunc capability independently; struct_ops argument conversion is not enabled by this change.

---
 arch/riscv/net/bpf_jit_comp64.c | 40 +++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
arch/riscv/net/bpf_jit_comp64.c +40 -0
index b1084f30f3ce..c97d13a3eae4 100644
@@ -714,6 +714,37 @@ static int sign_extend(u8 rd, u8 rs, u8 sz, bool sign, struct rv_jit_context *ct
 	return 0;
 }
 
+/*
+ * Rebase the __arena args of a kfunc call to arena kernel addresses,
+ * aN = kern_vm_start + (u32)aN, with RV_REG_ARENA holding kern_vm_start.
+ * A nullable arg preserves NULL by skipping the add, tested on the
+ * truncated value as arena NULL is offset 0.
+ */
+static int emit_kfunc_arena_args(struct rv_jit_context *ctx,
+				 const struct btf_func_model *fm)
+{
+	int i;
+
+	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
+		u8 flags = fm->arg_flags[i];
+		u8 reg = bpf_to_rv_reg(BPF_REG_1 + i, ctx);
+
+		if (!(flags & BTF_FMODEL_ARENA_ARG))
+			continue;
+		if (WARN_ON_ONCE(!ctx->arena_vm_start))
+			return -EINVAL;
+
+		emit_zextw(reg, reg, ctx);
+		if (flags & BTF_FMODEL_NULLABLE_ARG) {
+			/* Skip the fixed-width add so that NULL stays NULL. */
+			emit(rv_beq(reg, RV_REG_ZERO, 4), ctx);
+		}
+		emit(rv_add(reg, RV_REG_ARENA, reg), ctx);
+	}
+
+	return 0;
+}
+
 #define BPF_FIXUP_OFFSET_MASK   GENMASK(26, 0)
 #define BPF_FIXUP_REG_MASK      GENMASK(31, 27)
 #define REG_DONT_CLEAR_MARKER	0	/* RV_REG_ZERO unused in pt_regmap */
@@ -1834,6 +1865,10 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
 				if (sign_extend(reg, reg, fm->arg_size[idx], sign, ctx))
 					return -EINVAL;
 			}
+
+			ret = emit_kfunc_arena_args(ctx, fm);
+			if (ret)
+				return ret;
 		}
 
 		/* restore TCC to RV_REG_TCC before bpf2bpf call */
@@ -2132,6 +2167,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return true;
 }
 
+bool bpf_jit_supports_arena_kfunc_args(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_ptr_xchg(void)
 {
 	return true;
Signature · 1 line
2.53.0
maintainer
#9 lore raw

Re: [PATCH bpf-next v1 03/14] bpf, riscv: JIT arena kfunc argument rebasing

Quoted 70 lines · Kumar Kartikeya Dwivedi

A BPF arena pointer is represented as a zero-extended 32-bit offset while a kfunc receives a kernel address. Kfuncs whose BTF argument names carry the __arena suffix therefore need the JIT to rebase those offsets immediately before the native call.

RV_REG_ARENA already holds kern_vm_start whenever the program uses an arena. Zero-extend each tagged argument and add that base:

zext.w aN, aN
add aN, s7, aN

For an __arena__nullable argument, branch over the fixed-width add when the truncated offset is zero so that NULL remains NULL. An unconditionally tagged zero is intentionally converted to the arena base.

The sequence is emitted as native code after the BPF instruction stream has been blinded, and its size depends only on the function model and enabled ISA extensions. Advertise the kfunc capability independently; struct_ops argument conversion is not enabled by this change.

---
  arch/riscv/net/bpf_jit_comp64.c | 40 +++++++++++++++++++++++++++++++++
  1 file changed, 40 insertions(+)
arch/riscv/net/bpf_jit_comp64.c +31 -0
index b1084f30f3ce..c97d13a3eae4 100644
@@ -714,6 +714,37 @@ static int sign_extend(u8 rd, u8 rs, u8 sz, bool sign, struct rv_jit_context *ct
  	return 0;
  }
 
+/*
+ * Rebase the __arena args of a kfunc call to arena kernel addresses,
+ * aN = kern_vm_start + (u32)aN, with RV_REG_ARENA holding kern_vm_start.
+ * A nullable arg preserves NULL by skipping the add, tested on the
+ * truncated value as arena NULL is offset 0.
+ */
+static int emit_kfunc_arena_args(struct rv_jit_context *ctx,
+				 const struct btf_func_model *fm)
+{
+	int i;
+
+	for (i = 0; i < min_t(int, fm->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
+		u8 flags = fm->arg_flags[i];
+		u8 reg = bpf_to_rv_reg(BPF_REG_1 + i, ctx);
+
+		if (!(flags & BTF_FMODEL_ARENA_ARG))
+			continue;
+		if (WARN_ON_ONCE(!ctx->arena_vm_start))
+			return -EINVAL;
+
+		emit_zextw(reg, reg, ctx);
+		if (flags & BTF_FMODEL_NULLABLE_ARG) {
+			/* Skip the fixed-width add so that NULL stays NULL. */
+			emit(rv_beq(reg, RV_REG_ZERO, 4), ctx);
+		}
+		emit(rv_add(reg, RV_REG_ARENA, reg), ctx);
+	}
+
+	return 0;
+}
+
  #define BPF_FIXUP_OFFSET_MASK   GENMASK(26, 0)
  #define BPF_FIXUP_REG_MASK      GENMASK(31, 27)
  #define REG_DONT_CLEAR_MARKER	0	/* RV_REG_ZERO unused in pt_regmap */

arch/riscv/net/bpf_jit_comp64.c int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, > if (sign_extend(reg, reg, fm->arg_size[idx], sign, ctx)) · line 1865

Quoted 1 line
arch/riscv/net/bpf_jit_comp64.c +0 -0
@@ -1834,6 +1865,10 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, >   				if (sign_extend(reg, reg, fm->arg_size[idx], sign, ctx))
  					return -EINVAL;

sign_extend will emit for both signed and unsigned 4-byte kfunc arguments. The current patch therefore generates a redundant sign-ext prior to zero-ext for arena pointers.

It would be cleaner to merge the two passes into a single loop:

for (idx = 0; idx < fm->nr_args; idx++) {
    u8 flags = fm->arg_flags[idx];
    bool sign = flags & BTF_FMODEL_SIGNED_ARG;
    u8 reg = bpf_to_rv_reg(BPF_REG_1 + idx, ctx);

    if (flags & BTF_FMODEL_ARENA_ARG) {
        xxx
        continue;
    }

    if (sign_extend(reg, reg, fm->arg_size[idx], sign, ctx))
        return -EINVAL;
}
Quoted 20 lines
arch/riscv/net/bpf_jit_comp64.c +9 -0
  			}
+
+			ret = emit_kfunc_arena_args(ctx, fm);
+			if (ret)
+				return ret;
  		}
 
  		/* restore TCC to RV_REG_TCC before bpf2bpf call */
@@ -2132,6 +2167,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
  	return true;
  }
 
+bool bpf_jit_supports_arena_kfunc_args(void)
+{
+	return true;
+}
+
  bool bpf_jit_supports_ptr_xchg(void)
  {
  	return true;
author maintainer
#10 lore raw

PATCH 4/14 bpf, riscv: Convert struct_ops arena arguments in the trampoline

In reply to Kumar Kartikeya Dwivedi

A struct_ops callback receives native kernel addresses, while its BPF program expects an arena pointer argument as a zero-extended 32-bit offset. Convert arguments marked with BTF_FMODEL_ARENA_ARG while the trampoline copies them into the BPF context.

bpf_tramp_arena_base() supplies the known base only for the single-program indirect trampoline. Materialize its low 32 bits once in t2, subtract it from each tagged argument through t1, and zero-extend the result before storing it. For a nullable argument, preserve the full native pointer in t1 and branch over the variable-length subtraction sequence when it is NULL.

Walk the function model by argument while keeping a separate ABI slot index. This keeps the arena flags aligned with the correct native register or stack slot when an earlier argument occupies two slots, including the case where a 16-byte argument straddles a7 and the stack. Registered and stack-passed arena pointers use the same conversion helper.

bpf_tramp_arena_base() returns zero for tracing trampolines, so their emitted argument-save sequence is unchanged. An indirect trampoline cannot call the original function, which ensures a converted pointer never escapes back into a native callback. Advertise the struct_ops capability independently now that the reverse conversion is implemented.

---
 arch/riscv/net/bpf_jit_comp64.c | 93 +++++++++++++++++++++++++++++----
 1 file changed, 82 insertions(+), 11 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +82 -11
index c97d13a3eae4..8db992a285f6 100644
@@ -888,20 +888,75 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
 	return ret;
 }
 
-static void store_args(int nr_arg_slots, int args_off, int stack_args_off,
+/*
+ * Convert an arena kernel address into the arena pointer form on its way
+ * into the BPF ctx, dst = (u32)(src - kern_vm_start). A nullable arg
+ * preserves NULL, tested on the full 64-bit kernel pointer. The final
+ * zero-extension makes the stored value satisfy the JIT invariant for arena
+ * pointer registers.
+ */
+static void emit_arena_arg_conv(u8 dst, u8 src, bool nullable, u8 base,
+				struct rv_jit_context *ctx)
+{
+	int branch_off = 0;
+
+	if (nullable) {
+		if (dst != src)
+			emit_mv(dst, src, ctx);
+		branch_off = ctx->ninsns;
+		/* Patched below once the variable-length conversion is emitted. */
+		emit(rv_nop(), ctx);
+		src = dst;
+	}
+
+	emit_sub(dst, src, base, ctx);
+	emit_zextw(dst, dst, ctx);
+
+	if (nullable && ctx->insns) {
+		u32 insn = rv_beq(dst, RV_REG_ZERO, ctx->ninsns - branch_off);
+
+		*(u32 *)(ctx->insns + branch_off) = insn;
+	}
+}
+
+static void store_args(const struct btf_func_model *m, int args_off,
+		       int stack_args_off, u64 arena_base,
 		       struct rv_jit_context *ctx)
 {
-	int i;
+	int i, j, slot = 0;
 
-	for (i = 0; i < nr_arg_slots; i++) {
-		if (i < RV_MAX_REG_ARGS) {
-			emit_sd(RV_REG_FP, -args_off, RV_REG_A0 + i, ctx);
-		} else {
-			emit_ld(RV_REG_T1, stack_args_off +
-				(i - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
-			emit_sd(RV_REG_FP, -args_off, RV_REG_T1, ctx);
+	/* Only the low 32 bits of the base take part in the subtraction. */
+	if (arena_base)
+		emit_imm(RV_REG_T2, (s32)(u32)arena_base, ctx);
+
+	/*
+	 * Walk arguments and slots together so a 16-byte argument consumes two
+	 * ABI locations before the flags for the following argument are used.
+	 */
+	for (i = 0; i < m->nr_args; i++) {
+		bool arena_arg = arena_base && (m->arg_flags[i] & BTF_FMODEL_ARENA_ARG);
+		bool nullable = m->arg_flags[i] & BTF_FMODEL_NULLABLE_ARG;
+		int slots = round_up(m->arg_size[i], 8) / 8;
+
+		for (j = 0; j < slots; j++, slot++) {
+			u8 src;
+
+			if (slot < RV_MAX_REG_ARGS) {
+				src = RV_REG_A0 + slot;
+			} else {
+				emit_ld(RV_REG_T1, stack_args_off +
+					(slot - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
+				src = RV_REG_T1;
+			}
+
+			if (arena_arg) {
+				emit_arena_arg_conv(RV_REG_T1, src, nullable,
+						    RV_REG_T2, ctx);
+				src = RV_REG_T1;
+			}
+			emit_sd(RV_REG_FP, -args_off, src, ctx);
+			args_off -= 8;
 		}
-		args_off -= 8;
 	}
 }
 
@@ -1039,9 +1094,20 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 	bool is_struct_ops = is_struct_ops_tramp(fentry);
 	void *orig_call = func_addr;
 	bool save_ret;
+	u64 arena_base;
 	u64 func_meta;
 	u32 insn;
 
+	/*
+	 * F_INDIRECT is only compatible with F_RET_FENTRY_RET. In particular,
+	 * an indirect trampoline never calls the original function with the
+	 * arena arguments converted into their BPF representation.
+	 */
+	WARN_ON_ONCE((flags & BPF_TRAMP_F_INDIRECT) &&
+		     (flags & ~(BPF_TRAMP_F_INDIRECT | BPF_TRAMP_F_RET_FENTRY_RET)));
+
+	arena_base = bpf_tramp_arena_base(m, tnodes, flags);
+
 	/* Two types of generated trampoline stack layout:
 	 *
 	 * 1. trampoline called from function entry
@@ -1189,7 +1255,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 	 * SP, which the trampoline keeps as FP. The fentry path pushes the
 	 * parent frame first, so its incoming stack arguments start at FP + 16.
 	 */
-	store_args(nr_arg_slots, args_off, is_struct_ops ? 0 : 16, ctx);
+	store_args(m, args_off, is_struct_ops ? 0 : 16, arena_base, ctx);
 
 	if (bpf_fsession_cnt(tnodes)) {
 		/* clear all session cookies' value */
@@ -2172,6 +2238,11 @@ bool bpf_jit_supports_arena_kfunc_args(void)
 	return true;
 }
 
+bool bpf_jit_supports_arena_struct_ops_args(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_ptr_xchg(void)
 {
 	return true;
Signature · 1 line
2.53.0
#11 lore raw

Re: [PATCH bpf-next v1 04/14] bpf, riscv: Convert struct_ops arena arguments in the trampoline

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Unaligned memory access in JIT instruction patching --

--- Patch [4]: [PATCH bpf-next v1 04/14] bpf, riscv: Convert struct_ops arena arguments in the trampoline --- commit 0283a3765a698e09bc666805ea8cb746cf58f299
Author: Kumar Kartikeya Dwivedi <[email protected]>

bpf, riscv: Convert struct_ops arena arguments in the trampoline

This commit converts struct_ops callback arena arguments into the arena pointer form when copied into the BPF context. It walks the function model by argument, adjusts the pointers via bpf_tramp_arena_base(), and handles nullable arguments.

Quoted 4 lines
arch/riscv/net/bpf_jit_comp64.c +0 -0
index c97d13a3eae4e..8db992a285f6a 100644

[ ... ]

Quoted 23 lines
arch/riscv/net/bpf_jit_comp64.c +18 -1
@@ -888,20 +888,75 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
 	return ret;
 }
 
-static void store_args(int nr_arg_slots, int args_off, int stack_args_off,
+/*
+ * Convert an arena kernel address into the arena pointer form on its way
+ * into the BPF ctx, dst = (u32)(src - kern_vm_start). A nullable arg
+ * preserves NULL, tested on the full 64-bit kernel pointer. The final
+ * zero-extension makes the stored value satisfy the JIT invariant for arena
+ * pointer registers.
+ */
+static void emit_arena_arg_conv(u8 dst, u8 src, bool nullable, u8 base,
+				struct rv_jit_context *ctx)
+{
+	int branch_off = 0;
+
+	if (nullable) {
+		if (dst != src)
+			emit_mv(dst, src, ctx);
+		branch_off = ctx->ninsns;
+		/* Patched below once the variable-length conversion is emitted. */
+		emit(rv_nop(), ctx);

arch/riscv/net/bpf_jit_comp64.c int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t, · line 918

Quoted 10 lines
arch/riscv/net/bpf_jit_comp64.c +10 -0
+		src = dst;
+	}
+
+	emit_sub(dst, src, base, ctx);
+	emit_zextw(dst, dst, ctx);
+
+	if (nullable && ctx->insns) {
+		u32 insn = rv_beq(dst, RV_REG_ZERO, ctx->ninsns - branch_off);
+
+		*(u32 *)(ctx->insns + branch_off) = insn;

[Severity: High]
This is a pre-existing issue, but does this direct 32-bit store cause an unaligned memory access trap?

If earlier instructions in the JIT context use 16-bit compressed instructions, ctx->ninsns (and therefore branch_off) becomes an odd number. In that case, ctx->insns + branch_off points to a 2-byte aligned address, not a 4-byte aligned one.

On RISC-V hardware that does not support unaligned memory accesses, casting an unaligned u16 pointer to a u32 pointer and performing a 32-bit store triggers a CPU exception.

Could this be safely written as two sequential 16-bit writes to avoid the trap?

Quoted 2 lines
arch/riscv/net/bpf_jit_comp64.c +2 -0
+	}
+}

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
maintainer
#12 lore raw

Re: [PATCH bpf-next v1 04/14] bpf, riscv: Convert struct_ops arena arguments in the trampoline

In reply to Kumar Kartikeya Dwivedi
Quoted 86 lines · Kumar Kartikeya Dwivedi

A struct_ops callback receives native kernel addresses, while its BPF program expects an arena pointer argument as a zero-extended 32-bit offset. Convert arguments marked with BTF_FMODEL_ARENA_ARG while the trampoline copies them into the BPF context.

bpf_tramp_arena_base() supplies the known base only for the single-program indirect trampoline. Materialize its low 32 bits once in t2, subtract it from each tagged argument through t1, and zero-extend the result before storing it. For a nullable argument, preserve the full native pointer in t1 and branch over the variable-length subtraction sequence when it is NULL.

Walk the function model by argument while keeping a separate ABI slot index. This keeps the arena flags aligned with the correct native register or stack slot when an earlier argument occupies two slots, including the case where a 16-byte argument straddles a7 and the stack. Registered and stack-passed arena pointers use the same conversion helper.

bpf_tramp_arena_base() returns zero for tracing trampolines, so their emitted argument-save sequence is unchanged. An indirect trampoline cannot call the original function, which ensures a converted pointer never escapes back into a native callback. Advertise the struct_ops capability independently now that the reverse conversion is implemented.

---
  arch/riscv/net/bpf_jit_comp64.c | 93 +++++++++++++++++++++++++++++----
  1 file changed, 82 insertions(+), 11 deletions(-)
arch/riscv/net/bpf_jit_comp64.c +36 -9
index c97d13a3eae4..8db992a285f6 100644
@@ -888,20 +888,75 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
  	return ret;
  }
 
-static void store_args(int nr_arg_slots, int args_off, int stack_args_off,
+/*
+ * Convert an arena kernel address into the arena pointer form on its way
+ * into the BPF ctx, dst = (u32)(src - kern_vm_start). A nullable arg
+ * preserves NULL, tested on the full 64-bit kernel pointer. The final
+ * zero-extension makes the stored value satisfy the JIT invariant for arena
+ * pointer registers.
+ */
+static void emit_arena_arg_conv(u8 dst, u8 src, bool nullable, u8 base,
+				struct rv_jit_context *ctx)
+{
+	int branch_off = 0;
+
+	if (nullable) {
+		if (dst != src)
+			emit_mv(dst, src, ctx);
+		branch_off = ctx->ninsns;
+		/* Patched below once the variable-length conversion is emitted. */
+		emit(rv_nop(), ctx);
+		src = dst;
+	}
+
+	emit_sub(dst, src, base, ctx);
+	emit_zextw(dst, dst, ctx);
+
+	if (nullable && ctx->insns) {
+		u32 insn = rv_beq(dst, RV_REG_ZERO, ctx->ninsns - branch_off);
+
+		*(u32 *)(ctx->insns + branch_off) = insn;
+	}
+}
+
+static void store_args(const struct btf_func_model *m, int args_off,
+		       int stack_args_off, u64 arena_base,
  		       struct rv_jit_context *ctx)
  {
-	int i;
+	int i, j, slot = 0;
 
-	for (i = 0; i < nr_arg_slots; i++) {
-		if (i < RV_MAX_REG_ARGS) {
-			emit_sd(RV_REG_FP, -args_off, RV_REG_A0 + i, ctx);
-		} else {
-			emit_ld(RV_REG_T1, stack_args_off +
-				(i - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
-			emit_sd(RV_REG_FP, -args_off, RV_REG_T1, ctx);
+	/* Only the low 32 bits of the base take part in the subtraction. */
+	if (arena_base)

arch/riscv/net/bpf_jit_comp64.c int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t, · line 939

Quoted 10 lines
arch/riscv/net/bpf_jit_comp64.c +10 -0
+		emit_imm(RV_REG_T2, (s32)(u32)arena_base, ctx);
+
+	/*
+	 * Walk arguments and slots together so a 16-byte argument consumes two
+	 * ABI locations before the flags for the following argument are used.
+	 */
+	for (i = 0; i < m->nr_args; i++) {
+		bool arena_arg = arena_base && (m->arg_flags[i] & BTF_FMODEL_ARENA_ARG);
+		bool nullable = m->arg_flags[i] & BTF_FMODEL_NULLABLE_ARG;
+		int slots = round_up(m->arg_size[i], 8) / 8;

small nits. maybe name arg_slots will looks better.

Quoted 66 lines
arch/riscv/net/bpf_jit_comp64.c +36 -2
+
+		for (j = 0; j < slots; j++, slot++) {
+			u8 src;
+
+			if (slot < RV_MAX_REG_ARGS) {
+				src = RV_REG_A0 + slot;
+			} else {
+				emit_ld(RV_REG_T1, stack_args_off +
+					(slot - RV_MAX_REG_ARGS) * 8, RV_REG_FP, ctx);
+				src = RV_REG_T1;
+			}
+
+			if (arena_arg) {
+				emit_arena_arg_conv(RV_REG_T1, src, nullable,
+						    RV_REG_T2, ctx);
+				src = RV_REG_T1;
+			}
+			emit_sd(RV_REG_FP, -args_off, src, ctx);
+			args_off -= 8;
  		}
-		args_off -= 8;
  	}
  }
 
@@ -1039,9 +1094,20 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
  	bool is_struct_ops = is_struct_ops_tramp(fentry);
  	void *orig_call = func_addr;
  	bool save_ret;
+	u64 arena_base;
  	u64 func_meta;
  	u32 insn;
 
+	/*
+	 * F_INDIRECT is only compatible with F_RET_FENTRY_RET. In particular,
+	 * an indirect trampoline never calls the original function with the
+	 * arena arguments converted into their BPF representation.
+	 */
+	WARN_ON_ONCE((flags & BPF_TRAMP_F_INDIRECT) &&
+		     (flags & ~(BPF_TRAMP_F_INDIRECT | BPF_TRAMP_F_RET_FENTRY_RET)));
+
+	arena_base = bpf_tramp_arena_base(m, tnodes, flags);
+
  	/* Two types of generated trampoline stack layout:
  	 *
  	 * 1. trampoline called from function entry
@@ -1189,7 +1255,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
  	 * SP, which the trampoline keeps as FP. The fentry path pushes the
  	 * parent frame first, so its incoming stack arguments start at FP + 16.
  	 */
-	store_args(nr_arg_slots, args_off, is_struct_ops ? 0 : 16, ctx);
+	store_args(m, args_off, is_struct_ops ? 0 : 16, arena_base, ctx);
 
  	if (bpf_fsession_cnt(tnodes)) {
  		/* clear all session cookies' value */
@@ -2172,6 +2238,11 @@ bool bpf_jit_supports_arena_kfunc_args(void)
  	return true;
  }
 
+bool bpf_jit_supports_arena_struct_ops_args(void)
+{
+	return true;
+}
+
  bool bpf_jit_supports_ptr_xchg(void)
  {
  	return true;
author maintainer
#13 lore raw

PATCH 5/14 bpf, s390: JIT arena kfunc argument rebasing

In reply to Kumar Kartikeya Dwivedi

Kfuncs with __arena arguments expect kernel addresses, while BPF programs carry arena pointers as zero-extended 32-bit offsets. The s390 JIT does not translate those offsets at the call boundary, so advertising arena kfunc argument support would pass invalid addresses to the callee.

Use the kfunc BTF model after the existing argument width normalization to find arena arguments. Load kern_vm_start from the existing arena literal once per call, truncate each tagged argument with llgfr, and add the base. For __arena__nullable, test the truncated value and skip the add so offset zero remains NULL.

Advertise the kfunc-specific arena argument capability. Struct_ops trampoline conversion is independent and remains disabled until its own support is added.

Cc Ilya Leoshkevich <[email protected]>
Cc Heiko Carstens <[email protected]>
Cc Vasily Gorbik <[email protected]>
---
 arch/s390/net/bpf_jit_comp.c | 49 ++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
arch/s390/net/bpf_jit_comp.c +49 -0
index c46872b071ce..20b6e53999b3 100644
@@ -946,6 +946,48 @@ static int sign_zero_extend(struct bpf_jit *jit, int r, u8 size, u8 flags)
 	}
 }
 
+/*
+ * Rebase the __arena args of a kfunc call to arena kernel addresses,
+ * rN = kern_vm_start + (u32)rN. A nullable arg preserves NULL by skipping
+ * the add, tested on the truncated value as arena NULL is offset 0.
+ */
+static int emit_kfunc_arena_args(struct bpf_jit *jit, struct bpf_prog *fp,
+				 const struct btf_func_model *m)
+{
+	bool base_loaded = false;
+	int i;
+
+	for (i = 0; i < min_t(int, m->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) {
+		u8 flags = m->arg_flags[i];
+		int arg = BPF_REG_1 + i;
+
+		if (!(flags & BTF_FMODEL_ARENA_ARG))
+			continue;
+		if (WARN_ON_ONCE(!fp->aux->arena))
+			return -EINVAL;
+
+		if (!base_loaded) {
+			/* lgrl %w0,kern_arena */
+			EMIT6_PCREL_RILB(0xc4080000, REG_W0, jit->kern_arena);
+			base_loaded = true;
+		}
+
+		/* llgfr %arg,%arg: truncate and clear the upper 32 bits */
+		EMIT4(0xb9160000, arg, arg);
+		if (flags & BTF_FMODEL_NULLABLE_ARG) {
+			/* ltgr %arg,%arg */
+			EMIT4(0xb9020000, arg, arg);
+			/* brc 8,1f */
+			EMIT4_PCREL_RIC(0xa7040000, 8, jit->prg + 8);
+		}
+		/* agr %arg,%w0 */
+		EMIT4(0xb9080000, arg, REG_W0);
+		/* 1: */
+	}
+
+	return 0;
+}
+
 /*
  * Compile one eBPF instruction into s390x code
  *
@@ -1867,6 +1909,8 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 						     m->arg_flags[j]))
 					return -1;
 			}
+			if (emit_kfunc_arena_args(jit, fp, m))
+				return -1;
 		}
 
 		if ((void *)func == arch_bpf_timed_may_goto) {
@@ -2459,6 +2503,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_arena_kfunc_args(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_far_kfunc_call(void)
 {
 	return true;
Signature · 1 line
2.53.0
author maintainer
#14 lore raw

PATCH 6/14 bpf, s390: Convert struct_ops arena arguments

In reply to Kumar Kartikeya Dwivedi

A struct_ops callback receives a native kernel arena address, while its BPF program expects the corresponding zero-extended 32-bit arena offset in the ctx. The s390 trampoline currently copies native arguments verbatim, so it cannot support callbacks whose stub marks an argument with __arena.

Obtain the arena base for the single-program indirect trampoline and convert each tagged argument while copying it into the BPF ctx. Test a nullable source as a full 64-bit kernel pointer, subtract the low 32 bits of kern_vm_start, and zero-extend the result. This preserves NULL and provides the register form required by arena loads.

Keep the native argument index separate from the BPF ctx slot index. The former selects r2-r6 or one caller stack slot per s390 ABI argument, while the latter still expands 16-byte arguments to two slots. This also converts arena pointers passed after the fifth argument without shifting later values.

bpf_tramp_arena_base() returns a base only for an indirect struct_ops trampoline. Assert the incompatible flag combinations so converted arguments can never be passed back to the original kernel function, and advertise the struct_ops-specific arena argument capability.

Cc Ilya Leoshkevich <[email protected]>
Cc Heiko Carstens <[email protected]>
Cc Vasily Gorbik <[email protected]>
---
 arch/s390/net/bpf_jit_comp.c | 69 ++++++++++++++++++++++++++++++++++--
 1 file changed, 66 insertions(+), 3 deletions(-)
arch/s390/net/bpf_jit_comp.c +66 -3
index 20b6e53999b3..964f1ad82637 100644
@@ -2607,6 +2607,34 @@ static void load_imm64(struct bpf_jit *jit, int dst_reg, u64 val)
 	EMIT6_IMM(0xc00d0000, dst_reg, val);
 }
 
+/*
+ * Convert an arena kernel address into the arena pointer form on its way
+ * into the BPF ctx, dst = (u32)(src - kern_vm_start). A nullable arg
+ * preserves NULL, tested on the full 64-bit kernel pointer. The 32-bit
+ * subtraction followed by zero-extension keeps the upper half clear.
+ */
+static void emit_arena_arg_conv(struct bpf_jit *jit, int dst, int src,
+				bool nullable, u32 base_lo)
+{
+	if (dst != src) {
+		/* lgr %dst,%src */
+		EMIT4(0xb9040000, dst, src);
+	}
+	if (nullable) {
+		/* ltgr %dst,%dst */
+		EMIT4(0xb9020000, dst, dst);
+		/* brc 8,1f */
+		EMIT4_PCREL_RIC(0xa7040000, 8, jit->prg + 16);
+	}
+	/* llilf %w1,base_lo */
+	EMIT6_IMM(0xc00f0000, REG_W1, base_lo);
+	/* sr %dst,%w1 */
+	EMIT2(0x1b00, dst, REG_W1);
+	/* llgfr %dst,%dst */
+	EMIT4(0xb9160000, dst, dst);
+	/* 1: */
+}
+
 static void emit_store_stack_imm64(struct bpf_jit *jit, int tmp_reg, int stack_off, u64 imm)
 {
 	load_imm64(jit, tmp_reg, imm);
@@ -2740,6 +2768,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 	int cookie_cnt, cookie_off, fsession_cnt;
 	struct bpf_jit *jit = &tjit->common;
 	int arg, bpf_arg_off;
+	u64 arena_base;
 	u64 func_meta;
 	int i, j;
 
@@ -2749,6 +2778,16 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 	if (nr_stack_args > MAX_NR_STACK_ARGS)
 		return -ENOTSUPP;
 
+	/*
+	 * F_INDIRECT is only compatible with F_RET_FENTRY_RET. Arena conversion
+	 * relies on the indirect trampoline never calling the original function
+	 * with converted arguments.
+	 */
+	WARN_ON_ONCE((flags & BPF_TRAMP_F_INDIRECT) &&
+		     (flags & ~(BPF_TRAMP_F_INDIRECT | BPF_TRAMP_F_RET_FENTRY_RET)));
+
+	arena_base = bpf_tramp_arena_base(m, tnodes, flags);
+
 	/* Return to %r14 in the struct_ops case. */
 	if (flags & BPF_TRAMP_F_INDIRECT)
 		flags |= BPF_TRAMP_F_SKIP_FRAME;
@@ -2829,14 +2868,33 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 			      (i - MAX_NR_REG_ARGS) * sizeof(u64);
 		bpf_arg_off = tjit->bpf_args_off + j * sizeof(u64);
 		if (m->arg_size[i] <= 8) {
-			if (i < MAX_NR_REG_ARGS)
+			bool arena_arg = arena_base &&
+					 (m->arg_flags[i] & BTF_FMODEL_ARENA_ARG);
+			bool nullable = m->arg_flags[i] & BTF_FMODEL_NULLABLE_ARG;
+
+			if (arena_arg) {
+				if (i < MAX_NR_REG_ARGS) {
+					emit_arena_arg_conv(jit, REG_W0, arg, nullable,
+							    (u32)arena_base);
+				} else {
+					/* lg %w0,arg(%r15) */
+					EMIT6_DISP_LH(0xe3000000, 0x0004, REG_W0,
+						      REG_0, REG_15, arg);
+					emit_arena_arg_conv(jit, REG_W0, REG_W0,
+							    nullable, (u32)arena_base);
+				}
+				/* stg %w0,bpf_arg_off(%r15) */
+				EMIT6_DISP_LH(0xe3000000, 0x0024, REG_W0,
+						      REG_0, REG_15, bpf_arg_off);
+			} else if (i < MAX_NR_REG_ARGS) {
 				/* stg %arg,bpf_arg_off(%r15) */
 				EMIT6_DISP_LH(0xe3000000, 0x0024, arg,
-					      REG_0, REG_15, bpf_arg_off);
-			else
+						      REG_0, REG_15, bpf_arg_off);
+			} else {
 				/* mvc bpf_arg_off(8,%r15),arg(%r15) */
 				_EMIT6(0xd207f000 | bpf_arg_off,
 				       0xf000 | arg);
+			}
 			j += 1;
 		} else {
 			if (i < MAX_NR_REG_ARGS) {
@@ -3092,6 +3150,11 @@ bool bpf_jit_supports_subprog_tailcalls(void)
 	return true;
 }
 
+bool bpf_jit_supports_arena_struct_ops_args(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_arena(void)
 {
 	return true;
Signature · 1 line
2.53.0
#15 lore raw

Re: [PATCH bpf-next v1 06/14] bpf, s390: Convert struct_ops arena arguments

This isn't a bug, but could slfi %dst,base_lo replace the llilf %w1 + sr %dst,%w1 pair here, dropping the per-argument reload of the base and the %r1 clobber?
Quoted 17 lines
arch/s390/net/bpf_jit_comp.c +9 -0
index 20b6e53999b3f..964f1ad82637b 100644
@@ -2607,6 +2607,34 @@ static void load_imm64(struct bpf_jit *jit, int dst_reg, u64 val)
 	EMIT6_IMM(0xc00d0000, dst_reg, val);
 }
 
+/*
+ * Convert an arena kernel address into the arena pointer form on its way
+ * into the BPF ctx, dst = (u32)(src - kern_vm_start). A nullable arg
+ * preserves NULL, tested on the full 64-bit kernel pointer. The 32-bit
+ * subtraction followed by zero-extension keeps the upper half clear.
+ */
+static void emit_arena_arg_conv(struct bpf_jit *jit, int dst, int src,
+				bool nullable, u32 base_lo)
+{

[ ... ]

arch/s390/net/bpf_jit_comp.c static void load_imm64(struct bpf_jit *jit, int dst_reg, u64 val) · line 2632

Quoted 4 lines
arch/s390/net/bpf_jit_comp.c +4 -0
+	/* llilf %w1,base_lo */
+	EMIT6_IMM(0xc00f0000, REG_W1, base_lo);
+	/* sr %dst,%w1 */
+	EMIT2(0x1b00, dst, REG_W1);

This isn't a bug, but could slfi %dst,base_lo replace the llilf %w1 + sr %dst,%w1 pair here, dropping the per-argument reload of the base and the %r1 clobber?

Quoted 4 lines
arch/s390/net/bpf_jit_comp.c +4 -0
+	/* llgfr %dst,%dst */
+	EMIT4(0xb9160000, dst, dst);
+	/* 1: */
+}

[ ... ]

Quoted 18 lines
arch/s390/net/bpf_jit_comp.c +14 -1
@@ -2829,14 +2868,33 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 		bpf_arg_off = tjit->bpf_args_off + j * sizeof(u64);
 		if (m->arg_size[i] <= 8) {
-			if (i < MAX_NR_REG_ARGS)
+			bool arena_arg = arena_base &&
+					 (m->arg_flags[i] & BTF_FMODEL_ARENA_ARG);
+			bool nullable = m->arg_flags[i] & BTF_FMODEL_NULLABLE_ARG;
+
+			if (arena_arg) {
+				if (i < MAX_NR_REG_ARGS) {
+					emit_arena_arg_conv(jit, REG_W0, arg, nullable,
+							    (u32)arena_base);
+				} else {
+					/* lg %w0,arg(%r15) */
+					EMIT6_DISP_LH(0xe3000000, 0x0004, REG_W0,
+						      REG_0, REG_15, arg);
+					emit_arena_arg_conv(jit, REG_W0, REG_W0,
+							    nullable, (u32)arena_base);

arch/s390/net/bpf_jit_comp.c static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, · line 2891

Quoted 10 lines
arch/s390/net/bpf_jit_comp.c +6 -2
+				}
+				/* stg %w0,bpf_arg_off(%r15) */
+				EMIT6_DISP_LH(0xe3000000, 0x0024, REG_W0,
+						      REG_0, REG_15, bpf_arg_off);
+			} else if (i < MAX_NR_REG_ARGS) {
 				/* stg %arg,bpf_arg_off(%r15) */
 				EMIT6_DISP_LH(0xe3000000, 0x0024, arg,
-					      REG_0, REG_15, bpf_arg_off);
-			else
+						      REG_0, REG_15, bpf_arg_off);

This isn't a bug, but was the extra tab on the REG_0, REG_15, bpf_arg_off continuation lines intended? Realigning them to the open parenthesis (as the nearby lg %w0,arg(%r15) already is) would keep the diff to the functional change.

Quoted 5 lines
arch/s390/net/bpf_jit_comp.c +2 -0
+			} else {
 				/* mvc bpf_arg_off(8,%r15),arg(%r15) */
 				_EMIT6(0xd207f000 | bpf_arg_off,
 				       0xf000 | arg);
+			}

[ ... ]

Quoted 5 lines

Keep the native argument index separate from the BPF ctx slot index. The former selects r2-r6 or one caller stack slot per s390 ABI argument, while the latter still expands 16-byte arguments to two slots. This also converts arena pointers passed after the fifth argument without shifting later values.

This isn't a bug, but since i and j were already separate before this change, could this paragraph be trimmed to just the new part - that arena pointers beyond the fifth argument are loaded, converted and stored in place?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32538351576

author maintainer
#16 lore raw

PATCH 7/14 bpf, loongarch: Fix stack arguments for indirect trampolines

In reply to Kumar Kartikeya Dwivedi

LoongArch passes arguments beyond a0-a7 at the caller stack pointer. The trampoline store_args() helper always reads those arguments at FP + 16, which is correct for an fentry trampoline: its prologue leaves FP 16 bytes below the stack pointer at trampoline entry after accounting for the saved parent and traced-function frames.

A struct_ops indirect trampoline is entered through a function pointer and only saves its own RA and FP before setting FP to the entry stack pointer. Its stack arguments therefore start at FP, not FP + 16. As a result, every stack-passed struct_ops argument is currently read two slots late.

Select the source offset based on whether the trampoline is indirect. This also prepares the stack-passed arena argument path to consume the actual pointer slot.

Fixes c9ebe2016de9 ("LoongArch: BPF: Support up to 12 function arguments for trampoline")
Cc Tiezhu Yang <[email protected]>
Cc Huacai Chen <[email protected]>
---
 arch/loongarch/net/bpf_jit.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
arch/loongarch/net/bpf_jit.c +5 -4
index 29c281bef28e..d193293a0fd2 100644
@@ -1662,17 +1662,18 @@ int bpf_arch_text_invalidate(void *dst, size_t len)
 	return ret;
 }
 
-static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off)
+static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off, bool is_struct_ops)
 {
+	int stack_args_off = is_struct_ops ? 0 : 16;
 	int i;
 
 	for (i = 0; i < nr_arg_slots; i++) {
 		if (i < LOONGARCH_MAX_REG_ARGS)
 			emit_insn(ctx, std, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off);
 		else {
-			/* Skip slots for T0 and FP of traced function */
+			/* Skip the saved T0 and FP slots for a traced function. */
 			emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP,
-				  16 + (i - LOONGARCH_MAX_REG_ARGS) * 8);
+				  stack_args_off + (i - LOONGARCH_MAX_REG_ARGS) * 8);
 			emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -args_off);
 		}
 		args_off -= 8;
@@ -1995,7 +1996,7 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
 	func_meta = nr_arg_slots;
 	emit_store_stack_imm64(ctx, LOONGARCH_GPR_T1, -func_meta_off, func_meta);
 
-	store_args(ctx, nr_arg_slots, args_off);
+	store_args(ctx, nr_arg_slots, args_off, is_struct_ops);
 
 	if (bpf_fsession_cnt(tnodes)) {
 		/* clear all session cookies' value */
Signature · 1 line
2.53.0
#17 lore raw

Re: [PATCH bpf-next v1 07/14] bpf, loongarch: Fix stack arguments for indirect trampolines

This isn't a bug, but does the updated comment still accurately describe what happens in both cases?
Quoted 16 lines
arch/loongarch/net/bpf_jit.c +2 -1
index 29c281bef28ef..d193293a0fd25 100644
@@ -1662,17 +1662,18 @@ int bpf_arch_text_invalidate(void *dst, size_t len)
 	return ret;
 }
 
-static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off)
+static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off, bool is_struct_ops)
 {
+	int stack_args_off = is_struct_ops ? 0 : 16;
 	int i;
 
 	for (i = 0; i < nr_arg_slots; i++) {
 		if (i < LOONGARCH_MAX_REG_ARGS)

arch/loongarch/net/bpf_jit.c int bpf_arch_text_invalidate(void *dst, size_t len) · line 1679

Quoted 10 lines
arch/loongarch/net/bpf_jit.c +2 -2
 			emit_insn(ctx, std, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off);
 		else {
-			/* Skip slots for T0 and FP of traced function */
+			/* Skip the saved T0 and FP slots for a traced function. */
 			emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP,
-				  16 + (i - LOONGARCH_MAX_REG_ARGS) * 8);
+				  stack_args_off + (i - LOONGARCH_MAX_REG_ARGS) * 8);
 			emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -args_off);
 		}
 		args_off -= 8;

This isn't a bug, but does the updated comment still accurately describe what happens in both cases?

When is_struct_ops is true, stack_args_off is 0, so the code loads stack arguments directly from FP+0 without skipping anything. The indirect trampoline prologue saves RA and FP below FP, not above it, so there are no saved slots to skip in that case.

The comment now says "Skip the saved T0 and FP slots for a traced function," but it sits on a line that executes for both traced functions (where the skip happens) and indirect trampolines (where it doesn't).

Could the comment be moved or reworded to reflect that the skipping only applies to the fentry case? For instance, a comment on the stack_args_off initialization could explain both frame layouts.

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32538351576

maintainer Acked-by Tested-by on v1 7/14
#18 lore raw

Re: [PATCH bpf-next v1 07/14] bpf, loongarch: Fix stack arguments for indirect trampolines

In reply to Kumar Kartikeya Dwivedi
Quoted 50 lines · Kumar Kartikeya Dwivedi

LoongArch passes arguments beyond a0-a7 at the caller stack pointer. The trampoline store_args() helper always reads those arguments at FP + 16, which is correct for an fentry trampoline: its prologue leaves FP 16 bytes below the stack pointer at trampoline entry after accounting for the saved parent and traced-function frames.

A struct_ops indirect trampoline is entered through a function pointer and only saves its own RA and FP before setting FP to the entry stack pointer. Its stack arguments therefore start at FP, not FP + 16. As a result, every stack-passed struct_ops argument is currently read two slots late.

Select the source offset based on whether the trampoline is indirect. This also prepares the stack-passed arena argument path to consume the actual pointer slot.

Fixes c9ebe2016de9 ("LoongArch: BPF: Support up to 12 function arguments for trampoline")
Cc Tiezhu Yang <[email protected]>
Cc Huacai Chen <[email protected]>
---
  arch/loongarch/net/bpf_jit.c | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)
arch/loongarch/net/bpf_jit.c +4 -3
index 29c281bef28e..d193293a0fd2 100644
@@ -1662,17 +1662,18 @@ int bpf_arch_text_invalidate(void *dst, size_t len)
  	return ret;
  }
 
-static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off)
+static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off, bool is_struct_ops)
  {
+	int stack_args_off = is_struct_ops ? 0 : 16;
  	int i;
 
  	for (i = 0; i < nr_arg_slots; i++) {
  		if (i < LOONGARCH_MAX_REG_ARGS)
  			emit_insn(ctx, std, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off);
  		else {
-			/* Skip slots for T0 and FP of traced function */
+			/* Skip the saved T0 and FP slots for a traced function. */
  			emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP,
-				  16 + (i - LOONGARCH_MAX_REG_ARGS) * 8);
+				  stack_args_off + (i - LOONGARCH_MAX_REG_ARGS) * 8);
  			emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -args_off);
  		}
  		args_off -= 8;

arch/loongarch/net/bpf_jit.c static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i · line 2002

Quoted 8 lines
arch/loongarch/net/bpf_jit.c +1 -1
@@ -1995,7 +1996,7 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
  	func_meta = nr_arg_slots;
  	emit_store_stack_imm64(ctx, LOONGARCH_GPR_T1, -func_meta_off, func_meta);
 
-	store_args(ctx, nr_arg_slots, args_off);
+	store_args(ctx, nr_arg_slots, args_off, is_struct_ops);
 
  	if (bpf_fsession_cnt(tnodes)) {
  		/* clear all session cookies' value */

According to the suggestion from bot+bpf-ci, adding the following comment before "int stack_args_off = is_struct_ops ? 0 : 16;" and removing the pre-existing comment inside the else branch would be much better:

/*
* Regular fentry trampolines need to skip the 16-byte saved T0 and FP
* slots of the traced function. Indirect struct_ops trampolines place
* incoming stack arguments directly at FP.
*/

This clarifies both frame layouts without causing any ambiguity inside the loop.

Other than that, the code looks good to me:

By the way, I wrote a selftest for this patch:

selftests/bpf: Add test for indirect struct_ops trampoline

https://lore.kernel.org/bpf/[email protected]/

Without this patch:

  $ sudo ./test_progs -t struct_ops_trampoline
  ...
  run_struct_ops_trampoline:FAIL:check_stack_passed_arg9 unexpected 
check_stack_passed_arg9: actual 69570864 != expected 9999
  #471     struct_ops_trampoline:FAIL
  Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED

With this patch:

  $ sudo ./test_progs -t struct_ops_trampoline
  #471     struct_ops_trampoline:OK
  Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED

Thanks,
Tiezhu

author maintainer
#19 lore raw

Re: [PATCH bpf-next v1 07/14] bpf, loongarch: Fix stack arguments for indirect trampolines

Quoted 96 lines · Tiezhu Yang
Quoted 50 lines · Kumar Kartikeya Dwivedi

LoongArch passes arguments beyond a0-a7 at the caller stack pointer. The trampoline store_args() helper always reads those arguments at FP + 16, which is correct for an fentry trampoline: its prologue leaves FP 16 bytes below the stack pointer at trampoline entry after accounting for the saved parent and traced-function frames.

A struct_ops indirect trampoline is entered through a function pointer and only saves its own RA and FP before setting FP to the entry stack pointer. Its stack arguments therefore start at FP, not FP + 16. As a result, every stack-passed struct_ops argument is currently read two slots late.

Select the source offset based on whether the trampoline is indirect. This also prepares the stack-passed arena argument path to consume the actual pointer slot.

Fixes c9ebe2016de9 ("LoongArch: BPF: Support up to 12 function arguments for trampoline")
Cc Tiezhu Yang <[email protected]>
Cc Huacai Chen <[email protected]>
---
  arch/loongarch/net/bpf_jit.c | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)
arch/loongarch/net/bpf_jit.c +4 -3
index 29c281bef28e..d193293a0fd2 100644
@@ -1662,17 +1662,18 @@ int bpf_arch_text_invalidate(void *dst, size_t len)
  	return ret;
  }
 
-static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off)
+static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off, bool is_struct_ops)
  {
+	int stack_args_off = is_struct_ops ? 0 : 16;
  	int i;
 
  	for (i = 0; i < nr_arg_slots; i++) {
  		if (i < LOONGARCH_MAX_REG_ARGS)
  			emit_insn(ctx, std, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off);
  		else {
-			/* Skip slots for T0 and FP of traced function */
+			/* Skip the saved T0 and FP slots for a traced function. */
  			emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP,
-				  16 + (i - LOONGARCH_MAX_REG_ARGS) * 8);
+				  stack_args_off + (i - LOONGARCH_MAX_REG_ARGS) * 8);
  			emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -args_off);
  		}
  		args_off -= 8;

arch/loongarch/net/bpf_jit.c static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i · line 2002

Quoted 8 lines
arch/loongarch/net/bpf_jit.c +1 -1
@@ -1995,7 +1996,7 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
  	func_meta = nr_arg_slots;
  	emit_store_stack_imm64(ctx, LOONGARCH_GPR_T1, -func_meta_off, func_meta);
 
-	store_args(ctx, nr_arg_slots, args_off);
+	store_args(ctx, nr_arg_slots, args_off, is_struct_ops);
 
  	if (bpf_fsession_cnt(tnodes)) {
  		/* clear all session cookies' value */

According to the suggestion from bot+bpf-ci, adding the following comment before "int stack_args_off = is_struct_ops ? 0 : 16;" and removing the pre-existing comment inside the else branch would be much better:

/*
* Regular fentry trampolines need to skip the 16-byte saved T0 and FP
* slots of the traced function. Indirect struct_ops trampolines place
* incoming stack arguments directly at FP.
*/

This clarifies both frame layouts without causing any ambiguity inside the loop.

Other than that, the code looks good to me:

By the way, I wrote a selftest for this patch:

selftests/bpf: Add test for indirect struct_ops trampoline

https://lore.kernel.org/bpf/[email protected]/

Without this patch:

   $ sudo ./test_progs -t struct_ops_trampoline
   ...
   run_struct_ops_trampoline:FAIL:check_stack_passed_arg9 unexpected
check_stack_passed_arg9: actual 69570864 != expected 9999
   #471     struct_ops_trampoline:FAIL
   Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED
Quoted 7 lines

With this patch:

   $ sudo ./test_progs -t struct_ops_trampoline
   #471     struct_ops_trampoline:OK
   Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED

Thanks a lot for the test. Please also take a look at the argument rebase patches once you folks have more cycles. Will definitely need your help and reviews since they are mostly vibe coded.

Quoted 3 lines

Thanks,
Tiezhu

maintainer
#20 lore raw

Re: [PATCH bpf-next v1 07/14] bpf, loongarch: Fix stack arguments for indirect trampolines

Quoted 1 line · Kumar Kartikeya Dwivedi

On Fri Aug 28, 2026 at 6:33 AM CEST, Tiezhu Yang wrote:

...

Quoted 11 lines
Quoted 7 lines

With this patch:

    $ sudo ./test_progs -t struct_ops_trampoline
    #471     struct_ops_trampoline:OK
    Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED

Thanks a lot for the test. Please also take a look at the argument rebase patches once you folks have more cycles. Will definitely need your help and reviews since they are mostly vibe coded.

Hi Kumar,

I have sent a standalone prerequisite fix for the pre-existing issue:

LoongArch: BPF: Fix off-by-one error for insn_is_cast_user
https://lore.kernel.org/loongarch/[email protected]/

IMO, when you respin, please rebase your patches 07, 08, and 09 on top of this fix for your v2 series.

Additionally, please send the v2 as a standalone series to the loongarch mailing list [email protected] and the reviewer Hengqi Chen <[email protected]> too, as the LoongArch BPF JIT patches are typically routed through the loongarch-next tree.

https://lore.kernel.org/loongarch/CAAhV-H55YcyXRewC+0YzTHXb1jab0GkAV13EF9nFbTx=UkX4ww@mail.gmail.com/

Thanks,
Tiezhu

author maintainer
#21 lore raw

Re: [PATCH bpf-next v1 07/14] bpf, loongarch: Fix stack arguments for indirect trampolines

Quoted 28 lines · Tiezhu Yang
Quoted 1 line · Kumar Kartikeya Dwivedi

On Fri Aug 28, 2026 at 6:33 AM CEST, Tiezhu Yang wrote:

...

Quoted 11 lines
Quoted 7 lines

With this patch:

    $ sudo ./test_progs -t struct_ops_trampoline
    #471     struct_ops_trampoline:OK
    Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED

Thanks a lot for the test. Please also take a look at the argument rebase patches once you folks have more cycles. Will definitely need your help and reviews since they are mostly vibe coded.

Hi Kumar,

I have sent a standalone prerequisite fix for the pre-existing issue:

LoongArch: BPF: Fix off-by-one error for insn_is_cast_user
https://lore.kernel.org/loongarch/[email protected]/

IMO, when you respin, please rebase your patches 07, 08, and 09 on top of this fix for your v2 series.

Quoted 7 lines

Additionally, please send the v2 as a standalone series to the loongarch mailing list [email protected] and the reviewer Hengqi Chen <[email protected]> too, as the LoongArch BPF JIT patches are typically routed through the loongarch-next tree.

https://lore.kernel.org/loongarch/CAAhV-H55YcyXRewC+0YzTHXb1jab0GkAV13EF9nFbTx=UkX4ww@mail.gmail.com/

Ok, I will send v2 without loongarch changes and then split out and send them separately later on (or let me know if you'd like to commandeer them, I'm happy to do it either way).

Thanks

author maintainer
#22 lore raw

PATCH 8/14 bpf, loongarch: JIT arena kfunc argument rebasing

In reply to Kumar Kartikeya Dwivedi

Kfunc parameters marked with BTF_FMODEL_ARENA_ARG carry a 32-bit arena offset in the BPF register, while the native kfunc expects a directly dereferenceable kernel address. Without JIT conversion, moving arena allocation kfuncs from KF_ARENA_ARGS to BTF argument metadata would make LoongArch pass offsets to functions which dereference them.

The LoongArch JIT already keeps the arena kernel mapping base in s6 for arena memory accesses. Reuse it while preparing a pseudo-kfunc call: clear the upper 32 bits of each marked argument, preserve a nullable zero by branching over the addition, and otherwise add the base in s6. Keep the existing ABI extension path unchanged for ordinary kfunc arguments.

Advertise the kfunc-specific arena argument capability independently of struct_ops trampoline conversion.

---
 arch/loongarch/net/bpf_jit.c | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)
arch/loongarch/net/bpf_jit.c +21 -3
index d193293a0fd2..33cabaa7353f 100644
@@ -283,6 +283,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return true;
 }
 
+bool bpf_jit_supports_arena_kfunc_args(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_far_kfunc_call(void)
 {
 	return true;
@@ -1195,9 +1200,22 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 
 			for (i = 0; i < m->nr_args; i++) {
 				u8 reg = regmap[BPF_REG_1 + i];
-				bool sign = m->arg_flags[i] & BTF_FMODEL_SIGNED_ARG;
-
-				emit_abi_ext(ctx, reg, m->arg_size[i], sign);
+				u8 flags = m->arg_flags[i];
+
+				if (flags & BTF_FMODEL_ARENA_ARG) {
+					if (WARN_ON_ONCE(!ctx->arena_vm_start))
+						return -EINVAL;
+
+					/* rN = kern_vm_start + (u32)rN */
+					emit_zext_32(ctx, reg, true);
+					if (flags & BTF_FMODEL_NULLABLE_ARG)
+						emit_insn(ctx, beq, reg, LOONGARCH_GPR_ZERO, 2);
+					emit_insn(ctx, addd, reg, reg, REG_ARENA);
+					continue;
+				}
+
+				emit_abi_ext(ctx, reg, m->arg_size[i],
+					     flags & BTF_FMODEL_SIGNED_ARG);
 			}
 		}
 
Signature · 1 line
2.53.0
#23 lore raw

Re: [PATCH bpf-next v1 08/14] bpf, loongarch: JIT arena kfunc argument rebasing

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The insn_is_cast_user JIT implementation on LoongArch fails to preserve the nullable zero (NULL) due to an off-by-one error in the branch offset. --

--- Patch [8]: [PATCH bpf-next v1 08/14] bpf, loongarch: JIT arena kfunc argument rebasing --- commit dafba7b807dd9770b1b1fe373622d157b3fde2ab
Author: Kumar Kartikeya Dwivedi <[email protected]>

bpf, loongarch: JIT arena kfunc argument rebasing

Kfunc parameters marked with BTF_FMODEL_ARENA_ARG carry a 32-bit arena offset in the BPF register, while the native kfunc expects a directly dereferenceable kernel address. This patch implements the JIT conversion for LoongArch to correctly add the arena base address to these offsets.

Quoted 4 lines
arch/loongarch/net/bpf_jit.c +0 -0
index d193293a0fd25..33cabaa7353f9 100644

[ ... ]

Quoted 7 lines
arch/loongarch/net/bpf_jit.c +0 -3
@@ -1195,9 +1200,22 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 
 			for (i = 0; i < m->nr_args; i++) {
 				u8 reg = regmap[BPF_REG_1 + i];
-				bool sign = m->arg_flags[i] & BTF_FMODEL_SIGNED_ARG;
-
-				emit_abi_ext(ctx, reg, m->arg_size[i], sign);

arch/loongarch/net/bpf_jit.c static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext · line 1212

Quoted 10 lines
arch/loongarch/net/bpf_jit.c +10 -0
+				u8 flags = m->arg_flags[i];
+
+				if (flags & BTF_FMODEL_ARENA_ARG) {
+					if (WARN_ON_ONCE(!ctx->arena_vm_start))
+						return -EINVAL;
+
+					/* rN = kern_vm_start + (u32)rN */
+					emit_zext_32(ctx, reg, true);
+					if (flags & BTF_FMODEL_NULLABLE_ARG)
+						emit_insn(ctx, beq, reg, LOONGARCH_GPR_ZERO, 2);

[Severity: High]
This isn't a bug introduced by this patch, but seeing the correct offset of 2 used here to skip the subsequent addition brings up a question about the pre-existing arena pointer cast implementation.

Does the pre-existing code for insn_is_cast_user in build_insn() have an off-by-one error?

When handling the cast instruction, the code does:

arch/loongarch/net/bpf_jit.c:build_insn() {
    ...
    emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 1);
    emit_insn(ctx, or, t1, dst, t1);
    move_reg(ctx, dst, t1);
    ...
}

Because the branch offset is 1 instead of 2, won't it branch to PC + 4 and execute the or instruction instead of skipping it? If so, a NULL pointer would be incorrectly converted to the arena base address instead of remaining NULL.

Quoted 8 lines
arch/loongarch/net/bpf_jit.c +6 -0
+					emit_insn(ctx, addd, reg, reg, REG_ARENA);
+					continue;
+				}
+
+				emit_abi_ext(ctx, reg, m->arg_size[i],
+					     flags & BTF_FMODEL_SIGNED_ARG);
 			}
 		}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
author maintainer
#24 lore raw

PATCH 9/14 bpf, loongarch: Convert struct_ops arena arguments in trampolines

In reply to Kumar Kartikeya Dwivedi

A struct_ops callback receives native kernel addresses from its caller, but an arena BPF program expects each argument marked with BTF_FMODEL_ARENA_ARG to be a 32-bit offset from its arena mapping. LoongArch currently copies the native pointer into the BPF ctx unchanged.

Obtain the mapping base through bpf_tramp_arena_base() and materialize it in t2 while saving arguments. Copy a marked argument through t1, preserve a nullable native NULL by branching over the subtraction, and otherwise subtract the base before clearing the upper 32 bits. Truncating the full address difference produces the required arena offset.

Walk the function model by argument while maintaining a separate ABI slot index. This keeps argument flags aligned when an earlier small struct spans two slots, while preserving the existing register and stack slot layout. The preceding indirect-trampoline fix supplies the correct stack source for arguments beyond a0-a7.

bpf_tramp_arena_base() only returns a base for the single-program indirect trampoline. Assert that such a trampoline cannot call the original function, which expects unconverted kernel addresses, and advertise the struct_ops-specific arena argument capability.

---
 arch/loongarch/net/bpf_jit.c | 73 ++++++++++++++++++++++++++++++------
 1 file changed, 61 insertions(+), 12 deletions(-)
arch/loongarch/net/bpf_jit.c +61 -12
index 33cabaa7353f..5020414bfd9f 100644
@@ -288,6 +288,11 @@ bool bpf_jit_supports_arena_kfunc_args(void)
 	return true;
 }
 
+bool bpf_jit_supports_arena_struct_ops_args(void)
+{
+	return true;
+}
+
 bool bpf_jit_supports_far_kfunc_call(void)
 {
 	return true;
@@ -1680,21 +1685,55 @@ int bpf_arch_text_invalidate(void *dst, size_t len)
 	return ret;
 }
 
-static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off, bool is_struct_ops)
+/*
+ * Convert an arena kernel address into a 32-bit arena offset while copying it
+ * into the BPF ctx. A nullable argument preserves a native NULL.
+ */
+static void emit_arena_arg_conv(struct jit_ctx *ctx, int dst, int src, bool nullable, int base)
+{
+	if (dst != src)
+		move_reg(ctx, dst, src);
+	if (nullable)
+		emit_insn(ctx, beq, dst, LOONGARCH_GPR_ZERO, 2);
+	emit_insn(ctx, subd, dst, dst, base);
+	emit_zext_32(ctx, dst, true);
+}
+
+static void store_args(struct jit_ctx *ctx, const struct btf_func_model *m, int args_off,
+		       bool is_struct_ops, u64 arena_base)
 {
 	int stack_args_off = is_struct_ops ? 0 : 16;
-	int i;
+	int i, slot = 0;
+
+	if (arena_base)
+		move_imm(ctx, LOONGARCH_GPR_T2, arena_base, false);
 
-	for (i = 0; i < nr_arg_slots; i++) {
-		if (i < LOONGARCH_MAX_REG_ARGS)
-			emit_insn(ctx, std, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off);
-		else {
-			/* Skip the saved T0 and FP slots for a traced function. */
-			emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP,
-				  stack_args_off + (i - LOONGARCH_MAX_REG_ARGS) * 8);
-			emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -args_off);
+	for (i = 0; i < m->nr_args; i++) {
+		bool arena_arg = arena_base && (m->arg_flags[i] & BTF_FMODEL_ARENA_ARG);
+		bool nullable = m->arg_flags[i] & BTF_FMODEL_NULLABLE_ARG;
+		int slots = round_up(m->arg_size[i], 8) / 8;
+
+		while (slots-- > 0) {
+			int src;
+
+			if (slot < LOONGARCH_MAX_REG_ARGS) {
+				src = LOONGARCH_GPR_A0 + slot;
+			} else {
+				/* Skip the saved T0 and FP slots for a traced function. */
+				emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP,
+					  stack_args_off +
+					  (slot - LOONGARCH_MAX_REG_ARGS) * 8);
+				src = LOONGARCH_GPR_T1;
+			}
+			if (arena_arg) {
+				emit_arena_arg_conv(ctx, LOONGARCH_GPR_T1, src, nullable,
+						    LOONGARCH_GPR_T2);
+				src = LOONGARCH_GPR_T1;
+			}
+			emit_insn(ctx, std, src, LOONGARCH_GPR_FP, -args_off);
+			slot++;
+			args_off -= 8;
 		}
-		args_off -= 8;
 	}
 }
 
@@ -1868,6 +1907,7 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
 	struct bpf_tramp_nodes *fexit = &tnodes[BPF_TRAMP_FEXIT];
 	struct bpf_tramp_nodes *fmod_ret = &tnodes[BPF_TRAMP_MODIFY_RETURN];
 	u32 **branches = NULL;
+	u64 arena_base;
 
 	/*
 	 * FP + 8       [ RA to parent func ] return address to parent
@@ -1922,6 +1962,15 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
 	if (flags & (BPF_TRAMP_F_ORIG_STACK | BPF_TRAMP_F_SHARE_IPMODIFY))
 		return -ENOTSUPP;
 
+	/*
+	 * An indirect trampoline never calls the original function. Arena
+	 * conversion relies on this because the original takes kernel addresses.
+	 */
+	WARN_ON_ONCE((flags & BPF_TRAMP_F_INDIRECT) &&
+		     (flags & ~(BPF_TRAMP_F_INDIRECT | BPF_TRAMP_F_RET_FENTRY_RET)));
+
+	arena_base = bpf_tramp_arena_base(m, tnodes, flags);
+
 	/* Room of trampoline frame to store return address and frame pointer */
 	stack_size = 16;
 
@@ -2014,7 +2063,7 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
 	func_meta = nr_arg_slots;
 	emit_store_stack_imm64(ctx, LOONGARCH_GPR_T1, -func_meta_off, func_meta);
 
-	store_args(ctx, nr_arg_slots, args_off, is_struct_ops);
+	store_args(ctx, m, args_off, is_struct_ops, arena_base);
 
 	if (bpf_fsession_cnt(tnodes)) {
 		/* clear all session cookies' value */
Signature · 1 line
2.53.0
#25 lore raw

Re: [PATCH bpf-next v1 09/14] bpf, loongarch: Convert struct_ops arena arguments in trampolines

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The cast_user JIT implementation on LoongArch fails to preserve NULL pointers due to an incorrect branch offset. --

--- Patch [9]: [PATCH bpf-next v1 09/14] bpf, loongarch: Convert struct_ops arena arguments in trampolines --- commit ef7bc691896c4955d9e0b94ff8e49c81e26f20ef
Author: Kumar Kartikeya Dwivedi <[email protected]>

bpf, loongarch: Convert struct_ops arena arguments in trampolines

This patch allows an arena BPF program on LoongArch to receive native kernel addresses converted to 32-bit arena offsets for struct_ops callbacks. It introduces emit_arena_arg_conv() to handle the conversion while preserving native NULL pointers.

Quoted 4 lines
arch/loongarch/net/bpf_jit.c +0 -0
index 33cabaa7353f9..5020414bfd9f2 100644

[ ... ]

Quoted 5 lines
arch/loongarch/net/bpf_jit.c +0 -1
@@ -1680,21 +1685,55 @@ int bpf_arch_text_invalidate(void *dst, size_t len)
 	return ret;
 }
 
-static void store_args(struct jit_ctx *ctx, int nr_arg_slots, int args_off, bool is_struct_ops)

arch/loongarch/net/bpf_jit.c int bpf_arch_text_invalidate(void *dst, size_t len) · line 1697

Quoted 10 lines
arch/loongarch/net/bpf_jit.c +10 -0
+/*
+ * Convert an arena kernel address into a 32-bit arena offset while copying it
+ * into the BPF ctx. A nullable argument preserves a native NULL.
+ */
+static void emit_arena_arg_conv(struct jit_ctx *ctx, int dst, int src, bool nullable, int base)
+{
+	if (dst != src)
+		move_reg(ctx, dst, src);
+	if (nullable)
+		emit_insn(ctx, beq, dst, LOONGARCH_GPR_ZERO, 2);

[Severity: High]
This is a pre-existing issue, but seeing the correct offset of 2 used here for the beq instruction made me notice a potential bug in build_insn() when handling cast_user:

arch/loongarch/net/bpf_jit.c:build_insn() {
...

    emit_insn(ctx, beq, t1, LOONGARCH_GPR_ZERO, 1);
    emit_insn(ctx, or, t1, dst, t1);
    move_reg(ctx, dst, t1);

...
}

Does this pre-existing offset of 1 cause the branch to jump to the immediately next instruction (PC + 4), failing to skip the 'or' instruction?

If so, would a NULL user pointer casted via cast_user incorrectly receive the value of user_vm_start instead of remaining NULL?

Could this lead to invalid pointers being passed to userspace, potentially causing logic errors or data corruption if userspace believes the pointer is valid and writes to the start of the arena?

Quoted 3 lines
arch/loongarch/net/bpf_jit.c +3 -0
+	emit_insn(ctx, subd, dst, dst, base);
+	emit_zext_32(ctx, dst, true);
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
author maintainer
#26 lore raw

PATCH 10/14 bpf, powerpc: JIT arena kfunc argument rebasing

In reply to Kumar Kartikeya Dwivedi

Kfunc parameters annotated with __arena carry a 32-bit arena offset in a BPF register. The kernel function expects a directly dereferenceable kernel address, so the JIT must add the arena kernel mapping base before making the call. Nullable parameters must preserve offset zero as NULL.

PowerPC64 already keeps the arena kernel base in r26 for arena memory accesses. Reuse it in the kfunc ABI preparation path: zero-extend each arena argument to 32 bits, skip the addition for nullable zero, and otherwise add r26. Advertise the kfunc-argument capability on PowerPC64 so arena allocation kfuncs can move to suffix annotations without losing PowerPC support.

Cc Hari Bathini <[email protected]>
Cc Christophe Leroy <[email protected]>
Cc Naveen N. Rao <[email protected]>
---
 arch/powerpc/net/bpf_jit_comp.c   |  5 +++++
 arch/powerpc/net/bpf_jit_comp64.c | 18 +++++++++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)
arch/powerpc/net/bpf_jit_comp.c +5 -0
index 7b07b43575f1..911088cbabb9 100644
@@ -527,6 +527,11 @@ bool bpf_jit_supports_kfunc_call(void)
 	return IS_ENABLED(CONFIG_PPC64);
 }
 
+bool bpf_jit_supports_arena_kfunc_args(void)
+{
+	return IS_ENABLED(CONFIG_PPC64);
+}
+
 bool bpf_jit_supports_private_stack(void)
 {
 	return IS_ENABLED(CONFIG_PPC64);
arch/powerpc/net/bpf_jit_comp64.c +17 -1
index fc9db691e820..fc235fbfbeb9 100644
@@ -624,8 +624,24 @@ static int prepare_for_kfunc_call(const struct bpf_prog *fp, u32 *image,
 	for (i = 0; i < m->nr_args; i++) {
 		/* Note that BPF ABI only allows up to 5 args for kfuncs */
 		u32 reg = bpf_to_ppc(BPF_REG_1 + i), size = m->arg_size[i];
+		u8 flags = m->arg_flags[i];
 
-		if (!(m->arg_flags[i] & BTF_FMODEL_SIGNED_ARG)) {
+		if (flags & BTF_FMODEL_ARENA_ARG) {
+			if (WARN_ON_ONCE(!ctx->arena_vm_start))
+				return -1;
+
+			/* rN = kern_vm_start + (u32)rN */
+			if (zero_extend(image, ctx, reg, reg, 4))
+				return -1;
+			if (flags & BTF_FMODEL_NULLABLE_ARG) {
+				EMIT(PPC_RAW_CMPLDI(reg, 0));
+				PPC_BCC_CONST_SHORT(COND_EQ, 8);
+			}
+			EMIT(PPC_RAW_ADD(reg, reg, bpf_to_ppc(ARENA_VM_START)));
+			continue;
+		}
+
+		if (!(flags & BTF_FMODEL_SIGNED_ARG)) {
 			if (zero_extend(image, ctx, reg, reg, size))
 				return -1;
 		} else {
Signature · 1 line
2.53.0
#27 lore raw

Re: [PATCH bpf-next v1 10/14] bpf, powerpc: JIT arena kfunc argument rebasing

This isn't a bug, but the comment above prepare_for_kfunc_call lists:
Quoted 15 lines
arch/powerpc/net/bpf_jit_comp64.c +5 -1
index fc9db691e8203..fc235fbfbeb97 100644
@@ -624,8 +624,24 @@ static int prepare_for_kfunc_call(const struct bpf_prog *fp, u32 *image,
 	for (i = 0; i < m->nr_args; i++) {
 		/* Note that BPF ABI only allows up to 5 args for kfuncs */
 		u32 reg = bpf_to_ppc(BPF_REG_1 + i), size = m->arg_size[i];
+		u8 flags = m->arg_flags[i];
 
-		if (!(m->arg_flags[i] & BTF_FMODEL_SIGNED_ARG)) {
+		if (flags & BTF_FMODEL_ARENA_ARG) {
+			if (WARN_ON_ONCE(!ctx->arena_vm_start))
+				return -1;
+

arch/powerpc/net/bpf_jit_comp64.c static int prepare_for_kfunc_call(const struct bpf_prog *fp, u32 *image, · line 642

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp64.c +10 -0
+			/* rN = kern_vm_start + (u32)rN */
+			if (zero_extend(image, ctx, reg, reg, 4))
+				return -1;
+			if (flags & BTF_FMODEL_NULLABLE_ARG) {
+				EMIT(PPC_RAW_CMPLDI(reg, 0));
+				PPC_BCC_CONST_SHORT(COND_EQ, 8);
+			}
+			EMIT(PPC_RAW_ADD(reg, reg, bpf_to_ppc(ARENA_VM_START)));
+			continue;
+		}

This isn't a bug, but the comment above prepare_for_kfunc_call lists:

/*
* Handle powerpc ABI expectations from caller:
* - Unsigned arguments are zero-extended.
* - Signed arguments are sign-extended.
*/

After this change the function also rebases __arena arguments onto the arena kernel base. Would it be worth adding that as a third bullet?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32538351576

author maintainer
#28 lore raw

PATCH 11/14 bpf: Replace arena kfunc argument flags with suffixes

In reply to Kumar Kartikeya Dwivedi

The arena allocation kfuncs still identify pointer arguments with KF_ARENA_ARG2. These flags cover only the first two parameters and duplicate the __arena suffix mechanism used by other kfuncs.

Annotate the optional allocation address with __arena__nullable. Mark the free and reserve addresses with __arena so a valid address whose low 32 bits are zero is rebased unconditionally instead of becoming NULL.

The JIT now passes kernel arena addresses to these kfuncs. Translate them back to the lower-32-bit user addresses expected by the existing arena helpers by subtracting kern_vm_start. This preserves allocation-anywhere, freeing the first page of a 4 GiB arena, and reservation at address zero.

Drop KF_ARENA_ARG1 and KF_ARENA_ARG2 from the kernel interface and remove the flags from the arena kfunc sets. KF_ARENA_RET remains responsible for annotating the allocation return value.

Keep the affected selftests synchronized with the conversion. Associate an arena before the iterator map-pointer failures so they still reach the intended diagnostics, account for the extra nullable branch in JIT labels, and treat 1ULL << 32 as the same allocation-anywhere request as NULL after the required 32-bit truncation.

---
 include/linux/btf.h                           |  2 -
 kernel/bpf/arena.c                            | 40 ++++++++++++++-----
 .../selftests/bpf/progs/arena_kfunc_jit.c     | 16 ++++----
 .../selftests/bpf/progs/verifier_arena.c      |  6 +++
 .../bpf/progs/verifier_arena_large.c          |  4 +-
 5 files changed, 46 insertions(+), 22 deletions(-)
include/linux/btf.h +0 -2
index 89d5a5c4f117..65e5f11dc27e 100644
@@ -76,8 +76,6 @@
 #define KF_RCU_PROTECTED (1 << 11) /* kfunc should be protected by rcu cs when they are invoked */
 #define KF_FASTCALL     (1 << 12) /* kfunc supports bpf_fastcall protocol */
 #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
-#define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
-#define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
 #define KF_IMPLICIT_ARGS (1 << 16) /* kfunc has implicit arguments supplied by the verifier */
 #define KF_SPINLOCK_SAFE (1 << 17) /* kfunc is allowed inside bpf_spin_lock-ed region */
 
kernel/bpf/arena.c +29 -11
index 7b6847200b43..6c34a0d34b3f 100644
@@ -1044,18 +1044,28 @@ static void arena_free_irq(struct irq_work *iw)
 	schedule_work(&arena->free_work);
 }
 
+static long arena_kaddr_to_uaddr(struct bpf_arena *arena, const void *addr)
+{
+	if (!addr)
+		return 0;
+
+	return (long)addr - bpf_arena_get_kern_vm_start(arena);
+}
+
 __bpf_kfunc_start_defs();
 
-__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__ign, u32 page_cnt,
-					int node_id, u64 flags)
+__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__arena__nullable,
+					u32 page_cnt, int node_id, u64 flags)
 {
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long addr;
 
 	if (map->map_type != BPF_MAP_TYPE_ARENA || flags || !page_cnt)
 		return NULL;
 
-	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
+	addr = arena_kaddr_to_uaddr(arena, addr__arena__nullable);
+	return (void *)arena_alloc_pages(arena, addr, page_cnt, node_id, true);
 }
 
 void *bpf_arena_alloc_pages_non_sleepable(void *p__map, void *addr__ign, u32 page_cnt,
@@ -1082,14 +1092,20 @@ void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn
 	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
 }
 
-__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+/*
+ * A valid arena address can have zero low 32 bits, so ptr must be rebased
+ * unconditionally instead of being treated as nullable.
+ */
+__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__arena, u32 page_cnt)
 {
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt)
 		return;
-	arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	arena_free_pages(arena, ptr, page_cnt, true);
 }
 
 void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_cnt)
@@ -1102,10 +1118,11 @@ void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_c
 	arena_free_pages(arena, (long)ptr__ign, page_cnt, false);
 }
 
-__bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+__bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__arena, u32 page_cnt)
 {
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;
 
 	if (map->map_type != BPF_MAP_TYPE_ARENA)
 		return -EINVAL;
@@ -1113,14 +1130,15 @@ __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c
 	if (!page_cnt)
 		return 0;
 
-	return arena_reserve_pages(arena, (long)ptr__ign, page_cnt);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	return arena_reserve_pages(arena, ptr, page_cnt);
 }
 __bpf_kfunc_end_defs();
 
 BTF_KFUNCS_START(arena_kfuncs)
-BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
-BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
-BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_SPINLOCK_SAFE)
 BTF_KFUNCS_END(arena_kfuncs)
 
 static const struct btf_kfunc_id_set common_kfunc_set = {
tools/testing/selftests/bpf/progs/arena_kfunc_jit.c +8 -8
index b5a01cbc33a7..c9af35c683b3 100644
@@ -49,15 +49,15 @@ __arch_x86_64
 __jited("...")
 __jited("	movl	%edi, %edi")
 __jited("	testl	%edi, %edi")
-__jited("	je	L0")
+__jited("	je	L1")
 __jited("	addq	%r12, %rdi")
-__jited("L0:	callq	{{.*}}")
+__jited("L1:	callq	{{.*}}")
 __arch_arm64
 __jited("...")
 __jited("	mov	w0, w0")
-__jited("	cbz	w0, L0")
+__jited("	cbz	w0, L1")
 __jited("	add	x0, x28, w0, uxtw")
-__jited("L0:	{{.*}}")
+__jited("L1:	{{.*}}")
 __success
 int arena_arg_jit_nullable(void *ctx)
 {
@@ -79,9 +79,9 @@ __jited("	movl	%ecx, %ecx")
 __jited("	addq	%r12, %rcx")
 __jited("	movl	%r8d, %r8d")
 __jited("	testl	%r8d, %r8d")
-__jited("	je	L0")
+__jited("	je	L1")
 __jited("	addq	%r12, %r8")
-__jited("L0:	callq	{{.*}}")
+__jited("L1:	callq	{{.*}}")
 __arch_arm64
 __jited("...")
 __jited("	add	x0, x28, w0, uxtw")
@@ -89,9 +89,9 @@ __jited("	add	x1, x28, w1, uxtw")
 __jited("	add	x2, x28, w2, uxtw")
 __jited("	add	x3, x28, w3, uxtw")
 __jited("	mov	w4, w4")
-__jited("	cbz	w4, L0")
+__jited("	cbz	w4, L1")
 __jited("	add	x4, x28, w4, uxtw")
-__jited("L0:	{{.*}}")
+__jited("L1:	{{.*}}")
 __success
 int arena_arg_jit_args5(void *ctx)
 {
tools/testing/selftests/bpf/progs/verifier_arena.c +6 -0
index 815f342eb4b0..d76490e059f9 100644
@@ -445,6 +445,8 @@ int iter_maps1(struct bpf_iter__bpf_map *ctx)
 
 	if (!map)
 		return 0;
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
 	bpf_arena_alloc_pages(map, NULL, map->max_entries, 0, 0);
 	return 0;
 }
@@ -455,6 +457,8 @@ int iter_maps2(struct bpf_iter__bpf_map *ctx)
 {
 	struct seq_file *seq = ctx->meta->seq;
 
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
 	bpf_arena_alloc_pages((void *)seq, NULL, 1, 0, 0);
 	return 0;
 }
@@ -467,6 +471,8 @@ int iter_maps3(struct bpf_iter__bpf_map *ctx)
 
 	if (!map)
 		return 0;
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
 	bpf_arena_alloc_pages(map->inner_map_meta, NULL, map->max_entries, 0, 0);
 	return 0;
 }
tools/testing/selftests/bpf/progs/verifier_arena_large.c +3 -1
index 6ab8730d4878..f6515e0e9b17 100644
@@ -49,8 +49,10 @@ int big_alloc1(void *ctx)
 
 	no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE,
 					1, NUMA_NO_NODE, 0);
-	if (no_page)
+	/* Only the low 32 bits contribute, so this is equivalent to NULL. */
+	if (!no_page)
 		return 3;
+	bpf_arena_free_pages(&arena, (void __arena *)no_page, 1);
 	if (*page1 != 1)
 		return 4;
 	if (*page2 != 2)
Signature · 1 line
2.53.0
#29 lore raw

Re: [PATCH bpf-next v1 11/14] bpf: Replace arena kfunc argument flags with suffixes

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Missing address translation in non-sleepable arena kfuncs leads to arena memory corruption. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Missing address translation in non-sleepable arena kfuncs leads to arena memory corruption. --

--- Patch [11]: [PATCH bpf-next v1 11/14] bpf: Replace arena kfunc argument flags with suffixes --- commit a7446937b514c39a83585718956aa0b543092b07
Author: Kumar Kartikeya Dwivedi <[email protected]>

bpf: Replace arena kfunc argument flags with suffixes

This commit replaces the KF_ARENA_ARG1 and KF_ARENA_ARG2 flags with __arena and __arena__nullable BTF suffixes for arena kfuncs. It also updates the main kfuncs to translate JIT-provided kernel addresses back to user addresses using arena_kaddr_to_uaddr().

Quoted 4 lines
kernel/bpf/arena.c +0 -0
index 7b6847200b431..6c34a0d34b3f9 100644

[ ... ]

Quoted 22 lines
kernel/bpf/arena.c +11 -2
@@ -1044,18 +1044,28 @@ static void arena_free_irq(struct irq_work *iw)
 	schedule_work(&arena->free_work);
 }
 
+static long arena_kaddr_to_uaddr(struct bpf_arena *arena, const void *addr)
+{
+	if (!addr)
+		return 0;
+
+	return (long)addr - bpf_arena_get_kern_vm_start(arena);
+}
+
 __bpf_kfunc_start_defs();
 
-__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__ign, u32 page_cnt,
-					int node_id, u64 flags)
+__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__arena__nullable,
+					u32 page_cnt, int node_id, u64 flags)
 {
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long addr;

kernel/bpf/arena.c static void arena_free_irq(struct irq_work *iw) · line 1071

Quoted 10 lines
kernel/bpf/arena.c +2 -1
 
 	if (map->map_type != BPF_MAP_TYPE_ARENA || flags || !page_cnt)
 		return NULL;
 
-	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
+	addr = arena_kaddr_to_uaddr(arena, addr__arena__nullable);
+	return (void *)arena_alloc_pages(arena, addr, page_cnt, node_id, true);
 }
 
 void *bpf_arena_alloc_pages_non_sleepable(void *p__map, void *addr__ign, u32 page_cnt,

[Severity: High]
Does this code miss address translation when the verifier dynamically replaces the kfunc with its non-sleepable or sleepable variants?

When a BPF program calls bpf_arena_alloc_pages() from within a non-sleepable context, the verifier replaces the call with bpf_arena_alloc_pages_non_sleepable().

Since the BPF JIT adds kern_vm_start to the pointer argument based on the __arena__nullable annotation on the main kfunc, won't bpf_arena_alloc_pages_non_sleepable() receive a kernel address in addr__ign?

Because bpf_arena_alloc_pages_non_sleepable() just casts addr__ign without calling arena_kaddr_to_uaddr(), it appears this will pass a kernel address directly to arena_alloc_pages(), potentially causing arena memory corruption or silent allocation failures.

[ ... ]

Quoted 14 lines
kernel/bpf/arena.c +6 -1
@@ -1082,14 +1092,20 @@ void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn
 	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
 }
 
-__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+/*
+ * A valid arena address can have zero low 32 bits, so ptr must be rebased
+ * unconditionally instead of being treated as nullable.
+ */
+__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__arena, u32 page_cnt)
 {
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;

kernel/bpf/arena.c void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn · line 1111

Quoted 10 lines
kernel/bpf/arena.c +3 -2
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt)
 		return;
-	arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	arena_free_pages(arena, ptr, page_cnt, true);
 }
 
 void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_cnt)

[Severity: High]
Does this have the same issue with the non-sleepable variant of free?

When bpf_arena_free_pages_non_sleepable() is substituted by the verifier, it will receive a JIT-rebased kernel address but will pass ptr__ign directly to arena_free_pages() without address translation.

Will this cause the wrong pages inside the arena to be freed?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
#30 lore raw

Re: [PATCH bpf-next v1 11/14] bpf: Replace arena kfunc argument flags with suffixes

In reply to Kumar Kartikeya Dwivedi
The commit message states that marking the allocation address with __arena__nullable preserves allocation-anywhere and freeing the first page of a 4 GiB arena. Does the nullable annotation actually…
Quoted 4 lines
kernel/bpf/arena.c +0 -0
index 7b6847200b431..6c34a0d34b3f9 100644

[ ... ]

Quoted 20 lines
kernel/bpf/arena.c +10 -2
@@ -1044,18 +1044,28 @@ static void arena_free_irq(struct irq_work *iw)
 	schedule_work(&arena->free_work);
 }
 
+static long arena_kaddr_to_uaddr(struct bpf_arena *arena, const void *addr)
+{
+	if (!addr)
+		return 0;
+
+	return (long)addr - bpf_arena_get_kern_vm_start(arena);
+}
+
 __bpf_kfunc_start_defs();
 
-__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__ign, u32 page_cnt,
-					int node_id, u64 flags)
+__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__arena__nullable,
+					u32 page_cnt, int node_id, u64 flags)
 {
 	struct bpf_map *map = p__map;

kernel/bpf/arena.c static void arena_free_irq(struct irq_work *iw) · line 1069

Quoted 10 lines
kernel/bpf/arena.c +3 -1
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long addr;
 
 	if (map->map_type != BPF_MAP_TYPE_ARENA || flags || !page_cnt)
 		return NULL;
 
-	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
+	addr = arena_kaddr_to_uaddr(arena, addr__arena__nullable);
+	return (void *)arena_alloc_pages(arena, addr, page_cnt, node_id, true);
 }

The commit message states that marking the allocation address with __arena__nullable preserves allocation-anywhere and freeing the first page of a 4 GiB arena. Does the nullable annotation actually preserve freeing the first page?

A 4 GiB-aligned arena is constructible (arena_map_alloc() only requires map_extra to be page-aligned and not cross a 32-bit boundary, both of which hold for a 4 GiB-aligned map_extra with vm_range == SZ_4G). For such an arena, the pointer that bpf_arena_alloc_pages() returns for pgoff 0 is:

    clear_lo32(arena->user_vm_start) + 0 == user_vm_start

whose low 32 bits are zero.

When a program passes that pointer back, the JIT emits for the nullable argument (arch/x86/net/bpf_jit_comp.c emit_kfunc_arena_args):

    mov edi,edi;
    test edi,edi;
    je +3;
    add rdi,r12

so the truncated value 0 takes the branch and NULL reaches the kfunc. arena_kaddr_to_uaddr() then returns 0 and arena_alloc_pages() takes the else branch:

    } else {
        ret = pgoff = range_tree_find(&arena->rt, page_cnt);

which picks an arbitrary free range instead of pgoff 0. The kfunc returns a valid but different address and the program has no way to tell its placement request was ignored.

Before this commit the parameter was addr__ign, so it was neither verifier-checked nor JIT-rebased and the full 64-bit user address reached the kernel: 'if (uaddr)' was true (uaddr == user_vm_start != 0), compute_pgoff() returned:

    (u32)(user_vm_start - (u32)user_vm_start) >> PAGE_SHIFT == 0

and the page was allocated exactly where requested.

The commit message claims the conversion preserves allocation-anywhere, freeing the first page of a 4 GiB arena, and reservation at address zero. It mentions treating 1ULL << 32 as the same allocation-anywhere request as NULL after the required 32-bit truncation. However, it does not address that allocating at the first page of a 4 GiB-aligned arena stops working.

The verifier_arena_large.c hunk in this same patch flips:

tools/testing/selftests/bpf/progs/verifier_arena_large.c int big_alloc1(void *ctx) · line 55

Quoted 13 lines
tools/testing/selftests/bpf/progs/verifier_arena_large.c +3 -1
index 6ab8730d48782..f6515e0e9b174 100644
@@ -49,8 +49,10 @@ int big_alloc1(void *ctx)
 
 	no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE,
 					1, NUMA_NO_NODE, 0);
-	if (no_page)
+	/* Only the low 32 bits contribute, so this is equivalent to NULL. */
+	if (!no_page)
 		return 3;
+	bpf_arena_free_pages(&arena, (void __arena *)no_page, 1);

which also drops the only coverage for an out-of-range allocation address being refused.

Does distinguishing 'no address preference' from 'address with zero low 32 bits' need an out-of-band encoding (e.g. a flags bit) rather than the nullable suffix?

Quoted 22 lines

commit a3ff9c5abfeb66a3cd71d6298d7f8a13e7fc8fe3
Author: Kumar Kartikeya Dwivedi <[email protected]>

bpf: Replace arena kfunc argument flags with suffixes

The arena allocation kfuncs still identify pointer arguments with KF_ARENA_ARG2. These flags cover only the first two parameters and duplicate the __arena suffix mechanism used by other kfuncs.

Annotate the optional allocation address with __arena__nullable. Mark the free and reserve addresses with __arena so a valid address whose low 32 bits are zero is rebased unconditionally instead of becoming NULL.

The JIT now passes kernel arena addresses to these kfuncs. Translate them back to the lower-32-bit user addresses expected by the existing arena helpers by subtracting kern_vm_start. This preserves allocation-anywhere, freeing the first page of a 4 GiB arena, and reservation at address zero.

Drop KF_ARENA_ARG1 and KF_ARENA_ARG2 from the kernel interface and remove the flags from the arena kfunc sets. KF_ARENA_RET remains responsible for annotating the allocation return value.

Quoted 7 lines

Keep the affected selftests synchronized with the conversion. Associate an arena before the iterator map-pointer failures so they still reach the intended diagnostics, account for the extra nullable branch in JIT labels, and treat 1ULL << 32 as the same allocation-anywhere request as NULL after the required 32-bit truncation.

This looks like a bug fix for arena kfunc argument handling. Should this include:

Fixes 9a8b57f2bd4d ("bpf: Split arena kfunc and struct_ops JIT capabilities")

The JIT arena rebasing enabled in commit 9a8b57f2bd4d caused the JIT to pass kernel addresses to arena kfuncs, but the kfuncs still expected 32-bit user addresses. This led to incorrect address handling, including valid addresses with zero low 32 bits being treated as NULL.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32538351576

maintainer
#31 lore raw

Re: [PATCH bpf-next v1 11/14] bpf: Replace arena kfunc argument flags with suffixes

In reply to Kumar Kartikeya Dwivedi

On Sat, 2026-08-22 at 01:35 +0200, Kumar Kartikeya Dwivedi wrote:

...

Quoted 4 lines
kernel/bpf/arena.c +0 -0
index 7b6847200b43..6c34a0d34b3f 100644

...

Quoted 8 lines
kernel/bpf/arena.c +3 -1
@@ -1082,14 +1092,20 @@ void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn
 	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
 }
 
-__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+/*
+ * A valid arena address can have zero low 32 bits, so ptr must be rebased
+ * unconditionally instead of being treated as nullable.

kernel/bpf/arena.c void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn · line 1106

Quoted 10 lines
kernel/bpf/arena.c +4 -1
+ */
+__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__arena, u32 page_cnt)
 {
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt)
 		return;                                      ^^^^^^^^^^^^

This is a behavioral change, as far as I understand. Is it intended?

Quoted 6 lines
kernel/bpf/arena.c +2 -1
-	arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	arena_free_pages(arena, ptr, page_cnt, true);
 }
 
 void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_cnt)

...

tools/testing/selftests/bpf/progs/verifier_arena.c int iter_maps1(struct bpf_iter__bpf_map *ctx) · line 449

Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +2 -0
index 815f342eb4b0..d76490e059f9 100644
@@ -445,6 +445,8 @@ int iter_maps1(struct bpf_iter__bpf_map *ctx)
 
 	if (!map)
 		return 0;
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);

Is this behavioral change intended?

Quoted 3 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +0 -0
 	bpf_arena_alloc_pages(map, NULL, map->max_entries, 0, 0);
 	return 0;
 }

...

author maintainer
#32 lore raw

Re: [PATCH bpf-next v1 11/14] bpf: Replace arena kfunc argument flags with suffixes

Quoted 27 lines · Eduard Zingerman

On Sat, 2026-08-22 at 01:35 +0200, Kumar Kartikeya Dwivedi wrote:

...

Quoted 4 lines
kernel/bpf/arena.c +0 -0
index 7b6847200b43..6c34a0d34b3f 100644

...

Quoted 15 lines
kernel/bpf/arena.c +6 -1
@@ -1082,14 +1092,20 @@ void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn
      return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
 }
 
-__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+/*
+ * A valid arena address can have zero low 32 bits, so ptr must be rebased
+ * unconditionally instead of being treated as nullable.
+ */
+__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__arena, u32 page_cnt)
 {
      struct bpf_map *map = p__map;
      struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+     long ptr;
 
Quoted 6 lines

kernel/bpf/arena.c void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn

Quoted 3 lines
kernel/bpf/arena.c +1 -1
-     if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+     if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt)
              return;                                      ^^^^^^^^^^^^

This is a behavioral change, as far as I understand. Is it intended?

Well, I guess I should have added it to the commit log, but rejecting off=0 leaves no way to express freeing the first page.
I am not sure whether accepting the base address now would cause any issues. We can change this to __arena__nullable to preserve current behavior I guess.

Quoted 16 lines
Quoted 7 lines
kernel/bpf/arena.c +2 -1
-     arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
+     ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+     arena_free_pages(arena, ptr, page_cnt, true);
 }
 
 void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_cnt)

...

Quoted 6 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +0 -0
index 815f342eb4b0..d76490e059f9 100644
@@ -445,6 +445,8 @@ int iter_maps1(struct bpf_iter__bpf_map *ctx)
 
Quoted 7 lines

tools/testing/selftests/bpf/progs/verifier_arena.c int iter_maps1(struct bpf_iter__bpf_map *ctx) · line 449

Quoted 4 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +2 -0
      if (!map)
              return 0;
+     /* Associate an arena before testing the generic map-pointer path. */
+     bpf_arena_reserve_pages(&arena, NULL, 0);

Is this behavioral change intended?

I think we have this problem elsewhere, but we need to know what arena a program is associated to (addr_space_cast e.g. gets rejected whenever it cannot see an associated arena map for a prog, so people add fake references).

The second bit is that programs could allocate pages for random arena maps they got access to for other programs, but probably cannot anymore... Earlier, that was determined using the map argument at runtime, now the burden shifts to verifier time. If we want to the conversion to happen it seems to be inevitable.

Quoted 5 lines
Quoted 3 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +0 -0
      bpf_arena_alloc_pages(map, NULL, map->max_entries, 0, 0);
      return 0;
 }

...

reviewer
#33 lore raw

Re: [PATCH bpf-next v1 11/14] bpf: Replace arena kfunc argument flags with suffixes

In reply to Kumar Kartikeya Dwivedi
Quoted 3 lines · Kumar Kartikeya Dwivedi

The arena allocation kfuncs still identify pointer arguments with KF_ARENA_ARG2. These flags cover only the first two parameters and duplicate the __arena suffix mechanism used by other kfuncs.

This is a little petty, but the paragraph mischaracterizes what happened :) It's not like the flags duplicate an existing mechanism for an unknown/dumb reason. There was no __arena and it was the only mechanism until very recently.

Rephrase please?

Quoted 14 lines

Annotate the optional allocation address with __arena__nullable. Mark the free and reserve addresses with __arena so a valid address whose low 32 bits are zero is rebased unconditionally instead of becoming NULL.

The JIT now passes kernel arena addresses to these kfuncs. Translate them back to the lower-32-bit user addresses expected by the existing arena helpers by subtracting kern_vm_start. This preserves allocation-anywhere, freeing the first page of a 4 GiB arena, and reservation at address zero.

Drop KF_ARENA_ARG1 and KF_ARENA_ARG2 from the kernel interface and remove the flags from the arena kfunc sets. KF_ARENA_RET remains responsible for annotating the allocation return value.

Quoted 5 lines

Keep the affected selftests synchronized with the conversion. Associate an arena before the iterator map-pointer failures so they still reach the intended diagnostics, account for the extra nullable branch in JIT labels, and treat 1ULL << 32 as the same allocation-anywhere request as NULL after the required 32-bit truncation.

It's been brought to my attention a few times [1], that we (the linux contributors) would strongly prefer the commit messages to explain *why* the change is being made, instead of *what*.

And the llms tend to do the exact opposite.

This applies to other patches in the series as well, particularly the cover letter.

[1] https://lore.kernel.org/bpf/20260730212613.GJamvBdRJ5H98hTWS8@fat_crate.local/

Quoted 97 lines
---
  include/linux/btf.h                           |  2 -
  kernel/bpf/arena.c                            | 40 ++++++++++++++-----
  .../selftests/bpf/progs/arena_kfunc_jit.c     | 16 ++++----
  .../selftests/bpf/progs/verifier_arena.c      |  6 +++
  .../bpf/progs/verifier_arena_large.c          |  4 +-
  5 files changed, 46 insertions(+), 22 deletions(-)
include/linux/btf.h +0 -2
index 89d5a5c4f117..65e5f11dc27e 100644
@@ -76,8 +76,6 @@
  #define KF_RCU_PROTECTED (1 << 11) /* kfunc should be protected by rcu cs when they are invoked */
  #define KF_FASTCALL     (1 << 12) /* kfunc supports bpf_fastcall protocol */
  #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
-#define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
-#define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
  #define KF_IMPLICIT_ARGS (1 << 16) /* kfunc has implicit arguments supplied by the verifier */
  #define KF_SPINLOCK_SAFE (1 << 17) /* kfunc is allowed inside bpf_spin_lock-ed region */
 
kernel/bpf/arena.c +24 -7
index 7b6847200b43..6c34a0d34b3f 100644
@@ -1044,18 +1044,28 @@ static void arena_free_irq(struct irq_work *iw)
  	schedule_work(&arena->free_work);
  }
 
+static long arena_kaddr_to_uaddr(struct bpf_arena *arena, const void *addr)
+{
+	if (!addr)
+		return 0;
+
+	return (long)addr - bpf_arena_get_kern_vm_start(arena);
+}
+
  __bpf_kfunc_start_defs();
 
-__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__ign, u32 page_cnt,
-					int node_id, u64 flags)
+__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__arena__nullable,
+					u32 page_cnt, int node_id, u64 flags)
  {
  	struct bpf_map *map = p__map;
  	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long addr;
 
  	if (map->map_type != BPF_MAP_TYPE_ARENA || flags || !page_cnt)
  		return NULL;
 
-	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
+	addr = arena_kaddr_to_uaddr(arena, addr__arena__nullable);
+	return (void *)arena_alloc_pages(arena, addr, page_cnt, node_id, true);
  }
 
  void *bpf_arena_alloc_pages_non_sleepable(void *p__map, void *addr__ign, u32 page_cnt,
@@ -1082,14 +1092,20 @@ void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn
  	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
  }
 
-__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+/*
+ * A valid arena address can have zero low 32 bits, so ptr must be rebased
+ * unconditionally instead of being treated as nullable.
+ */
+__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__arena, u32 page_cnt)
  {
  	struct bpf_map *map = p__map;
  	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt)
  		return;
-	arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	arena_free_pages(arena, ptr, page_cnt, true);
  }
 
  void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_cnt)
@@ -1102,10 +1118,11 @@ void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_c
  	arena_free_pages(arena, (long)ptr__ign, page_cnt, false);
  }
 
-__bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+__bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__arena, u32 page_cnt)
  {
  	struct bpf_map *map = p__map;
  	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;
 
  	if (map->map_type != BPF_MAP_TYPE_ARENA)
  		return -EINVAL;

kernel/bpf/arena.c __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c · line 1134

Quoted 6 lines
kernel/bpf/arena.c +2 -1
@@ -1113,14 +1130,15 @@ __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c
  	if (!page_cnt)
  		return 0;
 
-	return arena_reserve_pages(arena, (long)ptr__ign, page_cnt);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	return arena_reserve_pages(arena, ptr, page_cnt);

This patch does a couple unrelated things clumped together.

There is a kfunc arena arg identification mechanism change: flag to suffix. And there is behavioral change: rebasing the pointer etc.

I think it'd be easier to review and would lead to a cleaner git history if these were separated. In particular, I'd expect the flag deprecation itself to not require any changes in the jit tests.

Quoted 108 lines
kernel/bpf/arena.c +3 -3
  }
  __bpf_kfunc_end_defs();
 
  BTF_KFUNCS_START(arena_kfuncs)
-BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
-BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
-BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_SPINLOCK_SAFE)
  BTF_KFUNCS_END(arena_kfuncs)
 
  static const struct btf_kfunc_id_set common_kfunc_set = {
tools/testing/selftests/bpf/progs/arena_kfunc_jit.c +8 -8
index b5a01cbc33a7..c9af35c683b3 100644
@@ -49,15 +49,15 @@ __arch_x86_64
  __jited("...")
  __jited("	movl	%edi, %edi")
  __jited("	testl	%edi, %edi")
-__jited("	je	L0")
+__jited("	je	L1")
  __jited("	addq	%r12, %rdi")
-__jited("L0:	callq	{{.*}}")
+__jited("L1:	callq	{{.*}}")
  __arch_arm64
  __jited("...")
  __jited("	mov	w0, w0")
-__jited("	cbz	w0, L0")
+__jited("	cbz	w0, L1")
  __jited("	add	x0, x28, w0, uxtw")
-__jited("L0:	{{.*}}")
+__jited("L1:	{{.*}}")
  __success
  int arena_arg_jit_nullable(void *ctx)
  {
@@ -79,9 +79,9 @@ __jited("	movl	%ecx, %ecx")
  __jited("	addq	%r12, %rcx")
  __jited("	movl	%r8d, %r8d")
  __jited("	testl	%r8d, %r8d")
-__jited("	je	L0")
+__jited("	je	L1")
  __jited("	addq	%r12, %r8")
-__jited("L0:	callq	{{.*}}")
+__jited("L1:	callq	{{.*}}")
  __arch_arm64
  __jited("...")
  __jited("	add	x0, x28, w0, uxtw")
@@ -89,9 +89,9 @@ __jited("	add	x1, x28, w1, uxtw")
  __jited("	add	x2, x28, w2, uxtw")
  __jited("	add	x3, x28, w3, uxtw")
  __jited("	mov	w4, w4")
-__jited("	cbz	w4, L0")
+__jited("	cbz	w4, L1")
  __jited("	add	x4, x28, w4, uxtw")
-__jited("L0:	{{.*}}")
+__jited("L1:	{{.*}}")
  __success
  int arena_arg_jit_args5(void *ctx)
  {
tools/testing/selftests/bpf/progs/verifier_arena.c +6 -0
index 815f342eb4b0..d76490e059f9 100644
@@ -445,6 +445,8 @@ int iter_maps1(struct bpf_iter__bpf_map *ctx)
 
  	if (!map)
  		return 0;
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
  	bpf_arena_alloc_pages(map, NULL, map->max_entries, 0, 0);
  	return 0;
  }
@@ -455,6 +457,8 @@ int iter_maps2(struct bpf_iter__bpf_map *ctx)
  {
  	struct seq_file *seq = ctx->meta->seq;
 
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
  	bpf_arena_alloc_pages((void *)seq, NULL, 1, 0, 0);
  	return 0;
  }
@@ -467,6 +471,8 @@ int iter_maps3(struct bpf_iter__bpf_map *ctx)
 
  	if (!map)
  		return 0;
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
  	bpf_arena_alloc_pages(map->inner_map_meta, NULL, map->max_entries, 0, 0);
  	return 0;
  }
tools/testing/selftests/bpf/progs/verifier_arena_large.c +3 -1
index 6ab8730d4878..f6515e0e9b17 100644
@@ -49,8 +49,10 @@ int big_alloc1(void *ctx)
 
  	no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE,
  					1, NUMA_NO_NODE, 0);
-	if (no_page)
+	/* Only the low 32 bits contribute, so this is equivalent to NULL. */
+	if (!no_page)
  		return 3;
+	bpf_arena_free_pages(&arena, (void __arena *)no_page, 1);
  	if (*page1 != 1)
  		return 4;
  	if (*page2 != 2)
author maintainer
#34 lore raw

Re: [PATCH bpf-next v1 11/14] bpf: Replace arena kfunc argument flags with suffixes

Quoted 5 lines · Ihor Solodrai
Quoted 3 lines · Kumar Kartikeya Dwivedi

The arena allocation kfuncs still identify pointer arguments with KF_ARENA_ARG2. These flags cover only the first two parameters and duplicate the __arena suffix mechanism used by other kfuncs.

Quoted 7 lines

This is a little petty, but the paragraph mischaracterizes what happened :) It's not like the flags duplicate an existing mechanism for an unknown/dumb reason. There was no __arena and it was the only mechanism until very recently.

Rephrase please?

Sure, will reword.

Quoted 24 lines
Quoted 14 lines

Annotate the optional allocation address with __arena__nullable. Mark the free and reserve addresses with __arena so a valid address whose low 32 bits are zero is rebased unconditionally instead of becoming NULL.

The JIT now passes kernel arena addresses to these kfuncs. Translate them back to the lower-32-bit user addresses expected by the existing arena helpers by subtracting kern_vm_start. This preserves allocation-anywhere, freeing the first page of a 4 GiB arena, and reservation at address zero.

Drop KF_ARENA_ARG1 and KF_ARENA_ARG2 from the kernel interface and remove the flags from the arena kfunc sets. KF_ARENA_RET remains responsible for annotating the allocation return value.

Quoted 5 lines

Keep the affected selftests synchronized with the conversion. Associate an arena before the iterator map-pointer failures so they still reach the intended diagnostics, account for the extra nullable branch in JIT labels, and treat 1ULL << 32 as the same allocation-anywhere request as NULL after the required 32-bit truncation.

It's been brought to my attention a few times [1], that we (the linux contributors) would strongly prefer the commit messages to explain *why* the change is being made, instead of *what*.

Quoted 8 lines

And the llms tend to do the exact opposite.

This applies to other patches in the series as well, particularly the cover letter.

[1]
https://lore.kernel.org/bpf/20260730212613.GJamvBdRJ5H98hTWS8@fat_crate.local/

I'll rewrite the commit log. I mostly just sent this out more as an RFC (except we pw kicks RFCs out of CI queue), so it's rough in several places.

Quoted 106 lines
Quoted 97 lines
---
  include/linux/btf.h                           |  2 -
  kernel/bpf/arena.c                            | 40 ++++++++++++++-----
  .../selftests/bpf/progs/arena_kfunc_jit.c     | 16 ++++----
  .../selftests/bpf/progs/verifier_arena.c      |  6 +++
  .../bpf/progs/verifier_arena_large.c          |  4 +-
  5 files changed, 46 insertions(+), 22 deletions(-)
include/linux/btf.h +0 -2
index 89d5a5c4f117..65e5f11dc27e 100644
@@ -76,8 +76,6 @@
  #define KF_RCU_PROTECTED (1 << 11) /* kfunc should be protected by rcu cs when they are invoked */
  #define KF_FASTCALL     (1 << 12) /* kfunc supports bpf_fastcall protocol */
  #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
-#define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
-#define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
  #define KF_IMPLICIT_ARGS (1 << 16) /* kfunc has implicit arguments supplied by the verifier */
  #define KF_SPINLOCK_SAFE (1 << 17) /* kfunc is allowed inside bpf_spin_lock-ed region */
 
kernel/bpf/arena.c +24 -7
index 7b6847200b43..6c34a0d34b3f 100644
@@ -1044,18 +1044,28 @@ static void arena_free_irq(struct irq_work *iw)
  	schedule_work(&arena->free_work);
  }
 
+static long arena_kaddr_to_uaddr(struct bpf_arena *arena, const void *addr)
+{
+	if (!addr)
+		return 0;
+
+	return (long)addr - bpf_arena_get_kern_vm_start(arena);
+}
+
  __bpf_kfunc_start_defs();
 
-__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__ign, u32 page_cnt,
-					int node_id, u64 flags)
+__bpf_kfunc void *bpf_arena_alloc_pages(void *p__map, void *addr__arena__nullable,
+					u32 page_cnt, int node_id, u64 flags)
  {
  	struct bpf_map *map = p__map;
  	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long addr;
 
  	if (map->map_type != BPF_MAP_TYPE_ARENA || flags || !page_cnt)
  		return NULL;
 
-	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
+	addr = arena_kaddr_to_uaddr(arena, addr__arena__nullable);
+	return (void *)arena_alloc_pages(arena, addr, page_cnt, node_id, true);
  }
 
  void *bpf_arena_alloc_pages_non_sleepable(void *p__map, void *addr__ign, u32 page_cnt,
@@ -1082,14 +1092,20 @@ void *bpf_arena_alloc_pages_sleepable(void *p__map, void *addr__ign, u32 page_cn
  	return (void *)arena_alloc_pages(arena, (long)addr__ign, page_cnt, node_id, true);
  }
 
-__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+/*
+ * A valid arena address can have zero low 32 bits, so ptr must be rebased
+ * unconditionally instead of being treated as nullable.
+ */
+__bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__arena, u32 page_cnt)
  {
  	struct bpf_map *map = p__map;
  	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt)
  		return;
-	arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	arena_free_pages(arena, ptr, page_cnt, true);
  }
 
  void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_cnt)
@@ -1102,10 +1118,11 @@ void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_c
  	arena_free_pages(arena, (long)ptr__ign, page_cnt, false);
  }
 
-__bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_cnt)
+__bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__arena, u32 page_cnt)
  {
  	struct bpf_map *map = p__map;
  	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	long ptr;
 
  	if (map->map_type != BPF_MAP_TYPE_ARENA)
  		return -EINVAL;

kernel/bpf/arena.c __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c · line 1134

Quoted 6 lines
kernel/bpf/arena.c +2 -1
@@ -1113,14 +1130,15 @@ __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c
  	if (!page_cnt)
  		return 0;
 
-	return arena_reserve_pages(arena, (long)ptr__ign, page_cnt);
+	ptr = arena_kaddr_to_uaddr(arena, ptr__arena);
+	return arena_reserve_pages(arena, ptr, page_cnt);

This patch does a couple unrelated things clumped together.

Quoted 7 lines

There is a kfunc arena arg identification mechanism change: flag to suffix. And there is behavioral change: rebasing the pointer etc.

I think it'd be easier to review and would lead to a cleaner git history if these were separated. In particular, I'd expect the flag deprecation itself to not require any changes in the jit tests.

Yep, makes sense. Will split into separate commits.

Quoted 108 lines
Quoted 108 lines
kernel/bpf/arena.c +3 -3
  }
  __bpf_kfunc_end_defs();
 
  BTF_KFUNCS_START(arena_kfuncs)
-BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
-BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
-BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_SPINLOCK_SAFE)
  BTF_KFUNCS_END(arena_kfuncs)
 
  static const struct btf_kfunc_id_set common_kfunc_set = {
tools/testing/selftests/bpf/progs/arena_kfunc_jit.c +8 -8
index b5a01cbc33a7..c9af35c683b3 100644
@@ -49,15 +49,15 @@ __arch_x86_64
  __jited("...")
  __jited("	movl	%edi, %edi")
  __jited("	testl	%edi, %edi")
-__jited("	je	L0")
+__jited("	je	L1")
  __jited("	addq	%r12, %rdi")
-__jited("L0:	callq	{{.*}}")
+__jited("L1:	callq	{{.*}}")
  __arch_arm64
  __jited("...")
  __jited("	mov	w0, w0")
-__jited("	cbz	w0, L0")
+__jited("	cbz	w0, L1")
  __jited("	add	x0, x28, w0, uxtw")
-__jited("L0:	{{.*}}")
+__jited("L1:	{{.*}}")
  __success
  int arena_arg_jit_nullable(void *ctx)
  {
@@ -79,9 +79,9 @@ __jited("	movl	%ecx, %ecx")
  __jited("	addq	%r12, %rcx")
  __jited("	movl	%r8d, %r8d")
  __jited("	testl	%r8d, %r8d")
-__jited("	je	L0")
+__jited("	je	L1")
  __jited("	addq	%r12, %r8")
-__jited("L0:	callq	{{.*}}")
+__jited("L1:	callq	{{.*}}")
  __arch_arm64
  __jited("...")
  __jited("	add	x0, x28, w0, uxtw")
@@ -89,9 +89,9 @@ __jited("	add	x1, x28, w1, uxtw")
  __jited("	add	x2, x28, w2, uxtw")
  __jited("	add	x3, x28, w3, uxtw")
  __jited("	mov	w4, w4")
-__jited("	cbz	w4, L0")
+__jited("	cbz	w4, L1")
  __jited("	add	x4, x28, w4, uxtw")
-__jited("L0:	{{.*}}")
+__jited("L1:	{{.*}}")
  __success
  int arena_arg_jit_args5(void *ctx)
  {
tools/testing/selftests/bpf/progs/verifier_arena.c +6 -0
index 815f342eb4b0..d76490e059f9 100644
@@ -445,6 +445,8 @@ int iter_maps1(struct bpf_iter__bpf_map *ctx)
 
  	if (!map)
  		return 0;
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
  	bpf_arena_alloc_pages(map, NULL, map->max_entries, 0, 0);
  	return 0;
  }
@@ -455,6 +457,8 @@ int iter_maps2(struct bpf_iter__bpf_map *ctx)
  {
  	struct seq_file *seq = ctx->meta->seq;
 
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
  	bpf_arena_alloc_pages((void *)seq, NULL, 1, 0, 0);
  	return 0;
  }
@@ -467,6 +471,8 @@ int iter_maps3(struct bpf_iter__bpf_map *ctx)
 
  	if (!map)
  		return 0;
+	/* Associate an arena before testing the generic map-pointer path. */
+	bpf_arena_reserve_pages(&arena, NULL, 0);
  	bpf_arena_alloc_pages(map->inner_map_meta, NULL, map->max_entries, 0, 0);
  	return 0;
  }
tools/testing/selftests/bpf/progs/verifier_arena_large.c +3 -1
index 6ab8730d4878..f6515e0e9b17 100644
@@ -49,8 +49,10 @@ int big_alloc1(void *ctx)
 
  	no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE,
  					1, NUMA_NO_NODE, 0);
-	if (no_page)
+	/* Only the low 32 bits contribute, so this is equivalent to NULL. */
+	if (!no_page)
  		return 3;
+	bpf_arena_free_pages(&arena, (void __arena *)no_page, 1);
  	if (*page1 != 1)
  		return 4;
  	if (*page2 != 2)
author maintainer
#35 lore raw

PATCH 12/14 resolve_btfids: Drop KF_ARENA_ARG flag support

In reply to Kumar Kartikeya Dwivedi

Arena argument suffixes now describe the address-space contract at any parameter position, while the kernel no longer publishes KF_ARENA_ARG1 or KF_ARENA_ARG2. Keeping private copies in resolve_btfids would leave an obsolete second annotation mechanism and silently accept flags the kernel no longer understands.

Derive arena arguments exclusively from their suffixes and retain KF_ARENA_RET for functions returning arena pointers. Update the resolver selftest to distinguish return-only flags, unannotated pointer arguments, and suffix-annotated arguments.

---
 tools/bpf/resolve_btfids/main.c               | 22 ++-----------------
 .../selftests/bpf/prog_tests/resolve_btfids.c | 22 ++++++-------------
 2 files changed, 9 insertions(+), 35 deletions(-)
tools/bpf/resolve_btfids/main.c +2 -20
index 37d7e7224207..3dbf329edb46 100644
@@ -177,8 +177,6 @@ struct object {
 
 #define KF_FASTCALL	(1 << 12)
 #define KF_ARENA_RET	(1 << 13)
-#define KF_ARENA_ARG1	(1 << 14)
-#define KF_ARENA_ARG2	(1 << 15)
 #define KF_IMPLICIT_ARGS (1 << 16)
 #define KF_IMPL_SUFFIX "_impl"
 #define TYPE_ATTR_ARENA "address_space(1)"
@@ -1317,22 +1315,6 @@ static int process_kfunc_with_implicit_args(struct btf2btf_context *ctx, struct
 	return 0;
 }
 
-static bool is_arena_arg(const struct btf *btf, const struct kfunc *kfunc,
-			 const struct btf_param *param, u32 idx)
-{
-	if (is_arena_param(btf, param))
-		return true;
-
-	switch (idx) {
-	case 0:
-		return kfunc->flags & KF_ARENA_ARG1;
-	case 1:
-		return kfunc->flags & KF_ARENA_ARG2;
-	default:
-		return false;
-	}
-}
-
 static s32 arena_tag_ptr(struct btf *btf, u32 ptr_id, struct kfunc *kfunc)
 {
 	const struct btf_type *ptr = btf__type_by_id(btf, ptr_id);
@@ -1382,7 +1364,7 @@ static s32 add_arena_tagged_proto(struct btf *btf, struct kfunc *kfunc)
 	int err, i;
 
 	for (i = 0; i < nr_params; i++) {
-		if (is_arena_arg(btf, kfunc, &params[i], i)) {
+		if (is_arena_param(btf, &params[i])) {
 			has_arena_arg = true;
 			break;
 		}
@@ -1420,7 +1402,7 @@ static s32 add_arena_tagged_proto(struct btf *btf, struct kfunc *kfunc)
 	for (i = 0; i < nr_params; i++) {
 		t = btf__type_by_id(btf, new_proto_id);
 		tag_params = btf_params(t);
-		if (!is_arena_arg(btf, kfunc, &tag_params[i], i))
+		if (!is_arena_param(btf, &tag_params[i]))
 			continue;
 
 		id = arena_tag_ptr(btf, tag_params[i].type, kfunc);
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +7 -15
index 3f9949e8227d..a4381c7fa7da 100644
@@ -23,13 +23,6 @@
 #ifndef KF_ARENA_RET
 #define KF_ARENA_RET  (1 << 13)
 #endif
-#ifndef KF_ARENA_ARG1
-#define KF_ARENA_ARG1 (1 << 14)
-#endif
-#ifndef KF_ARENA_ARG2
-#define KF_ARENA_ARG2 (1 << 15)
-#endif
-
 struct symbol {
 	const char	*name;
 	int		 type;
@@ -57,9 +50,8 @@ struct kfunc_symbol {
 static struct kfunc_symbol kfunc_symbols[] = {
 	{ "kfunc_a", -1, 0, 0, false },
 	{ "kfunc_b", -1, KF_FASTCALL, 0, false },
-	{ "kfunc_c", -1, KF_ARENA_RET | KF_ARENA_ARG1 | KF_ARENA_ARG2,
-	  ARENA_ARG(0) | ARENA_ARG(1), true },
-	{ "kfunc_d", -1, KF_ARENA_ARG2, ARENA_ARG(1), false },
+	{ "kfunc_c", -1, KF_ARENA_RET, 0, true },
+	{ "kfunc_d", -1, 0, 0, false },
 	{ "kfunc_e", -1, 0, ARENA_ARG(0) | ARENA_ARG(1) | ARENA_ARG(2) |
 	  ARENA_ARG(3) | ARENA_ARG(4), false },
 	{ "kfunc_f", -1, 0, ARENA_ARG(1), false },
@@ -111,8 +103,8 @@ BTF_SET_END(test_set)
 BTF_KFUNCS_START(test_kfunc_set)
 BTF_ID_FLAGS(func, kfunc_a)
 BTF_ID_FLAGS(func, kfunc_b, KF_FASTCALL)
-BTF_ID_FLAGS(func, kfunc_c, KF_ARENA_RET | KF_ARENA_ARG1 | KF_ARENA_ARG2)
-BTF_ID_FLAGS(func, kfunc_d, KF_ARENA_ARG2)
+BTF_ID_FLAGS(func, kfunc_c, KF_ARENA_RET)
+BTF_ID_FLAGS(func, kfunc_d)
 BTF_ID_FLAGS(func, kfunc_e)
 BTF_ID_FLAGS(func, kfunc_f)
 BTF_ID_FLAGS(func, kfunc_g, KF_ARENA_RET)
@@ -126,8 +118,8 @@ BTF_KFUNCS_START(test_kfunc_set_rev)
 BTF_ID_FLAGS(func, kfunc_g, KF_ARENA_RET)
 BTF_ID_FLAGS(func, kfunc_f)
 BTF_ID_FLAGS(func, kfunc_e)
-BTF_ID_FLAGS(func, kfunc_d, KF_ARENA_ARG2)
-BTF_ID_FLAGS(func, kfunc_c, KF_ARENA_RET | KF_ARENA_ARG1 | KF_ARENA_ARG2)
+BTF_ID_FLAGS(func, kfunc_d)
+BTF_ID_FLAGS(func, kfunc_c, KF_ARENA_RET)
 BTF_ID_FLAGS(func, kfunc_b, KF_FASTCALL)
 BTF_ID_FLAGS(func, kfunc_a)
 BTF_KFUNCS_END(test_kfunc_set_rev)
@@ -315,7 +307,7 @@ void test_resolve_btfids(void)
 	}
 
 	/*
-	 * Check resolve_btfids wrapped exactly the arena-flagged or suffixed
+	 * Check resolve_btfids wrapped exactly the arena return or suffixed
 	 * return/args with the address_space(1) type attribute, and left other
 	 * pointers/returns untouched.
 	 */
Signature · 1 line
2.53.0
#36 lore raw

Re: [PATCH bpf-next v1 12/14] resolve_btfids: Drop KF_ARENA_ARG flag support

Does the reworded comment accurately describe the mechanism? After this patch there are exactly two annotation sources: KF_ARENA_RET for the return value, and the __arena/__arena__nullable…
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0 -0
index 37d7e7224207d..3dbf329edb469 100644

[ ... ]

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 3f9949e8227df..a4381c7fa7da4 100644

[ ... ]

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c void test_resolve_btfids(void) · line 313

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -315,7 +307,7 @@ void test_resolve_btfids(void)
 	}
 
 	/*
-	 * Check resolve_btfids wrapped exactly the arena-flagged or suffixed
+	 * Check resolve_btfids wrapped exactly the arena return or suffixed
 	 * return/args with the address_space(1) type attribute, and left other
 	 * pointers/returns untouched.
 	 */

Does the reworded comment accurately describe the mechanism? After this patch there are exactly two annotation sources: KF_ARENA_RET for the return value, and the __arena/__arena__nullable parameter-name suffixes for arguments.

A return value has no parameter name, so it cannot be suffix-annotated. Looking at add_arena_tagged_proto():

    if (kfunc->flags & KF_ARENA_RET) {
            ret_type_id = arena_tag_ptr(btf, ret_type_id, kfunc);
    ...
            if (!is_arena_param(btf, &tag_params[i]))
                    continue;

is_arena_param() is only ever applied to btf_params(), never to proto->type.

The new text 'the arena return or suffixed return/args' mentions the return twice and asserts a 'suffixed return', which the code cannot produce. The old wording ('arena-flagged or suffixed return/args') was defensible because 'arena-flagged' collectively covered KF_ARENA_RET, KF_ARENA_ARG1 and KF_ARENA_ARG2; the reword keeps the '/args' shorthand but drops the term that made it coherent.

Could this be reworded to 'the KF_ARENA_RET return and the suffixed args' to match what the code below actually checks?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32538351576

maintainer
#37 lore raw

Re: [PATCH bpf-next v1 12/14] resolve_btfids: Drop KF_ARENA_ARG flag support

In reply to Kumar Kartikeya Dwivedi

On Sat, 2026-08-22 at 01:35 +0200, Kumar Kartikeya Dwivedi wrote:

...

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 3f9949e8227d..a4381c7fa7da 100644

...

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c struct kfunc_symbol { · line 54

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +2 -3
@@ -57,9 +50,8 @@ struct kfunc_symbol {
 static struct kfunc_symbol kfunc_symbols[] = {
 	{ "kfunc_a", -1, 0, 0, false },
 	{ "kfunc_b", -1, KF_FASTCALL, 0, false },
-	{ "kfunc_c", -1, KF_ARENA_RET | KF_ARENA_ARG1 | KF_ARENA_ARG2,
-	  ARENA_ARG(0) | ARENA_ARG(1), true },
-	{ "kfunc_d", -1, KF_ARENA_ARG2, ARENA_ARG(1), false },
+	{ "kfunc_c", -1, KF_ARENA_RET, 0, true },
+	{ "kfunc_d", -1, 0, 0, false },

This changes test cases for kfunc_{c,d}, which previously were expected to have arena args. Why?

Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
 	{ "kfunc_e", -1, 0, ARENA_ARG(0) | ARENA_ARG(1) | ARENA_ARG(2) |
 	  ARENA_ARG(3) | ARENA_ARG(4), false },
 	{ "kfunc_f", -1, 0, ARENA_ARG(1), false },

...

author maintainer
#38 lore raw

Re: [PATCH bpf-next v1 12/14] resolve_btfids: Drop KF_ARENA_ARG flag support

Quoted 12 lines · Eduard Zingerman

On Sat, 2026-08-22 at 01:35 +0200, Kumar Kartikeya Dwivedi wrote:

...

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 3f9949e8227d..a4381c7fa7da 100644

...

Quoted 12 lines

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c struct kfunc_symbol { · line 54

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +2 -3
@@ -57,9 +50,8 @@ struct kfunc_symbol {
 static struct kfunc_symbol kfunc_symbols[] = {
      { "kfunc_a", -1, 0, 0, false },
      { "kfunc_b", -1, KF_FASTCALL, 0, false },
-     { "kfunc_c", -1, KF_ARENA_RET | KF_ARENA_ARG1 | KF_ARENA_ARG2,
-       ARENA_ARG(0) | ARENA_ARG(1), true },
-     { "kfunc_d", -1, KF_ARENA_ARG2, ARENA_ARG(1), false },
+     { "kfunc_c", -1, KF_ARENA_RET, 0, true },
+     { "kfunc_d", -1, 0, 0, false },

This changes test cases for kfunc_{c,d}, which previously were expected to have arena args. Why?

They got it because they were tagged with KF_ARENA_ARG{1,2}. They don't have suffix otherwise, so they don't get it now. I can add the suffixes though in v2.

Quoted 6 lines
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
      { "kfunc_e", -1, 0, ARENA_ARG(0) | ARENA_ARG(1) | ARENA_ARG(2) |
        ARENA_ARG(3) | ARENA_ARG(4), false },
      { "kfunc_f", -1, 0, ARENA_ARG(1), false },

...

reviewer Acked-by on v1 12/14
#39 lore raw

Re: [PATCH bpf-next v1 12/14] resolve_btfids: Drop KF_ARENA_ARG flag support

In reply to Kumar Kartikeya Dwivedi
Quoted 6 lines · Kumar Kartikeya Dwivedi

Arena argument suffixes now describe the address-space contract at any parameter position, while the kernel no longer publishes KF_ARENA_ARG1 or KF_ARENA_ARG2. Keeping private copies in resolve_btfids would leave an obsolete second annotation mechanism and silently accept flags the kernel no longer understands.

Quoted 6 lines

Derive arena arguments exclusively from their suffixes and retain KF_ARENA_RET for functions returning arena pointers. Update the resolver selftest to distinguish return-only flags, unannotated pointer arguments, and suffix-annotated arguments.

Assuming Eduard's point re selftest is going to be addressed:

Quoted 6 lines
---
  tools/bpf/resolve_btfids/main.c               | 22 ++-----------------
  .../selftests/bpf/prog_tests/resolve_btfids.c | 22 ++++++-------------
  2 files changed, 9 insertions(+), 35 deletions(-)

[...]

author maintainer
#40 lore raw

PATCH 13/14 selftests/bpf: Exercise arena arguments on every capable JIT

In reply to Kumar Kartikeya Dwivedi

The arena kfunc runtime tests are restricted to x86-64 and arm64, so new JIT implementations can compile without exercising rebasing, nullable arguments, five-register calls, or the verifier capability gate.

Run those tests on RISC-V, s390, LoongArch, and PowerPC64 as well. Teach test_loader about PowerPC64 so it can select the newly annotated programs.

Enable the struct_ops arena tests on RISC-V, s390, and LoongArch, whose indirect trampolines now implement the separate reverse-conversion path. PowerPC64 remains covered only for kfunc arguments because it does not advertise the struct_ops capability.

---
 .../bpf/prog_tests/test_struct_ops_arena.c    | 16 ++++++---
 .../testing/selftests/bpf/progs/arena_kfunc.c | 36 +++++++++++++++++++
 tools/testing/selftests/bpf/progs/bpf_misc.h  |  1 +
 tools/testing/selftests/bpf/test_loader.c     |  5 +++
 4 files changed, 53 insertions(+), 5 deletions(-)
tools/testing/selftests/bpf/prog_tests/test_struct_ops_arena.c +11 -5
index 7f9f54ba3fbe..7f44b00af06d 100644
@@ -6,7 +6,13 @@
 #include "struct_ops_arena_attach.skel.h"
 #include "struct_ops_arena_fail.skel.h"
 
-#if defined(__x86_64__) || defined(__aarch64__)
+#if defined(__x86_64__) || defined(__aarch64__) || \
+	(defined(__riscv) && __riscv_xlen == 64) || defined(__s390x__) || \
+	defined(__loongarch__)
+#define HAVE_ARENA_STRUCT_OPS_ARGS
+#endif
+
+#ifdef HAVE_ARENA_STRUCT_OPS_ARGS
 /*
  * Attach callbacks with __arena and __arena__nullable arguments and drive
  * them through the bpf_testmod_ops3_call_test_arena*() kfuncs.
@@ -111,11 +117,11 @@ static void arena_arg_attach(void)
 void serial_test_struct_ops_arena(void)
 {
 	/*
-	 * Arena struct_ops arguments need JIT support, currently x86-64 and
-	 * arm64 only. Elsewhere verification fails with "JIT does not support
-	 * arena arguments", so the programs cannot even load.
+	 * Arena struct_ops arguments need JIT support. Elsewhere verification
+	 * fails with "JIT does not support arena arguments", so the programs
+	 * cannot even load.
 	 */
-#if defined(__x86_64__) || defined(__aarch64__)
+#ifdef HAVE_ARENA_STRUCT_OPS_ARGS
 	if (test__start_subtest("arena_arg"))
 		arena_arg();
 	if (test__start_subtest("arena_arg_fail"))
tools/testing/selftests/bpf/progs/arena_kfunc.c +36 -0
index bf0d304e0e59..8ee2bd4d6633 100644
@@ -28,6 +28,10 @@ volatile u64 stash;
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __success __retval(0)
 int arena_arg_forms(void *ctx)
 {
@@ -72,6 +76,10 @@ int arena_arg_forms(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __success __retval(0)
 int arena_arg_rebase(void *ctx)
 {
@@ -114,6 +122,10 @@ int arena_arg_rebase(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __success __retval(0)
 int arena_args5(void *ctx)
 {
@@ -146,6 +158,10 @@ int arena_args5(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __success __retval(0)
 int arena_arg_mixed(void *ctx)
 {
@@ -174,6 +190,10 @@ int arena_arg_mixed(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __success __retval(0)
 int arena_arg_unpopulated(void *ctx)
 {
@@ -195,6 +215,10 @@ int arena_arg_unpopulated(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __failure __msg("arena pointer requires a program with an associated arena")
 int arena_arg_no_arena(void *ctx)
 {
@@ -205,6 +229,10 @@ int arena_arg_no_arena(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __failure __msg("is not a pointer to arena or scalar")
 int arena_arg_bad_reg(void *ctx)
 {
@@ -221,6 +249,10 @@ int arena_arg_bad_reg(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __failure __msg("arena pointer cannot be a stack argument")
 int arena_arg_stack(void *ctx)
 {
@@ -232,6 +264,10 @@ int arena_arg_stack(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __description("arena_arg_stack: not supported, dummy test")
 __success
 int arena_arg_stack(void *ctx)
tools/testing/selftests/bpf/progs/bpf_misc.h +1 -0
index 5eacf1b43252..c35359c97e44 100644
@@ -159,6 +159,7 @@
 #define __arch_riscv64		__arch("RISCV64")
 #define __arch_s390x		__arch("s390x")
 #define __arch_loongarch	__arch("LOONGARCH")
+#define __arch_powerpc64	__arch("POWERPC64")
 #define __caps_unpriv(caps)	__test_tag("test_caps_unpriv=" EXPAND_QUOTE(caps))
 #define __load_if_JITed()	__test_tag("load_mode=jited")
 #define __load_if_no_JITed()	__test_tag("load_mode=no_jited")
tools/testing/selftests/bpf/test_loader.c +5 -0
index 07807757b518..221393f2a8ef 100644
@@ -378,6 +378,7 @@ enum arch {
 	ARCH_RISCV64	= 0x8,
 	ARCH_S390X	= 0x10,
 	ARCH_LOONGARCH	= 0x20,
+	ARCH_POWERPC64	= 0x40,
 };
 
 static int get_current_arch(void)
@@ -392,6 +393,8 @@ static int get_current_arch(void)
 	return ARCH_S390X;
 #elif defined(__loongarch__)
 	return ARCH_LOONGARCH;
+#elif defined(__powerpc64__)
+	return ARCH_POWERPC64;
 #endif
 	return ARCH_UNKNOWN;
 }
@@ -585,6 +588,8 @@ static int parse_test_spec(struct test_loader *tester,
 				arch = ARCH_S390X;
 			} else if (strcmp(val, "LOONGARCH") == 0) {
 				arch = ARCH_LOONGARCH;
+			} else if (strcmp(val, "POWERPC64") == 0) {
+				arch = ARCH_POWERPC64;
 			} else {
 				PRINT_FAIL("bad arch spec: '%s'\n", val);
 				err = -EINVAL;
Signature · 1 line
2.53.0
#41 lore raw

Re: [PATCH bpf-next v1 13/14] selftests/bpf: Exercise arena arguments on every capable JIT

Can arena_arg_unpopulated run safely on the newly added architectures?
Quoted 4 lines
tools/testing/selftests/bpf/progs/arena_kfunc.c +0 -0
index bf0d304e0e59c..8ee2bd4d66331 100644

[ ... ]

tools/testing/selftests/bpf/progs/arena_kfunc.c int arena_arg_unpopulated(void *ctx)

Quoted 21 lines
tools/testing/selftests/bpf/progs/arena_kfunc.c +5 -1
@@ -174,6 +190,10 @@ int arena_arg_mixed(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __success __retval(0)
 int arena_arg_unpopulated(void *ctx)
 {
@@ -187,7 +207,7 @@ int arena_arg_unpopulated(void *ctx)
 	 * and returns VM_FAULT_SIGBUS. That condition is diagnosed by
 	 * bpf_prog_test_run_syscall() as -EFAULT propagated from ctx_access_ok().
 	 */
-	val = (u64 *)bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+	val = (u64 __arena *)bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
 	if (!val)
 		return 3;
 
@@ -195,9 +215,13 @@ int arena_arg_unpopulated(void *ctx)

Can arena_arg_unpopulated run safely on the newly added architectures?

This test deliberately passes an unpopulated arena address to the kernel (stash = (u64)val + PAGE_SIZE, then bpf_kfunc_arena_arg_test((u64 *)stash)). The kernel dereferences this address in bpf_testmod.c:

  old = *val__arena;
  *val__arena = old + 1;

For this to work without crashing, the page fault handler must call bpf_arena_handle_page_fault(), which installs the per-arena scratch page.

Looking at the fault handler integration, bpf_arena_handle_page_fault() is only wired up on x86-64 (arch/x86/mm/fault.c) and arm64 (arch/arm64/mm/fault.c). The other architectures use the __weak stub that always returns false, meaning the fault is never recovered.

Without a scratch page, the fault on an unmapped vmalloc address becomes an unrecoverable kernel oops. The new JIT capability (bpf_jit_supports_arena_kfunc_args) only covers rebasing the argument at the call site, not recovering from unpopulated-page faults.

Should this test remain restricted to x86-64 and arm64, or does the recovery path need to be implemented on the new architectures first?

[ ... ]

Quoted 12 lines
tools/testing/selftests/bpf/progs/arena_kfunc.c +4 -0
@@ -221,6 +249,10 @@ int arena_arg_bad_reg(void *ctx)
 SEC("syscall")
 __arch_x86_64
 __arch_arm64
+__arch_riscv64
+__arch_s390x
+__arch_loongarch
+__arch_powerpc64
 __failure __msg("arena pointer cannot be a stack argument")
 int arena_arg_stack(void *ctx)
 {

tools/testing/selftests/bpf/progs/arena_kfunc.c int arena_arg_stack(void *ctx) · line 261

Quoted 2 lines
tools/testing/selftests/bpf/progs/arena_kfunc.c +0 -0
@@ -228,6 +260,10 @@ int arena_arg_stack(void *ctx)
 	return bpf_kfunc_arena_stack_arg_test(1, 2, 3, 4, 5, (u64 *)1);
 }

Will arena_arg_stack produce the expected failure message on the new arches?

The test expects the verifier to reject the program with "arena pointer cannot be a stack argument". However, this message is only reachable after the verifier checks whether the JIT supports stack arguments.

bpf_kfunc_arena_stack_arg_test() takes 6 parameters, which exceeds MAX_BPF_FUNC_REG_ARGS (5). In kernel/bpf/verifier.c, gen_kfunc_arg_proto() checks this first:

  if (nargs > MAX_BPF_FUNC_REG_ARGS && !bpf_jit_supports_stack_args()) {
      verbose(env, "JIT does not support kfunc %s() with %d args\n", ...);
      return -ENOTSUPP;
  }

Only after this gate does the per-argument loop inspect individual arguments and emit "arena pointer cannot be a stack argument".

bpf_jit_supports_stack_args() returns true only on x86-64 and arm64. On riscv64, s390, loongarch, and powerpc64, it returns false (the __weak stub in kernel/bpf/core.c). So the program does fail to load on those arches, but with "JIT does not support kfunc bpf_kfunc_arena_stack_arg_test() with 6 args" instead of the expected message.

test_loader's validate_msgs() will report this as a test failure since it does not match the __msg annotation. Should this test stay limited to x86-64 and arm64?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32538351576

reviewer
#42 lore raw

Re: [PATCH bpf-next v1 13/14] selftests/bpf: Exercise arena arguments on every capable JIT

In reply to Kumar Kartikeya Dwivedi
Quoted 11 lines · Kumar Kartikeya Dwivedi

The arena kfunc runtime tests are restricted to x86-64 and arm64, so new JIT implementations can compile without exercising rebasing, nullable arguments, five-register calls, or the verifier capability gate.

Run those tests on RISC-V, s390, LoongArch, and PowerPC64 as well. Teach test_loader about PowerPC64 so it can select the newly annotated programs.

Enable the struct_ops arena tests on RISC-V, s390, and LoongArch, whose indirect trampolines now implement the separate reverse-conversion path. PowerPC64 remains covered only for kfunc arguments because it does not advertise the struct_ops capability.

I'd avoid adding these unless you actually ran it.

Have you tried non-x86 / arm64 arches at least in emulation?

Quoted 10 lines
---
  .../bpf/prog_tests/test_struct_ops_arena.c    | 16 ++++++---
  .../testing/selftests/bpf/progs/arena_kfunc.c | 36 +++++++++++++++++++
  tools/testing/selftests/bpf/progs/bpf_misc.h  |  1 +
  tools/testing/selftests/bpf/test_loader.c     |  5 +++
  4 files changed, 53 insertions(+), 5 deletions(-)

[...]

author maintainer
#43 lore raw

Re: [PATCH bpf-next v1 13/14] selftests/bpf: Exercise arena arguments on every capable JIT

Quoted 8 lines · Ihor Solodrai
Quoted 7 lines · Kumar Kartikeya Dwivedi

The arena kfunc runtime tests are restricted to x86-64 and arm64, so new JIT implementations can compile without exercising rebasing, nullable arguments, five-register calls, or the verifier capability gate.

Run those tests on RISC-V, s390, LoongArch, and PowerPC64 as well. Teach test_loader about PowerPC64 so it can select the newly annotated programs.

Quoted 9 lines
Quoted 4 lines

Enable the struct_ops arena tests on RISC-V, s390, and LoongArch, whose indirect trampolines now implement the separate reverse-conversion path. PowerPC64 remains covered only for kfunc arguments because it does not advertise the struct_ops capability.

I'd avoid adding these unless you actually ran it.

Have you tried non-x86 / arm64 arches at least in emulation?

No, I'm relying on individual JIT maintainers to green flag this set from their side. I think that should suffice. It would be best to do the conversion all at once, and we don't really have a time constraint, so can wait for all JIT folks to chip in with their reviews and tags before we land this.

Quoted 10 lines
Quoted 10 lines
---
  .../bpf/prog_tests/test_struct_ops_arena.c    | 16 ++++++---
  .../testing/selftests/bpf/progs/arena_kfunc.c | 36 +++++++++++++++++++
  tools/testing/selftests/bpf/progs/bpf_misc.h  |  1 +
  tools/testing/selftests/bpf/test_loader.c     |  5 +++
  4 files changed, 53 insertions(+), 5 deletions(-)

[...]

author maintainer
#44 lore raw

PATCH 14/14 docs/bpf: Document split arena argument JIT capabilities

In reply to Kumar Kartikeya Dwivedi

Arena kfunc calls and struct_ops callbacks perform opposite conversions at different JIT boundaries and are now advertised independently.

Document the architectures supporting each capability so users can tell where arena allocation kfuncs and arena struct_ops arguments can load.

---
 Documentation/bpf/kfuncs.rst | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
Documentation/bpf/kfuncs.rst +9 -6
index 89dea6b0b024..65e56491cae1 100644
@@ -301,9 +301,10 @@ An example is given below::
         }
 
 Calling such a kfunc requires the program to use an arena map and a JIT with
-arena argument support (currently x86-64 and arm64); verification fails
-otherwise. The program can pass any value without compromising the kernel. A
-value that does not point into the arena is a program bug.
+arena kfunc argument support (currently x86-64, arm64, RISC-V, s390,
+LoongArch, and PowerPC64); verification fails otherwise. The program can pass
+any value without compromising the kernel. A value that does not point into
+the arena is a program bug.
 
 The suffixes have the same meaning on the arguments of struct_ops stub
 functions, with the conversion running in the opposite direction. The
@@ -311,9 +312,11 @@ kernel caller passes the kernel arena address and the trampoline converts
 it while saving the arguments, so the callback receives an arena pointer
 it can dereference directly. With ``__arena`` the kernel caller must not
 pass NULL. With ``__arena__nullable`` a NULL kernel pointer arrives as NULL.
-However, there is no obligation to prove to the verifier that such a pointer is
-non-NULL before use, in-line with existing semantics of arena pointers used in
-a program (or obtained from any other source).
+Struct_ops arena argument conversion is a separate JIT capability, currently
+provided by x86-64, arm64, RISC-V, s390, and LoongArch. However, there is no
+obligation to prove to the verifier that such a pointer is non-NULL before use,
+in-line with existing semantics of arena pointers used in a program (or
+obtained from any other source).
 
 .. _BPF_kfunc_nodef:
 
Signature · 1 line
2.53.0