Skip to contentSkip to conversation

Fix generic __uninit kfunc output buffers

PATCH Superseded Kumar Kartikeya Dwivedi 5 participants 12 messages Sep 15 – Sep 16

Subsystems (6)

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

cover Fix generic __uninit kfunc output buffers

Generic __uninit kfunc arguments are output buffers. Their stack liveness already treats them as writes, but argument checking still requires readable contents and does not record definite initialization after the call. Check these arguments as write-only and reuse helper output-buffer tracking, with one entry per argument slot.

Kumar Kartikeya Dwivedi (3):
bpf: Fix generic __uninit kfunc output buffers selftests/bpf: Allow privileged preparation for capability tests selftests/bpf: Cover generic __uninit kfunc output buffers

 Documentation/bpf/kfuncs.rst                  |  27 +-
 include/linux/bpf_verifier.h                  |  11 +-
 kernel/bpf/verifier.c                         |  70 ++-
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 tools/testing/selftests/bpf/progs/bpf_misc.h  |   9 +-
 .../bpf/progs/verifier_kfunc_uninit.c         | 399 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  56 +++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |   9 +
 tools/testing/selftests/bpf/test_loader.c     |  41 +-
 9 files changed, 577 insertions(+), 47 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c

base-commit: 5ef40d69b38a93bc9951dadb1a15c85c597e1a40

Signature · 1 line
2.53.0
author maintainer
#1 lore raw

PATCH 1/3 bpf: Fix generic __uninit kfunc output buffers

Stack liveness treats __uninit kfunc arguments as writes, but generic memory argument validation still checks them as both reads and writes. A checkpoint can consequently poison an output buffer before the call and cause its read check to reject the program. Allowing that read would not suffice: ordinary clobber checks leave poisoned bytes poisoned after the call.

Check generic __uninit memory arguments as write-only and reuse helper raw memory handling to record their definite initialization. Track output sizes per argument slot so that a call can initialize multiple buffers, including arguments passed on the stack. Apply the writes after validating all arguments, preserving input checks when an input aliases an output. Keep variable offsets and sizes conservative, disabling raw mode only for the affected argument.

Preserve MEM_UNINIT when a scalar-only struct pointer is resolved to a fixed-size memory argument. In stack liveness, inspect the resolved BTF parameter rather than its ABI slot when looking up the annotation, since a preceding by-value parameter can occupy multiple slots.

Document that generic output buffers must be fully initialized on every return path, including error returns and padding.

---
 Documentation/bpf/kfuncs.rst | 27 ++++++++++----
 include/linux/bpf_verifier.h | 11 +++---
 kernel/bpf/verifier.c        | 70 +++++++++++++++++++++++++-----------
 3 files changed, 76 insertions(+), 32 deletions(-)
Documentation/bpf/kfuncs.rst +20 -7
index 6c2c048dccef..71fb0ca72d69 100644
@@ -164,19 +164,32 @@ suffix should be used.
 2.3.3 __uninit Annotation
 -------------------------
 
-This annotation is used to indicate that the argument will be treated as
-uninitialized.
+Use ``__uninit`` on a pointer parameter for an output that the kfunc
+initializes without reading its incoming contents.
 
-An example is given below::
+For generic memory buffers, the kfunc must initialize every byte in the
+declared range on every return path, including error returns and struct
+padding. The range is determined by the pointed-to type or the associated
+``__sz`` or ``__szk`` size argument.
+
+The annotation does not change the accepted pointer types. A stack-backed
+struct passed as a generic memory buffer must still be scalar-only.
+
+A stack buffer with a verifier-known constant offset and size may be
+uninitialized before the call and is considered initialized afterwards.
+For variable offsets or sizes, callers with neither ``CAP_PERFMON`` nor
+``CAP_SYS_ADMIN`` must initialize the potentially accessed stack range before
+the call.
+
+For dynptr parameters, ``__uninit`` indicates that the kfunc constructs a
+dynptr in the supplied storage. For example::
 
-        __bpf_kfunc int bpf_dynptr_from_skb(..., struct bpf_dynptr_kern *ptr__uninit)
+        __bpf_kfunc int bpf_dynptr_from_skb(..., struct bpf_dynptr *ptr__uninit)
         {
         ...
         }
 
-Here, the dynptr will be treated as an uninitialized dynptr. Without this
-annotation, the verifier will reject the program if the dynptr passed in is
-not initialized.
+Without this annotation, a dynptr argument must already be initialized.
 
 2.3.4 __nullable Annotation
 ---------------------------
include/linux/bpf_verifier.h +6 -5
index cf85141ea167..9ddbb20ec1e9 100644
@@ -1547,12 +1547,13 @@ struct ref_obj_desc {
 };
 
 /*
- * A memory argument a call fills in. The verifier allows the stack to be uninitialized if
- * the range is a known constant. Stack slots are marked as STACK_MISC by check_mem_access().
+ * Memory arguments a call fills in, indexed by argument slot. The verifier allows the
+ * stack to be uninitialized if the range is a known constant. Stack slots are marked as
+ * STACK_MISC by check_mem_access() after all arguments have been checked.
  */
 struct arg_raw_mem_desc {
-	u8 regno;
-	int size;
+	u16 mask;
+	int size[MAX_BPF_FUNC_ARGS];
 };
 
 /* Size of PTR_TO_MEM returned, taken from a constant allocation-size argument */
@@ -1579,6 +1580,7 @@ struct bpf_call_arg_meta {
 	struct bpf_dynptr_desc dynptr;
 	struct ref_obj_desc ref_obj;
 	struct ret_mem_desc ret_mem;
+	struct arg_raw_mem_desc arg_raw_mem;
 
 	/* Only set by kfunc */
 	bool r0_rdonly;
@@ -1617,7 +1619,6 @@ struct bpf_call_arg_meta {
 	s64 const_map_key;
 	struct btf *ret_btf;
 	struct btf_field *kptr_field;
-	struct arg_raw_mem_desc arg_raw_mem;
 };
 
 int bpf_get_helper_proto(struct bpf_verifier_env *env, int func_id,
kernel/bpf/verifier.c +50 -20

149 lines not on this page · raw message

Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix generic __uninit kfunc output buffers

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] bpf: UBSAN out-of-bounds and NULL pointer arithmetic in mark_raw_stack() --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] bpf: UBSAN out-of-bounds and NULL pointer arithmetic in mark_raw_stack() --

commit fc77f675fb39113d85b64af5b859a8ccdbd2caef
Author: Kumar Kartikeya Dwivedi <[email protected]>

bpf: Fix generic __uninit kfunc output buffers

This commit updates the verifier to check generic __uninit memory arguments as write-only, tracking output sizes per argument slot so a call can initialize multiple buffers, including those passed on the stack. It reuses helper raw memory handling to record their definite initialization after validating all arguments.

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index 6c6b8d8520cdf..070ec5daeec33 100644

[ ... ]

Quoted 8 lines
kernel/bpf/verifier.c +4 -0
@@ -9550,6 +9556,33 @@ static int check_func_args(struct bpf_verifier_env *env, struct bpf_call_arg_met
 	return 0;
 }
 
+static int mark_raw_stack(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
+			  int insn_idx)
+{
+	struct bpf_func_state *caller = cur_func(env);

kernel/bpf/verifier.c static int check_func_args(struct bpf_verifier_env *env, struct bpf_call_arg_met · line 9572

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+	u32 slot;
+	int i, err;
+
+	/*
+	 * Validate every argument before initializing outputs: an input argument
+	 * may alias an output buffer. Use the normal stack-write checks to discard
+	 * stale spills and preserve the rules for special stack objects.
+	 */
+	for (slot = 0; slot < MAX_BPF_FUNC_ARGS; slot++) {
+		struct bpf_reg_state *reg = get_func_arg_reg(caller, cur_regs(env), slot);

[Severity: High]
Does this unconditional call to get_func_arg_reg() for all MAX_BPF_FUNC_ARGS slots cause an out-of-bounds or NULL pointer arithmetic issue?

Looking at get_func_arg_reg(), it computes a pointer offset based on the caller's stack arguments:

kernel/bpf/verifier.c:get_func_arg_reg() {
    ...
    return &caller->stack_arg_regs[arg - MAX_BPF_FUNC_REG_ARGS];
}

Since this loop unconditionally evaluates get_func_arg_reg() for all 12 potential slots (MAX_BPF_FUNC_ARGS), even if meta->arg_raw_mem.size[slot] is 0, won't this trigger undefined behavior and a UBSAN panic for any helper or kfunc call where caller->stack_arg_regs is NULL or smaller than the computed offset?

Quoted 12 lines
kernel/bpf/verifier.c +12 -0
+		argno_t argno = argno_from_arg(slot + 1);
+
+		for (i = 0; i < meta->arg_raw_mem.size[slot]; i++) {
+			err = check_mem_access(env, insn_idx, reg, argno, i, BPF_B,
+					       BPF_WRITE, -1, false, false);
+			if (err)
+				return err;
+		}
+	}
+
+	return 0;
+}

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix generic __uninit kfunc output buffers

In reply to Kumar Kartikeya Dwivedi

[...]

kernel/bpf/verifier.c static int check_func_arg(struct bpf_verifier_env *env, u32 arg, u32 slot, u32 p · line 9241

Quoted 7 lines
kernel/bpf/verifier.c +2 -1
@@ -9232,7 +9237,8 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, u32 slot, u32 p
 
                access_type = fn->arg_type[arg - 1] & MEM_WRITE ? BPF_WRITE : BPF_READ;
                if (meta->btf)
-                       access_type = BPF_READ | BPF_WRITE;
+                       access_type = fn->arg_type[arg - 1] & MEM_UNINIT ?
+                                     BPF_WRITE : BPF_READ | BPF_WRITE;

This looks a bit convoluted. Consider doing this when generating the kfunc proto or at least something like below?

  if (arg_type & MEM_UNINIT)
        access_type = BPF_WRITE;
  else if (meta->btf)
        access_type = BPF_READ | BPF_WRITE;
  else
        access_type = arg_type & MEM_WRITE ? BPF_WRITE : BPF_READ;
Quoted 11 lines
kernel/bpf/verifier.c +4 -0
                zero_size_allowed = meta->btf || base_type(arg_type) == ARG_MEM_SIZE_OR_ZERO;
 
@@ -9550,6 +9556,33 @@ static int check_func_args(struct bpf_verifier_env *env, struct bpf_call_arg_met
        return 0;
 }
 
+static int mark_raw_stack(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
+                         int insn_idx)
+{
+       struct bpf_func_state *caller = cur_func(env);

kernel/bpf/verifier.c static int check_func_args(struct bpf_verifier_env *env, struct bpf_call_arg_met · line 9572

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+       u32 slot;
+       int i, err;
+
+       /*
+        * Validate every argument before initializing outputs: an input argument
+        * may alias an output buffer. Use the normal stack-write checks to discard
+        * stale spills and preserve the rules for special stack objects.
+        */
+       for (slot = 0; slot < MAX_BPF_FUNC_ARGS; slot++) {
+               struct bpf_reg_state *reg = get_func_arg_reg(caller, cur_regs(env), slot);

The bot has pointed out the need to skip inactive slots, but the rest of the patch looks good to me,

Quoted 13 lines
kernel/bpf/verifier.c +13 -0
+               argno_t argno = argno_from_arg(slot + 1);
+
+               for (i = 0; i < meta->arg_raw_mem.size[slot]; i++) {
+                       err = check_mem_access(env, insn_idx, reg, argno, i, BPF_B,
+                                              BPF_WRITE, -1, false, false);
+                       if (err)
+                               return err;
+               }
+       }
+
+       return 0;
+}
+
maintainer
#4 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix generic __uninit kfunc output buffers

In reply to Kumar Kartikeya Dwivedi

On Tue, 2026-09-15 at 16:10 +0200, Kumar Kartikeya Dwivedi wrote:

...

Quoted 17 lines

 2.3.4 __nullable Annotation
 ---------------------------

include/linux/bpf_verifier.h +0 -0
index cf85141ea167..9ddbb20ec1e9 100644
@@ -1547,12 +1547,13 @@ struct ref_obj_desc {
  };


 /*
- * A memory argument a call fills in. The verifier allows the stack to be uninitialized if
- * the range is a known constant. Stack slots are marked as STACK_MISC by check_mem_access().
+ * Memory arguments a call fills in, indexed by argument slot. The verifier allows the
+ * stack to be uninitialized if the range is a known constant. Stack slots are marked as
+ * STACK_MISC by check_mem_access() after all arguments have been checked.
  */
 struct arg_raw_mem_desc {

include/linux/bpf_verifier.h struct ref_obj_desc { · line 1555

Quoted 3 lines
include/linux/bpf_verifier.h +1 -2
-	u8 regno;
-	int size;
+	u16 mask;

Do we plan to add kfuncs with multiple __uninit parameters? If we do plan this, let's separate __uninit fix from the mask conversion. Otherwise let's keep regno.

Quoted 4 lines
include/linux/bpf_verifier.h +1 -0
+	int size[MAX_BPF_FUNC_ARGS];

 };

 /* Size of PTR_TO_MEM returned, taken from a constant allocation-size argument */

...

kernel/bpf/verifier.c static int check_stack_range_initialized( · line 6986

Quoted 12 lines
kernel/bpf/verifier.c +3 -1
index 6c6b8d8520cd..070ec5daeec3 100644
@@ -6981,7 +6981,9 @@ static int check_stack_range_initialized(
  	 */
  	bool allow_poison = access_size < 0 || clobber;
  	/* The call will initialize the memory; uninitialized stack allowed */
-	bool raw_mode = meta && meta->arg_raw_mem.regno == reg_from_argno(argno);
+	u32 arg_slot = abs(argno.argno) - 1;
+	bool raw_mode = meta && arg_slot < MAX_BPF_FUNC_ARGS &&
+		       (meta->arg_raw_mem.mask & BIT(arg_slot));

Thit BIT(abs(...)) trick is repeated several times, can it be abstracted?

...

author maintainer
#5 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix generic __uninit kfunc output buffers

Quoted 20 lines · Eduard Zingerman

On Tue, 2026-09-15 at 16:10 +0200, Kumar Kartikeya Dwivedi wrote:

...

Quoted 16 lines

 2.3.4 __nullable Annotation
 ---------------------------

include/linux/bpf_verifier.h +0 -0
index cf85141ea167..9ddbb20ec1e9 100644
@@ -1547,12 +1547,13 @@ struct ref_obj_desc {
  };


 /*
- * A memory argument a call fills in. The verifier allows the stack to be uninitialized if
- * the range is a known constant. Stack slots are marked as STACK_MISC by check_mem_access().
+ * Memory arguments a call fills in, indexed by argument slot. The verifier allows the
+ * stack to be uninitialized if the range is a known constant. Stack slots are marked as
+ * STACK_MISC by check_mem_access() after all arguments have been checked.
  */

Quoted 9 lines

include/linux/bpf_verifier.h struct ref_obj_desc { · line 1555

Quoted 4 lines

 struct arg_raw_mem_desc {

include/linux/bpf_verifier.h +1 -2
-	u8 regno;
-	int size;
+	u16 mask;

Do we plan to add kfuncs with multiple __uninit parameters? If we do plan this, let's separate __uninit fix from the mask conversion. Otherwise let's keep regno.

I separated the patches. I don't think are any consumers yet but it makes to support multiple such arguments, so just keeping the separate patch. If you disagree feel free to drop it when applying.

Quoted 22 lines
Quoted 4 lines
include/linux/bpf_verifier.h +1 -0
+	int size[MAX_BPF_FUNC_ARGS];

 };

 /* Size of PTR_TO_MEM returned, taken from a constant allocation-size argument */

...

kernel/bpf/verifier.c static int check_stack_range_initialized( · line 6986

Quoted 12 lines
kernel/bpf/verifier.c +3 -1
index 6c6b8d8520cd..070ec5daeec3 100644
@@ -6981,7 +6981,9 @@ static int check_stack_range_initialized(
  	 */
  	bool allow_poison = access_size < 0 || clobber;
  	/* The call will initialize the memory; uninitialized stack allowed */
-	bool raw_mode = meta && meta->arg_raw_mem.regno == reg_from_argno(argno);
+	u32 arg_slot = abs(argno.argno) - 1;
+	bool raw_mode = meta && arg_slot < MAX_BPF_FUNC_ARGS &&
+		       (meta->arg_raw_mem.mask & BIT(arg_slot));

Thit BIT(abs(...)) trick is repeated several times, can it be abstracted?

Quoted 1 line

...

For the other email, I decided to keep __prepare_priv, we need to resolve module ksym kfuncs but test the unpriv case. It is a bit of a hack and nobody will do this in practice, but it helps us retain coverage for the other case.

Addressed all other comments and will respin.

author maintainer
#6 lore raw

PATCH 2/3 selftests/bpf: Allow privileged preparation for capability tests

In reply to Kumar Kartikeya Dwivedi

The annotation-driven loader supports retaining selected capabilities with __caps_unpriv, but drops the others before libbpf prepares the object. Resolving bpf_testmod kfuncs requires enumerating module BTF and obtaining its file descriptors through BPF_BTF_GET_NEXT_ID and BPF_BTF_GET_FD_BY_ID. Both operations require CAP_SYS_ADMIN, so dropping it at this point prevents the tests from reaching program verification.

Resolve the module references while privileged, then drop capabilities before loading the programs.

Add an opt-in __prepare_priv annotation that calls bpf_object__prepare() with the fixture's initial capabilities, then applies __caps_unpriv for program loading. Preparation also creates maps and loads BTF, so making this the default would stop existing unprivileged tests from checking those operations at reduced capabilities. Keep their preparation path unchanged. The existing pre-execution callback runs after program loading and is too late for this.

Run cases retaining CAP_BPF even when unprivileged BPF is disabled, since that sysctl does not prohibit their program loads.

---
 tools/testing/selftests/bpf/progs/bpf_misc.h |  9 ++++-
 tools/testing/selftests/bpf/test_loader.c    | 41 +++++++++++++-------
 2 files changed, 35 insertions(+), 15 deletions(-)
tools/testing/selftests/bpf/progs/bpf_misc.h +8 -1
index eb88d9ce6c34..2ced1d751ace 100644
@@ -9,7 +9,8 @@
 #define QUOTE(str) #str
 #define EXPAND_QUOTE(str) QUOTE(str)
 
-/* This set of attributes controls behavior of the
+/*
+ * This set of attributes controls behavior of the
  * test_loader.c:test_loader__run_subtests().
  *
  * The test_loader sequentially loads each program in a skeleton.
@@ -131,6 +132,11 @@
  *                   Several __arch_* annotations could be specified at once.
  *                   When test case is not run on current arch it is marked as skipped.
  * __caps_unpriv     Specify the capabilities that should be set when running the test.
+ * __prepare_priv    In unprivileged mode, prepare the object with the fixture's
+ *                   initial capabilities before dropping them for program loading.
+ *                   Preparation includes map creation and BTF/kfunc resolution;
+ *                   these operations are not tested at the reduced capabilities.
+ *                   Program loading uses the normal __caps_unpriv selection.
  *
  * __linear_size     Specify the size of the linear area of non-linear skbs, or
  *                   0 for linear skbs.
@@ -166,6 +172,7 @@
 #define __arch_s390x		__arch("s390x")
 #define __arch_loongarch	__arch("LOONGARCH")
 #define __caps_unpriv(caps)	__test_tag("test_caps_unpriv=" EXPAND_QUOTE(caps))
+#define __prepare_priv		__test_tag("test_prepare_priv")
 #define __load_if_JITed()	__test_tag("load_mode=jited")
 #define __load_if_no_JITed()	__test_tag("load_mode=no_jited")
 #define __stderr(msg)		__test_tag("test_expect_stderr=" msg)
tools/testing/selftests/bpf/test_loader.c +27 -14

85 lines not on this page · raw message

Signature · 1 line
2.53.0
#7 lore raw

Re: [PATCH bpf-next v1 2/3] selftests/bpf: Allow privileged preparation for capability tests

Is the CAP_BPF escape hatch safe when the sysctl is actually on but get_unpriv_disabled() returned true for a different reason?
Quoted 4 lines
tools/testing/selftests/bpf/test_loader.c +0 -0
index 28724de06322..940b6d8e47ba 100644

[ ... ]

tools/testing/selftests/bpf/test_loader.c static bool can_execute_unpriv(struct test_loader *tester, struct test_spec *spe · line 1065

Quoted 9 lines
tools/testing/selftests/bpf/test_loader.c +1 -1
@@ -1049,7 +1059,7 @@ static bool can_execute_unpriv(struct test_loader *tester, struct test_spec *spe
 {
 	if (sysctl_unpriv_disabled < 0)
 		sysctl_unpriv_disabled = get_unpriv_disabled() ? 1 : 0;
-	if (sysctl_unpriv_disabled)
+	if (sysctl_unpriv_disabled && !(spec->unpriv.caps & (1ULL << CAP_BPF)))
 		return false;
 	if ((spec->prog_flags & BPF_F_ANY_ALIGNMENT) && !EFFICIENT_UNALIGNED_ACCESS)
 		return false;

Is the CAP_BPF escape hatch safe when the sysctl is actually on but get_unpriv_disabled() returned true for a different reason?

Looking at tools/testing/selftests/bpf/unpriv_helpers.c:get_unpriv_disabled(), that function returns true for three different conditions:

    if (disabled)
        return true;
    /*
     * Some unpriv tests rely on spectre mitigations being on.
     * If mitigations are off or status can't be determined
     * assume that unpriv tests are disabled.
     */
    mitigations_off = get_mitigations_off();
    if (mitigations_off < 0) { ...; return true; }
    return mitigations_off;

So it returns true when kernel.unprivileged_bpf_disabled=1, but also when the kernel was booted with mitigations=off, when CONFIG_CPU_MITIGATIONS=n, or when the config cannot be read.

The commit message justifies the change with "since that sysctl does not prohibit their program loads", which is only accurate for the first condition. For the mitigations arm, retaining CAP_BPF does not restore the verifier's speculative-execution analysis, because bpf_bypass_spec_v1() and bpf_bypass_spec_v4() in include/linux/bpf.h return true on cpu_mitigations_off() regardless of capabilities - only CAP_PERFMON is capability-gated there, and these cases deliberately drop that.

For example, tools/testing/selftests/bpf/progs/verifier_loops1.c:292 carries __caps_unpriv(CAP_BPF) on maybe_exit_scc_bug1(), whose entire purpose is the speculative path:

/*
* This test case triggered a bug in verifier.c:maybe_exit_scc().
* Speculative execution path reaches stack access instruction,
* ...
*/

Before this patch that @unpriv subtest was skipped on a mitigations=off host. It now runs there, but with bypass_spec_v1 set the verifier never walks the speculative branch, so the case silently stops exercising the regression it was written for and reports a pass either way - which is exactly the situation get_unpriv_disabled()'s mitigations check was added to avoid.

Should the two conditions be distinguished (e.g. have unpriv_helpers.c expose the sysctl state separately from the mitigations state) rather than bypassing the combined flag? Or at minimum, should the changelog mention that mitigations-off hosts are also affected?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34981336633

author maintainer
#8 lore raw

PATCH 3/3 selftests/bpf: Cover generic __uninit kfunc output buffers

In reply to Kumar Kartikeya Dwivedi

Exercise scalar, scalar-only struct and sized output buffers, both with uninitialized stack storage and with a checkpoint that poisons prior stores. Run the accepted programs and check the values written by the kfuncs.

Use the annotation-driven loader to repeat cases with CAP_BPF and CAP_NET_ADMIN, but without CAP_PERFMON or CAP_SYS_ADMIN. Prepare module BTF before dropping capabilities with __prepare_priv so the cases reach the verifier. Cover stack-passed output pointers and outputs following by-value arguments in both capability modes.

Cover multiple outputs, an independent output next to a variable-size buffer, variable offsets, and write-only maps. Check that read-only maps, out-of-bounds outputs, uninitialized aliased inputs, and reads beyond the initialized range retain their expected rejection.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_kfunc_uninit.c         | 399 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  56 +++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |   9 +
 4 files changed, 466 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 7732df9bc870..4a9affe81ee1 100644
@@ -55,6 +55,7 @@
 #include "verifier_iterating_callbacks.skel.h"
 #include "verifier_jeq_infer_not_null.skel.h"
 #include "verifier_jit_convergence.skel.h"
+#include "verifier_kfunc_uninit.skel.h"
 #include "verifier_kfunc_packet_access.skel.h"
 #include "verifier_ld_ind.skel.h"
 #include "verifier_ldsx.skel.h"
@@ -221,6 +222,7 @@ void test_verifier_int_ptr(void)              { RUN(verifier_int_ptr); }
 void test_verifier_iterating_callbacks(void)  { RUN(verifier_iterating_callbacks); }
 void test_verifier_jeq_infer_not_null(void)   { RUN(verifier_jeq_infer_not_null); }
 void test_verifier_jit_convergence(void)      { RUN(verifier_jit_convergence); }
+void test_verifier_kfunc_uninit(void)         { RUN_TESTS(verifier_kfunc_uninit); }
 void test_verifier_kfunc_packet_access(void)  { RUN_TESTS(verifier_kfunc_packet_access); }
 void test_verifier_load_acquire(void)         { RUN(verifier_load_acquire); }
 void test_verifier_ld_ind(void)               { RUN(verifier_ld_ind); }
tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c new file +399 -0

399 lines not on this page · raw message

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +56 -0

68 lines not on this page · raw message

tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +9 -0
index d3696d5254c9..2ec8192a102f 100644
@@ -289,6 +289,15 @@ __u64 bpf_kfunc_call_stack_arg_big(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
 void bpf_kfunc_call_test_pass_ctx(struct __sk_buff *skb) __ksym;
 void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) __ksym;
 void bpf_kfunc_call_test_pass2(struct prog_test_pass2 *p) __ksym;
+void bpf_kfunc_test_uninit_scalar(int *out__uninit) __ksym;
+void bpf_kfunc_test_uninit_struct(struct prog_test_pass1 *out__uninit) __ksym;
+void bpf_kfunc_test_uninit_mem(void *out__uninit, __u32 out__sz) __ksym;
+void bpf_kfunc_test_uninit_mem_const(void *out__uninit, __u32 out__szk) __ksym;
+void bpf_kfunc_test_uninit_multi(int *a__uninit, void *b__uninit, __u32 b__sz) __ksym;
+int bpf_kfunc_test_uninit_alias(int *out__uninit, const int *in) __ksym;
+void bpf_kfunc_test_uninit_pair(struct prog_test_pair_arg p, int *out__uninit) __ksym;
+void bpf_kfunc_test_uninit_stack(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
+				 int *out__uninit) __ksym;
 void bpf_kfunc_call_test_mem_len_fail2(__u64 *mem, int len) __ksym;
 
 void bpf_kfunc_call_test_destructive(void) __ksym;
Signature · 1 line
2.53.0
#9 lore raw

Re: [PATCH bpf-next v1 3/3] selftests/bpf: Cover generic __uninit kfunc output buffers

Does the commit message accurately describe the read-only map case?
Quoted 4 lines

Cover multiple outputs, an independent output next to a variable-size buffer, variable offsets, and write-only maps. Check that read-only maps, out-of-bounds outputs, uninitialized aliased inputs, and reads beyond the initialized range retain their expected rejection.

Does the commit message accurately describe the read-only map case?

The description states all four cases "retain their expected rejection", but read_only_map_output() tests a rejection that is newly introduced by the preceding fix 877a8f574a04, not one that is retained.

Before 877a8f574a04, check_func_arg() computed the access type for a generic memory argument as:

  if (...)
      access_type = BPF_READ | BPF_WRITE;

and 877a8f574a04 changed it to:

  access_type = arg_type & MEM_UNINIT ? BPF_WRITE : BPF_READ | BPF_WRITE;

check_map_access_type() in kernel/bpf/verifier.c tests with strict equality:

  if (type == BPF_WRITE && !(cap & BPF_MAP_CAN_WRITE)) {
      verbose(env, "write into map forbidden, ...");

With the pre-fix value BPF_READ|BPF_WRITE (== 3) neither branch fires, so passing a pointer into a BPF_F_RDONLY_PROG map (the .rodata map holding 'static const int read_only_output') as an __uninit output was accepted before the fix.

The __failure __msg("write into map forbidden") expectation therefore only holds after the fix; describing it as "retained" misstates what the new test covers (it is a new-behaviour test, arguably the most interesting user-visible tightening in the series).

[ ... ]

Quoted 5 lines
tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c new file +0 -0
new file mode 100644
index 000000000000..99c0ee562ea0

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c new file +6 -0
+SEC("tc")
+__success __retval(7)
+__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
+__prepare_priv
+__success_unpriv
+__naked void initialized_input_alias(void)

tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c line 288

Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c new file +10 -0
+{
+	asm volatile (
+		"*(u64 *)(r10 - 8) = 7;"
+		"r1 = r10;"
+		"r1 += -8;"
+		"r2 = r1;"
+		"call %[bpf_kfunc_test_uninit_alias];"
+		"exit;"
+		: : __imm(bpf_kfunc_test_uninit_alias) : __clobber_all);
+}

Will this test pass on big-endian targets?

initialized_input_alias() is the only test in this file whose expected return value depends on host endianness, so it will fail on big-endian targets (s390x, which is a supported BPF selftest arch - see tools/testing/selftests/bpf/DENYLIST.s390x, and whose JIT supports kfunc calls via arch/s390/net/bpf_jit_comp.c bpf_jit_supports_kfunc_call()).

The program stores an 8-byte value and then has the kfunc read only 4 bytes from the low address of that store:

  "*(u64 *)(r10 - 8) = 7;"   /* 8-byte store */
  "r1 = r10;" "r1 += -8;"    /* out__uninit = r10-8 */
  "r2 = r1;"                 /* in          = r10-8 (aliases out) */

"call %[bpf_kfunc_test_uninit_alias];"
"exit;"

The kfunc in tools/testing/selftests/bpf/test_kmods/bpf_testmod.c is:

  __bpf_kfunc int bpf_kfunc_test_uninit_alias(int *out__uninit, const int *in)
  {
      int value = get_unaligned(in);
      put_unaligned(42, out__uninit);
      return value;
  }

so it does a native-endian 4-byte load at r10-8 and returns it in r0, which is the program's return value.

On little-endian the bytes at r10-8..r10-5 are 07 00 00 00, so value == 7, matching __retval(7). On big-endian the u64 7 is laid out as
00 00 00 00 00 00 00 07, so the 4 bytes at r10-8 are 00 00 00 00, value == 0, and __retval(7) fails.

Every other executed test in the file is endian-neutral because it reads back exactly the int-sized/byte-pattern values the kfunc wrote. Only this test relies on the layout of a wider store.

The existing convention in this directory is to guard exactly this pattern, e.g. tools/testing/selftests/bpf/progs/verifier_spill_fill.c:

  #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
      "r4 = *(u16*)(r10 - 8);"
  #else
      "r4 = *(u16*)(r10 - 6);"
  #endif

Since the kfunc's 'in' argument is a const int * (4-byte fixed-size ARG_PTR_TO_MEM), the simplest endian-neutral fix is to initialize the input with a 4-byte store instead: "*(u32 *)(r10 - 8) = 7;". That still satisfies the input read check at r10-8..r10-5 and keeps the input/output aliasing that the test is designed to exercise.

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34981336633

maintainer
#10 lore raw

Re: [PATCH bpf-next v1 3/3] selftests/bpf: Cover generic __uninit kfunc output buffers

In reply to Kumar Kartikeya Dwivedi
Quoted 10 lines · Kumar Kartikeya Dwivedi

Exercise scalar, scalar-only struct and sized output buffers, both with uninitialized stack storage and with a checkpoint that poisons prior stores. Run the accepted programs and check the values written by the kfuncs.

Use the annotation-driven loader to repeat cases with CAP_BPF and CAP_NET_ADMIN, but without CAP_PERFMON or CAP_SYS_ADMIN. Prepare module BTF before dropping capabilities with __prepare_priv so the cases reach the verifier. Cover stack-passed output pointers and outputs following by-value arguments in both capability modes.

Quoted 7 lines

Cover multiple outputs, an independent output next to a variable-size buffer, variable offsets, and write-only maps. Check that read-only maps, out-of-bounds outputs, uninitialized aliased inputs, and reads beyond the initialized range retain their expected rejection.

---

I think this is a bit over-enthusiastic on adding more tests. For the actual fix only the following tests are necessary:
- struct_poisoned_at_checkpoint
- sized_buffer_poisoned_at_checkpoint
- uninitialized_input_alias

Also __prepare_priv is kind of a hack.
It works now, but I don't think libbpf was designed with an idea of dropping capabilities mid-flight in mind.

...

author maintainer
#11 lore raw
In reply to Kumar Kartikeya Dwivedi

+Cc Amery, sorry for missing it.

Quoted 25 lines · Kumar Kartikeya Dwivedi

Generic __uninit kfunc arguments are output buffers. Their stack liveness already treats them as writes, but argument checking still requires readable contents and does not record definite initialization after the call. Check these arguments as write-only and reuse helper output-buffer tracking, with one entry per argument slot.

Kumar Kartikeya Dwivedi (3):
bpf: Fix generic __uninit kfunc output buffers selftests/bpf: Allow privileged preparation for capability tests selftests/bpf: Cover generic __uninit kfunc output buffers

 Documentation/bpf/kfuncs.rst                  |  27 +-
 include/linux/bpf_verifier.h                  |  11 +-
 kernel/bpf/verifier.c                         |  70 ++-
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 tools/testing/selftests/bpf/progs/bpf_misc.h  |   9 +-
 .../bpf/progs/verifier_kfunc_uninit.c         | 399 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  56 +++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |   9 +
 tools/testing/selftests/bpf/test_loader.c     |  41 +-
 9 files changed, 577 insertions(+), 47 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c

base-commit: 5ef40d69b38a93bc9951dadb1a15c85c597e1a40