Skip to contentSkip to conversation

bpf: Scope callback arguments to their frame

PATCH Superseded Ihor Solodrai 4 participants 18 messages Oct 9 – Oct 29

Subsystems (4)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Ihor Solodrai

author reviewer
#0 lore raw

v2 cover bpf: Scope callback arguments to their frame

A callback-calling helper or kfunc can pass its callback a pointer that is only valid for the callback invocation, and the verifier has no way to express that lifetime.

Two helpers need fixing:

* bpf_user_ringbuf_drain() passes a CONST_PTR_TO_DYNPTR over a sample it releases as soon as the callback returns. Saving this pointer in callback_ctx leaves it pointing into reused kernel stack and allows arbitrary kernel read and write (bpf-next only, see patch #8).

* bpf_for_each_map_elem() over an array or per-CPU array map passes the address of a u32 held in bpf_for_each_array_elem()'s own frame, leaking four bytes of kernel stack.

Both arguments point into a helper's own frame, with nothing longer-lived to anchor them to. Rejecting spills of the dynptr pointer alone would still allow derived slices and clones to escape.

Introduce REF_TYPE_FRAME for this use case: a reference owned by a callee frame, dropped when the frame is popped. Tie each argument to this reference so callback return invalidates it and its descendants.

Callback setters repeat register defaults and map metadata setup, while reference release couples descendant invalidation to object release. The refactoring patches give common initialization one owner and separate invalidation from reference removal. Frame lifetime tracking can then reuse the existing reference machinery.

---

Patches #1-6 are non-functional preparation. Patch #7 introduces REF_TYPE_FRAME and its diagnostics; #8 and #10 apply it to the helpers, and #9 and #11 add selftests.

v1->v2:
* Add the refactoring patches: release frame references in one scan, and fold diagnostics into the REF_TYPE_FRAME introduction
* Replace temporary argument mask and deferred register scan with reference acquisition directly in the owning verifier state during callback setup (Eduard)
* Reuse callback_park_dynptr() for the nested drain test

v1: https://lore.kernel.org/r/[email protected]

---

Ihor Solodrai (11):
bpf: Set up callee state outside of setup_func_entry()
bpf: Pass the owning verifier state to callback setters
bpf: Append complete reference states
bpf: Separate reference removal from descendant invalidation
bpf: Share map-backed callback register setup
bpf: Rely on callback frame initialization defaults
bpf: Introduce REF_TYPE_FRAME in the verifier
bpf: Scope the bpf_user_ringbuf_drain() dynptr to its callback frame selftests/bpf: Cover the user ringbuf callback dynptr lifetime
bpf: Scope the bpf_for_each_map_elem() array key to the callback frame selftests/bpf: Cover callback-frame map key lifetime

 include/linux/bpf.h                           |   5 +
 include/linux/bpf_verifier.h                  |   6 +-
 kernel/bpf/arraymap.c                         |  18 +-
 kernel/bpf/diagnostics.c                      |   3 +
 kernel/bpf/diagnostics.h                      |   1 +
 kernel/bpf/states.c                           |   4 +
 kernel/bpf/verifier.c                         | 392 +++++++++---------
 .../selftests/bpf/progs/exceptions_fail.c     |  20 +
 .../selftests/bpf/progs/user_ringbuf_fail.c   | 135 ++++++
 .../bpf/progs/verifier_iterating_callbacks.c  |  91 ++++
 10 files changed, 476 insertions(+), 199 deletions(-)

base-commit: e1d84a37cba984388988d2f1ddc84561413f0db2

Signature · 1 line
2.56.0
author reviewer
#1 lore raw

PATCH v2 1/11 bpf: Set up callee state outside of setup_func_entry()

Callback argument setup may need to attach bookkeeping to the verifier state owning the callee frame. The frame should be current in that state before the arguments are set up.

For sync callbacks, the setter runs inside setup_func_entry() before the new frame becomes current. Async callback setters already run on a complete state, directly from push_callback_call().

Move sync callback argument setup to push_callback_call() after the frame is in place. Leave setup_func_entry() responsible for pushing the frame, and copy static-call arguments directly.

No functional change.

---
 kernel/bpf/verifier.c | 69 +++++++++++++------------------------------
 1 file changed, 20 insertions(+), 49 deletions(-)
kernel/bpf/verifier.c +20 -49
index 353bde9ae227..a0310e093880 100644
@@ -10821,32 +10821,25 @@ typedef int (*set_callee_state_fn)(struct bpf_verifier_env *env,
 				   struct bpf_func_state *callee,
 				   int insn_idx);
 
-static int set_callee_state(struct bpf_verifier_env *env,
-			    struct bpf_func_state *caller,
-			    struct bpf_func_state *callee, int insn_idx);
-
-static int setup_func_entry(struct bpf_verifier_env *env, int subprog, int callsite,
-			    set_callee_state_fn set_callee_state_cb,
-			    struct bpf_verifier_state *state)
+static struct bpf_func_state *setup_func_entry(struct bpf_verifier_env *env, int subprog,
+					       int callsite, struct bpf_verifier_state *state)
 {
-	struct bpf_func_state *caller, *callee;
-	int err;
+	struct bpf_func_state *callee;
 
 	if (state->curframe + 1 >= MAX_CALL_FRAMES) {
 		verbose(env, "the call stack of %d frames is too deep\n",
 			state->curframe + 2);
-		return -E2BIG;
+		return ERR_PTR(-E2BIG);
 	}
 
 	if (state->frame[state->curframe + 1]) {
 		verifier_bug(env, "Frame %d already allocated", state->curframe + 1);
-		return -EFAULT;
+		return ERR_PTR(-EFAULT);
 	}
 
-	caller = state->frame[state->curframe];
 	callee = kzalloc_obj(*callee, GFP_KERNEL_ACCOUNT);
 	if (!callee)
-		return -ENOMEM;
+		return ERR_PTR(-ENOMEM);
 	state->frame[state->curframe + 1] = callee;
 
 	/* callee cannot access r0, r6 - r9 for reading and has to write
@@ -10858,19 +10851,9 @@ static int setup_func_entry(struct bpf_verifier_env *env, int subprog, int calls
 			callsite,
 			state->curframe + 1 /* frameno within this callchain */,
 			subprog /* subprog number within this prog */);
-	err = set_callee_state_cb(env, caller, callee, callsite);
-	if (err)
-		goto err_out;
-
-	/* only increment it after check_reg_arg() finished */
 	state->curframe++;
 
-	return 0;
-
-err_out:
-	free_func_state(callee);
-	state->frame[state->curframe + 1] = NULL;
-	return err;
+	return callee;
 }
 
 static void gen_subprog_arg_proto(const struct bpf_subprog_info *sub, const struct btf *btf,
@@ -10990,10 +10973,6 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins
 	if (err == -EFAULT)
 		return err;
 
-	/* set_callee_state is used for direct subprog calls, but we are
-	 * interested in validating only BPF helpers that can call subprogs as
-	 * callbacks
-	 */
 	env->subprog_info[subprog].is_cb = true;
 	if (bpf_pseudo_kfunc_call(insn) &&
 	    !is_callback_calling_kfunc(insn->imm)) {
@@ -11044,8 +11023,11 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins
 	if (IS_ERR(callback_state))
 		return PTR_ERR(callback_state);
 
-	err = setup_func_entry(env, subprog, insn_idx, set_callee_state_cb,
-			       callback_state);
+	callee = setup_func_entry(env, subprog, insn_idx, callback_state);
+	if (IS_ERR(callee))
+		return PTR_ERR(callee);
+
+	err = set_callee_state_cb(env, caller, callee, insn_idx);
 	if (err)
 		return err;
 
@@ -11069,8 +11051,8 @@ static int check_static_func_call(struct bpf_verifier_env *env, int subprog,
 	struct bpf_verifier_state *state = env->cur_state;
 	struct bpf_subprog_info *caller_info;
 	u16 callee_incoming, stack_arg_cnt;
-	struct bpf_func_state *caller;
-	int err;
+	struct bpf_func_state *caller, *callee;
+	int i;
 
 	caller = state->frame[state->curframe];
 
@@ -11088,9 +11070,12 @@ static int check_static_func_call(struct bpf_verifier_env *env, int subprog,
 	 * For regular function entry setup new frame and continue
 	 * from that frame.
 	 */
-	err = setup_func_entry(env, subprog, *insn_idx, set_callee_state, state);
-	if (err)
-		return err;
+	callee = setup_func_entry(env, subprog, *insn_idx, state);
+	if (IS_ERR(callee))
+		return PTR_ERR(callee);
+
+	for (i = BPF_REG_1; i <= BPF_REG_5; i++)
+		callee->regs[i] = caller->regs[i];
 
 	bpf_diag_record_scrub(env, &caller->regs[BPF_REG_0], BPF_DIAG_MOD_CALLER_SAVED);
 	clear_caller_saved_regs(env, caller->regs);
@@ -11301,20 +11286,6 @@ int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 	return 0;
 }
 
-static int set_callee_state(struct bpf_verifier_env *env,
-			    struct bpf_func_state *caller,
-			    struct bpf_func_state *callee, int insn_idx)
-{
-	int i;
-
-	/* copy r1 - r5 args that callee can access.  The copy includes parent
-	 * pointers, which connects us up to the liveness chain
-	 */
-	for (i = BPF_REG_1; i <= BPF_REG_5; i++)
-		callee->regs[i] = caller->regs[i];
-	return 0;
-}
-
 static int set_map_elem_callback_state(struct bpf_verifier_env *env,
 				       struct bpf_func_state *caller,
 				       struct bpf_func_state *callee,
Signature · 1 line
2.56.0
author reviewer
#2 lore raw

PATCH v2 2/11 bpf: Pass the owning verifier state to callback setters

In reply to Ihor Solodrai

Callback setters receive the caller and callee frames, but not the verifier state that owns the callee. Synchronous callbacks use a queued state and asynchronous callbacks use a separate state, neither of which is env->cur_state.

Callback-local bookkeeping needs to be recorded in that owning state. Pass it through the callback setters and the map callback setup hook.

No functional change.

---
 include/linux/bpf.h   |  3 +++
 kernel/bpf/verifier.c | 16 +++++++++++++---
 2 files changed, 16 insertions(+), 3 deletions(-)
include/linux/bpf.h +3 -0
index 54144372281c..d5de9d49a168 100644
@@ -39,6 +39,7 @@
 #include <asm/rqspinlock.h>
 
 struct bpf_verifier_env;
+struct bpf_verifier_state;
 struct bpf_verifier_log;
 struct perf_event;
 struct bpf_prog;
@@ -178,6 +179,7 @@ struct bpf_map_ops {
 
 
 	int (*map_set_for_each_callback_args)(struct bpf_verifier_env *env,
+					      struct bpf_verifier_state *state,
 					      struct bpf_func_state *caller,
 					      struct bpf_func_state *callee);
 	long (*map_for_each_callback)(struct bpf_map *map,
@@ -3183,6 +3185,7 @@ int bpf_iter_map_fill_link_info(const struct bpf_iter_aux_info *aux,
 				struct bpf_link_info *info);
 
 int map_set_for_each_callback_args(struct bpf_verifier_env *env,
+				   struct bpf_verifier_state *state,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee);
 
kernel/bpf/verifier.c +13 -3
index a0310e093880..8d6812fee0c9 100644
@@ -10817,6 +10817,7 @@ static void invalidate_outgoing_stack_args(struct bpf_verifier_env *env,
 }
 
 typedef int (*set_callee_state_fn)(struct bpf_verifier_env *env,
+				   struct bpf_verifier_state *state,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee,
 				   int insn_idx);
@@ -11000,7 +11001,7 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins
 		callee->async_entry_cnt = state->frame[0]->async_entry_cnt + 1;
 
 		/* Convert bpf_timer_set_callback() args into timer callback args */
-		err = set_callee_state_cb(env, caller, callee, insn_idx);
+		err = set_callee_state_cb(env, async_cb, caller, callee, insn_idx);
 		if (err)
 			return err;
 
@@ -11027,7 +11028,7 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins
 	if (IS_ERR(callee))
 		return PTR_ERR(callee);
 
-	err = set_callee_state_cb(env, caller, callee, insn_idx);
+	err = set_callee_state_cb(env, callback_state, caller, callee, insn_idx);
 	if (err)
 		return err;
 
@@ -11257,6 +11258,7 @@ static int check_func_callx(struct bpf_verifier_env *env, struct bpf_insn *insn,
 }
 
 int map_set_for_each_callback_args(struct bpf_verifier_env *env,
+				   struct bpf_verifier_state *state,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee)
 {
@@ -11287,6 +11289,7 @@ int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 }
 
 static int set_map_elem_callback_state(struct bpf_verifier_env *env,
+				       struct bpf_verifier_state *state,
 				       struct bpf_func_state *caller,
 				       struct bpf_func_state *callee,
 				       int insn_idx)
@@ -11303,7 +11306,7 @@ static int set_map_elem_callback_state(struct bpf_verifier_env *env,
 		return -ENOTSUPP;
 	}
 
-	err = map->ops->map_set_for_each_callback_args(env, caller, callee);
+	err = map->ops->map_set_for_each_callback_args(env, state, caller, callee);
 	if (err)
 		return err;
 
@@ -11313,6 +11316,7 @@ static int set_map_elem_callback_state(struct bpf_verifier_env *env,
 }
 
 static int set_loop_callback_state(struct bpf_verifier_env *env,
+				   struct bpf_verifier_state *state,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee,
 				   int insn_idx)
@@ -11335,6 +11339,7 @@ static int set_loop_callback_state(struct bpf_verifier_env *env,
 }
 
 static int set_timer_callback_state(struct bpf_verifier_env *env,
+				    struct bpf_verifier_state *state,
 				    struct bpf_func_state *caller,
 				    struct bpf_func_state *callee,
 				    int insn_idx)
@@ -11370,6 +11375,7 @@ static int set_timer_callback_state(struct bpf_verifier_env *env,
 }
 
 static int set_find_vma_callback_state(struct bpf_verifier_env *env,
+				       struct bpf_verifier_state *state,
 				       struct bpf_func_state *caller,
 				       struct bpf_func_state *callee,
 				       int insn_idx)
@@ -11398,6 +11404,7 @@ static int set_find_vma_callback_state(struct bpf_verifier_env *env,
 }
 
 static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env,
+					   struct bpf_verifier_state *state,
 					   struct bpf_func_state *caller,
 					   struct bpf_func_state *callee,
 					   int insn_idx)
@@ -11421,6 +11428,7 @@ static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env,
 }
 
 static int set_rbtree_add_callback_state(struct bpf_verifier_env *env,
+					 struct bpf_verifier_state *state,
 					 struct bpf_func_state *caller,
 					 struct bpf_func_state *callee,
 					 int insn_idx)
@@ -11454,6 +11462,7 @@ static int set_rbtree_add_callback_state(struct bpf_verifier_env *env,
 }
 
 static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
+						 struct bpf_verifier_state *state,
 						 struct bpf_func_state *caller,
 						 struct bpf_func_state *callee,
 						 int insn_idx)
@@ -11489,6 +11498,7 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
 }
 
 static int set_rcu_callback_state(struct bpf_verifier_env *env,
+				  struct bpf_verifier_state *state,
 				  struct bpf_func_state *caller,
 				  struct bpf_func_state *callee,
 				  int insn_idx)
Signature · 1 line
2.56.0
author reviewer
#3 lore raw

PATCH v2 3/11 bpf: Append complete reference states

In reply to Ihor Solodrai

Reference acquisition exposes an uninitialized entry for each caller to fill after insertion.

Accept a complete reference entry and return its id after insertion. Keep lock and IRQ accounting and acquisition diagnostics in their callers. Take the target state directly so acquisition can also serve queued callback states.

No functional change.

---
 kernel/bpf/verifier.c | 73 +++++++++++++++++++++++--------------------
 1 file changed, 39 insertions(+), 34 deletions(-)
kernel/bpf/verifier.c +39 -34
index 8d6812fee0c9..62b991f3c39a 100644
@@ -1497,51 +1497,53 @@ static int grow_stack_arg_slots(struct bpf_verifier_env *env,
 	return 0;
 }
 
-/* Acquire a pointer id from the env and update the state->refs to include
- * this new pointer reference.
- * On success, returns a valid pointer id to associate with the register
- * On failure, returns a negative errno.
+/*
+ * Append @ref to @state->refs. Return its id, or a negative errno.
  */
-static struct bpf_reference_state *acquire_reference_state(struct bpf_verifier_env *env, int insn_idx)
+static int acquire_reference_state(struct bpf_verifier_state *state,
+				   const struct bpf_reference_state *ref)
 {
-	struct bpf_verifier_state *state = env->cur_state;
 	int new_ofs = state->acquired_refs;
 	int err;
 
 	err = resize_reference_state(state, state->acquired_refs + 1);
 	if (err)
-		return NULL;
-	state->refs[new_ofs].insn_idx = insn_idx;
-
-	return &state->refs[new_ofs];
+		return err;
+	state->refs[new_ofs] = *ref;
+	return ref->id;
 }
 
 static int acquire_reference(struct bpf_verifier_env *env, int insn_idx, int parent_id)
 {
-	struct bpf_reference_state *s;
+	struct bpf_reference_state ref = {
+		.type = REF_TYPE_PTR,
+		.id = ++env->id_gen,
+		.insn_idx = insn_idx,
+		.parent_id = parent_id,
+	};
+	int id;
 
-	s = acquire_reference_state(env, insn_idx);
-	if (!s)
-		return -ENOMEM;
-	s->type = REF_TYPE_PTR;
-	s->id = ++env->id_gen;
-	s->parent_id = parent_id;
-	bpf_diag_record_ref_acquire(env, insn_idx, s->id);
-	return s->id;
+	id = acquire_reference_state(env->cur_state, &ref);
+	if (id >= 0)
+		bpf_diag_record_ref_acquire(env, insn_idx, id);
+	return id;
 }
 
 static int acquire_lock_state(struct bpf_verifier_env *env, int insn_idx, enum ref_state_type type,
 			      int id, void *ptr)
 {
 	struct bpf_verifier_state *state = env->cur_state;
-	struct bpf_reference_state *s;
+	struct bpf_reference_state ref = {
+		.type = type,
+		.id = id,
+		.insn_idx = insn_idx,
+		.ptr = ptr,
+	};
+	int err;
 
-	s = acquire_reference_state(env, insn_idx);
-	if (!s)
-		return -ENOMEM;
-	s->type = type;
-	s->id = id;
-	s->ptr = ptr;
+	err = acquire_reference_state(state, &ref);
+	if (err < 0)
+		return err;
 
 	state->active_locks++;
 	state->active_lock_id = id;
@@ -1554,18 +1556,21 @@ static int acquire_lock_state(struct bpf_verifier_env *env, int insn_idx, enum r
 static int acquire_irq_state(struct bpf_verifier_env *env, int insn_idx)
 {
 	struct bpf_verifier_state *state = env->cur_state;
-	struct bpf_reference_state *s;
+	struct bpf_reference_state ref = {
+		.type = REF_TYPE_IRQ,
+		.id = ++env->id_gen,
+		.insn_idx = insn_idx,
+	};
+	int id;
 
-	s = acquire_reference_state(env, insn_idx);
-	if (!s)
-		return -ENOMEM;
-	s->type = REF_TYPE_IRQ;
-	s->id = ++env->id_gen;
+	id = acquire_reference_state(state, &ref);
+	if (id < 0)
+		return id;
 
-	state->active_irq_id = s->id;
+	state->active_irq_id = id;
 	bpf_diag_record_context(env, insn_idx, BPF_DIAG_CONTEXT_IRQ, true,
 				bpf_diag_irq_depth(state));
-	return s->id;
+	return id;
 }
 
 static void release_reference_state(struct bpf_verifier_state *state, int idx)
Signature · 1 line
2.56.0
author reviewer
#4 lore raw

PATCH v2 4/11 bpf: Separate reference removal from descendant invalidation

In reply to Ihor Solodrai

Reference release removes a program-owned reference and invalidates all values derived from it in one operation.

Frame-owned values will need the same descendant walk without object release semantics. Separate the walk from reference removal and pass its diagnostic reason explicitly. Keep object-reference removal and the existing release reason in release_reference().

No functional change.

---
 kernel/bpf/verifier.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
kernel/bpf/verifier.c +17 -12
index 62b991f3c39a..88b86f658e4a 100644
@@ -10668,8 +10668,9 @@ static int idstack_pop(struct bpf_idmap *idmap)
 	return idmap->map[--idmap->cnt].old;
 }
 
-/* Release id and objects derived from it iteratively in a DFS manner */
-static int release_reference(struct bpf_verifier_env *env, int id)
+/* Invalidate id and values derived from it iteratively in a DFS manner. */
+static int invalidate_reference(struct bpf_verifier_env *env, int id,
+				enum bpf_diag_mod_reason reason)
 {
 	u32 mask = (1 << STACK_SPILL) | (1 << STACK_DYNPTR);
 	struct bpf_verifier_state *vstate = env->cur_state;
@@ -10684,11 +10685,6 @@ static int release_reference(struct bpf_verifier_env *env, int id)
 	if (err)
 		return err;
 
-	if (find_reference_state(vstate, id)) {
-		err = release_reference_nomark(env, id);
-		WARN_ON_ONCE(err);
-	}
-
 	while ((id = idstack_pop(idstack))) {
 		/*
 		 * Child references are inaccessible after parent is released,
@@ -10724,14 +10720,12 @@ static int release_reference(struct bpf_verifier_env *env, int id)
 
 				if (reg->dynptr.first_slot)
 					dyn_stack--;
-				bpf_diag_record_scrub(env, &dyn_stack[0].spilled_ptr,
-						      BPF_DIAG_MOD_REF_RELEASE);
-				bpf_diag_record_scrub(env, &dyn_stack[1].spilled_ptr,
-						      BPF_DIAG_MOD_REF_RELEASE);
+				bpf_diag_record_scrub(env, &dyn_stack[0].spilled_ptr, reason);
+				bpf_diag_record_scrub(env, &dyn_stack[1].spilled_ptr, reason);
 				invalidate_dynptr(env, dyn_stack);
 				continue;
 			}
-			bpf_diag_record_scrub(env, reg, BPF_DIAG_MOD_REF_RELEASE);
+			bpf_diag_record_scrub(env, reg, reason);
 			if (!stack || stack->slot_type[BPF_REG_SIZE - 1] == STACK_SPILL)
 				mark_reg_invalid(env, reg);
 		}));
@@ -10740,6 +10734,17 @@ static int release_reference(struct bpf_verifier_env *env, int id)
 	return 0;
 }
 
+static int release_reference(struct bpf_verifier_env *env, int id)
+{
+	int err;
+
+	if (find_reference_state(env->cur_state, id)) {
+		err = release_reference_nomark(env, id);
+		WARN_ON_ONCE(err);
+	}
+	return invalidate_reference(env, id, BPF_DIAG_MOD_REF_RELEASE);
+}
+
 static void invalidate_non_owning_refs(struct bpf_verifier_env *env)
 {
 	struct bpf_func_state *unused;
Signature · 1 line
2.56.0
author reviewer
#5 lore raw

PATCH v2 5/11 bpf: Share map-backed callback register setup

In reply to Ihor Solodrai

Map iteration, timer, task-work and RCU callbacks repeat the same register type, offset and map-metadata setup for their map, key and value arguments.

Use one initializer for map-backed callback registers. Leave argument selection, value ids and callback-specific policy in each setter.

No functional change.

---
 kernel/bpf/verifier.c | 81 +++++++++++--------------------------------
 1 file changed, 21 insertions(+), 60 deletions(-)
kernel/bpf/verifier.c +21 -60
index 88b86f658e4a..8f2125c6e348 100644
@@ -11267,6 +11267,16 @@ static int check_func_callx(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	return check_static_func_call(env, subprog, insn_idx);
 }
 
+static void mark_map_callback_reg(struct bpf_reg_state *reg,
+				  const struct bpf_reg_state *map_reg,
+				  enum bpf_reg_type type)
+{
+	reg->type = type;
+	__mark_reg_known_zero(reg);
+	reg->map_ptr = map_reg->map_ptr;
+	reg->map_uid = map_reg->map_uid;
+}
+
 int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 				   struct bpf_verifier_state *state,
 				   struct bpf_func_state *caller,
@@ -11279,15 +11289,8 @@ int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 	 */
 	callee->regs[BPF_REG_1] = caller->regs[BPF_REG_1];
 
-	callee->regs[BPF_REG_2].type = PTR_TO_MAP_KEY;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_2]);
-	callee->regs[BPF_REG_2].map_ptr = caller->regs[BPF_REG_1].map_ptr;
-	callee->regs[BPF_REG_2].map_uid = caller->regs[BPF_REG_1].map_uid;
-
-	callee->regs[BPF_REG_3].type = PTR_TO_MAP_VALUE;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_3]);
-	callee->regs[BPF_REG_3].map_ptr = caller->regs[BPF_REG_1].map_ptr;
-	callee->regs[BPF_REG_3].map_uid = caller->regs[BPF_REG_1].map_uid;
+	mark_map_callback_reg(&callee->regs[BPF_REG_2], &caller->regs[BPF_REG_1], PTR_TO_MAP_KEY);
+	mark_map_callback_reg(&callee->regs[BPF_REG_3], &caller->regs[BPF_REG_1], PTR_TO_MAP_VALUE);
 	callee->regs[BPF_REG_3].id = ++env->id_gen;
 
 	/* pointer to stack or null */
@@ -11354,26 +11357,12 @@ static int set_timer_callback_state(struct bpf_verifier_env *env,
 				    struct bpf_func_state *callee,
 				    int insn_idx)
 {
-	struct bpf_map *map_ptr = caller->regs[BPF_REG_1].map_ptr;
-	u32 map_uid = caller->regs[BPF_REG_1].map_uid;
-
 	/* bpf_timer_set_callback(struct bpf_timer *timer, void *callback_fn);
 	 * callback_fn(struct bpf_map *map, void *key, void *value);
 	 */
-	callee->regs[BPF_REG_1].type = CONST_PTR_TO_MAP;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_1]);
-	callee->regs[BPF_REG_1].map_ptr = map_ptr;
-	callee->regs[BPF_REG_1].map_uid = map_uid;
-
-	callee->regs[BPF_REG_2].type = PTR_TO_MAP_KEY;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_2]);
-	callee->regs[BPF_REG_2].map_ptr = map_ptr;
-	callee->regs[BPF_REG_2].map_uid = map_uid;
-
-	callee->regs[BPF_REG_3].type = PTR_TO_MAP_VALUE;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_3]);
-	callee->regs[BPF_REG_3].map_ptr = map_ptr;
-	callee->regs[BPF_REG_3].map_uid = map_uid;
+	mark_map_callback_reg(&callee->regs[BPF_REG_1], &caller->regs[BPF_REG_1], CONST_PTR_TO_MAP);
+	mark_map_callback_reg(&callee->regs[BPF_REG_2], &caller->regs[BPF_REG_1], PTR_TO_MAP_KEY);
+	mark_map_callback_reg(&callee->regs[BPF_REG_3], &caller->regs[BPF_REG_1], PTR_TO_MAP_VALUE);
 	callee->regs[BPF_REG_3].id = ++env->id_gen;
 
 	/* unused */
@@ -11477,26 +11466,12 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
 						 struct bpf_func_state *callee,
 						 int insn_idx)
 {
-	struct bpf_map *map_ptr = caller->regs[BPF_REG_3].map_ptr;
-	u32 map_uid = caller->regs[BPF_REG_3].map_uid;
-
 	/*
 	 * callback_fn(struct bpf_map *map, void *key, void *value);
 	 */
-	callee->regs[BPF_REG_1].type = CONST_PTR_TO_MAP;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_1]);
-	callee->regs[BPF_REG_1].map_ptr = map_ptr;
-	callee->regs[BPF_REG_1].map_uid = map_uid;
-
-	callee->regs[BPF_REG_2].type = PTR_TO_MAP_KEY;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_2]);
-	callee->regs[BPF_REG_2].map_ptr = map_ptr;
-	callee->regs[BPF_REG_2].map_uid = map_uid;
-
-	callee->regs[BPF_REG_3].type = PTR_TO_MAP_VALUE;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_3]);
-	callee->regs[BPF_REG_3].map_ptr = map_ptr;
-	callee->regs[BPF_REG_3].map_uid = map_uid;
+	mark_map_callback_reg(&callee->regs[BPF_REG_1], &caller->regs[BPF_REG_3], CONST_PTR_TO_MAP);
+	mark_map_callback_reg(&callee->regs[BPF_REG_2], &caller->regs[BPF_REG_3], PTR_TO_MAP_KEY);
+	mark_map_callback_reg(&callee->regs[BPF_REG_3], &caller->regs[BPF_REG_3], PTR_TO_MAP_VALUE);
 	callee->regs[BPF_REG_3].id = ++env->id_gen;
 
 	/* unused */
@@ -11513,26 +11488,12 @@ static int set_rcu_callback_state(struct bpf_verifier_env *env,
 				  struct bpf_func_state *callee,
 				  int insn_idx)
 {
-	struct bpf_map *map_ptr = caller->regs[BPF_REG_2].map_ptr;
-	u32 map_uid = caller->regs[BPF_REG_2].map_uid;
-
 	/*
 	 * callback_fn(struct bpf_map *map, void *key, void *value);
 	 */
-	callee->regs[BPF_REG_1].type = CONST_PTR_TO_MAP;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_1]);
-	callee->regs[BPF_REG_1].map_ptr = map_ptr;
-	callee->regs[BPF_REG_1].map_uid = map_uid;
-
-	callee->regs[BPF_REG_2].type = PTR_TO_MAP_KEY;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_2]);
-	callee->regs[BPF_REG_2].map_ptr = map_ptr;
-	callee->regs[BPF_REG_2].map_uid = map_uid;
-
-	callee->regs[BPF_REG_3].type = PTR_TO_MAP_VALUE;
-	__mark_reg_known_zero(&callee->regs[BPF_REG_3]);
-	callee->regs[BPF_REG_3].map_ptr = map_ptr;
-	callee->regs[BPF_REG_3].map_uid = map_uid;
+	mark_map_callback_reg(&callee->regs[BPF_REG_1], &caller->regs[BPF_REG_2], CONST_PTR_TO_MAP);
+	mark_map_callback_reg(&callee->regs[BPF_REG_2], &caller->regs[BPF_REG_2], PTR_TO_MAP_KEY);
+	mark_map_callback_reg(&callee->regs[BPF_REG_3], &caller->regs[BPF_REG_2], PTR_TO_MAP_VALUE);
 
 	/* unused */
 	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
Signature · 1 line
2.56.0
author reviewer
#6 lore raw

PATCH v2 6/11 bpf: Rely on callback frame initialization defaults

In reply to Ihor Solodrai

Both callback entry paths call init_func_state(), which initializes all registers to NOT_INIT and the callback return range to zero.

Callback setters repeat resets for unused registers, and the timer setter repeats the default return range. Leave those defaults to frame initialization and set only arguments and callback-specific state in the setters.

No functional change.

---
 kernel/bpf/verifier.c | 31 -------------------------------
 1 file changed, 31 deletions(-)
kernel/bpf/verifier.c +0 -31
index 8f2125c6e348..67a61570a9ab 100644
@@ -11295,9 +11295,6 @@ int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 
 	/* pointer to stack or null */
 	callee->regs[BPF_REG_4] = caller->regs[BPF_REG_3];
-
-	/* unused */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
 	return 0;
 }
 
@@ -11341,11 +11338,6 @@ static int set_loop_callback_state(struct bpf_verifier_env *env,
 	callee->regs[BPF_REG_1].type = SCALAR_VALUE;
 	callee->regs[BPF_REG_2] = caller->regs[BPF_REG_3];
 
-	/* unused */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_3]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
-
 	callee->in_callback_fn = true;
 	callee->callback_ret_range = retval_range(0, 1);
 	return 0;
@@ -11365,11 +11357,7 @@ static int set_timer_callback_state(struct bpf_verifier_env *env,
 	mark_map_callback_reg(&callee->regs[BPF_REG_3], &caller->regs[BPF_REG_1], PTR_TO_MAP_VALUE);
 	callee->regs[BPF_REG_3].id = ++env->id_gen;
 
-	/* unused */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
 	callee->in_async_callback_fn = true;
-	callee->callback_ret_range = retval_range(0, 0);
 	return 0;
 }
 
@@ -11393,10 +11381,6 @@ static int set_find_vma_callback_state(struct bpf_verifier_env *env,
 
 	/* pointer to stack or null */
 	callee->regs[BPF_REG_3] = caller->regs[BPF_REG_4];
-
-	/* unused */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
 	callee->in_callback_fn = true;
 	callee->callback_ret_range = retval_range(0, 1);
 	return 0;
@@ -11412,15 +11396,9 @@ static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env,
 	 *			  callback_ctx, u64 flags);
 	 * callback_fn(const struct bpf_dynptr_t* dynptr, void *callback_ctx);
 	 */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_0]);
 	mark_dynptr_cb_reg(env, &callee->regs[BPF_REG_1], BPF_DYNPTR_TYPE_LOCAL);
 	callee->regs[BPF_REG_2] = caller->regs[BPF_REG_3];
 
-	/* unused */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_3]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
-
 	callee->in_callback_fn = true;
 	callee->callback_ret_range = retval_range(0, 1);
 	return 0;
@@ -11452,9 +11430,6 @@ static int set_rbtree_add_callback_state(struct bpf_verifier_env *env,
 	mark_reg_graph_node(callee->regs, BPF_REG_2, &field->graph_root);
 	ref_set_non_owning(env, &callee->regs[BPF_REG_2]);
 
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_3]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
 	callee->in_callback_fn = true;
 	callee->callback_ret_range = retval_range(0, 1);
 	return 0;
@@ -11474,9 +11449,6 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
 	mark_map_callback_reg(&callee->regs[BPF_REG_3], &caller->regs[BPF_REG_3], PTR_TO_MAP_VALUE);
 	callee->regs[BPF_REG_3].id = ++env->id_gen;
 
-	/* unused */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
 	callee->in_async_callback_fn = true;
 	callee->callback_ret_range = retval_range(S32_MIN, S32_MAX);
 	return 0;
@@ -11495,9 +11467,6 @@ static int set_rcu_callback_state(struct bpf_verifier_env *env,
 	mark_map_callback_reg(&callee->regs[BPF_REG_2], &caller->regs[BPF_REG_2], PTR_TO_MAP_KEY);
 	mark_map_callback_reg(&callee->regs[BPF_REG_3], &caller->regs[BPF_REG_2], PTR_TO_MAP_VALUE);
 
-	/* unused */
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
-	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_5]);
 	callee->in_async_callback_fn = true;
 	callee->callback_ret_range = retval_range(S32_MIN, S32_MAX);
 	return 0;
Signature · 1 line
2.56.0
author reviewer
#7 lore raw

PATCH v2 7/11 bpf: Introduce REF_TYPE_FRAME in the verifier

In reply to Ihor Solodrai

A callback-calling helper can pass its callback a pointer that is only valid for the duration of the callback invocation.

A callback can save such a value, or something derived from it, and make it outlive the frame. The BPF program can then reuse it after the helper returns.

The verifier tracks the argument's register type, but does not tie its lifetime to the callback frame. Invalidating a value together with everything derived from it is what invalidate_reference() already does, walking reg->parent_id across every frame and stack slot. What is missing is a type of reference that is not an object the program acquired and releases.

Introduce REF_TYPE_FRAME: a reference owned by a callee frame. A set_callee_state_fn can anchor an argument to such a reference with mark_frame_scoped_arg(), and prepare_func_exit() drops it when the frame is popped, invalidating the argument and everything derived from it through the existing walk. The anchored register is its own parent, so invalidate_reference() no longer queues a register as a descendant of itself.

---
 include/linux/bpf.h          |  2 +
 include/linux/bpf_verifier.h |  6 +--
 kernel/bpf/diagnostics.c     |  3 ++
 kernel/bpf/diagnostics.h     |  1 +
 kernel/bpf/states.c          |  4 ++
 kernel/bpf/verifier.c        | 80 +++++++++++++++++++++++++++++++++++-
 6 files changed, 92 insertions(+), 4 deletions(-)
include/linux/bpf.h +2 -0
index d5de9d49a168..e4498e6fa88a 100644
@@ -3188,6 +3188,8 @@ int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 				   struct bpf_verifier_state *state,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee);
+int mark_frame_scoped_arg(struct bpf_verifier_env *env, struct bpf_verifier_state *state,
+			  struct bpf_func_state *callee, u32 regno);
 
 int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value, u64 flags);
 int bpf_percpu_array_copy(struct bpf_map *map, void *key, void *value, u64 flags);
include/linux/bpf_verifier.h +3 -3
index c51083c761cf..54a4f440c33b 100644
@@ -272,15 +272,13 @@ struct bpf_stack_state {
 };
 
 struct bpf_reference_state {
-	/* Each reference object has a type. Ensure REF_TYPE_PTR is zero to
-	 * default to pointer reference on zero initialization of a state.
-	 */
 	enum ref_state_type {
 		REF_TYPE_PTR		= (1 << 1),
 		REF_TYPE_IRQ		= (1 << 2),
 		REF_TYPE_LOCK		= (1 << 3),
 		REF_TYPE_RES_LOCK 	= (1 << 4),
 		REF_TYPE_RES_LOCK_IRQ	= (1 << 5),
+		REF_TYPE_FRAME		= (1 << 6),
 		REF_TYPE_LOCK_MASK	= REF_TYPE_LOCK | REF_TYPE_RES_LOCK | REF_TYPE_RES_LOCK_IRQ,
 	} type;
 	/* Track each reference created with a unique id, even if the same
@@ -298,6 +296,8 @@ struct bpf_reference_state {
 		 * it matches on unlock.
 		 */
 		void *ptr;
+		/* For REF_TYPE_FRAME */
+		u32 frameno;
 	};
 };
 
kernel/bpf/diagnostics.c +3 -0
index 857b668212fb..ba1a3f07d58e 100644
@@ -2238,6 +2238,9 @@ static void diag_print_mod(struct bpf_verifier_env *env, const struct bpf_diag_h
 								   "resource release invalidated "
 								   "this value";
 		break;
+	case BPF_DIAG_MOD_FRAME_RELEASE:
+		reason = "the callback that owned this value returned";
+		break;
 	case BPF_DIAG_MOD_PKT_DATA_CHANGE:
 		reason = "packet data may have moved";
 		break;
kernel/bpf/diagnostics.h +1 -0
index a4102fb049ec..b5cab4d79c1a 100644
@@ -22,6 +22,7 @@ enum bpf_diag_mod_reason {
 	BPF_DIAG_MOD_SPILL,
 	BPF_DIAG_MOD_VAR_WRITE,
 	BPF_DIAG_MOD_REF_RELEASE,
+	BPF_DIAG_MOD_FRAME_RELEASE,
 	BPF_DIAG_MOD_PKT_DATA_CHANGE,
 	BPF_DIAG_MOD_NON_OWN_REF,
 	BPF_DIAG_MOD_CALLER_SAVED,
kernel/bpf/states.c +4 -0
index 18bf7b660c2f..1291824a3a7d 100644
@@ -899,6 +899,10 @@ static bool refsafe(struct bpf_verifier_state *old, struct bpf_verifier_state *c
 			break;
 		case REF_TYPE_IRQ:
 			break;
+		case REF_TYPE_FRAME:
+			if (old->refs[i].frameno != cur->refs[i].frameno)
+				return false;
+			break;
 		case REF_TYPE_LOCK:
 		case REF_TYPE_RES_LOCK:
 		case REF_TYPE_RES_LOCK_IRQ:
kernel/bpf/verifier.c +79 -1

110 lines not on this page · raw message

Signature · 1 line
2.56.0
author reviewer
#10 lore raw

PATCH v2 8/11 bpf: Scope the bpf_user_ringbuf_drain() dynptr to its callback frame

In reply to Ihor Solodrai

The dynptr argument to a bpf_user_ringbuf_drain() callback points to a descriptor on the helper's stack. The descriptor and the sample it describes are valid only during that callback invocation.

The callback can save the dynptr pointer, a slice, or a clone in the caller's frame and use it after the drain returns. An escaped descriptor pointer can allow arbitrary kernel read/write once the helper's stack is reused.

The verifier assigns the dynptr a type and an id, but does not tie it or its descendants to the callback's lifetime.

Anchor R1 to a frame reference so the argument and its descendants are invalidated when the callback frame is popped.

---
 kernel/bpf/verifier.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
kernel/bpf/verifier.c +10 -5
index c969fef2d452..edc78e50dc22 100644
@@ -713,11 +713,12 @@ static void mark_dynptr_stack_regs(struct bpf_verifier_env *env,
 	__mark_dynptr_reg(sreg2, type, false, id, parent_id);
 }
 
-static void mark_dynptr_cb_reg(struct bpf_verifier_env *env,
-			       struct bpf_reg_state *reg,
-			       enum bpf_dynptr_type type)
+static int mark_dynptr_cb_reg(struct bpf_verifier_env *env, struct bpf_verifier_state *state,
+			      struct bpf_func_state *callee, u32 regno,
+			      enum bpf_dynptr_type type)
 {
-	__mark_dynptr_reg(reg, type, true, ++env->id_gen, 0);
+	__mark_dynptr_reg(&callee->regs[regno], type, true, 0, 0);
+	return mark_frame_scoped_arg(env, state, callee, regno);
 }
 
 static int destroy_if_dynptr_stack_slot(struct bpf_verifier_env *env,
@@ -11456,11 +11457,15 @@ static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env,
 					   struct bpf_func_state *callee,
 					   int insn_idx)
 {
+	int err;
+
 	/* bpf_user_ringbuf_drain(struct bpf_map *map, void *callback_fn, void
 	 *			  callback_ctx, u64 flags);
 	 * callback_fn(const struct bpf_dynptr_t* dynptr, void *callback_ctx);
 	 */
-	mark_dynptr_cb_reg(env, &callee->regs[BPF_REG_1], BPF_DYNPTR_TYPE_LOCAL);
+	err = mark_dynptr_cb_reg(env, state, callee, BPF_REG_1, BPF_DYNPTR_TYPE_LOCAL);
+	if (err)
+		return err;
 	callee->regs[BPF_REG_2] = caller->regs[BPF_REG_3];
 
 	callee->in_callback_fn = true;
Signature · 1 line
2.56.0
author reviewer
#11 lore raw

PATCH v2 9/11 selftests/bpf: Cover the user ringbuf callback dynptr lifetime

In reply to Ihor Solodrai

The dynptr passed to a bpf_user_ringbuf_drain() callback, and values derived from it, must not be used after that callback returns.

Exercise escapes through the callback context, slices returned by bpf_dynptr_data() and bpf_dynptr_slice(), and clones stored in the caller's frame. Also cover an inner callback's dynptr escaping into an outer callback.

Require the callback-return diagnostic for escaped pointers and slices. Check the full bpf_throw() callback rejection so a frame-reference leak cannot satisfy the test accidentally.

---
 .../selftests/bpf/progs/exceptions_fail.c     |  20 +++
 .../selftests/bpf/progs/user_ringbuf_fail.c   | 135 ++++++++++++++++++
 2 files changed, 155 insertions(+)
tools/testing/selftests/bpf/progs/exceptions_fail.c +20 -0
index 22503cf62e9f..86a0667ba348 100644
@@ -31,6 +31,11 @@ struct {
 	__type(value, struct hmap_elem);
 } hmap SEC(".maps");
 
+struct {
+	__uint(type, BPF_MAP_TYPE_USER_RINGBUF);
+	__uint(max_entries, 4096);
+} user_ringbuf SEC(".maps");
+
 private(A) struct bpf_spin_lock lock;
 private(A) struct bpf_rb_root rbtree __contains(foo, node);
 
@@ -110,6 +115,21 @@ static int timer_cb(void *map, int *key, struct bpf_timer *timer)
 	return 0;
 }
 
+static long drain_cb(struct bpf_dynptr *dynptr, void *context)
+{
+	bpf_throw(0);
+	return 0;
+}
+
+SEC("?tc")
+__failure
+__msg("bpf_throw kfunc (insn {{[0-9]+}}) cannot be called from callback subprog {{[0-9]+}}")
+int reject_user_ringbuf_callback_throw(struct __sk_buff *ctx)
+{
+	bpf_user_ringbuf_drain(&user_ringbuf, drain_cb, NULL, 0);
+	return 0;
+}
+
 SEC("?tc")
 __failure __msg("cannot be called from callback subprog")
 int reject_async_callback_throw(struct __sk_buff *ctx)
tools/testing/selftests/bpf/progs/user_ringbuf_fail.c +135 -0

147 lines not on this page · raw message

Signature · 1 line
2.56.0
author reviewer
#12 lore raw

PATCH v2 10/11 bpf: Scope the bpf_for_each_map_elem() array key to the callback frame

In reply to Ihor Solodrai

bpf_for_each_map_elem() passes array and per-CPU array callbacks the address of a key on the helper's stack.

A callback can save that pointer in its context and dereference it after the helper returns. Loads through PTR_TO_MAP_KEY are not fault-protected, allowing a four-byte read-only leak of reused kernel stack.

The verifier tracks the pointer's type and key size, but does not tie its lifetime to the callback invocation.

Anchor R2 to a frame reference in the array map callback setup so the key and its copies are invalidated on callback return. Leave hash keys and map values unchanged, since their storage is not callback-local.

---
 kernel/bpf/arraymap.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
kernel/bpf/arraymap.c +16 -2
index 0fe9afd4a591..5e8669e311b6 100644
@@ -855,6 +855,20 @@ static u64 array_map_mem_usage(const struct bpf_map *map)
 	return usage;
 }
 
+static int array_map_set_for_each_callback_args(struct bpf_verifier_env *env,
+						struct bpf_verifier_state *state,
+						struct bpf_func_state *caller,
+						struct bpf_func_state *callee)
+{
+	int err;
+
+	err = map_set_for_each_callback_args(env, state, caller, callee);
+	if (err)
+		return err;
+
+	return mark_frame_scoped_arg(env, state, callee, BPF_REG_2);
+}
+
 BTF_ID_LIST_SINGLE(array_map_btf_ids, struct, bpf_array)
 const struct bpf_map_ops array_map_ops = {
 	.map_meta_equal = array_map_meta_equal,
@@ -875,7 +889,7 @@ const struct bpf_map_ops array_map_ops = {
 	.map_check_btf = array_map_check_btf,
 	.map_lookup_batch = generic_map_lookup_batch,
 	.map_update_batch = generic_map_update_batch,
-	.map_set_for_each_callback_args = map_set_for_each_callback_args,
+	.map_set_for_each_callback_args = array_map_set_for_each_callback_args,
 	.map_for_each_callback = bpf_for_each_array_elem,
 	.map_mem_usage = array_map_mem_usage,
 	.map_btf_id = &array_map_btf_ids[0],
@@ -900,7 +914,7 @@ const struct bpf_map_ops percpu_array_map_ops = {
 	.map_check_btf = array_map_check_btf,
 	.map_lookup_batch = generic_map_lookup_batch,
 	.map_update_batch = generic_map_update_batch,
-	.map_set_for_each_callback_args = map_set_for_each_callback_args,
+	.map_set_for_each_callback_args = array_map_set_for_each_callback_args,
 	.map_for_each_callback = bpf_for_each_array_elem,
 	.map_mem_usage = array_map_mem_usage,
 	.map_btf_id = &array_map_btf_ids[0],
Signature · 1 line
2.56.0
author reviewer
#13 lore raw

PATCH v2 11/11 selftests/bpf: Cover callback-frame map key lifetime

In reply to Ihor Solodrai

Array and per-CPU array callbacks to bpf_for_each_map_elem() receive a key on the helper's stack.

Check that saving this key in the callback context and dereferencing it after iteration is rejected.

Also require the same pattern to keep verifying for a hash key and an array value, whose storage outlives the callback. These controls guard the array-only and key-only scope of the lifetime restriction.

---
 .../bpf/progs/verifier_iterating_callbacks.c  | 91 +++++++++++++++++++
 1 file changed, 91 insertions(+)
tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c +91 -0
index 1fbcc5228306..2e0c56888953 100644
@@ -9,6 +9,20 @@ struct {
 	__type(value, __u64);
 } map SEC(".maps");
 
+struct {
+	__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+	__uint(max_entries, 8);
+	__type(key, __u32);
+	__type(value, __u64);
+} percpu_map SEC(".maps");
+
+struct {
+	__uint(type, BPF_MAP_TYPE_HASH);
+	__uint(max_entries, 8);
+	__type(key, __u32);
+	__type(value, __u64);
+} hash_map SEC(".maps");
+
 struct {
 	__uint(type, BPF_MAP_TYPE_USER_RINGBUF);
 	__uint(max_entries, 8);
@@ -800,4 +814,81 @@ __naked void check_add_const_regsafe_off(void)
 	: __clobber_common);
 }
 
+struct key_ctx {
+	__u32 *key;
+};
+
+static long park_key_cb(struct bpf_map *map, __u32 *key, __u64 *value,
+			void *context)
+{
+	struct key_ctx *c = context;
+
+	c->key = key;
+	return 0;
+}
+
+/* bpf_for_each_array_elem() passes a key from its own stack frame. */
+SEC("?raw_tp")
+__failure __msg("invalid mem access 'scalar'")
+int array_park_map_key(void *ctx)
+{
+	struct key_ctx c = {};
+
+	bpf_for_each_map_elem(&map, park_key_cb, &c, 0);
+	if (c.key)
+		return *c.key;
+	return 0;
+}
+
+SEC("?raw_tp")
+__failure __msg("invalid mem access 'scalar'")
+int percpu_array_park_map_key(void *ctx)
+{
+	struct key_ctx c = {};
+
+	bpf_for_each_map_elem(&percpu_map, park_key_cb, &c, 0);
+	if (c.key)
+		return *c.key;
+	return 0;
+}
+
+/* A hash key points into the element, which outlives the callback. */
+SEC("?raw_tp")
+__success
+int hash_park_map_key(void *ctx)
+{
+	struct key_ctx c = {};
+
+	bpf_for_each_map_elem(&hash_map, park_key_cb, &c, 0);
+	if (c.key)
+		return *c.key;
+	return 0;
+}
+
+struct value_ctx {
+	__u64 *value;
+};
+
+static long park_value_cb(struct bpf_map *map, __u32 *key, __u64 *value,
+			  void *context)
+{
+	struct value_ctx *c = context;
+
+	c->value = value;
+	return 0;
+}
+
+/* Only the key is frame-scoped; the element lives until map teardown. */
+SEC("?raw_tp")
+__success
+int array_park_map_value(void *ctx)
+{
+	struct value_ctx c = {};
+
+	bpf_for_each_map_elem(&map, park_value_cb, &c, 0);
+	if (c.value)
+		return *c.value;
+	return 0;
+}
+
 char _license[] SEC("license") = "GPL";
Signature · 1 line
2.56.0
#8 lore raw

Re: [PATCH bpf-next v2 07/11] bpf: Introduce REF_TYPE_FRAME in the verifier

In reply to Ihor Solodrai
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Critical] bpf: UAF due to dropped parent_id in dynptr_from_mem for frame-scoped args --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] bpf: UAF due to dropped parent_id in dynptr_from_mem for frame-scoped args --

commit 19d3a1bd15552653b24ece884c759a788c6abaf9
Author: Ihor Solodrai <[email protected]>

bpf: Introduce REF_TYPE_FRAME in the verifier

This commit introduces REF_TYPE_FRAME to track references owned by a callee frame. It allows a set_callee_state_fn to anchor an argument to a reference which is dropped when the frame is popped.

Quoted 8 lines
include/linux/bpf_verifier.h +0 -0
index c51083c761cf2..54a4f440c33b3 100644
@@ -272,15 +272,13 @@ struct bpf_stack_state {
 };
 
 struct bpf_reference_state {

include/linux/bpf_verifier.h struct bpf_stack_state { · line 281

Quoted 10 lines
include/linux/bpf_verifier.h +1 -3
-	/* Each reference object has a type. Ensure REF_TYPE_PTR is zero to
-	 * default to pointer reference on zero initialization of a state.
-	 */
 	enum ref_state_type {
 		REF_TYPE_PTR		= (1 << 1),
 		REF_TYPE_IRQ		= (1 << 2),
 		REF_TYPE_LOCK		= (1 << 3),
 		REF_TYPE_RES_LOCK 	= (1 << 4),
 		REF_TYPE_RES_LOCK_IRQ	= (1 << 5),
+		REF_TYPE_FRAME		= (1 << 6),
                ^^^^^^^^^^^^^^

[Severity: Critical]
Does this new reference type get ignored when the verifier checks if a register is referenced?

In kernel/bpf/verifier.c, find_reference_state() explicitly skips any reference type other than REF_TYPE_PTR:

kernel/bpf/verifier.c:find_reference_state() {
...

	for (i = 0; i < state->acquired_refs; i++) {
		if (state->refs[i].type != REF_TYPE_PTR)
			continue;
		if (state->refs[i].id == id)
			return true;
	}

...
}

Because find_reference_state() ignores REF_TYPE_FRAME, reg_is_referenced() returns false for frame-scoped arguments.

Could this lead to a regression if a callback passes a frame-scoped reference (like a PTR_TO_MAP_KEY array key or ringbuf dynptr slice) to a constructor like bpf_dynptr_from_mem() to create a local dynptr?

When check_helper_call() processes the constructor, reg_is_referenced() returns false, which leaves meta->ref_obj.id as 0:

kernel/bpf/verifier.c:check_helper_call() {
...

	if (reg_is_referenced(env, reg))
		update_ref_obj(&meta->ref_obj, reg);

...
}

Later, mark_stack_slots_dynptr() sets the dynptr's parent_id to this 0:

kernel/bpf/verifier.c:mark_stack_slots_dynptr() {
...

		/* Track parent's id if the parent is a referenced object */
		parent_id = ref_obj->id;

...
}

This strips the dynptr of its frame-scoped lifetime tie. When the callback exits, release_frame_references() won't invalidate the dynptr because its parent_id is 0.

Can the caller then extract a pointer from the dynptr using bpf_dynptr_data() and access memory after the callback frame is destroyed?

Is it possible that pointer cast functions like bpf_skc_to_tcp_sock() also suffer from the exact same logic flaw because they rely on find_reference_state(..., meta.ref_obj.id)?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
author reviewer
#9 lore raw

Re: [PATCH bpf-next v2 07/11] bpf: Introduce REF_TYPE_FRAME in the verifier

Quoted 48 lines · On 2026-10-09 1:48 p.m., [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] bpf: UAF due to dropped parent_id in dynptr_from_mem for frame-scoped args --

commit 19d3a1bd15552653b24ece884c759a788c6abaf9
Author: Ihor Solodrai <[email protected]>

bpf: Introduce REF_TYPE_FRAME in the verifier

This commit introduces REF_TYPE_FRAME to track references owned by a callee frame. It allows a set_callee_state_fn to anchor an argument to a reference which is dropped when the frame is popped.

Quoted 8 lines
include/linux/bpf_verifier.h +0 -0
index c51083c761cf2..54a4f440c33b3 100644
@@ -272,15 +272,13 @@ struct bpf_stack_state {
  };
 
  struct bpf_reference_state {

include/linux/bpf_verifier.h struct bpf_stack_state { · line 281

Quoted 10 lines
include/linux/bpf_verifier.h +1 -3
-	/* Each reference object has a type. Ensure REF_TYPE_PTR is zero to
-	 * default to pointer reference on zero initialization of a state.
-	 */
  	enum ref_state_type {
  		REF_TYPE_PTR		= (1 << 1),
  		REF_TYPE_IRQ		= (1 << 2),
  		REF_TYPE_LOCK		= (1 << 3),
  		REF_TYPE_RES_LOCK 	= (1 << 4),
  		REF_TYPE_RES_LOCK_IRQ	= (1 << 5),
+		REF_TYPE_FRAME		= (1 << 6),
                 ^^^^^^^^^^^^^^

[Severity: Critical]
Does this new reference type get ignored when the verifier checks if a register is referenced?

In kernel/bpf/verifier.c, find_reference_state() explicitly skips any reference type other than REF_TYPE_PTR:

kernel/bpf/verifier.c:find_reference_state() {
...

	for (i = 0; i < state->acquired_refs; i++) {
		if (state->refs[i].type != REF_TYPE_PTR)
			continue;
		if (state->refs[i].id == id)
			return true;
	}
Quoted 5 lines

...
}

Because find_reference_state() ignores REF_TYPE_FRAME, reg_is_referenced() returns false for frame-scoped arguments.

Yes, this is intentional.

Quoted 4 lines

Could this lead to a regression if a callback passes a frame-scoped reference (like a PTR_TO_MAP_KEY array key or ringbuf dynptr slice) to a constructor like bpf_dynptr_from_mem() to create a local dynptr?

No, bpf_dynptr_from_mem() accepts only PTR_TO_MAP_VALUE. check_helper_call() rejects other types:

	case BPF_FUNC_dynptr_from_mem:
		if (regs[BPF_REG_1].type != PTR_TO_MAP_VALUE) {
			[...]
			return -EACCES;
		}
		break;
Quoted 24 lines

When check_helper_call() processes the constructor, reg_is_referenced() returns false, which leaves meta->ref_obj.id as 0:

kernel/bpf/verifier.c:check_helper_call() {
...

	if (reg_is_referenced(env, reg))
		update_ref_obj(&meta->ref_obj, reg);

...
}

Later, mark_stack_slots_dynptr() sets the dynptr's parent_id to this 0:

kernel/bpf/verifier.c:mark_stack_slots_dynptr() {
...

		/* Track parent's id if the parent is a referenced object */
		parent_id = ref_obj->id;

...
}

This strips the dynptr of its frame-scoped lifetime tie. When the callback exits, release_frame_references() won't invalidate the dynptr because its parent_id is 0.

Quoted 6 lines

Can the caller then extract a pointer from the dynptr using bpf_dynptr_data() and access memory after the callback frame is destroyed?

Is it possible that pointer cast functions like bpf_skc_to_tcp_sock() also suffer from the exact same logic flaw because they rely on find_reference_state(..., meta.ref_obj.id)?

No, bpf_skc_to_tcp_sock() requires ARG_PTR_TO_BTF_ID_SOCK_COMMON. Its compatible register types, listed in btf_id_sock_common_types, exclude the frame-scoped arguments introduced by this series.

Quoted 1 line
#14 lore raw
In reply to Ihor Solodrai
Quoted 3 lines · Ihor Solodrai

Introduce REF_TYPE_FRAME for this use case: a reference owned by a callee frame, dropped when the frame is popped. Tie each argument to this reference so callback return invalidates it and its descendants.

Another idea...
The id is owned by the frame. Can we keep it in bpf_func_state like diag_frame_id ?
release_reference() already takes ids that are not in refs[]. See unmark_stack_slots_dynptr().
Clones copy parent_id and slices hang off dynptr's id, so both are reached.
All setters have callee already, so no need for patches 1-3, REF_TYPE_FRAME, frameno in refs[], the reg that is its own parent, etc. Something like this (compile tested only):

include/linux/bpf.h +1 -0
index 54144372281c..4f82b86cdfe2 100644
@@ -3185,6 +3185,7 @@ int bpf_iter_map_fill_link_info(const struct bpf_iter_aux_info *aux,
 int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee);
+void mark_frame_scoped_arg(struct bpf_verifier_env *env, struct bpf_func_state *callee, u32 regno);
 
 int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value, u64 flags);
 int bpf_percpu_array_copy(struct bpf_map *map, void *key, void *value, u64 flags);
include/linux/bpf_verifier.h +2 -0
index 112ca1511da4..8865ece94ef4 100644
@@ -364,6 +364,8 @@ struct bpf_func_state {
 	 * reused after returns, while this ID is preserved across state clones.
 	 */
 	u32 diag_frame_id;
+	/* parent_id of the args that are gone when this frame returns */
+	u32 frame_id;
 	/* subprog number == index within subprog_info
 	 * zero == main subprog
 	 */
kernel/bpf/arraymap.c +12 -2
index 0fe9afd4a591..3376cc81e5d9 100644
@@ -855,6 +855,16 @@ static u64 array_map_mem_usage(const struct bpf_map *map)
 	return usage;
 }
 
+static int array_map_set_for_each_callback_args(struct bpf_verifier_env *env,
+						struct bpf_func_state *caller,
+						struct bpf_func_state *callee)
+{
+	map_set_for_each_callback_args(env, caller, callee);
+	/* the key is on the stack of bpf_for_each_array_elem() */
+	mark_frame_scoped_arg(env, callee, BPF_REG_2);
+	return 0;
+}
+
 BTF_ID_LIST_SINGLE(array_map_btf_ids, struct, bpf_array)
 const struct bpf_map_ops array_map_ops = {
 	.map_meta_equal = array_map_meta_equal,
@@ -875,7 +885,7 @@ const struct bpf_map_ops array_map_ops = {
 	.map_check_btf = array_map_check_btf,
 	.map_lookup_batch = generic_map_lookup_batch,
 	.map_update_batch = generic_map_update_batch,
-	.map_set_for_each_callback_args = map_set_for_each_callback_args,
+	.map_set_for_each_callback_args = array_map_set_for_each_callback_args,
 	.map_for_each_callback = bpf_for_each_array_elem,
 	.map_mem_usage = array_map_mem_usage,
 	.map_btf_id = &array_map_btf_ids[0],
@@ -900,7 +910,7 @@ const struct bpf_map_ops percpu_array_map_ops = {
 	.map_check_btf = array_map_check_btf,
 	.map_lookup_batch = generic_map_lookup_batch,
 	.map_update_batch = generic_map_update_batch,
-	.map_set_for_each_callback_args = map_set_for_each_callback_args,
+	.map_set_for_each_callback_args = array_map_set_for_each_callback_args,
 	.map_for_each_callback = bpf_for_each_array_elem,
 	.map_mem_usage = array_map_mem_usage,
 	.map_btf_id = &array_map_btf_ids[0],
kernel/bpf/states.c +3 -0
index 953700822fa7..43d6682037fe 100644
@@ -995,6 +995,9 @@ static bool func_states_equal(struct bpf_verifier_env *env, struct bpf_func_stat
 	if (!old->no_stack_arg_load && cur->no_stack_arg_load)
 		return false;
 
+	if (!check_ids(old->frame_id, cur->frame_id, &env->idmap_scratch))
+		return false;
+
 	for (i = 0; i < MAX_BPF_REG; i++)
 		if (((1 << i) & live_regs) &&
 		    !regsafe(env, &old->regs[i], &cur->regs[i],
kernel/bpf/verifier.c +14 -0
index ec136cf377c1..bc12df55d2e2 100644
@@ -11465,6 +11465,14 @@ static int check_func_callx(struct bpf_verifier_env *env, struct bpf_insn *insn,
 	return check_static_func_call(env, subprog, insn_idx);
 }
 
+/* The arg points into the stack frame of the helper that calls the callback */
+void mark_frame_scoped_arg(struct bpf_verifier_env *env, struct bpf_func_state *callee, u32 regno)
+{
+	if (!callee->frame_id)
+		callee->frame_id = ++env->id_gen;
+	callee->regs[regno].parent_id = callee->frame_id;
+}
+
 int map_set_for_each_callback_args(struct bpf_verifier_env *env,
 				   struct bpf_func_state *caller,
 				   struct bpf_func_state *callee)
@@ -11631,6 +11639,7 @@ static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env,
 	 */
 	bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_0]);
 	mark_dynptr_cb_reg(env, &callee->regs[BPF_REG_1], BPF_DYNPTR_TYPE_LOCAL);
+	mark_frame_scoped_arg(env, callee, BPF_REG_1);
 	callee->regs[BPF_REG_2] = caller->regs[BPF_REG_3];
 
 	/* unused */
@@ -11885,6 +11894,11 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
 					     callee->allocated_stack / BPF_REG_SIZE - 1);
 	if (err)
 		return err;
+	if (callee->frame_id) {
+		err = release_reference(env, callee->frame_id);
+		if (err)
+			return err;
+	}
 
 	/* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,
 	 * there function call logic would reschedule callback visit. If iteration

Patches 5 and 6 are fine on their own. Pls send them separately.

pw-bot: cr

author reviewer
#15 lore raw
Quoted 5 lines · Alexei Starovoitov
Quoted 3 lines · Ihor Solodrai

Introduce REF_TYPE_FRAME for this use case: a reference owned by a callee frame, dropped when the frame is popped. Tie each argument to this reference so callback return invalidates it and its descendants.

Quoted 7 lines

Another idea...
The id is owned by the frame. Can we keep it in bpf_func_state like diag_frame_id ?
release_reference() already takes ids that are not in refs[]. See unmark_stack_slots_dynptr().
Clones copy parent_id and slices hang off dynptr's id, so both are reached.

Slices are not reached. A slice's parent_id is the dynptr id, and the walk from frame_id finds that id only while the dynptr is still in a register or a stack slot. R1 is caller-saved, so the bpf_dynptr_data() call that creates the slice usually clobbers it.

This can be fixed in your diff with:

kernel/bpf/verifier.c +1 -0
index bc12df55d2e2..57859c3baeeb 100644
@@ -11470,6 +11470,7 @@ void mark_frame_scoped_arg(struct bpf_verifier_env *env, struct bpf_func_state *
 {
 	if (!callee->frame_id)
 		callee->frame_id = ++env->id_gen;
+	callee->regs[regno].id = callee->frame_id;
 	callee->regs[regno].parent_id = callee->frame_id;
 }
 

i.e. the reg is its own parent again.

Quoted 2 lines

All setters have callee already, so no need for patches 1-3, REF_TYPE_FRAME, frameno in refs[], the reg that is its own parent, etc.

I don't quite understand the concern with the series as is. I'm guessing a new weird REF_TYPE in refs[] which is not an actual acquirable object gives an ick. Or is it the plumbing? I take it you agree that we need some kind of frame-tied lifetime tracking, right?

I added some refactoring patches in attempt to avoid growing bpf_func_state vs v1 (Eduard mentioned we create a lot of these objects), and that is what you've just suggested.

frame_id approach is fine with me, just want to understand the choices we are making.

Thanks!

Quoted 1 line

[...]

#16 lore raw
Quoted 19 lines · Ihor Solodrai

Slices are not reached. A slice's parent_id is the dynptr id, and the walk from frame_id finds that id only while the dynptr is still in a register or a stack slot. R1 is caller-saved, so the bpf_dynptr_data() call that creates the slice usually clobbers it.

This can be fixed in your diff with:

kernel/bpf/verifier.c +1 -0
index bc12df55d2e2..57859c3baeeb 100644
@@ -11470,6 +11470,7 @@ void mark_frame_scoped_arg(struct bpf_verifier_env *env, struct bpf_func_state *
 {
 	if (!callee->frame_id)
 		callee->frame_id = ++env->id_gen;
+	callee->regs[regno].id = callee->frame_id;
 	callee->regs[regno].parent_id = callee->frame_id;
 }
 
Quoted 1 line

i.e. the reg is its own parent again.

got it. you're right. Let's take your one-liner.
I think the reg->id != id hunk can still stay out. The reg is invalidated in the same pass, so the second pop of frame_id in invalidate_reference() finds nothing and the loop ends.

Quoted 5 lines

I don't quite understand the concern with the series as is. I'm guessing a new weird REF_TYPE in refs[] which is not an actual acquirable object gives an ick. Or is it the plumbing? I take it you agree that we need some kind of frame-tied lifetime tracking, right?

Yes, we need it. I guess it's the plumbing that bothers me a bit. The id belongs to the frame, but refs[] is in bpf_verifier_state and the callback runs in a pushed state. That's why setters had to move out of setup_func_entry(), take a state argument and handle errors, and why refsafe() and check_reference_leak() need REF_TYPE_FRAME cases. A u32 in the frame needs none of that.

Quoted 3 lines

I added some refactoring patches in attempt to avoid growing bpf_func_state vs v1 (Eduard mentioned we create a lot of these objects), and that is what you've just suggested.

Right, it grows. With padding that's 8 bytes on a 960 byte struct, 880 of which is regs[]. I feel it's cheaper than patches 1-3, but I don't insist.

maintainer
#17 lore raw
In reply to Ihor Solodrai
Quoted 44 lines · Ihor Solodrai

A callback-calling helper or kfunc can pass its callback a pointer that is only valid for the callback invocation, and the verifier has no way to express that lifetime.

Two helpers need fixing:

* bpf_user_ringbuf_drain() passes a CONST_PTR_TO_DYNPTR over a sample it releases as soon as the callback returns. Saving this pointer in callback_ctx leaves it pointing into reused kernel stack and allows arbitrary kernel read and write (bpf-next only, see patch #8).

* bpf_for_each_map_elem() over an array or per-CPU array map passes the address of a u32 held in bpf_for_each_array_elem()'s own frame, leaking four bytes of kernel stack.

Both arguments point into a helper's own frame, with nothing longer-lived to anchor them to. Rejecting spills of the dynptr pointer alone would still allow derived slices and clones to escape.

Introduce REF_TYPE_FRAME for this use case: a reference owned by a callee frame, dropped when the frame is popped. Tie each argument to this reference so callback return invalidates it and its descendants.

Callback setters repeat register defaults and map metadata setup, while reference release couples descendant invalidation to object release. The refactoring patches give common initialization one owner and separate invalidation from reference removal. Frame lifetime tracking can then reuse the existing reference machinery.

---

Patches #1-6 are non-functional preparation. Patch #7 introduces REF_TYPE_FRAME and its diagnostics; #8 and #10 apply it to the helpers, and #9 and #11 add selftests.

v1->v2:
* Add the refactoring patches: release frame references in one scan, and fold diagnostics into the REF_TYPE_FRAME introduction
* Replace temporary argument mask and deferred register scan with reference acquisition directly in the owning verifier state during callback setup (Eduard)
* Reuse callback_park_dynptr() for the nested drain test

Quoted 3 lines

v1: https://lore.kernel.org/r/[email protected]

---

The series lgtm, but it needs a respin because bpf-next had moved. Also, what Alexei suggests makes sense to me, it would be a bit simpler. Ihor, please take whatever approach you think is reasonable and respin. If you decide to respin as-is, feel free to add
"Acked-by: Eduard Zingerman <[email protected]>".