Skip to contentSkip to conversation

bpf: Extend the bpf_list family of APIs

PATCH RESEND Superseded Kaitao cheng 5 participants 37 messages May 12 – May 20

Subsystems (5)

v10 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v10 cover bpf: Extend the bpf_list family of APIs

In BPF, a list can only be used to implement a stack structure. Due to an incomplete API set, only FIFO or LIFO operations are supported. The patches enhance the BPF list API, making it more list-like.

Five new kfuncs have been added:
bpf_list_del: remove a node from the list
bpf_list_add_impl: insert a node after a given list node
bpf_list_is_first: check if a node is the first in the list
bpf_list_is_last: check if a node is the last in the list
bpf_list_empty: check if the list is empty

And add test cases for the aforementioned kfuncs.

Changes in v10:
- Remove the table-driven approach (Ihor Solodrai)
- Use the __nonown_allowed suffix for bpf_list_del/front/back
- Add test cases for __nonown_allowed

Changes in v9:
- Expand table-driven approach coverage (Emil Tsalapatis)
- Clear list node owner and unlink before drop (Emil Tsalapatis)
- Remove warnings caused by WARN_ON_ONCE() (Emil Tsalapatis)
- Introduce the __nonown_allowed suffix (Alexei Starovoitov)

Changes in v8:
- Use [patch v7 5/5] as the start of the patch series (Leon Hwang)
- Introduce double pointer prev_ptr in __bpf_list_del
(Kumar Kartikeya Dwivedi)
- Extract refactored __bpf_list_del/add into separate patches (Leon Hwang)
- Allow bpf_list_front/back result as the prev argument of bpf_list_add
- Split test cases (Leon Hwang)

Changes in v7:
- Replace bpf_list_node_is_edge with bpf_list_is_first/is_last
- Reimplement __bpf_list_del and __bpf_list_add (Kumar Kartikeya Dwivedi)
- Simplify test cases (Mykyta Yatsenko)

Changes in v6:
- Merge [patch v5 (2,4,6)/6] into [patch v6 4/5] (Leon Hwang)
- If list_head was 0-initialized, init it
- refactor kfunc checks to table-driven approach (Leon Hwang)

Changes in v5:
- Fix bpf_obj leak on bpf_list_add_impl error

Changes in v4:
- [patch v3 1/6] Revert to version v1 (Alexei Starovoitov)
- Change the parameters of bpf_list_add_impl to (head, new, prev, ...)

Changes in v3:
- Add a new lock_rec member to struct bpf_reference_state for lock holding detection.
- Add test cases to verify that the verifier correctly restricts calls to bpf_list_del when the spin_lock is not held.

Changes in v2:
- Remove the head parameter from bpf_list_del (Alexei Starovoitov)
- Add bpf_list_add/is_first/is_last/empty to API and test cases (Alexei Starovoitov)

Link to v9:
https://lore.kernel.org/all/[email protected]/

Link to v8:
https://lore.kernel.org/all/[email protected]/

Link to v7:
https://lore.kernel.org/all/[email protected]/

Link to v6:
https://lore.kernel.org/all/[email protected]/

Link to v5:
https://lore.kernel.org/all/[email protected]/

Link to v4:
https://lore.kernel.org/all/[email protected]/

Link to v3:
https://lore.kernel.org/all/[email protected]/

Link to v2:
https://lore.kernel.org/all/[email protected]/

Link to v1:
https://lore.kernel.org/all/[email protected]/

Kaitao Cheng (8):
bpf: refactor __bpf_list_del to take list node pointer
bpf: clear list node owner and unlink before drop
bpf: Introduce the bpf_list_del kfunc.
bpf: refactor __bpf_list_add to take insertion point via **prev_ptr
bpf: Add bpf_list_add to insert node after a given list node
bpf: add bpf_list_is_first/last/empty kfuncs
bpf: allow non-owning list-node args via __nonown_allowed selftests/bpf: Add test cases for
bpf_list_del/add/is_first/is_last/empty

 Documentation/bpf/kfuncs.rst                  |  22 +-
 kernel/bpf/helpers.c                          | 139 ++++--
 kernel/bpf/verifier.c                         |  44 +-
 .../selftests/bpf/progs/refcounted_kptr.c     | 421 ++++++++++++++++++
 4 files changed, 593 insertions(+), 33 deletions(-)
Signature · 1 line
2.50.1 (Apple Git-155)
#1 lore raw

PATCH RESEND v10 1/8 bpf: refactor __bpf_list_del to take list node pointer

From: Kaitao Cheng <[email protected]>

Refactor __bpf_list_del to accept (head, struct list_head *n) instead of (head, bool tail). The caller now passes the specific node to remove:
bpf_list_pop_front passes h->next, bpf_list_pop_back passes h->prev.

Prepares for introducing bpf_list_del(head, node) kfunc to remove an arbitrary node when the user holds ownership.

---
 kernel/bpf/helpers.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
kernel/bpf/helpers.c +13 -7
index baa12b24bb64..9cd7b028592c 100644
@@ -2550,22 +2550,24 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 	return bpf_list_push_back(head, node, meta__ign, off);
 }
 
-static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tail)
+static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head,
+					    struct list_head *n)
 {
-	struct list_head *n, *h = (void *)head;
+	struct list_head *h = (void *)head;
 	struct bpf_list_node_kern *node;
 
 	/* If list_head was 0-initialized by map, bpf_obj_init_field wasn't
 	 * called on its fields, so init here
 	 */
-	if (unlikely(!h->next))
+	if (unlikely(!h->next)) {
 		INIT_LIST_HEAD(h);
+		return NULL;
+	}
 	if (list_empty(h))
 		return NULL;
 
-	n = tail ? h->prev : h->next;
 	node = container_of(n, struct bpf_list_node_kern, list_head);
-	if (WARN_ON_ONCE(READ_ONCE(node->owner) != head))
+	if (unlikely(READ_ONCE(node->owner) != head))
 		return NULL;
 
 	list_del_init(n);
@@ -2575,12 +2577,16 @@ static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tai
 
 __bpf_kfunc struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head)
 {
-	return __bpf_list_del(head, false);
+	struct list_head *h = (void *)head;
+
+	return __bpf_list_del(head, h->next);
 }
 
 __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 {
-	return __bpf_list_del(head, true);
+	struct list_head *h = (void *)head;
+
+	return __bpf_list_del(head, h->prev);
 }
 
 __bpf_kfunc struct bpf_list_node *bpf_list_front(struct bpf_list_head *head)
Signature · 1 line
2.50.1 (Apple Git-155)
#2 lore raw

Re: [PATCH RESEND bpf-next v10 1/8] bpf: refactor __bpf_list_del to take list node pointer

The new code reads h->next before __bpf_list_del checks if the list is initialized. If the list is uninitialized (h->next is NULL), a NULL pointer is passed to __bpf_list_del.
Quoted 35 lines
kernel/bpf/helpers.c +7 -5
index baa12b24bb64..9cd7b028592c 100644
@@ -2550,22 +2550,24 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 	return bpf_list_push_back(head, node, meta__ign, off);
 }
 
-static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tail)
+static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head,
+					    struct list_head *n)
 {
-	struct list_head *n, *h = (void *)head;
+	struct list_head *h = (void *)head;
 	struct bpf_list_node_kern *node;
 
 	/* If list_head was 0-initialized by map, bpf_obj_init_field wasn't
 	 * called on its fields, so init here
 	 */
-	if (unlikely(!h->next))
+	if (unlikely(!h->next)) {
 		INIT_LIST_HEAD(h);
+		return NULL;
+	}
 	if (list_empty(h))
 		return NULL;
 
-	n = tail ? h->prev : h->next;
 	node = container_of(n, struct bpf_list_node_kern, list_head);
-	if (WARN_ON_ONCE(READ_ONCE(node->owner) != head))
+	if (unlikely(READ_ONCE(node->owner) != head))
 		return NULL;
 
 	list_del_init(n);

kernel/bpf/helpers.c static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tai · line 2583

Quoted 8 lines
kernel/bpf/helpers.c +3 -1
@@ -2575,12 +2577,16 @@ static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tai
 
 __bpf_kfunc struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head)
 {
-	return __bpf_list_del(head, false);
+	struct list_head *h = (void *)head;
+
+	return __bpf_list_del(head, h->next);
 }

The new code reads h->next before __bpf_list_del checks if the list is initialized. If the list is uninitialized (h->next is NULL), a NULL pointer is passed to __bpf_list_del.

While __bpf_list_del eventually initializes the list and returns NULL without dereferencing the NULL pointer, this is counterintuitive - passing a potentially garbage value that is later ignored.

Can you move the initialization check into the top-level kfuncs (bpf_list_pop_front and bpf_list_pop_back) to ensure the list_head passed to __bpf_list_del is always valid? You could initialize the list and return NULL in that case - we know it's empty.

This concern was raised by Emil Tsalapatis in v8:
https://lore.kernel.org/bpf/[email protected]/

kernel/bpf/helpers.c static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tai · line 2590

Quoted 8 lines
kernel/bpf/helpers.c +3 -1
 __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 {
-	return __bpf_list_del(head, true);
+	struct list_head *h = (void *)head;
+
+	return __bpf_list_del(head, h->prev);
 }

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

#3 lore raw

Re: [PATCH RESEND bpf-next v10 1/8] bpf: refactor __bpf_list_del to take list node pointer

I have clarified this issue in the [PATCH bpf-next v10 1/8] patch, with the link provided below:

https://lore.kernel.org/bpf/[email protected]/

Quoted 75 lines · 在 2026/5/12 14:41, [email protected] 写道:
Quoted 35 lines
kernel/bpf/helpers.c +7 -5
index baa12b24bb64..9cd7b028592c 100644
@@ -2550,22 +2550,24 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 	return bpf_list_push_back(head, node, meta__ign, off);
 }
 
-static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tail)
+static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head,
+					    struct list_head *n)
 {
-	struct list_head *n, *h = (void *)head;
+	struct list_head *h = (void *)head;
 	struct bpf_list_node_kern *node;
 
 	/* If list_head was 0-initialized by map, bpf_obj_init_field wasn't
 	 * called on its fields, so init here
 	 */
-	if (unlikely(!h->next))
+	if (unlikely(!h->next)) {
 		INIT_LIST_HEAD(h);
+		return NULL;
+	}
 	if (list_empty(h))
 		return NULL;
 
-	n = tail ? h->prev : h->next;
 	node = container_of(n, struct bpf_list_node_kern, list_head);
-	if (WARN_ON_ONCE(READ_ONCE(node->owner) != head))
+	if (unlikely(READ_ONCE(node->owner) != head))
 		return NULL;
 
 	list_del_init(n);

kernel/bpf/helpers.c static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tai · line 2583

Quoted 8 lines
kernel/bpf/helpers.c +3 -1
@@ -2575,12 +2577,16 @@ static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tai
 
 __bpf_kfunc struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head)
 {
-	return __bpf_list_del(head, false);
+	struct list_head *h = (void *)head;
+
+	return __bpf_list_del(head, h->next);
 }

The new code reads h->next before __bpf_list_del checks if the list is initialized. If the list is uninitialized (h->next is NULL), a NULL pointer is passed to __bpf_list_del.

While __bpf_list_del eventually initializes the list and returns NULL without dereferencing the NULL pointer, this is counterintuitive - passing a potentially garbage value that is later ignored.

Can you move the initialization check into the top-level kfuncs (bpf_list_pop_front and bpf_list_pop_back) to ensure the list_head passed to __bpf_list_del is always valid? You could initialize the list and return NULL in that case - we know it's empty.

This concern was raised by Emil Tsalapatis in v8:
https://lore.kernel.org/bpf/[email protected]/

kernel/bpf/helpers.c static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tai · line 2590

Quoted 8 lines
kernel/bpf/helpers.c +3 -1
 __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 {
-	return __bpf_list_del(head, true);
+	struct list_head *h = (void *)head;
+
+	return __bpf_list_del(head, h->prev);
 }

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

Signature · 2 lines
Thanks
Kaitao Cheng
maintainer Reviewed-by on v10 1/8
#4 lore raw

Re: [PATCH RESEND bpf-next v10 1/8] bpf: refactor __bpf_list_del to take list node pointer

In reply to Kaitao cheng
Quoted 11 lines · Kaitao cheng

From: Kaitao Cheng <[email protected]>

Refactor __bpf_list_del to accept (head, struct list_head *n) instead of (head, bool tail). The caller now passes the specific node to remove:
bpf_list_pop_front passes h->next, bpf_list_pop_back passes h->prev.

Prepares for introducing bpf_list_del(head, node) kfunc to remove an arbitrary node when the user holds ownership.

---

[...]

#5 lore raw

PATCH RESEND v10 2/8 bpf: clear list node owner and unlink before drop

In reply to Kaitao cheng

From: Kaitao Cheng <[email protected]>

When draining a BPF list_head, clear each node's owner pointer while still holding the spinlock, so concurrent readers always see a consistent owner.

Delink each node with list_del_init() before calling __bpf_obj_drop_impl(), preventing subsequent users who hold a reference count to the node from acquiring an invalid next node.

---
 kernel/bpf/helpers.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)
kernel/bpf/helpers.c +13 -9
index 9cd7b028592c..1e8754877dd1 100644
@@ -2247,10 +2247,11 @@ EXPORT_SYMBOL_GPL(bpf_base_func_proto);
 void bpf_list_head_free(const struct btf_field *field, void *list_head,
 			struct bpf_spin_lock *spin_lock)
 {
-	struct list_head *head = list_head, *orig_head = list_head;
+	struct list_head *head = list_head, drain, *pos, *n;
 
 	BUILD_BUG_ON(sizeof(struct list_head) > sizeof(struct bpf_list_head));
 	BUILD_BUG_ON(__alignof__(struct list_head) > __alignof__(struct bpf_list_head));
+	INIT_LIST_HEAD(&drain);
 
 	/* Do the actual list draining outside the lock to not hold the lock for
 	 * too long, and also prevent deadlocks if tracing programs end up
@@ -2261,20 +2262,23 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
 	__bpf_spin_lock_irqsave(spin_lock);
 	if (!head->next || list_empty(head))
 		goto unlock;
-	head = head->next;
+	list_for_each_safe(pos, n, head) {
+		WRITE_ONCE(container_of(pos,
+			struct bpf_list_node_kern, list_head)->owner, NULL);
+		list_move_tail(pos, &drain);
+	}
 unlock:
-	INIT_LIST_HEAD(orig_head);
+	INIT_LIST_HEAD(head);
 	__bpf_spin_unlock_irqrestore(spin_lock);
 
-	while (head != orig_head) {
-		void *obj = head;
-
-		obj -= field->graph_root.node_offset;
-		head = head->next;
+	while (!list_empty(&drain)) {
+		pos = drain.next;
+		list_del_init(pos);
 		/* The contained type can also have resources, including a
 		 * bpf_list_head which needs to be freed.
 		 */
-		__bpf_obj_drop_impl(obj, field->graph_root.value_rec, false);
+		__bpf_obj_drop_impl((char *)pos - field->graph_root.node_offset,
+				    field->graph_root.value_rec, false);
 	}
 }
 
Signature · 1 line
2.50.1 (Apple Git-155)
#6 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Does this create a race condition when a BPF program holds a refcount-acquired reference to a node being freed?
Quoted 30 lines
kernel/bpf/helpers.c +8 -3
index 9cd7b028592c..1e8754877dd1 100644
@@ -2247,10 +2247,11 @@ EXPORT_SYMBOL_GPL(bpf_base_func_proto);
 void bpf_list_head_free(const struct btf_field *field, void *list_head,
 			struct bpf_spin_lock *spin_lock)
 {
-	struct list_head *head = list_head, *orig_head = list_head;
+	struct list_head *head = list_head, drain, *pos, *n;
 
 	BUILD_BUG_ON(sizeof(struct list_head) > sizeof(struct bpf_list_head));
 	BUILD_BUG_ON(__alignof__(struct list_head) > __alignof__(struct bpf_list_head));
+	INIT_LIST_HEAD(&drain);
 
 	/* Do the actual list draining outside the lock to not hold the lock for
 	 * too long, and also prevent deadlocks if tracing programs end up
@@ -2261,20 +2262,23 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
 	__bpf_spin_lock_irqsave(spin_lock);
 	if (!head->next || list_empty(head))
 		goto unlock;
-	head = head->next;
+	list_for_each_safe(pos, n, head) {
+		WRITE_ONCE(container_of(pos,
+			struct bpf_list_node_kern, list_head)->owner, NULL);
+		list_move_tail(pos, &drain);
+	}
 unlock:
-	INIT_LIST_HEAD(orig_head);
+	INIT_LIST_HEAD(head);

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2276

Quoted 10 lines
kernel/bpf/helpers.c +3 -5
 	__bpf_spin_unlock_irqrestore(spin_lock);
 
-	while (head != orig_head) {
-		void *obj = head;
-
-		obj -= field->graph_root.node_offset;
-		head = head->next;
+	while (!list_empty(&drain)) {
+		pos = drain.next;
+		list_del_init(pos);

Does this create a race condition when a BPF program holds a refcount-acquired reference to a node being freed?

The patch clears each node's owner to NULL under spin_lock, moves the node to a local 'drain' list, then drops the lock. After unlock, the drain loop walks the local list and calls list_del_init(pos) before __bpf_obj_drop_impl().

Between spin_unlock and list_del_init(pos), each node in 'drain' has owner == NULL AND is still linked in 'drain'.

When a BPF program holds an owning or refcount-acquired reference to one of these nodes (node X), which is structurally supported because __bpf_obj_drop_impl() uses refcount_dec_and_test() and only frees at refcount 0, a concurrent push to a DIFFERENT bpf_list_head becomes a corruption:

CPU 0 (bpf_list_head_free, lock released) CPU 1 (BPF prog, refcount X)
----------------------------------------- ----------------------------
(owner of X == NULL, X linked in drain)

                                            bpf_list_push_back(other, X)
                                              __bpf_list_add: spin_lock()
                                              cmpxchg(X->owner, NULL,
                                                      POISON) -> OK
                                              list_add_tail(&X->list_head,
                                                            other_head)
                                                -> overwrites X->next,
                                                   X->prev, corrupts
                                                   other_head's chain
                                                   because X is still
                                                   stitched into drain

pos = drain.next; (may be X or neighbor using X's stale next)
list_del_init(pos); reads X->next/prev now pointing into other_head,
corrupts other_head's list and/or drain

Pre-patch, the owner pointer was left pointing at the old head (never cleared). Any concurrent push attempt cmpxchg(&node->owner, NULL, POISON) would fail and return -EINVAL, blocking re-insertion. This matches what bpf_rb_root_free() still does: it does NOT clear owner, so concurrent bpf_rbtree_add() cmpxchg fails.

The patch clears owner under the lock but still links the node into the unprotected local drain list, then drops the lock. The owner==NULL state now acts as a 'push is allowed' signal even though the node is still on a list, violating the invariant __bpf_list_add() relies on:

kernel/bpf/helpers.c:__bpf_list_add() {
    /* node->owner != NULL implies !list_empty(n), no need to separately
     * check the latter
     */
    if (cmpxchg(&node->owner, NULL, BPF_PTR_POISON)) {
            __bpf_obj_drop_impl((void *)n - off, rec, false);
            return -EINVAL;
    }

    tail ? list_add_tail(n, h) : list_add(n, h);
    WRITE_ONCE(node->owner, head);
}

Compounding this, 'drain' is a local stack variable: corruption can redirect list pointers into kernel stack memory of the freeing CPU, giving an unbounded list_add/list_del_init corruption vector.

Could you do list_del_init(pos) under spin_lock in the same critical section that clears owner? Swapping the order (list_del_init then WRITE_ONCE(owner, NULL), like __bpf_list_del() does), or leaving owner non-NULL like bpf_rb_root_free does so cmpxchg in concurrent push still fails?

The commit message's stated goal ('preventing subsequent users who hold a reference count to the node from acquiring an invalid next node') is only achievable if the delinking is done before unlock, not after.

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2283

Quoted 8 lines
kernel/bpf/helpers.c +2 -1
 		/* The contained type can also have resources, including a
 		 * bpf_list_head which needs to be freed.
 		 */
-		__bpf_obj_drop_impl(obj, field->graph_root.value_rec, false);
+		__bpf_obj_drop_impl((char *)pos - field->graph_root.node_offset,
+				    field->graph_root.value_rec, false);
 	}
 }

[email protected] raised a concern in v9 about whether this patch should include a Fixes tag. The CI bot suggested:

Fixes f0c5941ff5b2 ("bpf: Support bpf_list_head in map values")

since the concurrent reader issue was introduced when bpf_list_head support was added. The author responded in v10 that the issue only appears after the later patch 'bpf: Introduce the bpf_list_del kfunc' is merged, so no Fixes tag was added.

However, the reasoning suggests the Fixes tag should point to the bpf_list_del introduction rather than f0c5941ff5b2, and a Fixes tag may still be appropriate. Should this include a Fixes tag pointing to the commit that introduced bpf_list_del?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

maintainer
#7 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

On Tue, 2026-05-12 at 06:41 +0000, [email protected] wrote:

[...]

Quoted 15 lines

When a BPF program holds an owning or refcount-acquired reference to one of these nodes (node X), which is structurally supported because __bpf_obj_drop_impl() uses refcount_dec_and_test() and only frees at refcount 0, a concurrent push to a DIFFERENT bpf_list_head becomes a corruption:

CPU 0 (bpf_list_head_free, lock released) CPU 1 (BPF prog, refcount X)
----------------------------------------- ----------------------------
(owner of X == NULL, X linked in drain) bpf_list_push_back(other, X) __bpf_list_add: spin_lock() cmpxchg(X->owner, NULL, POISON) -> OK list_add_tail(&X->list_head, other_head)

Quoted 8 lines

                                                -> overwrites X->next, X->prev, corrupts other_head's chain because X is still stitched into drain
pos = drain.next; (may be X or neighbor using X's stale next)
list_del_init(pos); reads X->next/prev now pointing into other_head,
                       corrupts other_head's list and/or drain

Kaitao, this scenario seem plausible, could you please comment on it?

[...]

#8 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Quoted 29 lines · Eduard Zingerman

On Tue, 2026-05-12 at 06:41 +0000, [email protected] wrote:

[...]

Quoted 15 lines

When a BPF program holds an owning or refcount-acquired reference to one of these nodes (node X), which is structurally supported because __bpf_obj_drop_impl() uses refcount_dec_and_test() and only frees at refcount 0, a concurrent push to a DIFFERENT bpf_list_head becomes a corruption:

CPU 0 (bpf_list_head_free, lock released) CPU 1 (BPF prog, refcount X)
----------------------------------------- ----------------------------
(owner of X == NULL, X linked in drain) bpf_list_push_back(other, X) __bpf_list_add: spin_lock() cmpxchg(X->owner, NULL, POISON) -> OK list_add_tail(&X->list_head, other_head)

Quoted 8 lines

                                                -> overwrites X->next, X->prev, corrupts other_head's chain because X is still stitched into drain
pos = drain.next; (may be X or neighbor using X's stale next)
list_del_init(pos); reads X->next/prev now pointing into other_head,
                       corrupts other_head's list and/or drain

Quoted 1 line

Kaitao, this scenario seem plausible, could you please comment on it?

I think bot is correct.
This patch looks buggy.
It seems to me an optimization that breaks the concurrent logic. May be just drop this patch and reorder the other one, so that bot sees nonown suffix logic first.

#9 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Quoted 30 lines · Alexei Starovoitov
Quoted 29 lines · Eduard Zingerman

On Tue, 2026-05-12 at 06:41 +0000, [email protected] wrote:

[...]

Quoted 23 lines

When a BPF program holds an owning or refcount-acquired reference to one of these nodes (node X), which is structurally supported because __bpf_obj_drop_impl() uses refcount_dec_and_test() and only frees at refcount 0, a concurrent push to a DIFFERENT bpf_list_head becomes a corruption:

CPU 0 (bpf_list_head_free, lock released) CPU 1 (BPF prog, refcount X)
----------------------------------------- ----------------------------
(owner of X == NULL, X linked in drain) bpf_list_push_back(other, X) __bpf_list_add: spin_lock() cmpxchg(X->owner, NULL, POISON) -> OK list_add_tail(&X->list_head, other_head) -> overwrites X->next, X->prev, corrupts other_head's chain because X is still stitched into drain
pos = drain.next; (may be X or neighbor using X's stale next)
list_del_init(pos); reads X->next/prev now pointing into other_head,
                       corrupts other_head's list and/or drain

Quoted 7 lines
Quoted 1 line

Kaitao, this scenario seem plausible, could you please comment on it?

I think bot is correct.
This patch looks buggy.
It seems to me an optimization that breaks the concurrent logic. May be just drop this patch and reorder the other one, so that bot sees nonown suffix logic first.

This patch is still necessary because it addresses the problem discussed in this thread:
https://lore.kernel.org/all/[email protected]/

The patch does have a bug, however. To fix the issues we are seeing now, I propose the additional changes below and would appreciate feedback.

kernel/bpf/helpers.c +16 -2
@@ -2263,8 +2263,10 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        if (!head->next || list_empty(head))
                goto unlock;
        list_for_each_safe(pos, n, head) {
-               WRITE_ONCE(container_of(pos,
-                       struct bpf_list_node_kern, list_head)->owner, NULL);
+               struct bpf_list_node_kern *node;
+
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
+               WRITE_ONCE(node->owner, BPF_PTR_POISON);
                list_move_tail(pos, &drain);
        }
 unlock:
@@ -2272,8 +2274,12 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        __bpf_spin_unlock_irqrestore(spin_lock);
 
        while (!list_empty(&drain)) {
+               struct bpf_list_node_kern *node;
+
                pos = drain.next;
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
+               WRITE_ONCE(node->owner, NULL);
                /* The contained type can also have resources, including a
                 * bpf_list_head which needs to be freed.
                 */
@@ -2481,6 +2487,14 @@ static int __bpf_list_add(struct bpf_list_node_kern *node,
        if (unlikely(!h->next))
                INIT_LIST_HEAD(h);
 
+       /* bpf_list_head_free() marks nodes being detached with BPF_PTR_POISON
+        * before list_del_init().  cmpxchg(NULL, POISON) below would fail with
+        * that old value and fall into the generic error path, which wrongly
+        * calls __bpf_obj_drop_impl().  Reject POISON up front instead.
+        */
+       if (READ_ONCE(node->owner) == BPF_PTR_POISON)
+               return -EINVAL;
+
        /* node->owner != NULL implies !list_empty(n), no need to separately
         * check the latter
         */
Signature · 2 lines
Thanks
Kaitao Cheng
maintainer
#10 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Quoted 63 lines · Kaitao Cheng
Quoted 30 lines · Alexei Starovoitov
Quoted 29 lines · Eduard Zingerman

On Tue, 2026-05-12 at 06:41 +0000, [email protected] wrote:

[...]

Quoted 23 lines

When a BPF program holds an owning or refcount-acquired reference to one of these nodes (node X), which is structurally supported because __bpf_obj_drop_impl() uses refcount_dec_and_test() and only frees at refcount 0, a concurrent push to a DIFFERENT bpf_list_head becomes a corruption:

CPU 0 (bpf_list_head_free, lock released) CPU 1 (BPF prog, refcount X)
----------------------------------------- ----------------------------
(owner of X == NULL, X linked in drain) bpf_list_push_back(other, X) __bpf_list_add: spin_lock() cmpxchg(X->owner, NULL, POISON) -> OK list_add_tail(&X->list_head, other_head) -> overwrites X->next, X->prev, corrupts other_head's chain because X is still stitched into drain
pos = drain.next; (may be X or neighbor using X's stale next)
list_del_init(pos); reads X->next/prev now pointing into other_head,
                       corrupts other_head's list and/or drain

Quoted 7 lines
Quoted 1 line

Kaitao, this scenario seem plausible, could you please comment on it?

I think bot is correct.
This patch looks buggy.
It seems to me an optimization that breaks the concurrent logic. May be just drop this patch and reorder the other one, so that bot sees nonown suffix logic first.

This patch is still necessary because it addresses the problem discussed in this thread:
https://lore.kernel.org/all/[email protected]/

The patch does have a bug, however. To fix the issues we are seeing now, I propose the additional changes below and would appreciate feedback.

kernel/bpf/helpers.c +4 -2
@@ -2263,8 +2263,10 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        if (!head->next || list_empty(head))
                goto unlock;
        list_for_each_safe(pos, n, head) {
-               WRITE_ONCE(container_of(pos,
-                       struct bpf_list_node_kern, list_head)->owner, NULL);
+               struct bpf_list_node_kern *node;
+
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
+               WRITE_ONCE(node->owner, BPF_PTR_POISON);
                list_move_tail(pos, &drain);
        }
 unlock:

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2282

Quoted 9 lines
kernel/bpf/helpers.c +4 -0
@@ -2272,8 +2274,12 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        __bpf_spin_unlock_irqrestore(spin_lock);
 
        while (!list_empty(&drain)) {
+               struct bpf_list_node_kern *node;
+
                pos = drain.next;
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
+               WRITE_ONCE(node->owner, NULL);

I think this still leaves a short race window open.
Why does the .owner has field to be NULL?
Can the logic that implies for it to be NULL be extended to accept POISON as well?

Quoted 3 lines
kernel/bpf/helpers.c +0 -0
                /* The contained type can also have resources, including a
                 * bpf_list_head which needs to be freed.
                 */

[...]

#11 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Quoted 66 lines · Eduard Zingerman
Quoted 65 lines · Kaitao Cheng
Quoted 30 lines · Alexei Starovoitov
Quoted 29 lines · Eduard Zingerman

On Tue, 2026-05-12 at 06:41 +0000, [email protected] wrote:

[...]

Quoted 23 lines

When a BPF program holds an owning or refcount-acquired reference to one of these nodes (node X), which is structurally supported because __bpf_obj_drop_impl() uses refcount_dec_and_test() and only frees at refcount 0, a concurrent push to a DIFFERENT bpf_list_head becomes a corruption:

CPU 0 (bpf_list_head_free, lock released) CPU 1 (BPF prog, refcount X)
----------------------------------------- ----------------------------
(owner of X == NULL, X linked in drain) bpf_list_push_back(other, X) __bpf_list_add: spin_lock() cmpxchg(X->owner, NULL, POISON) -> OK list_add_tail(&X->list_head, other_head) -> overwrites X->next, X->prev, corrupts other_head's chain because X is still stitched into drain
pos = drain.next; (may be X or neighbor using X's stale next)
list_del_init(pos); reads X->next/prev now pointing into other_head,
                       corrupts other_head's list and/or drain

Quoted 7 lines
Quoted 1 line

Kaitao, this scenario seem plausible, could you please comment on it?

I think bot is correct.
This patch looks buggy.
It seems to me an optimization that breaks the concurrent logic. May be just drop this patch and reorder the other one, so that bot sees nonown suffix logic first.

This patch is still necessary because it addresses the problem discussed in this thread:
https://lore.kernel.org/all/[email protected]/

The patch does have a bug, however. To fix the issues we are seeing now, I propose the additional changes below and would appreciate feedback.

kernel/bpf/helpers.c +4 -2
@@ -2263,8 +2263,10 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        if (!head->next || list_empty(head))
                goto unlock;
        list_for_each_safe(pos, n, head) {
-               WRITE_ONCE(container_of(pos,
-                       struct bpf_list_node_kern, list_head)->owner, NULL);
+               struct bpf_list_node_kern *node;
+
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
+               WRITE_ONCE(node->owner, BPF_PTR_POISON);
                list_move_tail(pos, &drain);
        }
 unlock:
@@ -2272,8 +2274,12 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        __bpf_spin_unlock_irqrestore(spin_lock);
 
Quoted 12 lines

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2282

Quoted 7 lines
kernel/bpf/helpers.c +4 -0
        while (!list_empty(&drain)) {
+               struct bpf_list_node_kern *node;
+
                pos = drain.next;
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
+               WRITE_ONCE(node->owner, NULL);

I think this still leaves a short race window open.
Why does the .owner has field to be NULL?
Can the logic that implies for it to be NULL be extended to accept POISON as well?

Here, before setting owner to NULL, list_del_init() has already been executed, which means the node no longer belongs to any list. This should match the semantic meaning of owner == NULL.

Do you mean deleting WRITE_ONCE(node->owner, NULL) and preventing all subsequent __bpf_list_add() operations on this node?

Quoted 10 lines
Quoted 10 lines
kernel/bpf/helpers.c +0 -0
                /* The contained type can also have resources, including a
                 * bpf_list_head which needs to be freed.
                 */
@@ -2481,6 +2487,14 @@ static int __bpf_list_add(struct bpf_list_node_kern *node,
        if (unlikely(!h->next))
                INIT_LIST_HEAD(h);
 
Quoted 8 lines

kernel/bpf/helpers.c static int __bpf_list_add(struct bpf_list_node_kern *node, · line 2497

Quoted 8 lines
kernel/bpf/helpers.c +8 -0
+       /* bpf_list_head_free() marks nodes being detached with BPF_PTR_POISON
+        * before list_del_init().  cmpxchg(NULL, POISON) below would fail with
+        * that old value and fall into the generic error path, which wrongly
+        * calls __bpf_obj_drop_impl().  Reject POISON up front instead.
+        */
+       if (READ_ONCE(node->owner) == BPF_PTR_POISON)
+               return -EINVAL;
+

This code block is not needed; I will remove it.

Signature · 2 lines
Thanks
Kaitao Cheng
#12 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Quoted 82 lines · Kaitao Cheng
Quoted 66 lines · Eduard Zingerman
Quoted 65 lines · Kaitao Cheng
Quoted 30 lines · Alexei Starovoitov
Quoted 29 lines · Eduard Zingerman

On Tue, 2026-05-12 at 06:41 +0000, [email protected] wrote:

[...]

Quoted 23 lines

When a BPF program holds an owning or refcount-acquired reference to one of these nodes (node X), which is structurally supported because __bpf_obj_drop_impl() uses refcount_dec_and_test() and only frees at refcount 0, a concurrent push to a DIFFERENT bpf_list_head becomes a corruption:

CPU 0 (bpf_list_head_free, lock released) CPU 1 (BPF prog, refcount X)
----------------------------------------- ----------------------------
(owner of X == NULL, X linked in drain) bpf_list_push_back(other, X) __bpf_list_add: spin_lock() cmpxchg(X->owner, NULL, POISON) -> OK list_add_tail(&X->list_head, other_head) -> overwrites X->next, X->prev, corrupts other_head's chain because X is still stitched into drain
pos = drain.next; (may be X or neighbor using X's stale next)
list_del_init(pos); reads X->next/prev now pointing into other_head,
                       corrupts other_head's list and/or drain

Quoted 7 lines
Quoted 1 line

Kaitao, this scenario seem plausible, could you please comment on it?

I think bot is correct.
This patch looks buggy.
It seems to me an optimization that breaks the concurrent logic. May be just drop this patch and reorder the other one, so that bot sees nonown suffix logic first.

This patch is still necessary because it addresses the problem discussed in this thread:
https://lore.kernel.org/all/[email protected]/

The patch does have a bug, however. To fix the issues we are seeing now, I propose the additional changes below and would appreciate feedback.

kernel/bpf/helpers.c +4 -2
@@ -2263,8 +2263,10 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        if (!head->next || list_empty(head))
                goto unlock;
        list_for_each_safe(pos, n, head) {
-               WRITE_ONCE(container_of(pos,
-                       struct bpf_list_node_kern, list_head)->owner, NULL);
+               struct bpf_list_node_kern *node;
+
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
+               WRITE_ONCE(node->owner, BPF_PTR_POISON);
                list_move_tail(pos, &drain);
        }
 unlock:
@@ -2272,8 +2274,12 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        __bpf_spin_unlock_irqrestore(spin_lock);
 
Quoted 12 lines

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2282

Quoted 7 lines
kernel/bpf/helpers.c +4 -0
        while (!list_empty(&drain)) {
+               struct bpf_list_node_kern *node;
+
                pos = drain.next;
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
+               WRITE_ONCE(node->owner, NULL);

I think this still leaves a short race window open.
Why does the .owner has field to be NULL?
Can the logic that implies for it to be NULL be extended to accept POISON as well?

Quoted 6 lines

Here, before setting owner to NULL, list_del_init() has already been executed, which means the node no longer belongs to any list. This should match the semantic meaning of owner == NULL.

Do you mean deleting WRITE_ONCE(node->owner, NULL) and preventing all subsequent __bpf_list_add() operations on this node?

Hi Eduard Zingerman,

I’m not sure I fully understand your point. Could you please explain your suggestion in a bit more detail, or help clarify where the “short race window” you mentioned is left open?

Quoted 21 lines
Quoted 10 lines
Quoted 10 lines
kernel/bpf/helpers.c +0 -0
                /* The contained type can also have resources, including a
                 * bpf_list_head which needs to be freed.
                 */
@@ -2481,6 +2487,14 @@ static int __bpf_list_add(struct bpf_list_node_kern *node,
        if (unlikely(!h->next))
                INIT_LIST_HEAD(h);
 
Quoted 8 lines

kernel/bpf/helpers.c static int __bpf_list_add(struct bpf_list_node_kern *node, · line 2497

Quoted 8 lines
kernel/bpf/helpers.c +8 -0
+       /* bpf_list_head_free() marks nodes being detached with BPF_PTR_POISON
+        * before list_del_init().  cmpxchg(NULL, POISON) below would fail with
+        * that old value and fall into the generic error path, which wrongly
+        * calls __bpf_obj_drop_impl().  Reject POISON up front instead.
+        */
+       if (READ_ONCE(node->owner) == BPF_PTR_POISON)
+               return -EINVAL;
+

This code block is not needed; I will remove it.

Signature · 2 lines
Thanks
Kaitao Cheng
maintainer
#13 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

On Mon, 2026-05-18 at 11:02 +0800, Kaitao Cheng wrote:

[...]

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines

The patch does have a bug, however. To fix the issues we are seeing now, I propose the additional changes below and would appreciate feedback.

kernel/bpf/helpers.c +4 -2
@@ -2263,8 +2263,10 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        if (!head->next || list_empty(head))
                goto unlock;
        list_for_each_safe(pos, n, head) {
-               WRITE_ONCE(container_of(pos,
-                       struct bpf_list_node_kern, list_head)->owner, NULL);
+               struct bpf_list_node_kern *node;
+
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
+               WRITE_ONCE(node->owner, BPF_PTR_POISON);
                list_move_tail(pos, &drain);
        }
 unlock:
@@ -2272,8 +2274,12 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        __bpf_spin_unlock_irqrestore(spin_lock);
 
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2282

Quoted 7 lines
kernel/bpf/helpers.c +4 -0
        while (!list_empty(&drain)) {
+               struct bpf_list_node_kern *node;
+
                pos = drain.next;
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
+               WRITE_ONCE(node->owner, NULL);

Is CPU allowed to reorder the stores in list_del_init() and WRITE_ONCE()? If it is, I think there is a race here.

Thread #1:
enter bpf_list_head_free()
acquire H1 lock
list_move_tail(pos, &drain); // reordered
<-- ip here -->
WRITE_ONCE(node->owner, BPF_PTR_POISON); // reordered

Thread #2:

acquire H1 lock
n = bpf_refcount_acquire()
release H1 lock
acquire H2 lock
enter __bpf_list_add()
<-- ip here -->
cmpxchg(&node->owner, NULL, BPF_PTR_POISON)

[...]

#14 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Quoted 25 lines · Eduard Zingerman

On Mon, 2026-05-18 at 11:02 +0800, Kaitao Cheng wrote:

[...]

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines

The patch does have a bug, however. To fix the issues we are seeing now, I propose the additional changes below and would appreciate feedback.

kernel/bpf/helpers.c +4 -2
@@ -2263,8 +2263,10 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        if (!head->next || list_empty(head))
                goto unlock;
        list_for_each_safe(pos, n, head) {
-               WRITE_ONCE(container_of(pos,
-                       struct bpf_list_node_kern, list_head)->owner, NULL);
+               struct bpf_list_node_kern *node;
+
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
+               WRITE_ONCE(node->owner, BPF_PTR_POISON);
                list_move_tail(pos, &drain);
        }
 unlock:
@@ -2272,8 +2274,12 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        __bpf_spin_unlock_irqrestore(spin_lock);
 
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2282

Quoted 7 lines
kernel/bpf/helpers.c +4 -0
        while (!list_empty(&drain)) {
+               struct bpf_list_node_kern *node;
+
                pos = drain.next;
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
+               WRITE_ONCE(node->owner, NULL);

Is CPU allowed to reorder the stores in list_del_init() and WRITE_ONCE()? If it is, I think there is a race here.

Thanks for taking a close look at this. You are right that there is an ordering issue here, but I don't think the specific sequence illustrated by the example below is problematic.

Quoted 9 lines

Thread #1:
enter bpf_list_head_free()
acquire H1 lock
list_move_tail(pos, &drain); // reordered
<-- ip here -->
WRITE_ONCE(node->owner, BPF_PTR_POISON); // reordered

Thread #2:

Quoted 7 lines

acquire H1 lock
n = bpf_refcount_acquire()
release H1 lock
acquire H2 lock
enter __bpf_list_add()
<-- ip here -->
cmpxchg(&node->owner, NULL, BPF_PTR_POISON)

Even if the stores from list_move_tail(pos, &drain) become visible before WRITE_ONCE(node->owner, BPF_PTR_POISON), node->owner is not NULL in that window. Before the WRITE_ONCE(), it still points to H1. After the WRITE_ONCE(), it is BPF_PTR_POISON. In both cases, __bpf_list_add() will fail:

	cmpxchg(&node->owner, NULL, BPF_PTR_POISON)

because the old value is neither NULL nor expected to become NULL from this part of bpf_list_head_free().

However, I agree that your original concern about the ordering between list_del_init() and WRITE_ONCE(node->owner, NULL) is valid for the later drain loop:

	list_del_init(pos);
	WRITE_ONCE(node->owner, NULL);

Here owner == NULL is the signal that the node can be inserted into another list. Since WRITE_ONCE() does not provide release ordering, another CPU may observe owner == NULL and successfully acquire the node in __bpf_list_add() before the list_del_init() stores are visible. In that case __bpf_list_add() can link the node into H2, and the delayed stores from list_del_init() may then overwrite the node's list pointers and corrupt the H2 list.

So the fix should be to publish owner == NULL with release ordering after the node has been fully unlinked, for example:

```
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -2279,7 +2279,8 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
                pos = drain.next;
                node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
-               WRITE_ONCE(node->owner, NULL);
+               /* Ensure __bpf_list_add() sees the node as unlinked. */
+               smp_store_release(&node->owner, NULL);
                /* The contained type can also have resources, including a
                 * bpf_list_head which needs to be freed.
                 */
@@ -2607,7 +2608,8 @@ static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head,
                return NULL;

        list_del_init(n);
-       WRITE_ONCE(node->owner, NULL);
+       /* Ensure __bpf_list_add() sees the node as unlinked. */
+       smp_store_release(&node->owner, NULL);
        return (struct bpf_list_node *)n;
 }
```

The existing cmpxchg() in __bpf_list_add() is a successful RMW with return value, so it is fully ordered and is sufficient on the acquire side.

Signature · 2 lines
Thanks
Kaitao Cheng
maintainer
#15 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

Quoted 110 lines · Kaitao Cheng
Quoted 25 lines · Eduard Zingerman

On Mon, 2026-05-18 at 11:02 +0800, Kaitao Cheng wrote:

[...]

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines

The patch does have a bug, however. To fix the issues we are seeing now, I propose the additional changes below and would appreciate feedback.

kernel/bpf/helpers.c +4 -2
@@ -2263,8 +2263,10 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        if (!head->next || list_empty(head))
                goto unlock;
        list_for_each_safe(pos, n, head) {
-               WRITE_ONCE(container_of(pos,
-                       struct bpf_list_node_kern, list_head)->owner, NULL);
+               struct bpf_list_node_kern *node;
+
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
+               WRITE_ONCE(node->owner, BPF_PTR_POISON);
                list_move_tail(pos, &drain);
        }
 unlock:
@@ -2272,8 +2274,12 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
        __bpf_spin_unlock_irqrestore(spin_lock);
 
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2282

Quoted 7 lines
kernel/bpf/helpers.c +4 -0
        while (!list_empty(&drain)) {
+               struct bpf_list_node_kern *node;
+
                pos = drain.next;
+               node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
+               WRITE_ONCE(node->owner, NULL);

Is CPU allowed to reorder the stores in list_del_init() and WRITE_ONCE()? If it is, I think there is a race here.

Thanks for taking a close look at this. You are right that there is an ordering issue here, but I don't think the specific sequence illustrated by the example below is problematic.

Quoted 9 lines

Thread #1:
enter bpf_list_head_free()
acquire H1 lock
list_move_tail(pos, &drain); // reordered
<-- ip here -->
WRITE_ONCE(node->owner, BPF_PTR_POISON); // reordered

Thread #2:

Quoted 7 lines

acquire H1 lock
n = bpf_refcount_acquire()
release H1 lock
acquire H2 lock
enter __bpf_list_add()
<-- ip here -->
cmpxchg(&node->owner, NULL, BPF_PTR_POISON)

Even if the stores from list_move_tail(pos, &drain) become visible before WRITE_ONCE(node->owner, BPF_PTR_POISON), node->owner is not NULL in that window. Before the WRITE_ONCE(), it still points to H1. After the WRITE_ONCE(), it is BPF_PTR_POISON. In both cases, __bpf_list_add() will fail:

	cmpxchg(&node->owner, NULL, BPF_PTR_POISON)

because the old value is neither NULL nor expected to become NULL from this part of bpf_list_head_free().

However, I agree that your original concern about the ordering between list_del_init() and WRITE_ONCE(node->owner, NULL) is valid for the later drain loop:

	list_del_init(pos);
	WRITE_ONCE(node->owner, NULL);

Here owner == NULL is the signal that the node can be inserted into another list. Since WRITE_ONCE() does not provide release ordering, another CPU may observe owner == NULL and successfully acquire the node in __bpf_list_add() before the list_del_init() stores are visible. In that case __bpf_list_add() can link the node into H2, and the delayed stores from list_del_init() may then overwrite the node's list pointers and corrupt the H2 list.

So the fix should be to publish owner == NULL with release ordering after the node has been fully unlinked, for example:

```

kernel/bpf/helpers.c +4 -2
@@ -2279,7 +2279,8 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
                pos = drain.next;
                node = container_of(pos, struct bpf_list_node_kern, list_head);
                list_del_init(pos);
-               WRITE_ONCE(node->owner, NULL);
+               /* Ensure __bpf_list_add() sees the node as unlinked. */
+               smp_store_release(&node->owner, NULL);
                /* The contained type can also have resources, including a
                 * bpf_list_head which needs to be freed.
                 */
@@ -2607,7 +2608,8 @@ static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head,
                return NULL;
 
        list_del_init(n);
-       WRITE_ONCE(node->owner, NULL);
+       /* Ensure __bpf_list_add() sees the node as unlinked. */
+       smp_store_release(&node->owner, NULL);
        return (struct bpf_list_node *)n;
 }

```

Quoted 2 lines

The existing cmpxchg() in __bpf_list_add() is a successful RMW with return value, so it is fully ordered and is sufficient on the acquire side.

Hi Kaitao,

Thank you for the analysis. I agree with the smp_store_release() approach, could you please respin the series?

#16 lore raw

Re: [PATCH RESEND bpf-next v10 2/8] bpf: clear list node owner and unlink before drop

In reply to Kaitao cheng
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Critical] Race condition causing list corruption, infinite loop, and Use-After-Free in…

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] Race condition causing list corruption, infinite loop, and Use-After-Free in bpf_list_head_free.
- [Critical] Incomplete fix leaving bpf_rb_root_free vulnerable to Use-After-Free via dangling owner pointers.
- [High] Unbounded O(N) latency under IRQ-disabled spinlock in bpf_list_head_free.
--

commit 1bc93a76541e332d30d80430964ef29571e93654
Author: Kaitao Cheng <[email protected]>

bpf: clear list node owner and unlink before drop

This commit updates bpf_list_head_free to clear the owner pointer of each node and use a local drain list before dropping the objects, aiming to prevent concurrent readers from seeing an invalid next node.

Quoted 3 lines
kernel/bpf/helpers.c +0 -0

[ ... ]

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2270

Quoted 11 lines
kernel/bpf/helpers.c +5 -1
@@ -2261,20 +2262,23 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head,
 	__bpf_spin_lock_irqsave(spin_lock);
 	if (!head->next || list_empty(head))
 		goto unlock;
-	head = head->next;
+	list_for_each_safe(pos, n, head) {
+		WRITE_ONCE(container_of(pos,
+			struct bpf_list_node_kern, list_head)->owner, NULL);
+		list_move_tail(pos, &drain);
+	}
 unlock:

Does this introduce an unbounded delay with interrupts disabled?

The original code detached the head in O(1) time. Now there is an O(N) loop inside __bpf_spin_lock_irqsave(). Since BPF lists can grow quite large, could this cause soft lockups?

This also appears to contradict the comment just above the lock which explicitly mentions doing the actual list draining outside the lock to not hold it for too long.

kernel/bpf/helpers.c void bpf_list_head_free(const struct btf_field *field, void *list_head, · line 2276

Quoted 12 lines
kernel/bpf/helpers.c +4 -6
-	INIT_LIST_HEAD(orig_head);
+	INIT_LIST_HEAD(head);
 	__bpf_spin_unlock_irqrestore(spin_lock);
 
-	while (head != orig_head) {
-		void *obj = head;
-
-		obj -= field->graph_root.node_offset;
-		head = head->next;
+	while (!list_empty(&drain)) {
+		pos = drain.next;
+		list_del_init(pos);

Can this lead to list corruption and an infinite loop?

By setting the node's owner to NULL in the previous loop under the lock, the node appears unlinked to the BPF subsystem. Once the lock is released at the unlock label, a concurrent BPF program holding a reference to this node might successfully pass it to bpf_list_push_front() or similar, since the ownership check will now succeed.

If that happens, list_add() will overwrite the node's next and prev pointers. However, the local drain list header isn't updated. The drain.next pointer will still point to this hijacked node, and list_empty(&drain) will remain false forever, causing an infinite loop that repeatedly unlinks and drops the exact same node.

[ ... ]

On a related note, does bpf_rb_root_free() suffer from a similar issue regarding dangling owner pointers?

While not introduced by this patch, if a surviving RB node's original bpf_rb_root memory is freed and then reallocated as another bpf_rb_root, the node's owner pointer might falsely match the reallocated root.

A concurrent BPF program could then pass this node to bpf_rbtree_remove(), bypassing the ownership check and accessing freed memory during rb_erase_cached().

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#17 lore raw

PATCH RESEND v10 3/8 bpf: Introduce the bpf_list_del kfunc.

In reply to Kaitao cheng

From: Kaitao Cheng <[email protected]>

Allow users to remove any node from a linked list.

We have added an additional parameter bpf_list_head *head to bpf_list_del, as the verifier requires the head parameter to check whether the lock is being held.

---
 kernel/bpf/helpers.c  | 10 ++++++++++
 kernel/bpf/verifier.c |  6 +++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
kernel/bpf/helpers.c +10 -0
index 1e8754877dd1..51b6ea4bb8cb 100644
@@ -2593,6 +2593,15 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 	return __bpf_list_del(head, h->prev);
 }
 
+__bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head,
+					       struct bpf_list_node *node)
+{
+	struct bpf_list_node_kern *kn = (void *)node;
+
+	/* verifier guarantees node is a list node rather than list head */
+	return __bpf_list_del(head, &kn->list_head);
+}
+
 __bpf_kfunc struct bpf_list_node *bpf_list_front(struct bpf_list_head *head)
 {
 	struct list_head *h = (struct list_head *)head;
@@ -4725,6 +4734,7 @@ BTF_ID_FLAGS(func, bpf_list_push_back, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_push_back_impl)
 BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL)
kernel/bpf/verifier.c +5 -1
index 03f9e16c2abe..3c0e0076bd69 100644
@@ -10744,6 +10744,7 @@ enum special_kfunc_type {
 	KF_bpf_list_push_back,
 	KF_bpf_list_pop_front,
 	KF_bpf_list_pop_back,
+	KF_bpf_list_del,
 	KF_bpf_list_front,
 	KF_bpf_list_back,
 	KF_bpf_cast_to_kern_ctx,
@@ -10812,6 +10813,7 @@ BTF_ID(func, bpf_list_push_back_impl)
 BTF_ID(func, bpf_list_push_back)
 BTF_ID(func, bpf_list_pop_front)
 BTF_ID(func, bpf_list_pop_back)
+BTF_ID(func, bpf_list_del)
 BTF_ID(func, bpf_list_front)
 BTF_ID(func, bpf_list_back)
 BTF_ID(func, bpf_cast_to_kern_ctx)
@@ -11334,6 +11336,7 @@ static bool is_bpf_list_api_kfunc(u32 btf_id)
 	return is_bpf_list_push_kfunc(btf_id) ||
 	       btf_id == special_kfunc_list[KF_bpf_list_pop_front] ||
 	       btf_id == special_kfunc_list[KF_bpf_list_pop_back] ||
+	       btf_id == special_kfunc_list[KF_bpf_list_del] ||
 	       btf_id == special_kfunc_list[KF_bpf_list_front] ||
 	       btf_id == special_kfunc_list[KF_bpf_list_back];
 }
@@ -11456,7 +11459,8 @@ static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env,
 
 	switch (node_field_type) {
 	case BPF_LIST_NODE:
-		ret = is_bpf_list_push_kfunc(kfunc_btf_id);
+		ret = is_bpf_list_push_kfunc(kfunc_btf_id) ||
+		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_del];
 		break;
 	case BPF_RB_NODE:
 		ret = (is_bpf_rbtree_add_kfunc(kfunc_btf_id) ||
Signature · 1 line
2.50.1 (Apple Git-155)
#18 lore raw

Re: [PATCH RESEND bpf-next v10 3/8] bpf: Introduce the bpf_list_del kfunc.

Looking at bpf_list_pop_front() and bpf_list_pop_back() in the same file, both functions read from the list_head (h->next or h->prev) before passing it to __bpf_list_del(). If the list is…
Quoted 7 lines
kernel/bpf/helpers.c +0 -0
index 1e8754877dd1..51b6ea4bb8cb 100644
@@ -2593,6 +2593,15 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 	return __bpf_list_del(head, h->prev);
 }

kernel/bpf/helpers.c __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) · line 2604

Quoted 10 lines
kernel/bpf/helpers.c +9 -0
 
+__bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head,
+					       struct bpf_list_node *node)
+{
+	struct bpf_list_node_kern *kn = (void *)node;
+
+	/* verifier guarantees node is a list node rather than list head */
+	return __bpf_list_del(head, &kn->list_head);
+}
+

Looking at bpf_list_pop_front() and bpf_list_pop_back() in the same file, both functions read from the list_head (h->next or h->prev) before passing it to __bpf_list_del(). If the list is uninitialized, they end up passing potentially invalid pointer values to __bpf_list_del(), which then has to handle this by initializing the list and returning NULL.

While __bpf_list_del() handles this correctly, would it be clearer to move the initialization check into the top-level kfuncs (like bpf_list_pop_front/back) so that __bpf_list_del() always receives a valid list_head pointer rather than potentially garbage values that get ignored later?

Reference: https://lore.kernel.org/bpf/af50d188f6bc2de81fe3c321b26a645a9425ba9f2644cb6f8525d5bb637385f9@mail.kernel.org/

Quoted 8 lines
kernel/bpf/helpers.c +1 -0
@@ -4725,6 +4734,7 @@ BTF_ID_FLAGS(func, bpf_list_push_back, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_push_back_impl)
 BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL)

[ ... ]

Quoted 12 lines
kernel/bpf/verifier.c +1 -0
index 11054ad89c14..a6bcbe68cd72 100644
@@ -10744,6 +10744,7 @@ enum special_kfunc_type {
 	KF_bpf_list_push_back,
 	KF_bpf_list_pop_front,
 	KF_bpf_list_pop_back,
+	KF_bpf_list_del,
 	KF_bpf_list_front,
 	KF_bpf_list_back,
 	KF_bpf_cast_to_kern_ctx,

[ ... ]

kernel/bpf/verifier.c static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env, · line 11466

Quoted 10 lines
kernel/bpf/verifier.c +2 -1
@@ -11456,7 +11459,8 @@ static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env,
 
 	switch (node_field_type) {
 	case BPF_LIST_NODE:
-		ret = is_bpf_list_push_kfunc(kfunc_btf_id);
+		ret = is_bpf_list_push_kfunc(kfunc_btf_id) ||
+		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_del];
 		break;
 	case BPF_RB_NODE:
 		ret = (is_bpf_rbtree_add_kfunc(kfunc_btf_id) ||

Can bpf_list_del actually be called from BPF programs? Looking at the verifier's handling of KF_ARG_PTR_TO_LIST_NODE arguments in check_kfunc_args():

	case KF_ARG_PTR_TO_LIST_NODE:
		...
		if (!reg->ref_obj_id) {
			verbose(env, "allocated object must be referenced\n");
			return -EINVAL;
		}

This code path unconditionally requires an owning reference (ref_obj_id must be non-zero). However, once a node is added to the list via bpf_list_push_front/back, the owning reference is converted to a non-owning reference (ref_obj_id becomes 0). This means bpf_list_del cannot be called on nodes that are actually in the list.

The sister kfunc bpf_rbtree_remove handles this by having a special branch in check_kfunc_args() that accepts non-owning references for that specific kfunc. Does bpf_list_del need similar handling to allow removing nodes that are actually in the list?

Reference: https://lore.kernel.org/bpf/294242698d576e4fedf243ab055c6f9a984e55033933a9bd09808f62a7018e9d@mail.kernel.org/

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

#19 lore raw

Re: [PATCH RESEND bpf-next v10 3/8] bpf: Introduce the bpf_list_del kfunc.

I have clarified this issue in the [PATCH bpf-next v10 3/8] patch, with the link provided below:

https://lore.kernel.org/bpf/[email protected]/

Quoted 99 lines · 在 2026/5/12 14:41, [email protected] 写道:
Quoted 7 lines
kernel/bpf/helpers.c +0 -0
index 1e8754877dd1..51b6ea4bb8cb 100644
@@ -2593,6 +2593,15 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 	return __bpf_list_del(head, h->prev);
 }

kernel/bpf/helpers.c __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) · line 2604

Quoted 10 lines
kernel/bpf/helpers.c +9 -0
 
+__bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head,
+					       struct bpf_list_node *node)
+{
+	struct bpf_list_node_kern *kn = (void *)node;
+
+	/* verifier guarantees node is a list node rather than list head */
+	return __bpf_list_del(head, &kn->list_head);
+}
+

Looking at bpf_list_pop_front() and bpf_list_pop_back() in the same file, both functions read from the list_head (h->next or h->prev) before passing it to __bpf_list_del(). If the list is uninitialized, they end up passing potentially invalid pointer values to __bpf_list_del(), which then has to handle this by initializing the list and returning NULL.

While __bpf_list_del() handles this correctly, would it be clearer to move the initialization check into the top-level kfuncs (like bpf_list_pop_front/back) so that __bpf_list_del() always receives a valid list_head pointer rather than potentially garbage values that get ignored later?

Reference: https://lore.kernel.org/bpf/af50d188f6bc2de81fe3c321b26a645a9425ba9f2644cb6f8525d5bb637385f9@mail.kernel.org/

Quoted 8 lines
kernel/bpf/helpers.c +1 -0
@@ -4725,6 +4734,7 @@ BTF_ID_FLAGS(func, bpf_list_push_back, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_push_back_impl)
 BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL)

[ ... ]

Quoted 12 lines
kernel/bpf/verifier.c +1 -0
index 11054ad89c14..a6bcbe68cd72 100644
@@ -10744,6 +10744,7 @@ enum special_kfunc_type {
 	KF_bpf_list_push_back,
 	KF_bpf_list_pop_front,
 	KF_bpf_list_pop_back,
+	KF_bpf_list_del,
 	KF_bpf_list_front,
 	KF_bpf_list_back,
 	KF_bpf_cast_to_kern_ctx,

[ ... ]

kernel/bpf/verifier.c static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env, · line 11466

Quoted 10 lines
kernel/bpf/verifier.c +2 -1
@@ -11456,7 +11459,8 @@ static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env,
 
 	switch (node_field_type) {
 	case BPF_LIST_NODE:
-		ret = is_bpf_list_push_kfunc(kfunc_btf_id);
+		ret = is_bpf_list_push_kfunc(kfunc_btf_id) ||
+		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_del];
 		break;
 	case BPF_RB_NODE:
 		ret = (is_bpf_rbtree_add_kfunc(kfunc_btf_id) ||

Can bpf_list_del actually be called from BPF programs? Looking at the verifier's handling of KF_ARG_PTR_TO_LIST_NODE arguments in check_kfunc_args():

	case KF_ARG_PTR_TO_LIST_NODE:
		...
		if (!reg->ref_obj_id) {
			verbose(env, "allocated object must be referenced\n");
			return -EINVAL;
		}

This code path unconditionally requires an owning reference (ref_obj_id must be non-zero). However, once a node is added to the list via bpf_list_push_front/back, the owning reference is converted to a non-owning reference (ref_obj_id becomes 0). This means bpf_list_del cannot be called on nodes that are actually in the list.

The sister kfunc bpf_rbtree_remove handles this by having a special branch in check_kfunc_args() that accepts non-owning references for that specific kfunc. Does bpf_list_del need similar handling to allow removing nodes that are actually in the list?

Reference: https://lore.kernel.org/bpf/294242698d576e4fedf243ab055c6f9a984e55033933a9bd09808f62a7018e9d@mail.kernel.org/

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

Signature · 2 lines
Thanks
Kaitao Cheng
maintainer Reviewed-by on v10 3/8
#20 lore raw

Re: [PATCH RESEND bpf-next v10 3/8] bpf: Introduce the bpf_list_del kfunc.

In reply to Kaitao cheng
Quoted 10 lines · Kaitao cheng

From: Kaitao Cheng <[email protected]>

Allow users to remove any node from a linked list.

We have added an additional parameter bpf_list_head *head to bpf_list_del, as the verifier requires the head parameter to check whether the lock is being held.

---

[...]

kernel/bpf/verifier.c static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env, · line 11463

Quoted 7 lines
kernel/bpf/verifier.c +2 -1
@@ -11456,7 +11459,8 @@ static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env,
 
 	switch (node_field_type) {
 	case BPF_LIST_NODE:
-		ret = is_bpf_list_push_kfunc(kfunc_btf_id);
+		ret = is_bpf_list_push_kfunc(kfunc_btf_id) ||
+		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_del];

Nit: to keep everything in the same style, maybe add a is_bpf_list_del_kfunc() helper?

Quoted 3 lines
kernel/bpf/verifier.c +0 -0
 		break;
 	case BPF_RB_NODE:
 		ret = (is_bpf_rbtree_add_kfunc(kfunc_btf_id) ||
#21 lore raw

PATCH RESEND v10 4/8 bpf: refactor __bpf_list_add to take insertion point via **prev_ptr

In reply to Kaitao cheng

From: Kaitao Cheng <[email protected]>

Refactor __bpf_list_add to accept (node, head, struct list_head **prev_ptr, ..) instead of (node, head, bool tail, ..). Load prev from *prev_ptr after INIT_LIST_HEAD(h), so we never dereference an uninitialized h->prev when head was 0-initialized (e.g. push_back passes &h->prev).

When prev is not the list head, validate that prev is in the list via its owner.

Prepares for bpf_list_add(head, new, prev, ..) to insert after a given list node.

---
 kernel/bpf/helpers.c | 36 ++++++++++++++++++++++++++----------
 1 file changed, 26 insertions(+), 10 deletions(-)
kernel/bpf/helpers.c +26 -10
index 51b6ea4bb8cb..5388078f3171 100644
@@ -2471,9 +2471,11 @@ __bpf_kfunc void *bpf_refcount_acquire_impl(void *p__refcounted_kptr, void *meta
 
 static int __bpf_list_add(struct bpf_list_node_kern *node,
 			  struct bpf_list_head *head,
-			  bool tail, struct btf_record *rec, u64 off)
+			  struct list_head **prev_ptr,
+			  struct btf_record *rec, u64 off)
 {
 	struct list_head *n = &node->list_head, *h = (void *)head;
+	struct list_head *prev;
 
 	/* If list_head was 0-initialized by map, bpf_obj_init_field wasn't
 	 * called on its fields, so init here
@@ -2481,19 +2483,31 @@ static int __bpf_list_add(struct bpf_list_node_kern *node,
 	if (unlikely(!h->next))
 		INIT_LIST_HEAD(h);
 
+	prev = *prev_ptr;
+
+	/* When prev is not the list head, it must be a node in this list. */
+	if (prev != h) {
+		struct bpf_list_node_kern *prev_kn =
+			container_of(prev, struct bpf_list_node_kern, list_head);
+
+		if (unlikely(READ_ONCE(prev_kn->owner) != head))
+			goto fail;
+	}
+
 	/* node->owner != NULL implies !list_empty(n), no need to separately
 	 * check the latter
 	 */
-	if (cmpxchg(&node->owner, NULL, BPF_PTR_POISON)) {
-		/* Only called from BPF prog, no need to migrate_disable */
-		__bpf_obj_drop_impl((void *)n - off, rec, false);
-		return -EINVAL;
-	}
+	if (cmpxchg(&node->owner, NULL, BPF_PTR_POISON))
+		goto fail;
 
-	tail ? list_add_tail(n, h) : list_add(n, h);
+	list_add(n, prev);
 	WRITE_ONCE(node->owner, head);
-
 	return 0;
+
+fail:
+	/* Only called from BPF prog, no need to migrate_disable */
+	__bpf_obj_drop_impl((void *)n - off, rec, false);
+	return -EINVAL;
 }
 
 /**
@@ -2514,8 +2528,9 @@ __bpf_kfunc int bpf_list_push_front(struct bpf_list_head *head,
 				    u64 off)
 {
 	struct bpf_list_node_kern *n = (void *)node;
+	struct list_head *h = (void *)head;
 
-	return __bpf_list_add(n, head, false, meta ? meta->record : NULL, off);
+	return __bpf_list_add(n, head, &h, meta ? meta->record : NULL, off);
 }
 
 __bpf_kfunc int bpf_list_push_front_impl(struct bpf_list_head *head,
@@ -2543,8 +2558,9 @@ __bpf_kfunc int bpf_list_push_back(struct bpf_list_head *head,
 				   u64 off)
 {
 	struct bpf_list_node_kern *n = (void *)node;
+	struct list_head *h = (void *)head;
 
-	return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off);
+	return __bpf_list_add(n, head, &h->prev, meta ? meta->record : NULL, off);
 }
 
 __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
Signature · 1 line
2.50.1 (Apple Git-155)
maintainer Reviewed-by on v10 4/8
#22 lore raw

Re: [PATCH RESEND bpf-next v10 4/8] bpf: refactor __bpf_list_add to take insertion point via **prev_ptr

Quoted 7 lines · Kaitao cheng

From: Kaitao Cheng <[email protected]>

Refactor __bpf_list_add to accept (node, head, struct list_head **prev_ptr, ..) instead of (node, head, bool tail, ..). Load prev from *prev_ptr after INIT_LIST_HEAD(h), so we never dereference an uninitialized h->prev when head was 0-initialized (e.g. push_back passes &h->prev).

Quoted 8 lines

When prev is not the list head, validate that prev is in the list via its owner.

Prepares for bpf_list_add(head, new, prev, ..) to insert after a given list node.

---

[...]

#23 lore raw

PATCH RESEND v10 5/8 bpf: Add bpf_list_add to insert node after a given list node

In reply to Kaitao cheng

From: Kaitao Cheng <[email protected]>

Add a new kfunc bpf_list_add(head, new, prev, meta, off) that inserts 'new' after 'prev' in the BPF linked list. Both must be in the same list; 'prev' must already be in the list. The new node must be an owning reference (e.g. from bpf_obj_new); the kfunc consumes that reference and the node becomes non-owning once inserted.

We have added an additional parameter bpf_list_head *head to bpf_list_add, as the verifier requires the head parameter to check whether the lock is being held.

Returns 0 on success, -EINVAL if 'prev' is not in a list or 'new' is already in a list (or duplicate insertion). On failure, the kernel drops the passed-in node.

---
 kernel/bpf/helpers.c  | 11 +++++++++++
 kernel/bpf/verifier.c | 12 +++++++++---
 2 files changed, 20 insertions(+), 3 deletions(-)
kernel/bpf/helpers.c +11 -0
index 5388078f3171..2b8e8d4284a5 100644
@@ -2570,6 +2570,16 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 	return bpf_list_push_back(head, node, meta__ign, off);
 }
 
+__bpf_kfunc int bpf_list_add(struct bpf_list_head *head, struct bpf_list_node *new,
+			     struct bpf_list_node *prev, struct btf_struct_meta *meta,
+			     u64 off)
+{
+	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev;
+	struct list_head *prev_ptr = &p->list_head;
+
+	return __bpf_list_add(n, head, &prev_ptr, meta ? meta->record : NULL, off);
+}
+
 static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head,
 					    struct list_head *n)
 {
@@ -4748,6 +4758,7 @@ BTF_ID_FLAGS(func, bpf_list_push_front, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_push_front_impl)
 BTF_ID_FLAGS(func, bpf_list_push_back, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_push_back_impl)
+BTF_ID_FLAGS(func, bpf_list_add, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL)
kernel/bpf/verifier.c +9 -3
index 3c0e0076bd69..50f8732aa065 100644
@@ -10742,6 +10742,7 @@ enum special_kfunc_type {
 	KF_bpf_list_push_front,
 	KF_bpf_list_push_back_impl,
 	KF_bpf_list_push_back,
+	KF_bpf_list_add,
 	KF_bpf_list_pop_front,
 	KF_bpf_list_pop_back,
 	KF_bpf_list_del,
@@ -10811,6 +10812,7 @@ BTF_ID(func, bpf_list_push_front_impl)
 BTF_ID(func, bpf_list_push_front)
 BTF_ID(func, bpf_list_push_back_impl)
 BTF_ID(func, bpf_list_push_back)
+BTF_ID(func, bpf_list_add)
 BTF_ID(func, bpf_list_pop_front)
 BTF_ID(func, bpf_list_pop_back)
 BTF_ID(func, bpf_list_del)
@@ -10923,7 +10925,8 @@ static bool is_bpf_list_push_kfunc(u32 func_id)
 	return func_id == special_kfunc_list[KF_bpf_list_push_front] ||
 	       func_id == special_kfunc_list[KF_bpf_list_push_front_impl] ||
 	       func_id == special_kfunc_list[KF_bpf_list_push_back] ||
-	       func_id == special_kfunc_list[KF_bpf_list_push_back_impl];
+	       func_id == special_kfunc_list[KF_bpf_list_push_back_impl] ||
+	       func_id == special_kfunc_list[KF_bpf_list_add];
 }
 
 static bool is_bpf_rbtree_add_kfunc(u32 func_id)
@@ -19228,8 +19231,11 @@ int bpf_fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		int struct_meta_reg = BPF_REG_3;
 		int node_offset_reg = BPF_REG_4;
 
-		/* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */
-		if (is_bpf_rbtree_add_kfunc(desc->func_id)) {
+		/* list_add/rbtree_add have an extra arg (prev/less),
+		 * so args-to-fixup are in diff regs.
+		 */
+		if (desc->func_id == special_kfunc_list[KF_bpf_list_add] ||
+		    is_bpf_rbtree_add_kfunc(desc->func_id)) {
 			struct_meta_reg = BPF_REG_4;
 			node_offset_reg = BPF_REG_5;
 		}
Signature · 1 line
2.50.1 (Apple Git-155)
#24 lore raw

Re: [PATCH RESEND bpf-next v10 5/8] bpf: Add bpf_list_add to insert node after a given list node

The commit message states that 'prev' must already be in the list. How can a BPF program obtain such a reference and pass it to this kfunc?

kernel/bpf/helpers.c __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head, · line 2575

Quoted 10 lines
kernel/bpf/helpers.c +3 -0
@@ -2570,6 +2570,16 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 	return bpf_list_push_back(head, node, meta__ign, off);
 }
 
+__bpf_kfunc int bpf_list_add(struct bpf_list_head *head, struct bpf_list_node *new,
+			     struct bpf_list_node *prev, struct btf_struct_meta *meta,
+			     u64 off)

The commit message states that 'prev' must already be in the list. How can a BPF program obtain such a reference and pass it to this kfunc?

A node already in the list is tracked by the verifier as a non-owning reference (PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF with ref_obj_id == 0). The typical way to obtain this is via bpf_list_front() or bpf_list_back(), which return non-owning references.

However, check_kfunc_args() in kernel/bpf/verifier.c handles KF_ARG_PTR_TO_LIST_NODE by requiring owning references and rejects non-owning references.

Without a similar distinction for KF_ARG_PTR_TO_LIST_NODE (as exists for KF_ARG_PTR_TO_RB_NODE), is there any valid BPF register state that satisfies both the verifier check and the runtime semantics stated in the commit message?

The BPF CI bot reported this issue in v10. The author responded that this was fixed in [PATCH bpf-next v10 7/8], which introduces the __nonown_allowed annotation and corresponding verifier logic. However, patch 7/8 is not present in this commit (feed006fb969), so the parameter lacks the __nonown_allowed suffix and the verifier lacks the is_kfunc_arg_nonown_allowed() check.

Reference: https://lore.kernel.org/bpf/cb7368c6062232b6ad5f9ab1c6a97bc8530566a8b3db5c4deefdd5ea63d31538@mail.kernel.org/

Quoted 7 lines
kernel/bpf/helpers.c +7 -0
+{
+	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev;
+	struct list_head *prev_ptr = &p->list_head;
+
+	return __bpf_list_add(n, head, &prev_ptr, meta ? meta->record : NULL, off);
+}
+

[ ... ]

Quoted 3 lines
kernel/bpf/verifier.c +0 -0

[ ... ]

kernel/bpf/verifier.c struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) }; · line 19243

Quoted 9 lines
kernel/bpf/verifier.c +3 -2
@@ -19235,8 +19238,11 @@ struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) };
 		int struct_meta_reg = BPF_REG_3;
 		int node_offset_reg = BPF_REG_4;
 
-		/* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */
-		if (is_bpf_rbtree_add_kfunc(desc->func_id)) {
+		/* list_add/rbtree_add have an extra arg (prev/less),
+		 * so args-to-fixup are in diff regs.
+		 */

This isn't a bug, but the multi-line comment format doesn't follow the BPF subsystem style. The preferred form in kernel/bpf/ has the opening /* on its own line:

/*
* list_add/rbtree_add have an extra arg (prev/less),
* so args-to-fixup are in diff regs.
*/

kernel/bpf/verifier.c int bpf_fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 19241

Quoted 5 lines
kernel/bpf/verifier.c +2 -0
+		if (desc->func_id == special_kfunc_list[KF_bpf_list_add] ||
+		    is_bpf_rbtree_add_kfunc(desc->func_id)) {
 			struct_meta_reg = BPF_REG_4;
 			node_offset_reg = BPF_REG_5;
 		}

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

#25 lore raw

Re: [PATCH RESEND bpf-next v10 5/8] bpf: Add bpf_list_add to insert node after a given list node

Quoted 28 lines · 在 2026/5/12 14:41, [email protected] 写道:

kernel/bpf/helpers.c __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head, · line 2575

Quoted 10 lines
kernel/bpf/helpers.c +3 -0
@@ -2570,6 +2570,16 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 	return bpf_list_push_back(head, node, meta__ign, off);
 }
 
+__bpf_kfunc int bpf_list_add(struct bpf_list_head *head, struct bpf_list_node *new,
+			     struct bpf_list_node *prev, struct btf_struct_meta *meta,
+			     u64 off)

The commit message states that 'prev' must already be in the list. How can a BPF program obtain such a reference and pass it to this kfunc?

A node already in the list is tracked by the verifier as a non-owning reference (PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF with ref_obj_id == 0). The typical way to obtain this is via bpf_list_front() or bpf_list_back(), which return non-owning references.

However, check_kfunc_args() in kernel/bpf/verifier.c handles KF_ARG_PTR_TO_LIST_NODE by requiring owning references and rejects non-owning references.

Without a similar distinction for KF_ARG_PTR_TO_LIST_NODE (as exists for KF_ARG_PTR_TO_RB_NODE), is there any valid BPF register state that satisfies both the verifier check and the runtime semantics stated in the commit message?

Quoted 6 lines

The BPF CI bot reported this issue in v10. The author responded that this was fixed in [PATCH bpf-next v10 7/8], which introduces the __nonown_allowed annotation and corresponding verifier logic. However, patch 7/8 is not present in this commit (feed006fb969), so the parameter lacks the __nonown_allowed suffix and the verifier lacks the is_kfunc_arg_nonown_allowed() check.

If this is an issue that must be fixed, I will move [PATCH bpf-next v10 7/8] earlier in the next version.

Quoted 49 lines

Reference: https://lore.kernel.org/bpf/cb7368c6062232b6ad5f9ab1c6a97bc8530566a8b3db5c4deefdd5ea63d31538@mail.kernel.org/

Quoted 7 lines
kernel/bpf/helpers.c +7 -0
+{
+	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev;
+	struct list_head *prev_ptr = &p->list_head;
+
+	return __bpf_list_add(n, head, &prev_ptr, meta ? meta->record : NULL, off);
+}
+

[ ... ]

Quoted 3 lines
kernel/bpf/verifier.c +0 -0

[ ... ]

kernel/bpf/verifier.c struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) }; · line 19243

Quoted 9 lines
kernel/bpf/verifier.c +3 -2
@@ -19235,8 +19238,11 @@ struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) };
 		int struct_meta_reg = BPF_REG_3;
 		int node_offset_reg = BPF_REG_4;
 
-		/* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */
-		if (is_bpf_rbtree_add_kfunc(desc->func_id)) {
+		/* list_add/rbtree_add have an extra arg (prev/less),
+		 * so args-to-fixup are in diff regs.
+		 */

This isn't a bug, but the multi-line comment format doesn't follow the BPF subsystem style. The preferred form in kernel/bpf/ has the opening /* on its own line:

/*
* list_add/rbtree_add have an extra arg (prev/less),
* so args-to-fixup are in diff regs.
*/

kernel/bpf/verifier.c int bpf_fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 19241

Quoted 5 lines
kernel/bpf/verifier.c +2 -0
+		if (desc->func_id == special_kfunc_list[KF_bpf_list_add] ||
+		    is_bpf_rbtree_add_kfunc(desc->func_id)) {
 			struct_meta_reg = BPF_REG_4;
 			node_offset_reg = BPF_REG_5;
 		}

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

Signature · 2 lines
Thanks
Kaitao Cheng
#26 lore raw

Re: [PATCH RESEND bpf-next v10 5/8] bpf: Add bpf_list_add to insert node after a given list node

In reply to Kaitao cheng
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Critical] bpf_list_add drops the 'new' node on runtime failure, but the verifier unconditionally…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] bpf_list_add drops the 'new' node on runtime failure, but the verifier unconditionally converts 'new' to a non-owning reference, leading to a reliable Use-After-Free.
- [High] The bpf_list_add kfunc is marked with KF_IMPLICIT_ARGS but lacks the required _impl counterpart, causing the verifier to unconditionally reject it. --

commit 2a493a875461b6ae4bffbc4548a568dfaca01438
Author: Kaitao Cheng <[email protected]>

bpf: Add bpf_list_add to insert node after a given list node

This patch adds a new kfunc bpf_list_add() to insert a node after a given list node in a BPF linked list. The new node must be an owning reference which is consumed by the kfunc, and on failure, the kernel drops the passed-in node.

Quoted 7 lines
kernel/bpf/helpers.c +0 -0
index 5388078f31710..2b8e8d4284a54 100644
@@ -2570,6 +2570,16 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 	return bpf_list_push_back(head, node, meta__ign, off);
 }

kernel/bpf/helpers.c __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head, · line 2581

Quoted 10 lines
kernel/bpf/helpers.c +9 -0
 
+__bpf_kfunc int bpf_list_add(struct bpf_list_head *head, struct bpf_list_node *new,
+			     struct bpf_list_node *prev, struct btf_struct_meta *meta,
+			     u64 off)
+{
+	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev;
+	struct list_head *prev_ptr = &p->list_head;
+
+	return __bpf_list_add(n, head, &prev_ptr, meta ? meta->record : NULL, off);
+}

If __bpf_list_add() fails at runtime (e.g., because prev is not in the list indicated by head), it drops the 'new' node:

__bpf_list_add()
...
fail:

	__bpf_obj_drop_impl((void *)n - off, rec, false);
	return -EINVAL;

However, the verifier unconditionally converts 'new' into a non-owning reference during check_kfunc_call(). If __bpf_list_add() fails, the BPF program retains this non-owning reference to a freed node. Could this allow the program to trigger a use-after-free within the lock critical section?

[ ... ]

kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_list_push_front, KF_IMPLICIT_ARGS) · line 4761

Quoted 5 lines
kernel/bpf/helpers.c +1 -0
@@ -4748,6 +4758,7 @@ BTF_ID_FLAGS(func, bpf_list_push_front, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_push_front_impl)
 BTF_ID_FLAGS(func, bpf_list_push_back, KF_IMPLICIT_ARGS)
 BTF_ID_FLAGS(func, bpf_list_push_back_impl)
+BTF_ID_FLAGS(func, bpf_list_add, KF_IMPLICIT_ARGS)

Since bpf_list_add is marked with KF_IMPLICIT_ARGS, won't the verifier look for a bpf_list_add_impl counterpart during program load?

Without defining and registering a bpf_list_add_impl function, find_kfunc_impl_proto() will fail to find the required _impl suffix in the vmlinux BTF. Will this cause the verifier to unconditionally reject the kfunc?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
maintainer Reviewed-by on v10 5/8
#27 lore raw

Re: [PATCH RESEND bpf-next v10 5/8] bpf: Add bpf_list_add to insert node after a given list node

In reply to Kaitao cheng
Quoted 12 lines · Kaitao cheng

From: Kaitao Cheng <[email protected]>

Add a new kfunc bpf_list_add(head, new, prev, meta, off) that inserts 'new' after 'prev' in the BPF linked list. Both must be in the same list; 'prev' must already be in the list. The new node must be an owning reference (e.g. from bpf_obj_new); the kfunc consumes that reference and the node becomes non-owning once inserted.

We have added an additional parameter bpf_list_head *head to bpf_list_add, as the verifier requires the head parameter to check whether the lock is being held.

Quoted 6 lines

Returns 0 on success, -EINVAL if 'prev' is not in a list or 'new' is already in a list (or duplicate insertion). On failure, the kernel drops the passed-in node.

---

[...]

kernel/bpf/verifier.c int bpf_fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 19237

Quoted 10 lines
kernel/bpf/verifier.c +4 -2
@@ -19228,8 +19231,11 @@ int bpf_fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		int struct_meta_reg = BPF_REG_3;
 		int node_offset_reg = BPF_REG_4;
 
-		/* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */
-		if (is_bpf_rbtree_add_kfunc(desc->func_id)) {
+		/* list_add/rbtree_add have an extra arg (prev/less),
+		 * so args-to-fixup are in diff regs.
+		 */
+		if (desc->func_id == special_kfunc_list[KF_bpf_list_add] ||

Nit: same nit here, maybe add is_bpf_list_add_kfunc().

Quoted 4 lines
kernel/bpf/verifier.c +1 -0
+		    is_bpf_rbtree_add_kfunc(desc->func_id)) {
 			struct_meta_reg = BPF_REG_4;
 			node_offset_reg = BPF_REG_5;
 		}
#28 lore raw

PATCH RESEND v10 6/8 bpf: add bpf_list_is_first/last/empty kfuncs

In reply to Kaitao cheng

From: Kaitao Cheng <[email protected]>

Add three kfuncs for BPF linked list queries:
- bpf_list_is_first(head, node): true if node is the first in the list.
- bpf_list_is_last(head, node): true if node is the last in the list.
- bpf_list_empty(head): true if the list has no entries.

Currently, without these kfuncs, to implement the above functionality it is necessary to first call bpf_list_pop_front/back to retrieve the first or last node before checking whether the passed-in node was the first or last one. After the check, the node had to be pushed back into the list using bpf_list_push_front/back, which was very inefficient.

Now, with the bpf_list_is_first/last/empty kfuncs, we can directly check whether a node is the first, last, or whether the list is empty, without having to first retrieve the node.

---
 kernel/bpf/helpers.c  | 38 ++++++++++++++++++++++++++++++++++++++
 kernel/bpf/verifier.c | 15 +++++++++++++--
 2 files changed, 51 insertions(+), 2 deletions(-)
kernel/bpf/helpers.c +38 -0
index 2b8e8d4284a5..dfd465badd9d 100644
@@ -2648,6 +2648,41 @@ __bpf_kfunc struct bpf_list_node *bpf_list_back(struct bpf_list_head *head)
 	return (struct bpf_list_node *)h->prev;
 }
 
+__bpf_kfunc bool bpf_list_is_first(struct bpf_list_head *head, struct bpf_list_node *node)
+{
+	struct list_head *h = (struct list_head *)head;
+	struct bpf_list_node_kern *kn = (struct bpf_list_node_kern *)node;
+
+	if (READ_ONCE(kn->owner) != head)
+		return false;
+
+	return list_is_first(&kn->list_head, h);
+}
+
+__bpf_kfunc bool bpf_list_is_last(struct bpf_list_head *head, struct bpf_list_node *node)
+{
+	struct list_head *h = (struct list_head *)head;
+	struct bpf_list_node_kern *kn = (struct bpf_list_node_kern *)node;
+
+	if (READ_ONCE(kn->owner) != head)
+		return false;
+
+	return list_is_last(&kn->list_head, h);
+}
+
+__bpf_kfunc bool bpf_list_empty(struct bpf_list_head *head)
+{
+	struct list_head *h = (struct list_head *)head;
+
+	/* If list_head was 0-initialized by map, bpf_obj_init_field wasn't
+	 * called on its fields, so init here
+	 */
+	if (unlikely(!h->next))
+		INIT_LIST_HEAD(h);
+
+	return list_empty(h);
+}
+
 __bpf_kfunc struct bpf_rb_node *bpf_rbtree_remove(struct bpf_rb_root *root,
 						  struct bpf_rb_node *node)
 {
@@ -4764,6 +4799,9 @@ BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_front, KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_list_back, KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_list_is_first)
+BTF_ID_FLAGS(func, bpf_list_is_last)
+BTF_ID_FLAGS(func, bpf_list_empty)
 BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE)
 BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL)
kernel/bpf/verifier.c +13 -2
index 50f8732aa065..ca33f35bc3eb 100644
@@ -10748,6 +10748,9 @@ enum special_kfunc_type {
 	KF_bpf_list_del,
 	KF_bpf_list_front,
 	KF_bpf_list_back,
+	KF_bpf_list_is_first,
+	KF_bpf_list_is_last,
+	KF_bpf_list_empty,
 	KF_bpf_cast_to_kern_ctx,
 	KF_bpf_rdonly_cast,
 	KF_bpf_rcu_read_lock,
@@ -10818,6 +10821,9 @@ BTF_ID(func, bpf_list_pop_back)
 BTF_ID(func, bpf_list_del)
 BTF_ID(func, bpf_list_front)
 BTF_ID(func, bpf_list_back)
+BTF_ID(func, bpf_list_is_first)
+BTF_ID(func, bpf_list_is_last)
+BTF_ID(func, bpf_list_empty)
 BTF_ID(func, bpf_cast_to_kern_ctx)
 BTF_ID(func, bpf_rdonly_cast)
 BTF_ID(func, bpf_rcu_read_lock)
@@ -11341,7 +11347,10 @@ static bool is_bpf_list_api_kfunc(u32 btf_id)
 	       btf_id == special_kfunc_list[KF_bpf_list_pop_back] ||
 	       btf_id == special_kfunc_list[KF_bpf_list_del] ||
 	       btf_id == special_kfunc_list[KF_bpf_list_front] ||
-	       btf_id == special_kfunc_list[KF_bpf_list_back];
+	       btf_id == special_kfunc_list[KF_bpf_list_back] ||
+	       btf_id == special_kfunc_list[KF_bpf_list_is_first] ||
+	       btf_id == special_kfunc_list[KF_bpf_list_is_last] ||
+	       btf_id == special_kfunc_list[KF_bpf_list_empty];
 }
 
 static bool is_bpf_rbtree_api_kfunc(u32 btf_id)
@@ -11463,7 +11472,9 @@ static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env,
 	switch (node_field_type) {
 	case BPF_LIST_NODE:
 		ret = is_bpf_list_push_kfunc(kfunc_btf_id) ||
-		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_del];
+		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_del] ||
+		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_is_first] ||
+		      kfunc_btf_id == special_kfunc_list[KF_bpf_list_is_last];
 		break;
 	case BPF_RB_NODE:
 		ret = (is_bpf_rbtree_add_kfunc(kfunc_btf_id) ||
Signature · 1 line
2.50.1 (Apple Git-155)
maintainer Reviewed-by on v10 6/8
#29 lore raw

Re: [PATCH RESEND bpf-next v10 6/8] bpf: add bpf_list_is_first/last/empty kfuncs

Quoted 13 lines · Kaitao cheng

From: Kaitao Cheng <[email protected]>

Add three kfuncs for BPF linked list queries:
- bpf_list_is_first(head, node): true if node is the first in the list.
- bpf_list_is_last(head, node): true if node is the last in the list.
- bpf_list_empty(head): true if the list has no entries.

Currently, without these kfuncs, to implement the above functionality it is necessary to first call bpf_list_pop_front/back to retrieve the first or last node before checking whether the passed-in node was the first or last one. After the check, the node had to be pushed back into the list using bpf_list_push_front/back, which was very inefficient.

Quoted 7 lines

Now, with the bpf_list_is_first/last/empty kfuncs, we can directly check whether a node is the first, last, or whether the list is empty, without having to first retrieve the node.

---

[...]

#30 lore raw

PATCH RESEND v10 7/8 bpf: allow non-owning list-node args via __nonown_allowed

In reply to Kaitao cheng

From: Kaitao Cheng <[email protected]>

KF_ARG_PTR_TO_LIST_NODE normally requires an owning reference (PTR_TO_BTF_ID | MEM_ALLOC with ref_obj_id). Introduce and use the __nonown_allowed annotation on selected list-node arguments so non-owning references with ref_obj_id==0 are accepted as well.

This enables passing bpf_list_front() / bpf_list_back() results to:

bpf_list_add() as insertion point (prev)
bpf_list_del() as deletion target (node)
bpf_list_is_first/last() as query target (node)

Verifier keeps existing owning-ref checks by default; only arguments annotated with __nonown_allowed bypass MEM_ALLOC/ref_obj_id checks and then follow the same list-node validation path.

---
 Documentation/bpf/kfuncs.rst | 22 ++++++++++++++++++++--
 kernel/bpf/helpers.c         | 20 +++++++++++---------
 kernel/bpf/verifier.c        | 13 +++++++++++++
 3 files changed, 44 insertions(+), 11 deletions(-)
Documentation/bpf/kfuncs.rst +20 -2
index 75e6c078e0e7..3a9db1108b95 100644
@@ -207,8 +207,26 @@ Here, the buffer may be NULL. If the buffer is not NULL, it must be at least
 buffer__szk bytes in size. The kfunc is responsible for checking if the buffer
 is NULL before using it.
 
-2.3.5 __str Annotation
-----------------------------
+2.3.5 __nonown_allowed Annotation
+---------------------------------
+
+This annotation is used to indicate that the parameter may be a non-owning reference.
+
+An example is given below::
+
+        __bpf_kfunc int bpf_list_add(..., struct bpf_list_node
+                                     *prev__nonown_allowed, ...)
+        {
+                ...
+        }
+
+For the ``prev__nonown_allowed`` parameter (resolved as ``KF_ARG_PTR_TO_LIST_NODE``),
+suffix ``__nonown_allowed`` retains the usual owning-pointer rules and also
+permits a non-owning reference with no ref_obj_id (e.g. the return value of
+bpf_list_front() / bpf_list_back()).
+
+2.3.6 __str Annotation
+----------------------
 This annotation is used to indicate that the argument is a constant string.
 
 An example is given below::
kernel/bpf/helpers.c +11 -9
index dfd465badd9d..f2f8705f0e9a 100644
@@ -2571,10 +2571,10 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 }
 
 __bpf_kfunc int bpf_list_add(struct bpf_list_head *head, struct bpf_list_node *new,
-			     struct bpf_list_node *prev, struct btf_struct_meta *meta,
-			     u64 off)
+			     struct bpf_list_node *prev__nonown_allowed,
+			     struct btf_struct_meta *meta, u64 off)
 {
-	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev;
+	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev__nonown_allowed;
 	struct list_head *prev_ptr = &p->list_head;
 
 	return __bpf_list_add(n, head, &prev_ptr, meta ? meta->record : NULL, off);
@@ -2620,9 +2620,9 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 }
 
 __bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head,
-					       struct bpf_list_node *node)
+				struct bpf_list_node *node__nonown_allowed)
 {
-	struct bpf_list_node_kern *kn = (void *)node;
+	struct bpf_list_node_kern *kn = (void *)node__nonown_allowed;
 
 	/* verifier guarantees node is a list node rather than list head */
 	return __bpf_list_del(head, &kn->list_head);
@@ -2648,10 +2648,11 @@ __bpf_kfunc struct bpf_list_node *bpf_list_back(struct bpf_list_head *head)
 	return (struct bpf_list_node *)h->prev;
 }
 
-__bpf_kfunc bool bpf_list_is_first(struct bpf_list_head *head, struct bpf_list_node *node)
+__bpf_kfunc bool bpf_list_is_first(struct bpf_list_head *head,
+				   struct bpf_list_node *node__nonown_allowed)
 {
 	struct list_head *h = (struct list_head *)head;
-	struct bpf_list_node_kern *kn = (struct bpf_list_node_kern *)node;
+	struct bpf_list_node_kern *kn = (struct bpf_list_node_kern *)node__nonown_allowed;
 
 	if (READ_ONCE(kn->owner) != head)
 		return false;
@@ -2659,10 +2660,11 @@ __bpf_kfunc bool bpf_list_is_first(struct bpf_list_head *head, struct bpf_list_n
 	return list_is_first(&kn->list_head, h);
 }
 
-__bpf_kfunc bool bpf_list_is_last(struct bpf_list_head *head, struct bpf_list_node *node)
+__bpf_kfunc bool bpf_list_is_last(struct bpf_list_head *head,
+				  struct bpf_list_node *node__nonown_allowed)
 {
 	struct list_head *h = (struct list_head *)head;
-	struct bpf_list_node_kern *kn = (struct bpf_list_node_kern *)node;
+	struct bpf_list_node_kern *kn = (struct bpf_list_node_kern *)node__nonown_allowed;
 
 	if (READ_ONCE(kn->owner) != head)
 		return false;
kernel/bpf/verifier.c +13 -0
index ca33f35bc3eb..08ab337866bf 100644
@@ -10502,6 +10502,11 @@ static bool is_kfunc_arg_nullable(const struct btf *btf, const struct btf_param
 	return btf_param_match_suffix(btf, arg, "__nullable");
 }
 
+static bool is_kfunc_arg_nonown_allowed(const struct btf *btf, const struct btf_param *arg)
+{
+	return btf_param_match_suffix(btf, arg, "__nonown_allowed");
+}
+
 static bool is_kfunc_arg_const_str(const struct btf *btf, const struct btf_param *arg)
 {
 	return btf_param_match_suffix(btf, arg, "__str");
@@ -12017,6 +12022,13 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 				return ret;
 			break;
 		case KF_ARG_PTR_TO_LIST_NODE:
+			if (is_kfunc_arg_nonown_allowed(btf, &args[i]) &&
+			    type_is_non_owning_ref(reg->type) && !reg->ref_obj_id) {
+				/* Allow bpf_list_front/back return value for
+				 * __nonown_allowed list-node arguments.
+				 */
+				goto check_ok;
+			}
 			if (reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) {
 				verbose(env, "%s expected pointer to allocated object\n",
 					reg_arg_name(env, argno));
@@ -12026,6 +12038,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 				verbose(env, "allocated object must be referenced\n");
 				return -EINVAL;
 			}
+check_ok:
 			ret = process_kf_arg_ptr_to_list_node(env, reg, argno, meta);
 			if (ret < 0)
 				return ret;
Signature · 1 line
2.50.1 (Apple Git-155)
#31 lore raw

Re: [PATCH RESEND bpf-next v10 7/8] bpf: allow non-owning list-node args via __nonown_allowed

This isn't a bug, but the second parameter alignment is inconsistent with the style used in the rest of this file. Most multi-line function declarations in helpers.c align continuation lines under…
Quoted 4 lines
Documentation/bpf/kfuncs.rst +0 -0
index 75e6c078e0e7..3a9db1108b95 100644

[ ... ]

Quoted 4 lines
kernel/bpf/helpers.c +0 -0
index dfd465badd9d..f2f8705f0e9a 100644

[ ... ]

kernel/bpf/helpers.c __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) · line 2625

Quoted 9 lines
kernel/bpf/helpers.c +2 -2
@@ -2620,9 +2620,9 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 }
 
 __bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head,
-					       struct bpf_list_node *node)
+				struct bpf_list_node *node__nonown_allowed)
 {
-	struct bpf_list_node_kern *kn = (void *)node;
+	struct bpf_list_node_kern *kn = (void *)node__nonown_allowed;

This isn't a bug, but the second parameter alignment is inconsistent with the style used in the rest of this file. Most multi-line function declarations in helpers.c align continuation lines under the first parameter. The other functions modified in this same patch (bpf_list_add, bpf_list_is_first, bpf_list_is_last) follow that convention, but this one does not.

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index 71d50415f743..4d78d834c609 100644

[ ... ]

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 12030

Quoted 10 lines
kernel/bpf/verifier.c +7 -0
@@ -12017,6 +12022,13 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 			break;
 		case KF_ARG_PTR_TO_LIST_NODE:
+			if (is_kfunc_arg_nonown_allowed(btf, &args[i]) &&
+			    type_is_non_owning_ref(reg->type) && !reg->ref_obj_id) {
+				/* Allow bpf_list_front/back return value for
+				 * __nonown_allowed list-node arguments.
+				 */
+				goto check_ok;
+			}

A subsystem pattern flags this as potentially concerning: The BPF subsystem guide requests that multi-line comments in kernel/bpf/ files follow the modern kernel style with the opening /* on its own line, even when the surrounding file uses the old style. The comment you added begins text on the same line as the opening delimiter. While the comment itself is clear and correct, the preferred formatting for new comments in this directory would be:

				/*
				 * Allow bpf_list_front/back return value for
				 * __nonown_allowed list-node arguments.
				 */

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/25716874656

#32 lore raw

Re: [PATCH RESEND bpf-next v10 7/8] bpf: allow non-owning list-node args via __nonown_allowed

In reply to Kaitao cheng
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Critical] bpf_list_add() dynamically frees the 'new' node on failure, but the verifier unconditionally…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] bpf_list_add() dynamically frees the 'new' node on failure, but the verifier unconditionally assumes success, enabling a Use-After-Free.
- [Critical] bpf_list_del() does not invalidate the passed non-owning reference, leaving a direct alias to potentially freed memory (Use-After-Free). --

commit abe3e2cc841c335580b071d1f616bb2c9d1deba4
Author: Kaitao Cheng <[email protected]>

bpf: allow non-owning list-node args via __nonown_allowed

This commit introduces the __nonown_allowed annotation for list-node arguments. It allows bpf_list_add, bpf_list_del, and bpf_list_is_first/last to accept non-owning references.

Quoted 3 lines
kernel/bpf/helpers.c +0 -0

[ ... ]

Quoted 4 lines
kernel/bpf/helpers.c +0 -0
@@ -2571,10 +2571,10 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 }
 
 __bpf_kfunc int bpf_list_add(struct bpf_list_head *head, struct bpf_list_node *new,

kernel/bpf/helpers.c __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head, · line 2580

Quoted 10 lines
kernel/bpf/helpers.c +3 -3
-			     struct bpf_list_node *prev, struct btf_struct_meta *meta,
-			     u64 off)
+			     struct bpf_list_node *prev__nonown_allowed,
+			     struct btf_struct_meta *meta, u64 off)
 {
-	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev;
+	struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev__nonown_allowed;
 	struct list_head *prev_ptr = &p->list_head;
 
 	return __bpf_list_add(n, head, &prev_ptr, meta ? meta->record : NULL, off);

If a BPF program passes a non-owning reference from a completely different list (protected by the same spinlock) to prev__nonown_allowed, can this cause a use-after-free?

At runtime, __bpf_list_add() detects that the prev node doesn't belong to the target head and aborts the insertion, calling __bpf_obj_drop_impl() on the new node to free it back to the allocator.

However, the verifier unconditionally assumes bpf_list_add() successfully consumed the new owning reference and safely converts it into a non-owning reference in its internal state.

Because the verifier considers this non-owning reference valid until the spinlock is unlocked, does this allow the BPF program to continuously dereference the freed memory while holding the lock?

[ ... ]

kernel/bpf/helpers.c __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) · line 2628

Quoted 12 lines
kernel/bpf/helpers.c +2 -2
@@ -2620,9 +2620,9 @@ __bpf_kfunc struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head)
 }
 
 __bpf_kfunc struct bpf_list_node *bpf_list_del(struct bpf_list_head *head,
-					       struct bpf_list_node *node)
+				struct bpf_list_node *node__nonown_allowed)
 {
-	struct bpf_list_node_kern *kn = (void *)node;
+	struct bpf_list_node_kern *kn = (void *)node__nonown_allowed;
 
 	/* verifier guarantees node is a list node rather than list head */
 	return __bpf_list_del(head, &kn->list_head);

Does passing a non-owning reference here create an alias that leads to a use-after-free?

When bpf_list_del() removes the node from the list, it returns a new owning reference (with ref_obj_id > 0) pointing to the exact same object.

The verifier does not explicitly invalidate the original non-owning reference argument (which has ref_obj_id == 0), so it remains fully valid in the verifier's state until the active spinlock is released.

If the BPF program subsequently frees the object by passing the returned owning reference to bpf_obj_drop(), won't the original non-owning reference remain as a direct, unchecked alias to the freed memory?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
maintainer Reviewed-by on v10 7/8
#33 lore raw

Re: [PATCH RESEND bpf-next v10 7/8] bpf: allow non-owning list-node args via __nonown_allowed

In reply to Kaitao cheng
Quoted 13 lines · Kaitao cheng

From: Kaitao Cheng <[email protected]>

KF_ARG_PTR_TO_LIST_NODE normally requires an owning reference (PTR_TO_BTF_ID | MEM_ALLOC with ref_obj_id). Introduce and use the __nonown_allowed annotation on selected list-node arguments so non-owning references with ref_obj_id==0 are accepted as well.

This enables passing bpf_list_front() / bpf_list_back() results to:

bpf_list_add() as insertion point (prev)
bpf_list_del() as deletion target (node)
bpf_list_is_first/last() as query target (node)

Quoted 6 lines

Verifier keeps existing owning-ref checks by default; only arguments annotated with __nonown_allowed bypass MEM_ALLOC/ref_obj_id checks and then follow the same list-node validation path.

---

[...]

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 12026

Quoted 6 lines
kernel/bpf/verifier.c +2 -0
@@ -12017,6 +12022,13 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
 				return ret;
 			break;
 		case KF_ARG_PTR_TO_LIST_NODE:
+			if (is_kfunc_arg_nonown_allowed(btf, &args[i]) &&
+			    type_is_non_owning_ref(reg->type) && !reg->ref_obj_id) {

^^^^^^^^^^^^^^^^
Nit: I think this check is redundant, type_is_non_owning_ref() should suffice.

Quoted 8 lines
kernel/bpf/verifier.c +5 -0
+				/* Allow bpf_list_front/back return value for
+				 * __nonown_allowed list-node arguments.
+				 */
+				goto check_ok;
+			}
 			if (reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) {
 				verbose(env, "%s expected pointer to allocated object\n",
 					reg_arg_name(env, argno));

[...]

maintainer
#34 lore raw

Re: [PATCH RESEND bpf-next v10 7/8] bpf: allow non-owning list-node args via __nonown_allowed

Quoted 14 lines · Eduard Zingerman
Quoted 13 lines · Kaitao cheng

From: Kaitao Cheng <[email protected]>

KF_ARG_PTR_TO_LIST_NODE normally requires an owning reference (PTR_TO_BTF_ID | MEM_ALLOC with ref_obj_id). Introduce and use the __nonown_allowed annotation on selected list-node arguments so non-owning references with ref_obj_id==0 are accepted as well.

This enables passing bpf_list_front() / bpf_list_back() results to:

bpf_list_add() as insertion point (prev)
bpf_list_del() as deletion target (node)
bpf_list_is_first/last() as query target (node)

Quoted 6 lines
Quoted 6 lines

Verifier keeps existing owning-ref checks by default; only arguments annotated with __nonown_allowed bypass MEM_ALLOC/ref_obj_id checks and then follow the same list-node validation path.

---

Also, I think it is worthwhile to move this patch before the patches that introduce kfuncs that use the __nonown_allowed annotation. (In case if this series goes to v11).

#35 lore raw

PATCH RESEND v10 8/8 selftests/bpf: Add test cases for bpf_list_del/add/is_first/is_last/empty

In reply to Kaitao cheng

From: Kaitao Cheng <[email protected]>

Extend refcounted_kptr with tests for bpf_list_add (including prev from bpf_list_front and bpf_refcount_acquire), bpf_list_del (including node from bpf_list_front, bpf_rbtree_remove and bpf_refcount_acquire), bpf_list_empty, bpf_list_is_first/last, and push_back on uninit head.

To verify the validity of bpf_list_del/add, the test also expects the verifier to reject calls to bpf_list_del/add made without holding the spin_lock.

---
 .../selftests/bpf/progs/refcounted_kptr.c     | 421 ++++++++++++++++++
 1 file changed, 421 insertions(+)
tools/testing/selftests/bpf/progs/refcounted_kptr.c +421 -0
index c847398837cc..21ae06797b18 100644
@@ -367,6 +367,427 @@ long insert_rbtree_and_stash__del_tree_##rem_tree(void *ctx)		\
 INSERT_STASH_READ(true, "insert_stash_read: remove from tree");
 INSERT_STASH_READ(false, "insert_stash_read: don't remove from tree");
 
+SEC("tc")
+__description("list_empty_test: list empty before add, non-empty after add")
+__success __retval(0)
+int list_empty_test(void *ctx)
+{
+	struct node_data *node_new;
+
+	bpf_spin_lock(&lock);
+	if (!bpf_list_empty(&head)) {
+		bpf_spin_unlock(&lock);
+		return -1;
+	}
+	bpf_spin_unlock(&lock);
+
+	node_new = bpf_obj_new(typeof(*node_new));
+	if (!node_new)
+		return -2;
+
+	bpf_spin_lock(&lock);
+	bpf_list_push_front(&head, &node_new->l);
+
+	if (bpf_list_empty(&head)) {
+		bpf_spin_unlock(&lock);
+		return -3;
+	}
+	bpf_spin_unlock(&lock);
+	return 0;
+}
+
+static struct node_data *__add_in_list(struct bpf_list_head *head,
+				       struct bpf_spin_lock *lock)
+{
+	struct node_data *node_new, *node_ref;
+
+	node_new = bpf_obj_new(typeof(*node_new));
+	if (!node_new)
+		return NULL;
+
+	node_ref = bpf_refcount_acquire(node_new);
+
+	bpf_spin_lock(lock);
+	bpf_list_push_front(head, &node_new->l);
+	bpf_spin_unlock(lock);
+	return node_ref;
+}
+
+SEC("tc")
+__description("list_is_edge_test1: is_first on first node, is_last on last node")
+__success __retval(0)
+int list_is_edge_test1(void *ctx)
+{
+	struct node_data *node_first, *node_last;
+	int err = 0;
+
+	node_last = __add_in_list(&head, &lock);
+	if (!node_last)
+		return -1;
+
+	node_first = __add_in_list(&head, &lock);
+	if (!node_first) {
+		bpf_obj_drop(node_last);
+		return -2;
+	}
+
+	bpf_spin_lock(&lock);
+	if (!bpf_list_is_first(&head, &node_first->l)) {
+		err = -3;
+		goto fail;
+	}
+	if (!bpf_list_is_last(&head, &node_last->l))
+		err = -4;
+
+fail:
+	bpf_spin_unlock(&lock);
+	bpf_obj_drop(node_first);
+	bpf_obj_drop(node_last);
+	return err;
+}
+
+SEC("tc")
+__description("list_is_edge_test2: accept list_front/list_back return value")
+__success __retval(0)
+int list_is_edge_test2(void *ctx)
+{
+	struct bpf_list_node *front, *back;
+	struct node_data *a, *b;
+	long err = 0;
+
+	a = __add_in_list(&head, &lock);
+	if (!a)
+		return -1;
+
+	b = __add_in_list(&head, &lock);
+	if (!b) {
+		bpf_obj_drop(a);
+		return -2;
+	}
+
+	bpf_spin_lock(&lock);
+	front = bpf_list_front(&head);
+	back = bpf_list_back(&head);
+	if (!front || !back) {
+		err = -3;
+		goto out_unlock;
+	}
+
+	if (!bpf_list_is_first(&head, front) || bpf_list_is_last(&head, front)) {
+		err = -4;
+		goto out_unlock;
+	}
+
+	if (!bpf_list_is_last(&head, back) || bpf_list_is_first(&head, back)) {
+		err = -5;
+		goto out_unlock;
+	}
+
+out_unlock:
+	bpf_spin_unlock(&lock);
+	bpf_obj_drop(a);
+	bpf_obj_drop(b);
+	return err;
+}
+
+SEC("tc")
+__description("list_is_edge_test3: single node is both first and last")
+__success __retval(0)
+int list_is_edge_test3(void *ctx)
+{
+	struct node_data *tmp;
+	struct bpf_list_node *node;
+	long err = 0;
+
+	tmp = __add_in_list(&head, &lock);
+	if (!tmp)
+		return -1;
+
+	bpf_spin_lock(&lock);
+	node = bpf_list_front(&head);
+	if (!node) {
+		bpf_spin_unlock(&lock);
+		bpf_obj_drop(tmp);
+		return -2;
+	}
+
+	if (!bpf_list_is_first(&head, node) || !bpf_list_is_last(&head, node))
+		err = -3;
+	bpf_spin_unlock(&lock);
+
+	bpf_obj_drop(tmp);
+	return err;
+}
+
+SEC("tc")
+__description("list_del_test1: del returns removed nodes")
+__success __retval(0)
+int list_del_test1(void *ctx)
+{
+	struct node_data *node_first, *node_last;
+	struct bpf_list_node *bpf_node_first, *bpf_node_last;
+	int err = 0;
+
+	node_last = __add_in_list(&head, &lock);
+	if (!node_last)
+		return -1;
+
+	node_first = __add_in_list(&head, &lock);
+	if (!node_first) {
+		bpf_obj_drop(node_last);
+		return -2;
+	}
+
+	bpf_spin_lock(&lock);
+	bpf_node_last = bpf_list_del(&head, &node_last->l);
+	bpf_node_first = bpf_list_del(&head, &node_first->l);
+	bpf_spin_unlock(&lock);
+
+	if (bpf_node_first)
+		bpf_obj_drop(container_of(bpf_node_first, struct node_data, l));
+	else
+		err = -3;
+
+	if (bpf_node_last)
+		bpf_obj_drop(container_of(bpf_node_last, struct node_data, l));
+	else
+		err = -4;
+
+	bpf_obj_drop(node_first);
+	bpf_obj_drop(node_last);
+	return err;
+}
+
+SEC("tc")
+__description("list_del_test2: remove an arbitrary node from the list")
+__success __retval(0)
+int list_del_test2(void *ctx)
+{
+	struct bpf_rb_node *rb;
+	struct bpf_list_node *l;
+	struct node_data *n;
+	long err;
+
+	err = __insert_in_tree_and_list(&head, &root, &lock);
+	if (err)
+		return err;
+
+	bpf_spin_lock(&lock);
+	rb = bpf_rbtree_first(&root);
+	if (!rb) {
+		bpf_spin_unlock(&lock);
+		return -4;
+	}
+
+	rb = bpf_rbtree_remove(&root, rb);
+	if (!rb) {
+		bpf_spin_unlock(&lock);
+		return -5;
+	}
+
+	n = container_of(rb, struct node_data, r);
+	l = bpf_list_del(&head, &n->l);
+	bpf_spin_unlock(&lock);
+	bpf_obj_drop(n);
+	if (!l)
+		return -6;
+
+	bpf_obj_drop(container_of(l, struct node_data, l));
+	return 0;
+}
+
+SEC("tc")
+__description("list_del_test3: list_del accepts list_front return value as node")
+__success __retval(0)
+int list_del_test3(void *ctx)
+{
+	struct node_data *tmp;
+	struct bpf_list_node *bpf_node, *l;
+	long err = 0;
+
+	tmp = __add_in_list(&head, &lock);
+	if (!tmp)
+		return -1;
+
+	bpf_spin_lock(&lock);
+	bpf_node = bpf_list_front(&head);
+	if (!bpf_node) {
+		bpf_spin_unlock(&lock);
+		err = -2;
+		goto fail;
+	}
+
+	l = bpf_list_del(&head, bpf_node);
+	bpf_spin_unlock(&lock);
+	if (!l) {
+		err = -3;
+		goto fail;
+	}
+
+	bpf_obj_drop(container_of(l, struct node_data, l));
+	bpf_obj_drop(tmp);
+	return 0;
+
+fail:
+	bpf_obj_drop(tmp);
+	return err;
+}
+
+SEC("tc")
+__description("list_add_test1: insert new node after prev")
+__success __retval(0)
+int list_add_test1(void *ctx)
+{
+	struct node_data *node_first;
+	struct node_data *new_node;
+	long err = 0;
+
+	node_first = __add_in_list(&head, &lock);
+	if (!node_first)
+		return -1;
+
+	new_node = bpf_obj_new(typeof(*new_node));
+	if (!new_node) {
+		err = -2;
+		goto fail;
+	}
+
+	bpf_spin_lock(&lock);
+	err = bpf_list_add(&head, &new_node->l, &node_first->l);
+	bpf_spin_unlock(&lock);
+	if (err) {
+		err = -3;
+		goto fail;
+	}
+
+fail:
+	bpf_obj_drop(node_first);
+	return err;
+}
+
+SEC("tc")
+__description("list_add_test2: list_add accepts list_front return value as prev")
+__success __retval(0)
+int list_add_test2(void *ctx)
+{
+	struct node_data *new_node, *tmp;
+	struct bpf_list_node *bpf_node;
+	long err = 0;
+
+	tmp = __add_in_list(&head, &lock);
+	if (!tmp)
+		return -1;
+
+	new_node = bpf_obj_new(typeof(*new_node));
+	if (!new_node) {
+		err = -2;
+		goto fail;
+	}
+
+	bpf_spin_lock(&lock);
+	bpf_node = bpf_list_front(&head);
+	if (!bpf_node) {
+		bpf_spin_unlock(&lock);
+		bpf_obj_drop(new_node);
+		err = -3;
+		goto fail;
+	}
+
+	err = bpf_list_add(&head, &new_node->l, bpf_node);
+	bpf_spin_unlock(&lock);
+	if (err) {
+		err = -4;
+		goto fail;
+	}
+
+fail:
+	bpf_obj_drop(tmp);
+	return err;
+}
+
+struct uninit_head_val {
+	struct bpf_spin_lock lock;
+	struct bpf_list_head head __contains(node_data, l);
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__type(key, int);
+	__type(value, struct uninit_head_val);
+	__uint(max_entries, 1);
+} uninit_head_map SEC(".maps");
+
+SEC("tc")
+__description("list_push_back_uninit_head: push_back on 0-initialized list head")
+__success __retval(0)
+int list_push_back_uninit_head(void *ctx)
+{
+	struct uninit_head_val *st;
+	struct node_data *node;
+	int ret = -1, key = 0;
+
+	st = bpf_map_lookup_elem(&uninit_head_map, &key);
+	if (!st)
+		return -1;
+
+	node = bpf_obj_new(typeof(*node));
+	if (!node)
+		return -1;
+
+	bpf_spin_lock(&st->lock);
+	ret = bpf_list_push_back(&st->head, &node->l);
+	bpf_spin_unlock(&st->lock);
+
+	return ret;
+}
+
+SEC("?tc")
+__failure __msg("bpf_spin_lock at off=32 must be held for bpf_list_head")
+long list_del_without_lock_fail(void *ctx)
+{
+	struct node_data *n;
+	struct bpf_list_node *l;
+
+	n = bpf_obj_new(typeof(*n));
+	if (!n)
+		return -1;
+
+	/* Error case: delete list node without holding lock */
+	l = bpf_list_del(&head, &n->l);
+	bpf_obj_drop(n);
+	if (!l)
+		return -2;
+	bpf_obj_drop(container_of(l, struct node_data, l));
+
+	return 0;
+}
+
+SEC("?tc")
+__failure __msg("bpf_spin_lock at off=32 must be held for bpf_list_head")
+long list_add_without_lock_fail(void *ctx)
+{
+	struct node_data *n, *prev;
+
+	n = bpf_obj_new(typeof(*n));
+	if (!n)
+		return -1;
+
+	prev = bpf_obj_new(typeof(*prev));
+	if (!prev) {
+		bpf_obj_drop(n);
+		return -1;
+	}
+
+	/* Error case: add list node without holding lock */
+	if (bpf_list_add(&head, &n->l, &prev->l)) {
+		bpf_obj_drop(prev);
+		bpf_obj_drop(n);
+		return -2;
+	}
+
+	return 0;
+}
+
 SEC("tc")
 __success
 long rbtree_refcounted_node_ref_escapes(void *ctx)
Signature · 1 line
2.50.1 (Apple Git-155)
#36 lore raw

Re: [PATCH RESEND bpf-next v10 8/8] selftests/bpf: Add test cases for bpf_list_del/add/is_first/is_last/empty

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Latent verifier errors (use-after-release and memory leak) in the negative test…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Latent verifier errors (use-after-release and memory leak) in the negative test list_add_without_lock_fail. --

commit 37523e4b94785be6995dc6c7cf660c200eca4aef
Author: Kaitao Cheng <[email protected]>

selftests/bpf: Add test cases for bpf_list_del/add/is_first/is_last/empty

This commit extends the refcounted_kptr selftests by adding new test cases for BPF linked list operations. It includes both successful path tests and negative tests verifying that the verifier correctly rejects operations made without holding the required spinlock.

Quoted 4 lines
tools/testing/selftests/bpf/progs/refcounted_kptr.c +0 -0
index c847398837ccd..21ae06797b183 100644

[ ... ]

Quoted 10 lines
tools/testing/selftests/bpf/progs/refcounted_kptr.c +10 -0
+SEC("?tc")
+__failure __msg("bpf_spin_lock at off=32 must be held for bpf_list_head")
+long list_add_without_lock_fail(void *ctx)
+{
+	struct node_data *n, *prev;
+
+	n = bpf_obj_new(typeof(*n));
+	if (!n)
+		return -1;
+

tools/testing/selftests/bpf/progs/refcounted_kptr.c long insert_rbtree_and_stash__del_tree_##rem_tree(void *ctx) \ · line 784

Quoted 10 lines
tools/testing/selftests/bpf/progs/refcounted_kptr.c +10 -0
+	prev = bpf_obj_new(typeof(*prev));
+	if (!prev) {
+		bpf_obj_drop(n);
+		return -1;
+	}
+
+	/* Error case: add list node without holding lock */
+	if (bpf_list_add(&head, &n->l, &prev->l)) {
+		bpf_obj_drop(prev);
+		bpf_obj_drop(n);

Since bpf_list_add() consumes the newly added node (&n->l), would this subsequent bpf_obj_drop(n) trigger a use-after-release error in the verifier if it evaluates past the missing spinlock?

tools/testing/selftests/bpf/progs/refcounted_kptr.c long insert_rbtree_and_stash__del_tree_##rem_tree(void *ctx) \ · line 596

Quoted 4 lines
tools/testing/selftests/bpf/progs/refcounted_kptr.c +4 -0
+		return -2;
+	}
+
+	return 0;

Similarly, prev is an owning reference acquired via bpf_obj_new(), but bpf_list_add() accepts a non-owning reference for the prev argument. Does this code leak the prev reference by returning 0 here without calling bpf_obj_drop(prev)?

While the test correctly fails today due to the missing spinlock as intended, these latent errors could make the test brittle and cause CI flakes if the verifier's evaluation order changes in the future.

Quoted 1 line

+}

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8