Skip to contentSkip to conversation

mm: list_lru: fix NULL deref when a folio moves to the root memcg

PATCH In review Shakeel Butt 4 participants 4 messages Nov 12

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm: list_lru: fix NULL deref when a folio moves to the root memcg

folio_memcg_list_lru_alloc() makes sure that a folio's memcg has its own lists in a list_lru, such as the THP deferred split queue. It looks up the folio's memcg twice: a fast path under RCU checks if the lists exist, and a slow path takes a reference and allocates them. The fast path returns early for the root memcg, which uses the global lists. The slow path does not check for the root.

The memcg can change between the two lookups: when a cgroup is removed, its folios are moved to the parent memcg. If the folio is moved up to the root in between, the slow path gets the root, and __memcg_list_lru_alloc() dereferences the root's parent, which is NULL.

Users see a NULL pointer dereference oops while a THP is allocated, such as in a page fault, in khugepaged or at swap-in. tmpfs huge pages are affected too, and for them the oops can also happen in reclaim, for example in kswapd. The task that hits the oops is killed, or the machine panics if panic_on_oops is set.

The race is narrow: the folio has to be moved up to the root right between the two lookups. For example, its cgroup is directly below the root and is removed, or has its memory controller disabled, at that moment.

Fix it by checking memcg_list_lru_allocated() again in the slow path, like the fast path does. It returns true for the root, which needs no lists of its own.

Fixes ae64f07a6a40 ("mm: list_lru: introduce folio_memcg_list_lru_alloc()")

---
Changes in v2:
- Check memcg_list_lru_allocated() instead of mem_cgroup_is_root(), like the fast path does, and update the comment to match (Muchun).
- Link to v1: https://lore.kernel.org/[email protected]

 mm/list_lru.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
mm/list_lru.c +9 -2
index d1d256822ccc..ca8de3793166 100644
@@ -645,9 +645,16 @@ int folio_memcg_list_lru_alloc(struct folio *folio, struct list_lru *lru,
 	if (likely(res))
 		return 0;
 
-	/* Allocation may block, pin the memcg */
+	/*
+	 * Allocation may block, so pin the memcg. If the folio's cgroup
+	 * was removed after the check above, the folio has moved up to a
+	 * parent memcg. That memcg may already have its lists, or may be
+	 * the root, which uses the global lists and has no parent. So
+	 * check again before allocating.
+	 */
 	memcg = get_mem_cgroup_from_folio(folio);
-	res = __memcg_list_lru_alloc(memcg, lru, gfp);
+	if (!memcg_list_lru_allocated(memcg, lru))
+		res = __memcg_list_lru_alloc(memcg, lru, gfp);
 	mem_cgroup_put(memcg);
 	return res;
 }
Signature · 1 line
2.53.0-Meta
reviewer Acked-by on v2 1/1
#1 lore raw
Quoted 27 lines · Shakeel Butt

folio_memcg_list_lru_alloc() makes sure that a folio's memcg has its own lists in a list_lru, such as the THP deferred split queue. It looks up the folio's memcg twice: a fast path under RCU checks if the lists exist, and a slow path takes a reference and allocates them. The fast path returns early for the root memcg, which uses the global lists. The slow path does not check for the root.

The memcg can change between the two lookups: when a cgroup is removed, its folios are moved to the parent memcg. If the folio is moved up to the root in between, the slow path gets the root, and __memcg_list_lru_alloc() dereferences the root's parent, which is NULL.

Users see a NULL pointer dereference oops while a THP is allocated, such as in a page fault, in khugepaged or at swap-in. tmpfs huge pages are affected too, and for them the oops can also happen in reclaim, for example in kswapd. The task that hits the oops is killed, or the machine panics if panic_on_oops is set.

The race is narrow: the folio has to be moved up to the root right between the two lookups. For example, its cgroup is directly below the root and is removed, or has its memory controller disabled, at that moment.

Quoted 7 lines

Fix it by checking memcg_list_lru_allocated() again in the slow path, like the fast path does. It returns true for the root, which needs no lists of its own.

Fixes ae64f07a6a40 ("mm: list_lru: introduce folio_memcg_list_lru_alloc()")

Thanks.

Reviewed-by on v2 1/1
#2 lore raw
In reply to Shakeel Butt
Quoted 23 lines · Shakeel Butt

folio_memcg_list_lru_alloc() makes sure that a folio's memcg has its own lists in a list_lru, such as the THP deferred split queue. It looks up the folio's memcg twice: a fast path under RCU checks if the lists exist, and a slow path takes a reference and allocates them. The fast path returns early for the root memcg, which uses the global lists. The slow path does not check for the root.

The memcg can change between the two lookups: when a cgroup is removed, its folios are moved to the parent memcg. If the folio is moved up to the root in between, the slow path gets the root, and __memcg_list_lru_alloc() dereferences the root's parent, which is NULL.

Users see a NULL pointer dereference oops while a THP is allocated, such as in a page fault, in khugepaged or at swap-in. tmpfs huge pages are affected too, and for them the oops can also happen in reclaim, for example in kswapd. The task that hits the oops is killed, or the machine panics if panic_on_oops is set.

The race is narrow: the folio has to be moved up to the root right between the two lookups. For example, its cgroup is directly below the root and is removed, or has its memory controller disabled, at that moment.

Quoted 8 lines

Fix it by checking memcg_list_lru_allocated() again in the slow path, like the fast path does. It returns true for the root, which needs no lists of its own.

Fixes ae64f07a6a40 ("mm: list_lru: introduce folio_memcg_list_lru_alloc()")

---

The fix looks good to me,

Quoted 19 lines

Changes in v2:
- Check memcg_list_lru_allocated() instead of mem_cgroup_is_root(), like the fast path does, and update the comment to match (Muchun).
- Link to v1: https://lore.kernel.org/[email protected]

 mm/list_lru.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
mm/list_lru.c +2 -1
index d1d256822ccc..ca8de3793166 100644
@@ -645,9 +645,16 @@ int folio_memcg_list_lru_alloc(struct folio *folio, struct list_lru *lru,
 	if (likely(res))
 		return 0;
 
-	/* Allocation may block, pin the memcg */
+	/*
+	 * Allocation may block, so pin the memcg. If the folio's cgroup

mm/list_lru.c int folio_memcg_list_lru_alloc(struct folio *folio, struct list_lru *lru, · line 658

Quoted 10 lines
mm/list_lru.c +7 -1
+	 * was removed after the check above, the folio has moved up to a
+	 * parent memcg. That memcg may already have its lists, or may be
+	 * the root, which uses the global lists and has no parent. So
+	 * check again before allocating.
+	 */
 	memcg = get_mem_cgroup_from_folio(folio);
-	res = __memcg_list_lru_alloc(memcg, lru, gfp);
+	if (!memcg_list_lru_allocated(memcg, lru))
+		res = __memcg_list_lru_alloc(memcg, lru, gfp);
 	mem_cgroup_put(memcg);

I was wondering how we make sure reparenting doesn't happen after this, but apparently it's fine as __memcg_list_lru_alloc() populates all list_lrus from the memcg up to the hierarchy before the root.

Signature · 2 lines
Cheers,
Harry / Hyeonggon
Reviewed-by on v2 1/1
#3 lore raw
In reply to Shakeel Butt
Quoted 23 lines · Shakeel Butt

folio_memcg_list_lru_alloc() makes sure that a folio's memcg has its own lists in a list_lru, such as the THP deferred split queue. It looks up the folio's memcg twice: a fast path under RCU checks if the lists exist, and a slow path takes a reference and allocates them. The fast path returns early for the root memcg, which uses the global lists. The slow path does not check for the root.

The memcg can change between the two lookups: when a cgroup is removed, its folios are moved to the parent memcg. If the folio is moved up to the root in between, the slow path gets the root, and __memcg_list_lru_alloc() dereferences the root's parent, which is NULL.

Users see a NULL pointer dereference oops while a THP is allocated, such as in a page fault, in khugepaged or at swap-in. tmpfs huge pages are affected too, and for them the oops can also happen in reclaim, for example in kswapd. The task that hits the oops is killed, or the machine panics if panic_on_oops is set.

The race is narrow: the folio has to be moved up to the root right between the two lookups. For example, its cgroup is directly below the root and is removed, or has its memory controller disabled, at that moment.

Quoted 7 lines

Fix it by checking memcg_list_lru_allocated() again in the slow path, like the fast path does. It returns true for the root, which needs no lists of its own.

Fixes ae64f07a6a40 ("mm: list_lru: introduce folio_memcg_list_lru_alloc()")