Skip to contentSkip to conversation

mm/memcg: fix UAF in drain_all_stock() async work during offline

PATCH In review Rik van Riel 5 participants 8 messages Aug 27 – Aug 28

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/memcg: fix UAF in drain_all_stock() async work during offline

drain_all_stock() queues drain work on remote CPUs via schedule_drain_work() -> queue_work_on(memcg_wq) and returns immediately without waiting. The worker, drain_local_memcg_stock() / drain_local_obj_stock(), dereferences per-CPU stock caches with READ_ONCE(stock->cached[i]) and does css_put() / obj_cgroup_put().

mem_cgroup_css_offline() calls drain_all_stock(memcg) to optimize reclamation latency, but never flushes memcg_wq. If that races with cgroup removal, free can happen while workers are still pending, causing UAF. The drain work could also have been queued by somebody else before offline started (e.g. high throttling), not just by the offline path itself.

Timeline illustrating the race:

CPU0 (rmdir + offline) CPU1 (charge cache holder)
------------------------- ----------------------------
cgroup_rmdir()

    cgroup_destroy_locked()
      kill_css_sync()
      ...

                                      refill_stock(victim)
                                        css_get(victim)
                                        WRITE_ONCE(cached[i]=victim)

percpu_ref kill confirmed, css_killed_ref_fn() called

css_killed_work_fn() [offline_wq]

    mem_cgroup_css_offline(victim)
      drain_all_stock(victim)
        is_memcg_drain_needed()
          READ_ONCE(cached) -> victim
        queue_work_on(CPU1, memcg_wq, work)
        // no flush!
      mem_cgroup_private_id_put()
    css_put() -> refcnt may hit 0

[RCU GP]
css_free_rwork_fn()

    mem_cgroup_free(victim)
    // victim struct freed

                                        // worker delayed by scheduler/
                                        // WQ concurrency
                                        drain_local_memcg_stock()
                                          old = READ_ONCE(cached[i])
                                          // UAF: old == freed victim
                                          memcg_uncharge(old)
                                          css_put(&old->css)

Fix by having the offline path wait for the workqueue to be done with the memcg, before freeing the memcg.

Found through a code audit with kres.

Fixes 591edfb10a94 ("mm: drain memcg stocks on css offlining")
Assisted-by Hermes:muse-spark-1.2 kres
---
 mm/memcontrol.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)
mm/memcontrol.c +25 -4
index 6dc4888a90f3..c95a1f6ec799 100644
@@ -2273,12 +2273,18 @@ static void schedule_drain_work(int cpu, struct work_struct *work)
  * Drains all per-CPU charge caches for given root_memcg resp. subtree
  * of the hierarchy under it.
  */
-void drain_all_stock(struct mem_cgroup *root_memcg)
+static void __drain_all_stock(struct mem_cgroup *root_memcg, bool sync)
 {
 	int cpu, curcpu;
 
-	/* If someone's already draining, avoid adding running more workers. */
-	if (!mutex_trylock(&percpu_charge_mutex))
+	/*
+	 *  If someone's already draining, avoid starting more workers.
+	 *  Synchronous callers need to guarantee all the last things
+	 *  are flushed, e.g. before a memcg is removed.
+	 */
+	if (sync)
+		mutex_lock(&percpu_charge_mutex);
+	else if (!mutex_trylock(&percpu_charge_mutex))
 		return;
 	/*
 	 * Notify other cpus that system-wide "drain" is running
@@ -2316,6 +2322,21 @@ void drain_all_stock(struct mem_cgroup *root_memcg)
 	mutex_unlock(&percpu_charge_mutex);
 }
 
+void drain_all_stock(struct mem_cgroup *root_memcg)
+{
+	__drain_all_stock(root_memcg, false);
+}
+
+void drain_all_stock_sync(struct mem_cgroup *root_memcg)
+{
+	/*
+	 * Make sure the workqueue is done with this memcg
+	 * before freeing it.
+	 */
+	__drain_all_stock(root_memcg, true);
+	flush_workqueue(memcg_wq);
+}
+
 static int memcg_hotplug_cpu_dead(unsigned int cpu)
 {
 	/* no need for the local lock */
@@ -4305,7 +4326,7 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
 	wb_memcg_offline(memcg);
 	lru_gen_offline_memcg(memcg);
 
-	drain_all_stock(memcg);
+	drain_all_stock_sync(memcg);
 
 	mem_cgroup_private_id_put(memcg, 1);
 }
Signature · 1 line
2.55.0
maintainer
#1 lore raw
Quoted 29 lines · Rik van Riel

drain_all_stock() queues drain work on remote CPUs via schedule_drain_work() -> queue_work_on(memcg_wq) and returns immediately without waiting. The worker, drain_local_memcg_stock() / drain_local_obj_stock(), dereferences per-CPU stock caches with READ_ONCE(stock->cached[i]) and does css_put() / obj_cgroup_put().

mem_cgroup_css_offline() calls drain_all_stock(memcg) to optimize reclamation latency, but never flushes memcg_wq. If that races with cgroup removal, free can happen while workers are still pending, causing UAF. The drain work could also have been queued by somebody else before offline started (e.g. high throttling), not just by the offline path itself.

Timeline illustrating the race:

CPU0 (rmdir + offline) CPU1 (charge cache holder)
------------------------- ----------------------------
cgroup_rmdir()

    cgroup_destroy_locked()
      kill_css_sync()
      ...

                                      refill_stock(victim)
                                        css_get(victim)
                                        WRITE_ONCE(cached[i]=victim)

percpu_ref kill confirmed, css_killed_ref_fn() called

css_killed_work_fn() [offline_wq]

Quoted 8 lines
    mem_cgroup_css_offline(victim)
      drain_all_stock(victim)
        is_memcg_drain_needed()
          READ_ONCE(cached) -> victim
        queue_work_on(CPU1, memcg_wq, work)
        // no flush!
      mem_cgroup_private_id_put()
    css_put() -> refcnt may hit 0

Why would refcnt hit 0? CPU1 stock has a reference.

Quoted 88 lines

[RCU GP]
css_free_rwork_fn()

    mem_cgroup_free(victim)
    // victim struct freed

                                        // worker delayed by scheduler/
                                        // WQ concurrency
                                        drain_local_memcg_stock()
                                          old = READ_ONCE(cached[i])
                                          // UAF: old == freed victim
                                          memcg_uncharge(old)
                                          css_put(&old->css)

Fix by having the offline path wait for the workqueue to be done with the memcg, before freeing the memcg.

Found through a code audit with kres.

Fixes 591edfb10a94 ("mm: drain memcg stocks on css offlining")
Assisted-by Hermes:muse-spark-1.2 kres
---
 mm/memcontrol.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)
mm/memcontrol.c +25 -4
index 6dc4888a90f3..c95a1f6ec799 100644
@@ -2273,12 +2273,18 @@ static void schedule_drain_work(int cpu, struct work_struct *work)
  * Drains all per-CPU charge caches for given root_memcg resp. subtree
  * of the hierarchy under it.
  */
-void drain_all_stock(struct mem_cgroup *root_memcg)
+static void __drain_all_stock(struct mem_cgroup *root_memcg, bool sync)
 {
 	int cpu, curcpu;
 
-	/* If someone's already draining, avoid adding running more workers. */
-	if (!mutex_trylock(&percpu_charge_mutex))
+	/*
+	 *  If someone's already draining, avoid starting more workers.
+	 *  Synchronous callers need to guarantee all the last things
+	 *  are flushed, e.g. before a memcg is removed.
+	 */
+	if (sync)
+		mutex_lock(&percpu_charge_mutex);
+	else if (!mutex_trylock(&percpu_charge_mutex))
 		return;
 	/*
 	 * Notify other cpus that system-wide "drain" is running
@@ -2316,6 +2322,21 @@ void drain_all_stock(struct mem_cgroup *root_memcg)
 	mutex_unlock(&percpu_charge_mutex);
 }
 
+void drain_all_stock(struct mem_cgroup *root_memcg)
+{
+	__drain_all_stock(root_memcg, false);
+}
+
+void drain_all_stock_sync(struct mem_cgroup *root_memcg)
+{
+	/*
+	 * Make sure the workqueue is done with this memcg
+	 * before freeing it.
+	 */
+	__drain_all_stock(root_memcg, true);
+	flush_workqueue(memcg_wq);
+}
+
 static int memcg_hotplug_cpu_dead(unsigned int cpu)
 {
 	/* no need for the local lock */
@@ -4305,7 +4326,7 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
 	wb_memcg_offline(memcg);
 	lru_gen_offline_memcg(memcg);
 
-	drain_all_stock(memcg);
+	drain_all_stock_sync(memcg);
 
 	mem_cgroup_private_id_put(memcg, 1);
 }
Signature · 1 line
2.55.0
maintainer
#2 lore raw
In reply to Rik van Riel
Quoted 18 lines · Rik van Riel

drain_all_stock() queues drain work on remote CPUs via schedule_drain_work() -> queue_work_on(memcg_wq) and returns immediately without waiting. The worker, drain_local_memcg_stock() / drain_local_obj_stock(), dereferences per-CPU stock caches with READ_ONCE(stock->cached[i]) and does css_put() / obj_cgroup_put().

mem_cgroup_css_offline() calls drain_all_stock(memcg) to optimize reclamation latency, but never flushes memcg_wq. If that races with cgroup removal, free can happen while workers are still pending, causing UAF. The drain work could also have been queued by somebody else before offline started (e.g. high throttling), not just by the offline path itself.

Timeline illustrating the race:

CPU0 (rmdir + offline) CPU1 (charge cache holder)
------------------------- ----------------------------
cgroup_rmdir()

Quoted 7 lines
    cgroup_destroy_locked()
      kill_css_sync()
      ...

                                      refill_stock(victim)
                                        css_get(victim)
                                        WRITE_ONCE(cached[i]=victim)

I'm really confused. CPU1 acquires a ref for the cached[i] pointer ^

Quoted 11 lines

percpu_ref kill confirmed, css_killed_ref_fn() called

css_killed_work_fn() [offline_wq]

    mem_cgroup_css_offline(victim)
      drain_all_stock(victim)
        is_memcg_drain_needed()
          READ_ONCE(cached) -> victim
        queue_work_on(CPU1, memcg_wq, work)
        // no flush!
      mem_cgroup_private_id_put()
    css_put() -> refcnt may hit 0

So how can it hit 0 here?

Quoted 4 lines

[RCU GP]
css_free_rwork_fn()

    mem_cgroup_free(victim)
    // victim struct freed

This won't run until we hit zero...

Quoted 8 lines
                                        // worker delayed by scheduler/
                                        // WQ concurrency
                                        drain_local_memcg_stock()
                                          old = READ_ONCE(cached[i])
                                          // UAF: old == freed victim
                                          memcg_uncharge(old)
                                          css_put(&old->css)

...which is here.

maintainer
#3 lore raw
In reply to Rik van Riel
Quoted 17 lines · On Thu, 27 Aug 2026 12:42:11 -0400 Rik van Riel <[email protected]> wrote:

drain_all_stock() queues drain work on remote CPUs via schedule_drain_work() -> queue_work_on(memcg_wq) and returns immediately without waiting. The worker, drain_local_memcg_stock() / drain_local_obj_stock(), dereferences per-CPU stock caches with READ_ONCE(stock->cached[i]) and does css_put() / obj_cgroup_put().

mem_cgroup_css_offline() calls drain_all_stock(memcg) to optimize reclamation latency, but never flushes memcg_wq. If that races with cgroup removal, free can happen while workers are still pending, causing UAF. The drain work could also have been queued by somebody else before offline started (e.g. high throttling), not just by the offline path itself.

Timeline illustrating the race:

...

Quoted 8 lines

Fix by having the offline path wait for the workqueue to be done with the memcg, before freeing the memcg.

Found through a code audit with kres.

Fixes 591edfb10a94 ("mm: drain memcg stocks on css offlining")
Assisted-by Hermes:muse-spark-1.2 kres

Sashiko might have found another thing in there:

	https://sashiko.dev/#/patchset/20260827124211.3b94b103@fangorn
#5 lore raw
In reply to Rik van Riel
kernel test robot noticed the following build warnings:

Hi Rik,

kernel test robot noticed the following build warnings:

[auto build test WARNING on akpm-mm/mm-everything]

url:    https://github.com/intel-lab-lkp/linux/commits/Rik-van-Riel/mm-memcg-fix-UAF-in-drain_all_stock-async-work-during-offline/20260827-124211
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20260827124211.3b94b103%40fangorn
patch subject: [PATCH] mm/memcg: fix UAF in drain_all_stock() async work during offline
config: x86_64-randconfig-074-20260828 (https://download.01.org/0day-ci/archive/20260828/[email protected]/config)
compiler: clang version 22.1.3 (https://github.com/llvm/llvm-project e9846648fd6183ee6d8cbdb4502213fcf902a211)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260828/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> mm/memcontrol.c:2380:6: warning: no previous prototype for function 'drain_all_stock_sync' [-Wmissing-prototypes]
    2380 | void drain_all_stock_sync(struct mem_cgroup *root_memcg)
         |      ^
   mm/memcontrol.c:2380:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
    2380 | void drain_all_stock_sync(struct mem_cgroup *root_memcg)
         | ^
         | static 

1 warning generated.

vim +/drain_all_stock_sync +2380 mm/memcontrol.c

2379

> 2380	void drain_all_stock_sync(struct mem_cgroup *root_memcg)

2381 {
2382 /*
2383 * Make sure the workqueue is done with this memcg
2384 * before freeing it.
2385 */
2386 __drain_all_stock(root_memcg, true);
2387 flush_workqueue(memcg_wq);
2388 }
2389

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

#6 lore raw
In reply to Rik van Riel
kernel test robot noticed the following build warnings:

Hi Rik,

kernel test robot noticed the following build warnings:

[auto build test WARNING on akpm-mm/mm-everything]

url:    https://github.com/intel-lab-lkp/linux/commits/Rik-van-Riel/mm-memcg-fix-UAF-in-drain_all_stock-async-work-during-offline/20260827-124211
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20260827124211.3b94b103%40fangorn
patch subject: [PATCH] mm/memcg: fix UAF in drain_all_stock() async work during offline
config: xtensa-randconfig-r073-20260828 (https://download.01.org/0day-ci/archive/20260828/[email protected]/config)
compiler: xtensa-linux-gcc (GCC) 16.1.0
smatch: v0.5.0-9187-g5189e3fb
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260828/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> mm/memcontrol.c:2380:6: warning: no previous prototype for 'drain_all_stock_sync' [-Wmissing-prototypes]

2380 | void drain_all_stock_sync(struct mem_cgroup *root_memcg)
| ^~~~~~~~~~~~~~~~~~~~

vim +/drain_all_stock_sync +2380 mm/memcontrol.c

2379

> 2380	void drain_all_stock_sync(struct mem_cgroup *root_memcg)

2381 {
2382 /*
2383 * Make sure the workqueue is done with this memcg
2384 * before freeing it.
2385 */
2386 __drain_all_stock(root_memcg, true);
2387 flush_workqueue(memcg_wq);
2388 }
2389

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

#7 lore raw
In reply to Rik van Riel
kernel test robot noticed the following build warnings:

Hi Rik,

kernel test robot noticed the following build warnings:

[auto build test WARNING on akpm-mm/mm-everything]

url:    https://github.com/intel-lab-lkp/linux/commits/Rik-van-Riel/mm-memcg-fix-UAF-in-drain_all_stock-async-work-during-offline/20260827-124211
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20260827124211.3b94b103%40fangorn
patch subject: [PATCH] mm/memcg: fix UAF in drain_all_stock() async work during offline
config: nios2-randconfig-r121-20260828 (https://download.01.org/0day-ci/archive/20260828/[email protected]/config)
compiler: nios2-linux-gcc (GCC) 8.5.0
sparse: v0.6.5-rc1
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260828/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> mm/memcontrol.c:2380:6: sparse: sparse: symbol 'drain_all_stock_sync' was not declared. Should it be static?

mm/memcontrol.c:4678:52: sparse: sparse: incompatible types in comparison expression (different address spaces):
mm/memcontrol.c:4678:52: sparse: struct task_struct [noderef] __rcu *
mm/memcontrol.c:4678:52: sparse: struct task_struct *

vim +/drain_all_stock_sync +2380 mm/memcontrol.c

2379

> 2380	void drain_all_stock_sync(struct mem_cgroup *root_memcg)

2381 {
2382 /*
2383 * Make sure the workqueue is done with this memcg
2384 * before freeing it.
2385 */
2386 __drain_all_stock(root_memcg, true);
2387 flush_workqueue(memcg_wq);
2388 }
2389

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki