make unused huge shrinker memcg aware

From: Qi Zheng <[email protected]>

Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") forgot to drop the super_fs_objects_eligible(), just fix it.

Quoted 5 lines · On Mon, 17 Aug 2026 17:03:25 +0800 Qi Zheng <[email protected]> wrote:

From: Qi Zheng <[email protected]>

Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") forgot to drop the super_fs_objects_eligible(), just fix it.

Oh. Why? Please can we have a description of what's wrong with the current code?

Hi Andrew,

Quoted 8 lines · Andrew Morton

Quoted 5 lines · On Mon, 17 Aug 2026 17:03:25 +0800 Qi Zheng <[email protected]> wrote:

From: Qi Zheng <[email protected]>

Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") forgot to drop the super_fs_objects_eligible(), just fix it.

Quoted 2 lines

Oh. Why? Please can we have a description of what's wrong with the current code?

As I replied in another thread [1], this fix and commit 0ef8faff490be together make up the complete patch originally sent by Usama [2], so they should be squashed together.

Without this fix, all memcg-aware
nr_cached_objects() will be skipped during memcg-level shrinking. Besides the newly added memcg-aware shmem huge shrinker, XFS will also run into issues.

Quoted 11 lines · On Sat, 29 Aug 2026 09:49:44 +0800 Qi Zheng <[email protected]> wrote:

Hi Andrew,

Quoted 8 lines · Andrew Morton

Quoted 5 lines · On Mon, 17 Aug 2026 17:03:25 +0800 Qi Zheng <[email protected]> wrote:

From: Qi Zheng <[email protected]>

Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") forgot to drop the super_fs_objects_eligible(), just fix it.

Quoted 6 lines

Quoted 2 lines

Oh. Why? Please can we have a description of what's wrong with the current code?

As I replied in another thread [1], this fix and commit 0ef8faff490be together make up the complete patch originally sent by Usama [2], so they should be squashed together.

0ef8faff490be is upstream so they can't be squashed!

Quoted 5 lines

Without this fix, all memcg-aware
nr_cached_objects() will be skipped during memcg-level shrinking. Besides the newly added memcg-aware shmem huge shrinker, XFS will also run into issues.

OK, please add this info to this patch's changelog. Please also decide whether a cc:stable should be added and if so, ensure the changelog makes clear why we're recommending a backport.

As part of this, let's expand on "XFS will also run into issues".

Thanks.

Quoted 5 lines

OK, please add this info to this patch's changelog. Please also decide whether a cc:stable should be added and if so, ensure the changelog makes clear why we're recommending a backport.

As part of this, let's expand on "XFS will also run into issues".

How about this instead:

---
Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") was meant to drop the blanket memcg gate in fs/super.c and let each ->nr_cached_objects() implementation decide for itself whether it is meaningful in per-memcg reclaim. However, when that patch was applied the removal of super_fs_objects_eligible() and its two call sites in super_cache_scan() / super_cache_count() was lost, so the helper is still gating every ->nr_cached_objects() hook and 0ef8faff490be is effectively a no-op.

Consequences of the leftover gate:

- XFS's inode-reclaim hook, which is intentionally driven from per-memcg contexts to free memcg-charged slab, is still short-circuited in fs/super.c — exactly the regression from commit 0baad6f9b997 ("fs/super: skip non-memcg-aware nr_cached_objects in memcg slab shrink") that 0ef8faff490be was written to undo. Memcg-charged XFS inode slab therefore keeps piling up under per-memcg pressure until global reclaim kicks in.

- Any future ->nr_cached_objects()/->free_cached_objects() that grows memcg awareness is likewise blocked before it can run, so filesystems cannot opt in to per-memcg reclaim on their own — defeating the whole point of pushing the gating decision down into the callbacks.

Drop the leftover helper and its call sites so the intent of 0ef8faff490be actually takes effect.

Fixes 0ef8faff490be ("fs: push nr_cached_objects memcg gating into

individual filesystems")
Cc: [email protected]
---

Can you fix up the commit message directly on your end? Let me know if you prefer me to spin a v5 instead.

Quoted 42 lines · On Mon, 31 Aug 2026 10:28:00 +0800 Qi Zheng <[email protected]> wrote:

Quoted 5 lines

OK, please add this info to this patch's changelog. Please also decide whether a cc:stable should be added and if so, ensure the changelog makes clear why we're recommending a backport.

As part of this, let's expand on "XFS will also run into issues".

How about this instead:

---
Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") was meant to drop the blanket memcg gate in fs/super.c and let each ->nr_cached_objects() implementation decide for itself whether it is meaningful in per-memcg reclaim. However, when that patch was applied the removal of super_fs_objects_eligible() and its two call sites in super_cache_scan() / super_cache_count() was lost, so the helper is still gating every ->nr_cached_objects() hook and 0ef8faff490be is effectively a no-op.

Consequences of the leftover gate:

- XFS's inode-reclaim hook, which is intentionally driven from per-memcg contexts to free memcg-charged slab, is still short-circuited in fs/super.c — exactly the regression from commit 0baad6f9b997 ("fs/super: skip non-memcg-aware nr_cached_objects in memcg slab shrink") that 0ef8faff490be was written to undo. Memcg-charged XFS inode slab therefore keeps piling up under per-memcg pressure until global reclaim kicks in.

- Any future ->nr_cached_objects()/->free_cached_objects() that grows memcg awareness is likewise blocked before it can run, so filesystems cannot opt in to per-memcg reclaim on their own — defeating the whole point of pushing the gating decision down into the callbacks.

Drop the leftover helper and its call sites so the intent of 0ef8faff490be actually takes effect.

Fixes 0ef8faff490be ("fs: push nr_cached_objects memcg gating into

individual filesystems")
Cc: [email protected]
---

Quoted 2 lines

Can you fix up the commit message directly on your end? Let me know if you prefer me to spin a v5 instead.

Thanks. I pasted this in and turned your [1/4] into a standalone patch for mm-hotfixes, with a cc:stable.

The remaining three patches I turned into a separate series for the next merge window.

This particular patch has no evidence of reviewer input yet?

From: Qi Zheng <[email protected]>
Subject: fs: fix missed removal of super_fs_objects_eligible()
Date: Mon, 17 Aug 2026 17:03:25 +0800

Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") was meant to drop the blanket memcg gate in fs/super.c and let each ->nr_cached_objects() implementation decide for itself whether it is meaningful in per-memcg reclaim. However, when that patch was applied the removal of super_fs_objects_eligible() and its two call sites in super_cache_scan() / super_cache_count() was lost, so the helper is still gating every ->nr_cached_objects() hook and 0ef8faff490be is effectively a no-op.

Consequences of the leftover gate:

- XFS's inode-reclaim hook, which is intentionally driven from per-memcg contexts to free memcg-charged slab, is still short-circuited in fs/super.c exactly the regression from commit 0baad6f9b997 ("fs/super: skip non-memcg-aware nr_cached_objects in memcg slab shrink") that 0ef8faff490be was written to undo. Memcg-charged XFS inode slab therefore keeps piling up under per-memcg pressure until global reclaim kicks in.

- Any future ->nr_cached_objects()/->free_cached_objects() that grows memcg awareness is likewise blocked before it can run, so filesystems cannot opt in to per-memcg reclaim on their own defeating the whole point of pushing the gating decision down into the callbacks.

Drop the leftover helper and its call sites so the intent of 0ef8faff490be actually takes effect.

Link https://lore.kernel.org/[email protected] open here

Link https://lore.kernel.org/3b038d373c70ebac7cdabfb0035bb91d1d6e6cfe.1786955972.git.zhengqi.arch@bytedance.com open here

Link https://lore.kernel.org/all/[email protected]/ [0] open here

Fixes 0ef8faff490b ("fs: push nr_cached_objects memcg gating into individual filesystems")

Signed-off-by Qi Zheng <[email protected]>

Cc Baolin Wang <[email protected]>

Cc Christian Brauner <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Christian Brauner <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Muchun Song <[email protected]>

Cc Roman Gushchin <[email protected]>

Cc Shakeel Butt <[email protected]>

Cc <[email protected]>

Signed-off-by Andrew Morton <[email protected]>

---

  fs/super.c |   18 ++----------------
  1 file changed, 2 insertions(+), 16 deletions(-)

fs/super.c renamed from fs/super.c~fs-fix-missed-removal-of-super_fs_objects_eligible +2 -16

@@ -172,19 +172,6 @@ static void super_wake(struct super_bloc
  }
 
  /*
- * The s_op->nr_cached_objects hooks (used for example by btrfs and xfs)
- * operate on filesystem-global state and ignore sc->memcg. Driving them
- * from per-memcg shrink_slab_memcg() invocations only burns CPU walking
- * per-cpu counters and queueing duplicate work: the actual reclaim happens on
- * the global path (kswapd or root direct reclaim) regardless. Restrict them
- * to that path.
- */
-static inline bool super_fs_objects_eligible(struct shrink_control *sc)
-{
-	return !sc->memcg || mem_cgroup_is_root(sc->memcg);
-}
-
-/*
   * One thing we have to be careful of with a per-sb shrinker is that we don't
   * drop the last active reference to the superblock from within the shrinker.
   * If that happens we could trigger unregistering the shrinker from within the

@@ -213,7 +200,7 @@ static unsigned long super_cache_scan(st
  	if (!super_trylock_shared(sb))
  		return SHRINK_STOP;
 
-	if (sb->s_op->nr_cached_objects && super_fs_objects_eligible(sc))
+	if (sb->s_op->nr_cached_objects)
  		fs_objects = sb->s_op->nr_cached_objects(sb, sc);
 
  	inodes = list_lru_shrink_count(&sb->s_inode_lru, sc);

@@ -274,8 +261,7 @@ static unsigned long super_cache_count(s
  		return 0;
  	smp_rmb();
 
-	if (sb->s_op && sb->s_op->nr_cached_objects &&
-	    super_fs_objects_eligible(sc))
+	if (sb->s_op && sb->s_op->nr_cached_objects)
  		total_objects = sb->s_op->nr_cached_objects(sb, sc);
 
  	total_objects += list_lru_shrink_count(&sb->s_dentry_lru, sc);

_

Quoted 37 lines · On Mon, 31 Aug 2026 10:28:00 +0800 Qi Zheng <[email protected]> wrote:

Quoted 5 lines

OK, please add this info to this patch's changelog. Please also decide whether a cc:stable should be added and if so, ensure the changelog makes clear why we're recommending a backport.

As part of this, let's expand on "XFS will also run into issues".

How about this instead:

---
Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") was meant to drop the blanket memcg gate in fs/super.c and let each ->nr_cached_objects() implementation decide for itself whether it is meaningful in per-memcg reclaim. However, when that patch was applied the removal of super_fs_objects_eligible() and its two call sites in super_cache_scan() / super_cache_count() was lost, so the helper is still gating every ->nr_cached_objects() hook and 0ef8faff490be is effectively a no-op.

Consequences of the leftover gate:

- XFS's inode-reclaim hook, which is intentionally driven from per-memcg contexts to free memcg-charged slab, is still short-circuited in fs/super.c — exactly the regression from commit 0baad6f9b997 ("fs/super: skip non-memcg-aware nr_cached_objects in memcg slab shrink") that 0ef8faff490be was written to undo. Memcg-charged XFS inode slab therefore keeps piling up under per-memcg pressure until global reclaim kicks in.

- Any future ->nr_cached_objects()/->free_cached_objects() that grows memcg awareness is likewise blocked before it can run, so filesystems cannot opt in to per-memcg reclaim on their own — defeating the whole point of pushing the gating decision down into the callbacks.

Drop the leftover helper and its call sites so the intent of 0ef8faff490be actually takes effect.

Quoted 7 lines

Fixes 0ef8faff490be ("fs: push nr_cached_objects memcg gating into

individual filesystems")
Cc: [email protected]
---

Can you fix up the commit message directly on your end? Let me know if you prefer me to spin a v5 instead.

Thanks. I pasted this in and turned your [1/4] into a standalone patch for mm-hotfixes, with a cc:stable.

The remaining three patches I turned into a separate series for the next merge window.

This particular patch has no evidence of reviewer input yet?

From: Qi Zheng <[email protected]>
Subject: fs: fix missed removal of super_fs_objects_eligible()
Date: Mon, 17 Aug 2026 17:03:25 +0800

Commit 0ef8faff490be ("fs: push nr_cached_objects memcg gating into individual filesystems") was meant to drop the blanket memcg gate in fs/super.c and let each ->nr_cached_objects() implementation decide for itself whether it is meaningful in per-memcg reclaim. However, when that patch was applied the removal of super_fs_objects_eligible() and its two call sites in super_cache_scan() / super_cache_count() was lost, so the helper is still gating every ->nr_cached_objects() hook and 0ef8faff490be is effectively a no-op.

Consequences of the leftover gate:

- XFS's inode-reclaim hook, which is intentionally driven from per-memcg contexts to free memcg-charged slab, is still short-circuited in fs/super.c exactly the regression from commit 0baad6f9b997 ("fs/super: skip non-memcg-aware nr_cached_objects in memcg slab shrink") that 0ef8faff490be was written to undo. Memcg-charged XFS inode slab therefore keeps piling up under per-memcg pressure until global reclaim kicks in.

- Any future ->nr_cached_objects()/->free_cached_objects() that grows memcg awareness is likewise blocked before it can run, so filesystems cannot opt in to per-memcg reclaim on their own defeating the whole point of pushing the gating decision down into the callbacks.

Drop the leftover helper and its call sites so the intent of 0ef8faff490be actually takes effect.

Link https://lore.kernel.org/[email protected] open here

Link https://lore.kernel.org/3b038d373c70ebac7cdabfb0035bb91d1d6e6cfe.1786955972.git.zhengqi.arch@bytedance.com open here

Link https://lore.kernel.org/all/[email protected]/ [0] open here

Fixes 0ef8faff490b ("fs: push nr_cached_objects memcg gating into individual filesystems")

Signed-off-by Qi Zheng <[email protected]>

Cc Baolin Wang <[email protected]>

Cc Christian Brauner <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Christian Brauner <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Muchun Song <[email protected]>

Cc Roman Gushchin <[email protected]>

Cc Shakeel Butt <[email protected]>

Cc <[email protected]>

Signed-off-by Andrew Morton <[email protected]>

---

From: Qi Zheng <[email protected]>

obj_cgroup_memcg() currently requires a non-NULL objcg, so callers that may hold a NULL objcg must guard the call with an explicit NULL check. This pattern is duplicated in folio_memcg(), folio_memcg_check(), mm/page_owner.c, and mm/zswap.c.

Teach obj_cgroup_memcg() to accept NULL and return NULL in that case, then remove the redundant NULL checks at the call sites. Also remove the mem_cgroup_from_entry() wrapper in zswap, which existed solely to provide this NULL-safe behaviour, and replace its two callers with direct obj_cgroup_memcg() calls.

No functional change intended.

Signed-off-by Qi Zheng <[email protected]>

From: Qi Zheng <[email protected]>

The shmem unused huge shrinker keeps a per-superblock list of inodes whose tail huge folio extends beyond i_size. Since that list is not memcg aware, reclaim triggered by one memcg can scan inodes from the whole superblock and split shmem huge folios charged to unrelated memcgs.

Convert the shrink list to a memcg-aware list_lru. Queue each inode on the list_lru sublist matching the memcg and node of the current tail huge folio, so non-root memcg reclaim only walks candidates charged to the reclaiming memcg. Global reclaim, root memcg reclaim and shmem quota reclaim keep global semantics.

Rather than pinning a struct mem_cgroup reference in shmem_inode_info, store a struct obj_cgroup reference instead. The list_lru add and delete paths resolve the current memcg from the objcg under RCU, so that memcg offline and list_lru entry migration remain consistent: list_lru migrates entries to the parent memcg sublist on offline, and obj_cgroup_memcg() follows the same reparenting, ensuring the correct sublist is always found at delete time. This avoids pinning a dying memcg through a long-lived CSS reference.

The list_lru still tracks inodes while the actual split target is the current tail huge folio, so validate the folio memcg/node during scan. If the folio no longer matches the reclaim context or splitting cannot proceed, requeue the inode according to the current tail folio; if the inode is no longer shrinkable, drop the scan entry.

This can be tested with the shrinker debugfs interface by allocating 32 tmpfs tail THPs in each of two memcgs, then scanning the sb-tmpfs shrinker with memcg A's cgroup id:

                before A scan    after A scan
   base         A=64M, B=64M     A=64M, B=64M (per-memcg count is skipped)
   patched      A=64M, B=64M     A=0,   B=64M

Signed-off-by Qi Zheng <[email protected]>

---

Changes in v4:
Changes in v3:
Changes in v2:

The shmem unused huge shrinker maintains a per-superblock list of inodes whose tail huge folio extends beyond i_size. Because this list is not memcg aware, reclaim triggered by memcg A can scan inodes across the entire superblock and split huge folios charged to unrelated memcg B, causing unexpected impact on it.

In the worst case, memcg A has no reclaimable shmem at all, making the reclaim entirely useless and incurring unnecessary latency. We observed this in production, where page lock contention during split caused multi-hundred-millisecond stalls:

tid 11340 comm scanner locked a page for 182264 us! kstack:

          unlock_page+1
          split_huge_page_to_list+3135
          shmem_unused_huge_shrink+767
          super_cache_scan+329
          do_shrink_slab+291
          shrink_slab+533
          shrink_node+400
          do_try_to_free_pages+206
          try_to_free_mem_cgroup_pages+262
          try_charge_memcg+591
          mem_cgroup_charge+136
          __handle_mm_fault+2431
          handle_mm_fault+194
          do_user_addr_fault+462
          __do_page_fault+176
          do_page_fault+48
          page_fault+62

Usama's recent patch [1] prevents the shmem unused shrinker from being invoked during memcg-level reclaim altogether, but this is overly
conservative: we can do better by reclaiming only the shmem charged to the reclaiming memcg.

This series converts the shrinker list to a memcg-aware list_lru, so that non-root memcg reclaim walks only candidates charged to the reclaiming memcg. Global reclaim, root memcg reclaim and shmem quota reclaim retain their existing global semantics.

To avoid pinning a dying memcg through a long-lived CSS reference, each inode stores an obj_cgroup reference instead of a mem_cgroup reference. The list_lru add/delete paths resolve the current memcg from the objcg under RCU, staying consistent with list_lru's own memcg migration on offline.

Quoted 3 lines · On Mon, 17 Aug 2026 17:03:24 +0800 Qi Zheng <[email protected]> wrote:

Changes in v4:
Changes in v3:
Changes in v2:

Thanks for the diligent versioning info. fyi, it is conventional to maintain this below the --- separator. It's not really the most important part of the [0/N]!

Quoted 8 lines

The shmem unused huge shrinker maintains a per-superblock list of inodes whose tail huge folio extends beyond i_size. Because this list is not memcg aware, reclaim triggered by memcg A can scan inodes across the entire superblock and split huge folios charged to unrelated memcg B, causing unexpected impact on it.

Quoted 4 lines

In the worst case, memcg A has no reclaimable shmem at all, making the reclaim entirely useless and incurring unnecessary latency. We observed this in production, where page lock contention during split caused multi-hundred-millisecond stalls:

Ugh. That's the most important part!

Quoted 29 lines

   tid 11340 comm scanner locked a page for 182264 us! kstack:
           unlock_page+1
           split_huge_page_to_list+3135
           shmem_unused_huge_shrink+767
           super_cache_scan+329
           do_shrink_slab+291
           shrink_slab+533
           shrink_node+400
           do_try_to_free_pages+206
           try_to_free_mem_cgroup_pages+262
           try_charge_memcg+591
           mem_cgroup_charge+136
           __handle_mm_fault+2431
           handle_mm_fault+194
           do_user_addr_fault+462
           __do_page_fault+176
           do_page_fault+48
           page_fault+62

Usama's recent patch [1] prevents the shmem unused shrinker from being invoked during memcg-level reclaim altogether, but this is overly
conservative: we can do better by reclaiming only the shmem charged to the reclaiming memcg.

This series converts the shrinker list to a memcg-aware list_lru, so that non-root memcg reclaim walks only candidates charged to the reclaiming memcg. Global reclaim, root memcg reclaim and shmem quota reclaim retain their existing global semantics.

Quoted 5 lines

To avoid pinning a dying memcg through a long-lived CSS reference, each inode stores an obj_cgroup reference instead of a mem_cgroup reference. The list_lru add/delete paths resolve the current memcg from the objcg under RCU, staying consistent with list_lru's own memcg migration on offline.

Sashiko said a few things and they look disturbing-if-true:

	https://sashiko.dev/#/patchset/[email protected]

(Apologies if this has already been considered - we don't have ways of tracking all this (yet, I hope) apart from personal memory and personal memorys are quite fried at present)