Skip to contentSkip to conversation

mm/mglru: fix ineffective memory protection for non-kswapd reclaim

RFC PATCH In review Ridong Chen 3 participants 7 messages Aug 26 – Aug 28 1 message not in the archive

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 mm/mglru: fix ineffective memory protection for non-kswapd reclaim

From: Ridong Chen <[email protected]>

memory.min/low is silently bypassed for MGLRU during global proactive reclaim (writing to the root memory.reclaim) and global direct reclaim. It can be reproduced as follows:

  # echo 7 > /sys/kernel/mm/lru_gen/enabled
  # cd /sys/fs/cgroup
  # mkdir -p a/b
  # echo 100M > a/memory.min
  # echo +memory > a/cgroup.subtree_control
  # echo 100M > a/b/memory.min
  # echo $$ > a/b/cgroup.procs
  # dd if=/dev/zero of=/tmp/testfile bs=1M count=200
  # cat a/b/memory.current
  222650368
  # echo 500M > memory.reclaim
  -bash: echo: write error: Resource temporarily unavailable
  # cat a/b/memory.current
  6070272

memory.min is 100M, yet reclaim drops a/b down to 6M, breaking the protection. The traditional LRU path is not affected because shrink_node() calls mem_cgroup_calculate_protection() for each memcg it visits during a top-down tree walk.

Commit 30d77b7eef01 ("mm/mglru: fix ineffective protection calculation") moved the protection computation into lru_gen_age_node(), which only runs for kswapd. Non-kswapd global reclaim reaches shrink_one() through lru_gen_shrink_node() -> shrink_many() without any protection computation, so emin/elow remain stale or zero.

Introduce mem_cgroup_protection_path() which computes emin/elow along the root-to-target path only by iterating through the cgroup ancestors array top-down. This avoids the full tree traversal that would be needed with mem_cgroup_calculate_protection(), limiting the cost to O(depth) per memcg - typically 3-5 levels.

Call it from shrink_one() for the non-kswapd path so that each memcg about to be shrunk has correct protection values.

Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
---
 include/linux/memcontrol.h |  7 +++++++
 mm/memcontrol.c            | 42 ++++++++++++++++++++++++++++++++++++++
 mm/vmscan.c                |  8 +++++++-
 3 files changed, 56 insertions(+), 1 deletion(-)
include/linux/memcontrol.h +7 -0
index 7d1c0ce189a8..26be8277d64b 100644
@@ -605,6 +605,8 @@ static inline void mem_cgroup_protection(struct mem_cgroup *root,
 
 void mem_cgroup_calculate_protection(struct mem_cgroup *root,
 				     struct mem_cgroup *memcg);
+void mem_cgroup_protection_path(struct mem_cgroup *root,
+				struct mem_cgroup *memcg);
 
 static inline bool mem_cgroup_unprotected(struct mem_cgroup *target,
 					  struct mem_cgroup *memcg)
@@ -1133,6 +1135,11 @@ static inline void mem_cgroup_calculate_protection(struct mem_cgroup *root,
 {
 }
 
+static inline void mem_cgroup_protection_path(struct mem_cgroup *root,
+					      struct mem_cgroup *memcg)
+{
+}
+
 static inline bool mem_cgroup_unprotected(struct mem_cgroup *target,
 					  struct mem_cgroup *memcg)
 {
mm/memcontrol.c +42 -0

48 lines not on this page · raw message

mm/vmscan.c +7 -1
index 6dff207ad8c6..e572d2742c8c 100644
@@ -5102,7 +5102,13 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc)
 	struct mem_cgroup *memcg = lruvec_memcg(lruvec);
 	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
 
-	/* lru_gen_age_node() called mem_cgroup_calculate_protection() */
+	/*
+	 * For kswapd, lru_gen_age_node() has already called
+	 * mem_cgroup_calculate_protection()
+	 */
+	if (!current_is_kswapd())
+		mem_cgroup_protection_path(NULL, memcg);
+
 	if (mem_cgroup_below_min(NULL, memcg))
 		return MEMCG_LRU_YOUNG;
 
Signature · 1 line
2.34.1
#2 lore raw
In reply to a message not in the archive
Quoted 6 lines · On 8/26/2026 9:47 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Integer overflow in effective_protection() calculations on systems with >= 16TB RAM. --

Thanks for the careful analysis. You're right that this is pre-existing in effective_protection() and not introduced by this patch.

I'm not sure it's worth fixing in practice, though — the overflow requires systems with >= 16TB of RAM, which I've never encountered myself. So I'd rather not fold a fix for it into this patch.

I'd like to hear the maintainers' thoughts on whether this is worth addressing separately.

Quoted 68 lines

commit 0b95b34e75bd94d348d96210ff25235304802608
Author: Ridong Chen <[email protected]>

mm/mglru: fix ineffective memory protection for non-kswapd reclaim

This commit fixes a bug where memory protection (memory.min/low) is silently bypassed during non-kswapd global reclaim. It introduces mem_cgroup_protection_path() to compute emin/elow correctly without a full tree walk.

Quoted 5 lines
mm/memcontrol.c +0 -0
index 11b85f4b6828b..a334974333cb6 100644
@@ -5197,6 +5197,48 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,

[ ... ]

mm/memcontrol.c void mem_cgroup_calculate_protection(struct mem_cgroup *root, · line 5238

Quoted 9 lines
mm/memcontrol.c +9 -0
+	rcu_read_lock();
+	for (i = root_level + 1; i <= cg->level; i++) {
+		struct mem_cgroup *cur;
+
+		cur = mem_cgroup_from_css(cgroup_css(cg->ancestors[i],
+						    &memory_cgrp_subsys));
+		if (cur)
+			page_counter_calculate_protection(&root->memory,
+					&cur->memory, recursive_protection);

[Severity: High]
This isn't a bug introduced by this patch, but since we are relying on page_counter_calculate_protection() here, is there a risk of an integer overflow during the protection calculation on systems with large memory?

Looking at effective_protection() in mm/page_counter.c, the function uses 64-bit unsigned long values to represent memory limits in pages:

mm/page_counter.c:effective_protection() {
     ...
	if (siblings_protected > parent_effective)
		return protected * parent_effective / siblings_protected;
     ...
}

If a system has >= 16TB of RAM, both protected and parent_effective can exceed 2^32 pages (since 16TB is 2^32 pages of 4KB). The multiplication protected * parent_effective will exceed 2^64 - 1 and wrap around.

This also seems to happen in a secondary distribution path in the same function:

mm/page_counter.c:effective_protection() {
     ...
		unclaimed = parent_effective - siblings_protected;
		unclaimed *= usage - protected;
     ...
}

Here, multiplying unclaimed and (usage - protected) can also multiply two page counts that each exceed 2^32, causing an overflow for the same reasons.

Could this overflow result in a drastically reduced effective protection value, effectively bypassing cgroup memory protection and causing cgroups to be unfairly reclaimed on machines with very large memory?

Quoted 3 lines
mm/memcontrol.c +3 -0
+	}
+	rcu_read_unlock();
+}
Signature · 2 lines
Best regards
Ridong
#5 lore raw
In reply to Ridong Chen
Quoted 32 lines · Ridong Chen

From: Ridong Chen <[email protected]>

memory.min/low is silently bypassed for MGLRU during global proactive reclaim (writing to the root memory.reclaim) and global direct reclaim. It can be reproduced as follows:

   # echo 7 > /sys/kernel/mm/lru_gen/enabled
   # cd /sys/fs/cgroup
   # mkdir -p a/b
   # echo 100M > a/memory.min
   # echo +memory > a/cgroup.subtree_control
   # echo 100M > a/b/memory.min
   # echo $$ > a/b/cgroup.procs
   # dd if=/dev/zero of=/tmp/testfile bs=1M count=200
   # cat a/b/memory.current
   222650368
   # echo 500M > memory.reclaim
   -bash: echo: write error: Resource temporarily unavailable
   # cat a/b/memory.current
   6070272

memory.min is 100M, yet reclaim drops a/b down to 6M, breaking the protection. The traditional LRU path is not affected because shrink_node() calls mem_cgroup_calculate_protection() for each memcg it visits during a top-down tree walk.

Commit 30d77b7eef01 ("mm/mglru: fix ineffective protection calculation") moved the protection computation into lru_gen_age_node(), which only runs for kswapd. Non-kswapd global reclaim reaches shrink_one() through lru_gen_shrink_node() -> shrink_many() without any protection computation, so emin/elow remain stale or zero.

Quoted 6 lines

Introduce mem_cgroup_protection_path() which computes emin/elow along the root-to-target path only by iterating through the cgroup ancestors array top-down. This avoids the full tree traversal that would be needed with mem_cgroup_calculate_protection(), limiting the cost to O(depth) per memcg - typically 3-5 levels.

Hi all,

This is an RFC patch.

My initial attempt to address this issue involved traversing the entire memcg tree, similar to what kswapd does. However, Sashiko raised concerns that this could introduce performance regressions [1].

To address that, this patch instead walks the cgroup ancestors array in a top-down manner, as outlined above, and I'd like to gather feedback on this approach.

[1] https://lore.kernel.org/linux-mm/[email protected]/

Quoted 109 lines

Call it from shrink_one() for the non-kswapd path so that each memcg about to be shrunk has correct protection values.

Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
---
  include/linux/memcontrol.h |  7 +++++++
  mm/memcontrol.c            | 42 ++++++++++++++++++++++++++++++++++++++
  mm/vmscan.c                |  8 +++++++-
  3 files changed, 56 insertions(+), 1 deletion(-)
include/linux/memcontrol.h +7 -0
index 7d1c0ce189a8..26be8277d64b 100644
@@ -605,6 +605,8 @@ static inline void mem_cgroup_protection(struct mem_cgroup *root,
 
  void mem_cgroup_calculate_protection(struct mem_cgroup *root,
  				     struct mem_cgroup *memcg);
+void mem_cgroup_protection_path(struct mem_cgroup *root,
+				struct mem_cgroup *memcg);
 
  static inline bool mem_cgroup_unprotected(struct mem_cgroup *target,
  					  struct mem_cgroup *memcg)
@@ -1133,6 +1135,11 @@ static inline void mem_cgroup_calculate_protection(struct mem_cgroup *root,
  {
  }
 
+static inline void mem_cgroup_protection_path(struct mem_cgroup *root,
+					      struct mem_cgroup *memcg)
+{
+}
+
  static inline bool mem_cgroup_unprotected(struct mem_cgroup *target,
  					  struct mem_cgroup *memcg)
  {
mm/memcontrol.c +42 -0
index 1271d390b617..c739db33b917 100644
@@ -5198,6 +5198,48 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
  	page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection);
  }
 
+/**
+ * mem_cgroup_protection_path - compute protection along root->memcg path
+ * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup)
+ * @memcg: the target memory cgroup
+ *
+ * Walk the ancestor path from @root down to @memcg and compute the effective
+ * protection at each level.  This is safe for isolated queries because it
+ * ensures parents are computed before children.
+ */
+void mem_cgroup_protection_path(struct mem_cgroup *root,
+				struct mem_cgroup *memcg)
+{
+	bool recursive_protection =
+		cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT;
+	struct cgroup *cg;
+	int root_level, i;
+
+	if (mem_cgroup_disabled())
+		return;
+
+	if (!root)
+		root = root_mem_cgroup;
+
+	if (memcg == root)
+		return;
+
+	root_level = root->css.cgroup->level;
+	cg = memcg->css.cgroup;
+
+	rcu_read_lock();
+	for (i = root_level + 1; i <= cg->level; i++) {
+		struct mem_cgroup *cur;
+
+		cur = mem_cgroup_from_css(cgroup_css(cg->ancestors[i],
+						    &memory_cgrp_subsys));
+		if (cur)
+			page_counter_calculate_protection(&root->memory,
+					&cur->memory, recursive_protection);
+	}
+	rcu_read_unlock();
+}
+
  static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg,
  			gfp_t gfp)
  {
mm/vmscan.c +7 -1
index 6dff207ad8c6..e572d2742c8c 100644
@@ -5102,7 +5102,13 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc)
  	struct mem_cgroup *memcg = lruvec_memcg(lruvec);
  	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
 
-	/* lru_gen_age_node() called mem_cgroup_calculate_protection() */
+	/*
+	 * For kswapd, lru_gen_age_node() has already called
+	 * mem_cgroup_calculate_protection()
+	 */
+	if (!current_is_kswapd())
+		mem_cgroup_protection_path(NULL, memcg);
+
  	if (mem_cgroup_below_min(NULL, memcg))
  		return MEMCG_LRU_YOUNG;
 
Signature · 2 lines
Best regards
Ridong
maintainer
#3 lore raw
In reply to Ridong Chen

Hi Ridong.

Quoted 11 lines · Ridong Chen
Quoted 6 lines · On 8/26/2026 9:47 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Integer overflow in effective_protection() calculations on systems with >= 16TB RAM. --

Thanks for the careful analysis. You're right that this is pre-existing in effective_protection() and not introduced by this patch.

Quoted 6 lines

I'm not sure it's worth fixing in practice, though — the overflow requires systems with >= 16TB of RAM, which I've never encountered myself. So I'd rather not fold a fix for it into this patch.

I'd like to hear the maintainers' thoughts on whether this is worth addressing separately.

Machines with >=16 TiB aren't completely out of reach [1]. Then configuring such protections to attain the overflow is another condition, so I'd consider this a nice to have but not a pressing fixup (separately, with thorough review of occurences where total page counts/bytes might get multiplied).

HTH,
Michal

[1] E.g. https://aws.amazon.com/ec2/instance-types/u7i/

maintainer
#6 lore raw
In reply to Ridong Chen
Quoted 32 lines · Ridong Chen

From: Ridong Chen <[email protected]>

memory.min/low is silently bypassed for MGLRU during global proactive reclaim (writing to the root memory.reclaim) and global direct reclaim. It can be reproduced as follows:

  # echo 7 > /sys/kernel/mm/lru_gen/enabled
  # cd /sys/fs/cgroup
  # mkdir -p a/b
  # echo 100M > a/memory.min
  # echo +memory > a/cgroup.subtree_control
  # echo 100M > a/b/memory.min
  # echo $$ > a/b/cgroup.procs
  # dd if=/dev/zero of=/tmp/testfile bs=1M count=200
  # cat a/b/memory.current
  222650368
  # echo 500M > memory.reclaim
  -bash: echo: write error: Resource temporarily unavailable
  # cat a/b/memory.current
  6070272

memory.min is 100M, yet reclaim drops a/b down to 6M, breaking the protection. The traditional LRU path is not affected because shrink_node() calls mem_cgroup_calculate_protection() for each memcg it visits during a top-down tree walk.

Commit 30d77b7eef01 ("mm/mglru: fix ineffective protection calculation") moved the protection computation into lru_gen_age_node(), which only runs for kswapd. Non-kswapd global reclaim reaches shrink_one() through lru_gen_shrink_node() -> shrink_many() without any protection computation, so emin/elow remain stale or zero.

Quoted 5 lines

Introduce mem_cgroup_protection_path() which computes emin/elow along the root-to-target path only by iterating through the cgroup ancestors array top-down. This avoids the full tree traversal that would be needed with mem_cgroup_calculate_protection(), limiting the cost to O(depth) per memcg - typically 3-5 levels.

Right, because of the memcg-lru...

Quoted 37 lines
mm/memcontrol.c +31 -0
@@ -5198,6 +5198,48 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
 	page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection);
 }
 
+/**
+ * mem_cgroup_protection_path - compute protection along root->memcg path
+ * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup)
+ * @memcg: the target memory cgroup
+ *
+ * Walk the ancestor path from @root down to @memcg and compute the effective
+ * protection at each level.  This is safe for isolated queries because it
+ * ensures parents are computed before children.
+ */
+void mem_cgroup_protection_path(struct mem_cgroup *root,
+				struct mem_cgroup *memcg)
+{
+	bool recursive_protection =
+		cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT;
+	struct cgroup *cg;
+	int root_level, i;
+
+	if (mem_cgroup_disabled())
+		return;
+
+	if (!root)
+		root = root_mem_cgroup;
+
+	if (memcg == root)
+		return;
+
+	root_level = root->css.cgroup->level;
+	cg = memcg->css.cgroup;
+
+	rcu_read_lock();
+	for (i = root_level + 1; i <= cg->level; i++) {

mm/memcontrol.c void mem_cgroup_calculate_protection(struct mem_cgroup *root, · line 5241

Quoted 10 lines
mm/memcontrol.c +10 -0
+		struct mem_cgroup *cur;
+
+		cur = mem_cgroup_from_css(cgroup_css(cg->ancestors[i],
+						    &memory_cgrp_subsys));
+		if (cur)
+			page_counter_calculate_protection(&root->memory,
+					&cur->memory, recursive_protection);
+	}
+	rcu_read_unlock();
+}

Please wrap this in a #ifdef CONFIG_LRU_GEN block.

#4 lore raw
In reply to Michal Koutný
Quoted 27 lines · Michal Koutný

Hi Ridong.

Quoted 11 lines · Ridong Chen
Quoted 6 lines · On 8/26/2026 9:47 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Integer overflow in effective_protection() calculations on systems with >= 16TB RAM. --

Thanks for the careful analysis. You're right that this is pre-existing in effective_protection() and not introduced by this patch.

Quoted 6 lines

I'm not sure it's worth fixing in practice, though — the overflow requires systems with >= 16TB of RAM, which I've never encountered myself. So I'd rather not fold a fix for it into this patch.

I'd like to hear the maintainers' thoughts on whether this is worth addressing separately.

Machines with >=16 TiB aren't completely out of reach [1]. Then configuring such protections to attain the overflow is another condition, so I'd consider this a nice to have but not a pressing fixup (separately, with thorough review of occurences where total page counts/bytes might get multiplied).

Quoted 4 lines

HTH,
Michal

[1] E.g. https://aws.amazon.com/ec2/instance-types/u7i/

Thanks Michal,

It's worth fixing now, I'll send a separate patch to address it.

Signature · 2 lines
Best regards
Ridong
#7 lore raw
In reply to Johannes Weiner
Quoted 33 lines · Johannes Weiner
Quoted 32 lines · Ridong Chen

From: Ridong Chen <[email protected]>

memory.min/low is silently bypassed for MGLRU during global proactive reclaim (writing to the root memory.reclaim) and global direct reclaim. It can be reproduced as follows:

   # echo 7 > /sys/kernel/mm/lru_gen/enabled
   # cd /sys/fs/cgroup
   # mkdir -p a/b
   # echo 100M > a/memory.min
   # echo +memory > a/cgroup.subtree_control
   # echo 100M > a/b/memory.min
   # echo $$ > a/b/cgroup.procs
   # dd if=/dev/zero of=/tmp/testfile bs=1M count=200
   # cat a/b/memory.current
   222650368
   # echo 500M > memory.reclaim
   -bash: echo: write error: Resource temporarily unavailable
   # cat a/b/memory.current
   6070272

memory.min is 100M, yet reclaim drops a/b down to 6M, breaking the protection. The traditional LRU path is not affected because shrink_node() calls mem_cgroup_calculate_protection() for each memcg it visits during a top-down tree walk.

Commit 30d77b7eef01 ("mm/mglru: fix ineffective protection calculation") moved the protection computation into lru_gen_age_node(), which only runs for kswapd. Non-kswapd global reclaim reaches shrink_one() through lru_gen_shrink_node() -> shrink_many() without any protection computation, so emin/elow remain stale or zero.

Quoted 8 lines
Quoted 5 lines

Introduce mem_cgroup_protection_path() which computes emin/elow along the root-to-target path only by iterating through the cgroup ancestors array top-down. This avoids the full tree traversal that would be needed with mem_cgroup_calculate_protection(), limiting the cost to O(depth) per memcg - typically 3-5 levels.

Right, because of the memcg-lru...

Yep, that's it.

Quoted 41 lines
Quoted 41 lines
mm/memcontrol.c +35 -0
@@ -5198,6 +5198,48 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
  	page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection);
  }
 
+/**
+ * mem_cgroup_protection_path - compute protection along root->memcg path
+ * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup)
+ * @memcg: the target memory cgroup
+ *
+ * Walk the ancestor path from @root down to @memcg and compute the effective
+ * protection at each level.  This is safe for isolated queries because it
+ * ensures parents are computed before children.
+ */
+void mem_cgroup_protection_path(struct mem_cgroup *root,
+				struct mem_cgroup *memcg)
+{
+	bool recursive_protection =
+		cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT;
+	struct cgroup *cg;
+	int root_level, i;
+
+	if (mem_cgroup_disabled())
+		return;
+
+	if (!root)
+		root = root_mem_cgroup;
+
+	if (memcg == root)
+		return;
+
+	root_level = root->css.cgroup->level;
+	cg = memcg->css.cgroup;
+
+	rcu_read_lock();
+	for (i = root_level + 1; i <= cg->level; i++) {
+		struct mem_cgroup *cur;
+
+		cur = mem_cgroup_from_css(cgroup_css(cg->ancestors[i],
+						    &memory_cgrp_subsys));
Quoted 8 lines

mm/memcontrol.c void mem_cgroup_calculate_protection(struct mem_cgroup *root, · line 5241

Quoted 6 lines
mm/memcontrol.c +6 -0
+		if (cur)
+			page_counter_calculate_protection(&root->memory,
+					&cur->memory, recursive_protection);
+	}
+	rcu_read_unlock();
+}

Please wrap this in a #ifdef CONFIG_LRU_GEN block.

Will add.

Signature · 2 lines
Best regards
Ridong