Skip to contentSkip to conversation

mm: avoid processing mlocked THPs in deferred split shrinker

PATCH In review Lance Yang 2 participants 4 messages Sep 8, 2025

Subsystems (2)
lore
Share & tools
lore

Lance Yang

author reviewer
#0 lore raw

PATCH 1/1 mm: avoid processing mlocked THPs in deferred split shrinker

From: Lance Yang <[email protected]>

When a new THP is faulted in or collapsed, it is unconditionally added to the deferred split queue. If this THP is subsequently mlocked, it remains on the queue but is removed from the LRU and marked unevictable.

During memory reclaim, deferred_split_scan() will still pick up this large folio. Because it's not partially mapped, it will proceed to call thp_underused() and then attempt to split_folio() to free all zero-filled subpages.

This is a pointless waste of CPU cycles. The folio is mlocked and unevictable, so any attempt to reclaim memory from it via splitting is doomed to fail.

So, let's add an early folio_test_mlocked() check to skip this case.

---
 mm/huge_memory.c | 3 +++
 1 file changed, 3 insertions(+)
mm/huge_memory.c +3 -0
index 77f0c3417973..d2e84015d6b4 100644
@@ -4183,6 +4183,9 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
 		bool underused = false;
 
 		if (!folio_test_partially_mapped(folio)) {
+			/* An mlocked folio is not a candidate for the shrinker. */
+			if (folio_test_mlocked(folio))
+				goto next;
 			underused = thp_underused(folio);
 			if (!underused)
 				goto next;
Signature · 1 line
2.49.0
#1 lore raw
Quoted 1 line · Lance Yang

From: Lance Yang <[email protected]>

Subject should likely be more specific:

mm: skip mlocked THPs that are underused early in deferred_split_scan()

Quoted 5 lines

When a new THP is faulted in or collapsed, it is unconditionally added to the deferred split queue. If this THP is subsequently mlocked, it remains on the queue but is removed from the LRU and marked unevictable.

Quoted 8 lines

During memory reclaim, deferred_split_scan() will still pick up this large folio. Because it's not partially mapped, it will proceed to call thp_underused() and then attempt to split_folio() to free all zero-filled subpages.

This is a pointless waste of CPU cycles. The folio is mlocked and unevictable, so any attempt to reclaim memory from it via splitting is doomed to fail.

I think the whole description is a bit misleading: we're not reclaiming
memory from fully-mapped THPs even when they are underused, because it
could violate mlock() semantics where we don't want a page fault+memory allocation on next access.

So something like the following might be clearer.

"When we stumble over a fully-mapped THP in the deferred shrinker, it does not make sense trying to detect whether it is underused, because try_to_map_unused_to_zeropage(), called while splitting the folio, will not actually replace any zero-ed pages by the shared zeropage.

Splitting the folio in that case does not make any sense, so let's not even scan if the folio is underused. "

If I run my reproducer from [1] and mlock() the pages just after allocating them, then I essentially get

AnonHugePages: 1048576 kB

converted to

Anonymous: 1048580 kB

Which makes sense (no memory optimized out) as discussed above.

[1] https://lkml.kernel.org/r/[email protected]

Quoted 13 lines

So, let's add an early folio_test_mlocked() check to skip this case.

---
  mm/huge_memory.c | 3 +++
  1 file changed, 3 insertions(+)
mm/huge_memory.c +0 -0
index 77f0c3417973..d2e84015d6b4 100644

mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4186

Quoted 4 lines
mm/huge_memory.c +1 -0
@@ -4183,6 +4183,9 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
  		bool underused = false;
 
  		if (!folio_test_partially_mapped(folio)) {
+			/* An mlocked folio is not a candidate for the shrinker. */

/*
* See try_to_map_unused_to_zeropage(): we cannot optimize zero-filled
* pages after splitting an mlocked folio.
*/

Quoted 5 lines
mm/huge_memory.c +2 -0
+			if (folio_test_mlocked(folio))
+				goto next;
  			underused = thp_underused(folio);
  			if (!underused)
  				goto next;
Signature · 3 lines
Cheers

David / dhildenb
author reviewer
#2 lore raw
Quoted 6 lines · David Hildenbrand
Quoted 1 line · Lance Yang

From: Lance Yang <[email protected]>

Subject should likely be more specific:

mm: skip mlocked THPs that are underused early in deferred_split_scan()

Right, that's a much better and more precise subject. Thanks!

Quoted 19 lines
Quoted 6 lines

When a new THP is faulted in or collapsed, it is unconditionally added to the deferred split queue. If this THP is subsequently mlocked, it remains on the queue but is removed from the LRU and marked unevictable.

Quoted 8 lines

During memory reclaim, deferred_split_scan() will still pick up this large folio. Because it's not partially mapped, it will proceed to call thp_underused() and then attempt to split_folio() to free all zero-filled subpages.

This is a pointless waste of CPU cycles. The folio is mlocked and unevictable, so any attempt to reclaim memory from it via splitting is doomed to fail.

I think the whole description is a bit misleading: we're not reclaiming
memory from fully-mapped THPs even when they are underused, because it
could violate mlock() semantics where we don't want a page fault+memory allocation on next access.

Quoted 6 lines

So something like the following might be clearer.

"When we stumble over a fully-mapped THP in the deferred shrinker, it does not make sense trying to detect whether it is underused, because try_to_map_unused_to_zeropage(), called while splitting the folio, will not actually replace any zero-ed pages by the shared zeropage.

Splitting the folio in that case does not make any sense, so let's not even scan if the folio is underused. "

Nice, that makes it much clearer. My understanding was indeed imprecise.

Quoted 5 lines

If I run my reproducer from [1] and mlock() the pages just after allocating them, then I essentially get

Quoted 7 lines

AnonHugePages:   1048576 kB

converted to

Anonymous:       1048580 kB

Which makes sense (no memory optimized out) as discussed above.

Yes, my reproducer also shows exactly that. It's clear a lot of work is done but no memory is actually optimized out ;)

Quoted 12 lines

[1] https://lkml.kernel.org/r/[email protected]

Quoted 8 lines

So, let's add an early folio_test_mlocked() check to skip this case.

---
  mm/huge_memory.c | 3 +++

  1 file changed, 3 insertions(+)
Quoted 14 lines

mm/huge_memory.c static unsigned long deferred_split_scan(struct · line 4186

Quoted 9 lines
mm/huge_memory.c +1 -0
index 77f0c3417973..d2e84015d6b4 100644
@@ -4183,6 +4183,9 @@ static unsigned long deferred_split_scan(struct 
 shrinker *shrink,
           bool underused = false;
           if (!folio_test_partially_mapped(folio)) {
+            /* An mlocked folio is not a candidate for the shrinker. */

/*
 * See try_to_map_unused_to_zeropage(): we cannot optimize zero-filled
 * pages after splitting an mlocked folio.
 */

Got it. I'll update the changelog and this comment as suggested.

Quoted 8 lines

mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4191

Quoted 6 lines
mm/huge_memory.c +2 -0
+            if (folio_test_mlocked(folio))
+                goto next;
               underused = thp_underused(folio);
               if (!underused)
                   goto next;

Cheers,
Lance

#3 lore raw
Quoted 6 lines
Quoted 6 lines

If I run my reproducer from [1] and mlock() the pages just after allocating them, then I essentially get

AnonHugePages:   1048576 kB

Quoted 8 lines
Quoted 5 lines

converted to

Anonymous:       1048580 kB

Which makes sense (no memory optimized out) as discussed above.

Yes, my reproducer also shows exactly that. It's clear a lot of work is done but no memory is actually optimized out ;)

I'm not really concerned about the scanning overhead. The real harm is splitting a THP without any benefit.

Signature · 3 lines
Cheers

David / dhildenb