mm: avoid processing mlocked THPs in deferred split shrinker
Subsystems (2)
Lance Yang
reviewerPATCH 1/1 mm: avoid processing mlocked THPs in deferred split shrinker
From: Lance Yang <[email protected]>
When a new THP is faulted in or collapsed, it is unconditionally added to the deferred split queue. If this THP is subsequently mlocked, it remains on the queue but is removed from the LRU and marked unevictable.
During memory reclaim, deferred_split_scan() will still pick up this large folio. Because it's not partially mapped, it will proceed to call thp_underused() and then attempt to split_folio() to free all zero-filled subpages.
This is a pointless waste of CPU cycles. The folio is mlocked and unevictable, so any attempt to reclaim memory from it via splitting is doomed to fail.
So, let's add an early folio_test_mlocked() check to skip this case.
--- mm/huge_memory.c | 3 +++ 1 file changed, 3 insertions(+)
mm/huge_memory.c +3 -0
@@ -4183,6 +4183,9 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,bool underused = false;if (!folio_test_partially_mapped (folio)) { +/* An mlocked folio is not a candidate for the shrinker. */ +if (folio_test_mlocked (folio)) + goto next; underused =thp_underused (folio);if (!underused) goto next;
Signature · 1 line
2.49.0
Quoted 1 line · Lance Yang
From: Lance Yang <[email protected]>
Subject should likely be more specific:
mm: skip mlocked THPs that are underused early in deferred_split_scan()
Quoted 5 lines
When a new THP is faulted in or collapsed, it is unconditionally added to the deferred split queue. If this THP is subsequently mlocked, it remains on the queue but is removed from the LRU and marked unevictable.
Quoted 8 lines
During memory reclaim,
deferred_split_scan()will still pick up this large folio. Because it's not partially mapped, it will proceed to callthp_underused()and then attempt tosplit_folio()to free all zero-filled subpages.This is a pointless waste of CPU cycles. The folio is mlocked and unevictable, so any attempt to reclaim memory from it via splitting is doomed to fail.
I think the whole description is a bit misleading: we're not reclaiming
memory from fully-mapped THPs even when they are underused, because it
could violate mlock() semantics where we don't want a page fault+memory allocation on next access.
So something like the following might be clearer.
"When we stumble over a fully-mapped THP in the deferred shrinker, it does not make sense trying to detect whether it is underused, because try_to_map_unused_to_zeropage(), called while splitting the folio, will not actually replace any zero-ed pages by the shared zeropage.
Splitting the folio in that case does not make any sense, so let's not even scan if the folio is underused. "
If I run my reproducer from [1] and mlock() the pages just after allocating them, then I essentially get
AnonHugePages: 1048576 kB
converted to
Anonymous: 1048580 kB
Which makes sense (no memory optimized out) as discussed above.
[1] https://lkml.kernel.org/r/[email protected]
Quoted 13 lines
So, let's add an early
folio_test_mlocked()check to skip this case.Signed-off-by Lance Yang <[email protected]>--- mm/huge_memory.c | 3 +++ 1 file changed, 3 insertions(+)mm/huge_memory.c +0
-0
mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4186
Quoted 4 lines
mm/huge_memory.c +1
-0@@ -4183,6 +4183,9 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,bool underused = false;if (!folio_test_partially_mapped (folio)) { +/* An mlocked folio is not a candidate for the shrinker. */
/*
* See try_to_map_unused_to_zeropage(): we cannot optimize zero-filled
* pages after splitting an mlocked folio.
*/
Quoted 5 lines
mm/huge_memory.c +2
-0+if (folio_test_mlocked (folio)) + goto next; underused =thp_underused (folio);if (!underused) goto next;
Signature · 3 lines
Cheers David / dhildenb
Lance Yang
reviewerQuoted 6 lines · David Hildenbrand
Quoted 1 line · Lance Yang
From: Lance Yang <[email protected]>
Subject should likely be more specific:
mm: skip mlocked THPs that are underused early in
deferred_split_scan()
Right, that's a much better and more precise subject. Thanks!
Quoted 19 lines
Quoted 6 lines
When a new THP is faulted in or collapsed, it is unconditionally added to the deferred split queue. If this THP is subsequently mlocked, it remains on the queue but is removed from the LRU and marked unevictable.
Quoted 8 lines
During memory reclaim,
deferred_split_scan()will still pick up this large folio. Because it's not partially mapped, it will proceed to callthp_underused()and then attempt tosplit_folio()to free all zero-filled subpages.This is a pointless waste of CPU cycles. The folio is mlocked and unevictable, so any attempt to reclaim memory from it via splitting is doomed to fail.
I think the whole description is a bit misleading: we're not reclaiming
memory from fully-mapped THPs even when they are underused, because it
could violatemlock()semantics where we don't want a page fault+memory allocation on next access.
Quoted 6 lines
So something like the following might be clearer.
"When we stumble over a fully-mapped THP in the deferred shrinker, it does not make sense trying to detect whether it is underused, because
try_to_map_unused_to_zeropage(), called while splitting the folio, will not actually replace any zero-ed pages by the shared zeropage.Splitting the folio in that case does not make any sense, so let's not even scan if the folio is underused. "
Nice, that makes it much clearer. My understanding was indeed imprecise.
Quoted 5 lines
If I run my reproducer from [1] and
mlock()the pages just after allocating them, then I essentially get
Quoted 7 lines
AnonHugePages: 1048576 kB
converted to
Anonymous: 1048580 kB
Which makes sense (no memory optimized out) as discussed above.
Yes, my reproducer also shows exactly that. It's clear a lot of work is done but no memory is actually optimized out ;)
Quoted 12 lines
[1] https://lkml.kernel.org/r/[email protected]
Quoted 8 lines
So, let's add an early
folio_test_mlocked()check to skip this case.Signed-off-by Lance Yang <[email protected]>---
mm/huge_memory.c| 3 +++1 file changed, 3 insertions(+)
Quoted 14 lines
mm/huge_memory.c static unsigned long deferred_split_scan(struct · line 4186
Quoted 9 lines
mm/huge_memory.c +1
-0@@ -4183,6 +4183,9 @@ static unsigned long deferred_split_scan(structshrinker *shrink, bool underused = false; if (!folio_test_partially_mapped(folio)) { + /* An mlocked folio is not a candidate for the shrinker. *//*
* Seetry_to_map_unused_to_zeropage(): we cannot optimize zero-filled
* pages after splitting an mlocked folio.
*/
Got it. I'll update the changelog and this comment as suggested.
Quoted 8 lines
mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4191
Quoted 6 lines
mm/huge_memory.c +2
-0+ if (folio_test_mlocked(folio)) + goto next; underused = thp_underused(folio); if (!underused) goto next;
Cheers,
Lance
Quoted 6 lines
Quoted 6 lines
If I run my reproducer from [1] and
mlock()the pages just after allocating them, then I essentially getAnonHugePages: 1048576 kB
Quoted 8 lines
Quoted 5 lines
converted to
Anonymous: 1048580 kB
Which makes sense (no memory optimized out) as discussed above.
Yes, my reproducer also shows exactly that. It's clear a lot of work is done but no memory is actually optimized out ;)
I'm not really concerned about the scanning overhead. The real harm is splitting a THP without any benefit.
Signature · 3 lines
Cheers David / dhildenb