mm: restrict zero-page remapping to underused THP splits
Subsystems (2)
RFC 1/1 mm: restrict zero-page remapping to underused THP splits
Since commit b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage via TTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.
When KSM is enabled with THP=always, this causes two regressions:
1. use_zero_pages=1: KSM calls try_to_merge_one_page() which triggers split_huge_page(). The split remaps all 512 zero-filled subpages to the shared zeropage at once, freeing the entire 2MB THP when KSM only intended to process a single 4KB page. This bypasses KSM's pages_to_scan rate limiting, causing ~1GB to be freed almost instantly.
2. use_zero_pages=0: The same split side-effect occurs through the stable/unstable tree merge paths. Each pages_to_scan iteration triggers an expensive split_huge_page() that silently frees 2MB, while the scanner wastes cycles on tree searches for zero-filled pages that were already freed as a side-effect.
Fix this by restricting TTU_USE_SHARED_ZEROPAGE to only the deferred split shrinker path (deferred_split_scan), which is the only caller that intentionally splits underused THPs to reclaim zero-filled subpages. Introduce folio_split_underused() as a dedicated entry point that passes is_underused_thp=true through __folio_split(), and use it from deferred_split_scan(). All other split callers (KSM, compaction, etc.) no longer get the zero-page remapping side-effect.
Reviewers notes: this patch is one of two potential approaches. This patch turns off the zero-page freeing that has been done since the noted commit, in all the other callers, only leaving the underused shrinker to do such behavior. We can also take the opposite approach of with something like split_huge_page_no_zeropage() and call this within KSM.
--- include/linux/huge_mm.h | 2 +- mm/huge_memory.c | 17 ++++++++++++----- 2 files changed, 13 insertions(+), 6 deletions(-)
include/linux/huge_mm.h +1 -1
@@ -378,7 +378,7 @@ int folio_check_splittable(struct folio *folio, unsigned int new_order, enum split_type split_type);int folio_split (struct folio *folio,unsigned int new_order,struct page *page,struct list_head *list); - +int folio_split_underused (struct folio *folio);static inline int split_huge_page_to_list_to_order (struct page *page,struct list_head *list,unsigned int new_order) {
mm/huge_memory.c +12 -5
@@ -4045,7 +4045,8 @@ static int __folio_freeze_and_split_unmapped(struct folio *folio, unsigned int n*/ static int __folio_split (struct folio *folio,unsigned int new_order,struct page *split_at,struct page *lock_at, -struct list_head *list,enum split_type split_type) +struct list_head *list,enum split_type split_type, +bool is_underused_thp) {XA_STATE (xas, &folio->mapping->i_pages, folio->index);struct folio *end_folio =folio_next (folio);
@@ -4174,7 +4175,7 @@ static int __folio_split(struct folio *folio, unsigned int new_order,if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && is_underused_thp) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page (folio,1 << old_order, ttu_flags);
@@ -4309,7 +4310,7 @@ int __split_huge_page_to_list_to_order(struct page *page, struct list_head *liststruct folio *folio =page_folio (page);return __folio_split (folio, new_order, &folio->page, page, list, -SPLIT_TYPE_UNIFORM ); +SPLIT_TYPE_UNIFORM , false); }/**
@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, -SPLIT_TYPE_NON_UNIFORM ); +SPLIT_TYPE_NON_UNIFORM ,false ); +} + +int folio_split_underused (struct folio *folio) +{ +return __folio_split (folio,0 , &folio->page, &folio->page, +NULL ,SPLIT_TYPE_NON_UNIFORM , true); }/**
@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, }if (!folio_trylock (folio)) goto requeue; -if (!split_folio (folio)) { +if (!folio_split_underused (folio)) { did_split = true;if (underused)count_vm_event (THP_UNDERUSED_SPLIT_PAGE );
Signature · 1 line
2.54.0
David Hildenbrand (Arm)
maintainerQuoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.
And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**
Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);
In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for VM_MERGABLE and ksm_run == KSM_RUN_MERGE in try_to_map_unused_to_zeropage() to just let KSM do its thing.
That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +8 -0
@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;
@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int
new_order,
if (nr_shmem_dropped)
shmem_uncharge(mapping->host, nr_shmem_dropped);- if (!ret && is_anon && !folio_is_device_private(folio))
+ if (!ret && is_anon && !folio_is_device_private(folio) &&
+ thp_underused_split_active())
ttu_flags = TTU_USE_SHARED_ZEROPAGE;
remap_page(folio, 1 << old_order, ttu_flags);
@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)
int num_zero_pages = 0, num_filled_pages = 0;
int i;- if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
+ if (!thp_underused_split_active())
return false;
if (folio_contain_hwpoisoned_page(folio))I tend to like (2), and maybe (3) on top. Opinions?
Signature · 3 lines
Cheers, David
Lance Yang
reviewerQuoted 113 lines · David Hildenbrand (Arm)
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))
Quoted 1 line
I tend to like (2), and maybe (3) on top. Opinions?
Cool! (2) + (3) sounds good to me ;)
For VM_MERGEABLE VMAs while KSM is running, makes sense to let KSM handle zero-filled pages itself. Without (2), the split path may remap many zero-filled subpages to the shared zeropage before KSM gets to them ... With (2), those subpages remain normal anon pages for KSM to process later, according to its own settings, such as use_zero_pages, and scan pacing, such as pages_to_scan.
For other VMAs, keeping the opportunistic shared zeropage remap seems useful while split_underused_thp is active. Once the THP is split, the underused shrinker cannot find it anymore :)
And, yes, if split_underused_thp is disabled, generic THP splits should not to do this extra scan/remap work; just leave those zero-filled pages alone, IMHO :D
Cheers, Lance
Usama Arif
reviewerQuoted 113 lines · On Fri, 8 May 2026 23:32:09 +0200 "David Hildenbrand (Arm)" <[email protected]>…
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))
Quoted 2 lines
I tend to like (2), and maybe (3) on top. Opinions?
Hello!
I think (3) definitely makes sense.
I have not had a deep look at KSM up until just now, so might be dumb to say all of below.. :)
What I see is that KSM scans THPs as 512 individual 4K subpages and splits the THP whenever it actually wants to merge a single 4K chunk. That seems like a lot of work for a single 4K?
One thing that came to my mind is to have a separate tree for THPs and only merge the THPs that have the same content, but the possibility of encoutering 2M pages with same content is extremely low? so this is probably a bad idea.
An alternative is, does it even make sense to process and split THPs by KSM in the way it works now? IMO this is a lot of work for a single 4K merge. Shrinker is designed to release memory when its needed, i.e. reclaim, at which point IMO free memory is more important than performance. But KSM runs all the time.. so constantly splitting THPs everytime a single 4K can be merged just hurts performance all the time. If someone cares about memory, they should be running the shrinker. Is a better alternative that KSM skips THPs, THP shrinker splits THPs into 4K subpages when memory is needed, and only then KSM gets those 4K subpages?
Above sounds like reworking KSM, but just wanted to put it out there.
(2) + (3) sounds like a good solution, but I wonder if above alternative of KSM just skipping THPs might be better?
David Hildenbrand (Arm)
maintainerQuoted 6 lines
Quoted 4 lines
I tend to like (2), and maybe (3) on top. Opinions?
Hello!
Hi!
Quoted 9 lines
I think (3) definitely makes sense.
I have not had a deep look at KSM up until just now, so might be dumb to say all of below.. :)
What I see is that KSM scans THPs as 512 individual 4K subpages and splits the THP whenever it actually wants to merge a single 4K chunk. That seems like a lot of work for a single 4K?
Yes, but that's what the users ask for: if there is a chance to deduplicate memory, it shall be deduplicated asap.
Quoted 4 lines
One thing that came to my mind is to have a separate tree for THPs and only merge the THPs that have the same content, but the possibility of encoutering 2M pages with same content is extremely low? so this is probably a bad idea.
Right, the probability is low, and it would change existing semantics, breaking existing users.
In addition, we would have to add large folio support for KSM, which I rather would avoid.
Quoted 7 lines
An alternative is, does it even make sense to process and split THPs by KSM in the way it works now? IMO this is a lot of work for a single 4K merge. Shrinker is designed to release memory when its needed, i.e. reclaim, at which point IMO free memory is more important than performance. But KSM runs all the time.. so constantly splitting THPs everytime a single 4K can be merged just hurts performance all the time.
Right, but that's what you get with KSM: bad performance if there is a chance to deduplicate :)
(and bad performance from scanning overhead)
Quoted 2 lines
If someone cares about memory,
they should be running the shrinker.
It's not just the zero page, but really any page content. The zero page is currently only "special" after we added conditional support to deduplicate to the shared zeropage in KSM. The shrinker doesn't help for any other page content besides zero-filled.
Further, the shrinker is something system-wide, whereby KSM is usually only enabled for selected VMAs (with some exceptions nowadays).
Also note that KSM deduplicates independent of the folio size: not just THPs, but really any (large) folio. Yes, it splits large folios, but that's really just to keep the T in THP.
Quoted 5 lines
Is a better alternative that KSM skips
THPs, THP shrinker splits THPs into 4K subpages when memory is needed, and only then KSM gets those 4K subpages?Above sounds like reworking KSM, but just wanted to put it out there.
Right, and it makes KSM more THP aware. Which is something I would avoid right now.
Quoted 3 lines
(2) + (3) sounds like a good solution, but I wonder if above alternative of KSM just skipping THPs might be better?
That would change the semantics where, for example, where we expect that memory was deduplicated after a KSM run.
VMs (where KSM is usually employed) are expected to be mostly backed by THPs:
except where we can deduplicate memory. Skipping THPs would essentially break the main use case for KSM :)
Does that make sense?
Signature · 3 lines
Cheers, David
Usama Arif
reviewerQuoted 74 lines · David Hildenbrand (Arm)
Quoted 6 lines
Quoted 4 lines
I tend to like (2), and maybe (3) on top. Opinions?
Hello!
Hi!
Quoted 9 lines
I think (3) definitely makes sense.
I have not had a deep look at KSM up until just now, so might be dumb to say all of below.. :)
What I see is that KSM scans THPs as 512 individual 4K subpages and splits the THP whenever it actually wants to merge a single 4K chunk. That seems like a lot of work for a single 4K?
Yes, but that's what the users ask for: if there is a chance to deduplicate memory, it shall be deduplicated asap.
Quoted 4 lines
One thing that came to my mind is to have a separate tree for THPs and only merge the THPs that have the same content, but the possibility of encoutering 2M pages with same content is extremely low? so this is probably a bad idea.
Right, the probability is low, and it would change existing semantics, breaking existing users.
In addition, we would have to add large folio support for KSM, which I rather would avoid.
Quoted 7 lines
An alternative is, does it even make sense to process and split THPs by KSM in the way it works now? IMO this is a lot of work for a single 4K merge. Shrinker is designed to release memory when its needed, i.e. reclaim, at which point IMO free memory is more important than performance. But KSM runs all the time.. so constantly splitting THPs everytime a single 4K can be merged just hurts performance all the time.
Right, but that's what you get with KSM: bad performance if there is a chance to deduplicate :)
(and bad performance from scanning overhead)
Quoted 2 lines
If someone cares about memory,
they should be running the shrinker.It's not just the zero page, but really any page content. The zero page is currently only "special" after we added conditional support to deduplicate to the shared zeropage in KSM. The shrinker doesn't help for any other page content besides zero-filled.
Further, the shrinker is something system-wide, whereby KSM is usually only enabled for selected VMAs (with some exceptions nowadays).
Also note that KSM deduplicates independent of the folio size: not just THPs, but really any (large) folio. Yes, it splits large folios, but that's really just to keep the T in THP.
Quoted 5 lines
Is a better alternative that KSM skips
THPs, THP shrinker splits THPs into 4K subpages when memory is needed, and only then KSM gets those 4K subpages?Above sounds like reworking KSM, but just wanted to put it out there.
Right, and it makes KSM more THP aware. Which is something I would avoid right now.
Quoted 3 lines
(2) + (3) sounds like a good solution, but I wonder if above alternative of KSM just skipping THPs might be better?
Quoted 9 lines
That would change the semantics where, for example, where we expect that memory was deduplicated after a KSM run.
VMs (where KSM is usually employed) are expected to be mostly backed by THPs:
except where we can deduplicate memory. Skipping THPs would essentially break the main use case for KSM :)Does that make sense?
Yes, all of above makes sense. But I feel like this means someone should not set THP policy to always and enable KSM together. In general I feel like KSM is not something that should be run on big servers, as hopefully you are not managing memory as 4K chunks for big machines and using a lot of THPs.
David Hildenbrand (Arm)
maintainerQuoted 81 lines · Usama Arif
Quoted 78 lines · David Hildenbrand (Arm)
Quoted 3 lines
Hello!
Hi!
Quoted 9 lines
I think (3) definitely makes sense.
I have not had a deep look at KSM up until just now, so might be dumb to say all of below.. :)
What I see is that KSM scans THPs as 512 individual 4K subpages and splits the THP whenever it actually wants to merge a single 4K chunk. That seems like a lot of work for a single 4K?
Yes, but that's what the users ask for: if there is a chance to deduplicate memory, it shall be deduplicated asap.
Quoted 4 lines
One thing that came to my mind is to have a separate tree for THPs and only merge the THPs that have the same content, but the possibility of encoutering 2M pages with same content is extremely low? so this is probably a bad idea.
Right, the probability is low, and it would change existing semantics, breaking existing users.
In addition, we would have to add large folio support for KSM, which I rather would avoid.
Quoted 7 lines
An alternative is, does it even make sense to process and split THPs by KSM in the way it works now? IMO this is a lot of work for a single 4K merge. Shrinker is designed to release memory when its needed, i.e. reclaim, at which point IMO free memory is more important than performance. But KSM runs all the time.. so constantly splitting THPs everytime a single 4K can be merged just hurts performance all the time.
Right, but that's what you get with KSM: bad performance if there is a chance to deduplicate :)
(and bad performance from scanning overhead)
Quoted 2 lines
If someone cares about memory,
they should be running the shrinker.It's not just the zero page, but really any page content. The zero page is currently only "special" after we added conditional support to deduplicate to the shared zeropage in KSM. The shrinker doesn't help for any other page content besides zero-filled.
Further, the shrinker is something system-wide, whereby KSM is usually only enabled for selected VMAs (with some exceptions nowadays).
Also note that KSM deduplicates independent of the folio size: not just THPs, but really any (large) folio. Yes, it splits large folios, but that's really just to keep the T in THP.
Quoted 5 lines
Is a better alternative that KSM skips
THPs, THP shrinker splits THPs into 4K subpages when memory is needed, and only then KSM gets those 4K subpages?Above sounds like reworking KSM, but just wanted to put it out there.
Right, and it makes KSM more THP aware. Which is something I would avoid right now.
Quoted 3 lines
(2) + (3) sounds like a good solution, but I wonder if above alternative of KSM just skipping THPs might be better?
That would change the semantics where, for example, where we expect that memory was deduplicated after a KSM run.
VMs (where KSM is usually employed) are expected to be mostly backed by THPs:
except where we can deduplicate memory. Skipping THPs would essentially break the main use case for KSM :)
Quoted 5 lines
Quoted 2 lines
Does that make sense?
Yes, all of above makes sense. But I feel like this means someone should not set THP policy to always and enable KSM together.
IIRC, QEMU will, as default, set MADV_HUGEPAGE and MADV_MERGEABLE :)
(KSM itself later has to be enabled manually on a system level)
Quoted 3 lines
In general I feel like KSM
is not something that should be run on big servers, as hopefully you are not managing memory as 4K chunks for big machines and using a lot of THPs.
Right. But the 4k chunks are movable and compaction can move them around to create THPs elsewhere.
Signature · 3 lines
Cheers, David
David Hildenbrand (Arm)
maintainerQuoted 53 lines · David Hildenbrand (Arm)
Quoted 52 lines · Usama Arif
Quoted 48 lines · David Hildenbrand (Arm)
Hi!
Yes, but that's what the users ask for: if there is a chance to deduplicate memory, it shall be deduplicated asap.
Right, the probability is low, and it would change existing semantics, breaking existing users.
In addition, we would have to add large folio support for KSM, which I rather would avoid.
Right, but that's what you get with KSM: bad performance if there is a chance to deduplicate :)
(and bad performance from scanning overhead)
It's not just the zero page, but really any page content. The zero page is currently only "special" after we added conditional support to deduplicate to the shared zeropage in KSM. The shrinker doesn't help for any other page content besides zero-filled.
Further, the shrinker is something system-wide, whereby KSM is usually only enabled for selected VMAs (with some exceptions nowadays).
Also note that KSM deduplicates independent of the folio size: not just THPs, but really any (large) folio. Yes, it splits large folios, but that's really just to keep the T in THP.
Right, and it makes KSM more THP aware. Which is something I would avoid right now.
That would change the semantics where, for example, where we expect that memory was deduplicated after a KSM run.
VMs (where KSM is usually employed) are expected to be mostly backed by THPs:
except where we can deduplicate memory. Skipping THPs would essentially break the main use case for KSM :)Does that make sense?
Quoted 6 lines
Quoted 2 lines
Yes, all of above makes sense. But I feel like this means someone should not set THP policy to always and enable KSM together.
IIRC, QEMU will, as default, set
MADV_HUGEPAGEandMADV_MERGEABLE:)(KSM itself later has to be enabled manually on a system level)
Digging around, RHEL documents that one might want to consider disabling THPs for performance:
"As KSM can reduce the occurence of transparent huge pages, you may want to disable it before enabling THP." [1]
But that doesn't mean that some people are using that combination.
In the end "some THPs" is better than "no THPs" :)
Signature · 3 lines
Cheers, David
Usama Arif
reviewerQuoted 51 lines · David Hildenbrand (Arm)
Quoted 50 lines · David Hildenbrand (Arm)
Quoted 49 lines · Usama Arif
Quoted 46 lines · David Hildenbrand (Arm)
Hi!
Yes, but that's what the users ask for: if there is a chance to deduplicate memory, it shall be deduplicated asap.
Right, the probability is low, and it would change existing semantics, breaking existing users.
In addition, we would have to add large folio support for KSM, which I rather would avoid.
Right, but that's what you get with KSM: bad performance if there is a chance to deduplicate :)
(and bad performance from scanning overhead)
It's not just the zero page, but really any page content. The zero page is currently only "special" after we added conditional support to deduplicate to the shared zeropage in KSM. The shrinker doesn't help for any other page content besides zero-filled.
Further, the shrinker is something system-wide, whereby KSM is usually only enabled for selected VMAs (with some exceptions nowadays).
Also note that KSM deduplicates independent of the folio size: not just THPs, but really any (large) folio. Yes, it splits large folios, but that's really just to keep the T in THP.
Right, and it makes KSM more THP aware. Which is something I would avoid right now.
That would change the semantics where, for example, where we expect that memory was deduplicated after a KSM run.
VMs (where KSM is usually employed) are expected to be mostly backed by THPs:
except where we can deduplicate memory. Skipping THPs would essentially break the main use case for KSM :)
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Does that make sense?
Yes, all of above makes sense. But I feel like this means someone should not set THP policy to always and enable KSM together.
IIRC, QEMU will, as default, set
MADV_HUGEPAGEandMADV_MERGEABLE:)
That is interesting... :)
Quoted 9 lines
Quoted 2 lines
(KSM itself later has to be enabled manually on a system level)
Digging around, RHEL documents that one might want to consider disabling THPs for performance:
"As KSM can reduce the occurence of transparent huge pages, you may want to disable it before enabling THP." [1]
Quoted 7 lines
But that doesn't mean that some people are using that combination.
In the end "some THPs" is better than "no THPs" :)
Thanks for sharing this and the link, its very useful.
Quoted 51 lines · David Hildenbrand (Arm)
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Quoted 7 lines
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
And what was the expected behavior before this commit? Did we just deal with the wasted memory?
Quoted 56 lines
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))
Quoted 1 line
I tend to like (2), and maybe (3) on top. Opinions?
I don't fully understand (2) but I definitely agree with (3).
Isn't (2) similar to my split_huge_page_no_zeropage() solution in that it only disables the behavior for KSM? but instead handled much further down in the call path? The "fix" commit sold this as a solution ONLY for the underutilized shrinker, but it is not that.
Either way I can test that solution. It seems clean when I gave it a second pass; it just seems to be hiding behavior deeper in the code, but at the proper location (the function that checks whether this action should occur).
Cheers,
-- Nico
Quoted 7 lines
--
Cheers,David
David Hildenbrand (Arm)
maintainerQuoted 55 lines · Nico Pache
Quoted 54 lines · David Hildenbrand (Arm)
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
Quoted 7 lines
Quoted 4 lines
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
And what was the expected behavior before this commit? Did we just deal with the wasted memory?
Before your change, splitting will always free memory, no matter who triggers splitting. So there is no wasted memory (regarding underused THPs).
With your change, if we happen to split before the deferred shrinker runs, we end up with zero-filled pages that waste memory. And reclaiming them (through the deferred shrinker) first requires another re-collapse to a THP.
Or am I misunderstanding your question?
Quoted 57 lines
Quoted 57 lines
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))
Quoted 8 lines
Quoted 1 line
I tend to like (2), and maybe (3) on top. Opinions?
I don't fully understand (2) but I definitely agree with (3).
Isn't (2) similar to my
split_huge_page_no_zeropage()solution in that it only disables the behavior for KSM? but instead handled much further down in the call path? The "fix" commit sold this as a solution ONLY for the underutilized shrinker, but it is not that.
It keeps other paths that happen to split the folio before the deferred shrinker runs unaffected (see above).
Signature · 3 lines
Cheers, David
Quoted 65 lines · David Hildenbrand (Arm)
Quoted 55 lines · Nico Pache
Quoted 54 lines · David Hildenbrand (Arm)
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
Quoted 7 lines
Quoted 4 lines
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
And what was the expected behavior before this commit? Did we just deal with the wasted memory?
Quoted 8 lines
Before your change, splitting will always free memory, no matter who triggers splitting. So there is no wasted memory (regarding underused THPs).
With your change, if we happen to split before the deferred shrinker runs, we end up with zero-filled pages that waste memory. And reclaiming them (through the deferred shrinker) first requires another re-collapse to a THP.
Or am I misunderstanding your question?
I meant before b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp"). Sorry, I should have been more specific.
As in before the underutilized shrinker (and commit b1f202060afe), we had the exact problem you describe above and no way to handle it. Correct?
I guess at reclaim THPs would be split and we would swap out a zero filled page?
Quoted 75 lines
Quoted 58 lines
Quoted 58 lines
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))Quoted 8 lines
Quoted 1 line
I tend to like (2), and maybe (3) on top. Opinions?
I don't fully understand (2) but I definitely agree with (3).
Isn't (2) similar to my
split_huge_page_no_zeropage()solution in that it only disables the behavior for KSM? but instead handled much further down in the call path? The "fix" commit sold this as a solution ONLY for the underutilized shrinker, but it is not that.It keeps other paths that happen to split the folio before the deferred shrinker runs unaffected (see above).
--
Cheers,David
David Hildenbrand (Arm)
maintainerQuoted 17 lines · Nico Pache
Quoted 6 lines · David Hildenbrand (Arm)
Quoted 3 lines · Nico Pache
And what was the expected behavior before this commit? Did we just deal with the wasted memory?
Quoted 8 lines
Before your change, splitting will always free memory, no matter who triggers splitting. So there is no wasted memory (regarding underused THPs).
With your change, if we happen to split before the deferred shrinker runs, we end up with zero-filled pages that waste memory. And reclaiming them (through the deferred shrinker) first requires another re-collapse to a THP.
Or am I misunderstanding your question?
Quoted 7 lines
I meant before
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"). Sorry, I should have been more specific.As in before the underutilized shrinker (and commit
b1f202060afe), we had the exact problem you describe above and no way to handle it. Correct?
Right. That's why the underused shrinker was added, to directly free pages that have been over-allocated with a THP, but are actually never "used" (remain zero).
Quoted 2 lines
I guess at reclaim THPs would be split and we would swap out a zero filled page?
We don't necessarily split during swapout, we try to swap out the THP if supported. But yes, that can happen.
I'm more thinking about other reasons to split a THP (e.g., migration, partial MADV_PAGEOUT/MADV_COLD/MADV_FREE), where the behavior would be changed.
Signature · 3 lines
Cheers, David
Lance Yang
reviewerQuoted 30 lines · David Hildenbrand (Arm)
Quoted 17 lines · Nico Pache
Quoted 6 lines · David Hildenbrand (Arm)
Quoted 3 lines · Nico Pache
And what was the expected behavior before this commit? Did we just deal with the wasted memory?
Quoted 8 lines
Before your change, splitting will always free memory, no matter who triggers splitting. So there is no wasted memory (regarding underused THPs).
With your change, if we happen to split before the deferred shrinker runs, we end up with zero-filled pages that waste memory. And reclaiming them (through the deferred shrinker) first requires another re-collapse to a THP.
Or am I misunderstanding your question?
Quoted 7 lines
I meant before
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"). Sorry, I should have been more specific.As in before the underutilized shrinker (and commit
b1f202060afe), we had the exact problem you describe above and no way to handle it. Correct?Right. That's why the underused shrinker was added, to directly free pages that have been over-allocated with a THP, but are actually never "used" (remain zero).
Quoted 1 line
Quoted 6 lines
Quoted 1 line
I guess at reclaim THPs would be split and we would swap out a zero filled page?
We don't necessarily split during swapout, we try to swap out the THP if supported. But yes, that can happen.
I'm more thinking about other reasons to split a THP (e.g., migration, partial MADV_PAGEOUT/MADV_COLD/MADV_FREE), where the behavior would be changed.
Yeah. Today any successful anon split gets TTU_USE_SHARED_ZEROPAGE. Would it be better to make KSM opt out, so other split paths keep the behavior they have today?
That way we do not have to worry about changing anything else at the same time :D
Cheers, Lance
David Hildenbrand (Arm)
maintainerQuoted 25 lines · Lance Yang
Quoted 16 lines · David Hildenbrand (Arm)
Quoted 10 lines · Nico Pache
On Tue, May 12, 2026 at 1:05 AM David Hildenbrand (Arm) <[email protected]> wrote:
I meant before
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"). Sorry, I should have been more specific.As in before the underutilized shrinker (and commit
b1f202060afe), we had the exact problem you describe above and no way to handle it. Correct?Right. That's why the underused shrinker was added, to directly free pages that have been over-allocated with a THP, but are actually never "used" (remain zero).
Quoted 1 line
Quoted 6 lines
Quoted 1 line
I guess at reclaim THPs would be split and we would swap out a zero filled page?
We don't necessarily split during swapout, we try to swap out the THP if supported. But yes, that can happen.
I'm more thinking about other reasons to split a THP (e.g., migration, partial MADV_PAGEOUT/MADV_COLD/MADV_FREE), where the behavior would be changed.
Quoted 6 lines
Yeah. Today any successful anon split gets
TTU_USE_SHARED_ZEROPAGE. Would it be better to make KSM opt out, so other split paths keep the behavior they have today?That way we do not have to worry about changing anything else at the same time :D
This is (1) here:
https://lore.kernel.org/all/[email protected]/
There will still be interaction between both mechanisms. So I'd prefer that in VM_MERGEABLE sections with KSM enabled, only KSM would take care of deduplicating.
Signature · 3 lines
Cheers, David
Lance Yang
reviewerQuoted 33 lines · David Hildenbrand (Arm)
Quoted 25 lines · Lance Yang
Quoted 16 lines · David Hildenbrand (Arm)
Quoted 10 lines · Nico Pache
On Tue, May 12, 2026 at 1:05 AM David Hildenbrand (Arm) <[email protected]> wrote:
I meant before
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"). Sorry, I should have been more specific.As in before the underutilized shrinker (and commit
b1f202060afe), we had the exact problem you describe above and no way to handle it. Correct?Right. That's why the underused shrinker was added, to directly free pages that have been over-allocated with a THP, but are actually never "used" (remain zero).
Quoted 1 line
Quoted 6 lines
Quoted 1 line
I guess at reclaim THPs would be split and we would swap out a zero filled page?
We don't necessarily split during swapout, we try to swap out the THP if supported. But yes, that can happen.
I'm more thinking about other reasons to split a THP (e.g., migration, partial MADV_PAGEOUT/MADV_COLD/MADV_FREE), where the behavior would be changed.
Quoted 6 lines
Yeah. Today any successful anon split gets
TTU_USE_SHARED_ZEROPAGE. Would it be better to make KSM opt out, so other split paths keep the behavior they have today?That way we do not have to worry about changing anything else at the same time :D
Quoted 6 lines
This is (1) here:
https://lore.kernel.org/all/[email protected]/
There will still be interaction between both mechanisms. So I'd prefer that in
VM_MERGEABLEsections with KSM enabled, only KSM would take care of deduplicating.
Cool. I'd go with that approach as well. It should avoid unexpected interaction between the two :)
That should keep VM_MERGEABLE VMAs under KSM's control, while leaving the other split paths unchanged for now.
Cheers, Lance
Quoted 114 lines · David Hildenbrand (Arm)
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))
Quoted 1 line
I tend to like (2), and maybe (3) on top. Opinions?
Coming back to this.
for (2), I have to export the KSM run state, which may be fine, but it introduces a race window. If a user disables KSM, a split occurs, and then re-enables it, the bug will present itself again.
Would it be better to just check if its VM_MERGEABLE?
(3) makes sense to me; I'll add that before sending the V1 :)
-- Nico
Quoted 7 lines
--
Cheers,David
David Hildenbrand (Arm)
maintainerQuoted 115 lines · Nico Pache
Quoted 114 lines · David Hildenbrand (Arm)
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))
Quoted 5 lines
Quoted 1 line
I tend to like (2), and maybe (3) on top. Opinions?
Coming back to this.
for (2), I have to export the KSM run state, which may be fine, but it
Right, you just need a simple helper.
Quoted 2 lines
introduces a race window. If a user disables KSM, a split occurs, and then re-enables it, the bug will present itself again.
Who cares?
Quoted 2 lines
Would it be better to just check if its
VM_MERGEABLE?
Some user space unconditionally sets VM_MERGEABLE, even if KSM is never enabled (IIRC QEMU, for example).
Signature · 3 lines
Cheers, David
Quoted 117 lines · David Hildenbrand (Arm)
Quoted 115 lines · Nico Pache
Quoted 114 lines · David Hildenbrand (Arm)
Quoted 5 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.And even when the underused scanner is effectively disabled on a system. Hm.
I don't quite like that we scan for zeropages when nobody even requested us to split because of zeropages.
I can see why we would want to scan for zeropages in a setup where the underused scanner is active, even when the split was triggered by someone/something else (below).
[...]
Quoted 17 lines
/**
(quoted diff) +7
-1@@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, - SPLIT_TYPE_NON_UNIFORM); + SPLIT_TYPE_NON_UNIFORM, false); +} + +int folio_split_underused(struct folio *folio) +{ + return __folio_split(folio, 0, &folio->page, &folio->page, + NULL, SPLIT_TYPE_NON_UNIFORM, true); } /**Review static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4572
Quoted 8 lines
(quoted diff) +1
-1@@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, } if (!folio_trylock(folio)) goto requeue; - if (!split_folio(folio)) { + if (!folio_split_underused(folio)) { did_split = true; if (underused) count_vm_event(THP_UNDERUSED_SPLIT_PAGE);In general, this looks clean.
But imagine the following: someone splits the THP for another reason: for example, because migration is unable to allocate a 2M THP, or because we have to split on swapout etc.
Not freeing the zero-filled pages means that these pages cannot be reclaimed anymore easily. We split a possibly underused THP but didn't free the memory.
The only way to free the memory would be to wait for another collapse, and then have the new THP be detected as underused.
Hm.
(1) As you say, the alternative is to let KSM say that it wants to handle the zero-filled pages itself. I'm not a the biggest fan of that approach. We still have two mechanisms interacting to some degree.
(2) Another approach is to just let KSM handle this in VMAs that are marked as mergable while KSM is active. That is, we check for
VM_MERGABLEandksm_run==KSM_RUN_MERGEintry_to_map_unused_to_zeropage()to just let KSM do its thing.That really just stops both mechanisms from interacting.
(3) Yet another approach I could think of (in general) is to disable the underused handling in a system where the underused splitting is entirely disabled.
mm/huge_memory.c +10
-1@@ -82,6 +82,14 @@ unsigned long huge_anon_orders_madvise __read_mostly;unsigned long huge_anon_orders_inherit__read_mostly ;static bool anon_orders_configured__initdata ; +static bool thp_underused_split_active (void ) +{ +if (!split_underused_thp) +return false; + +return khugepaged_max_ptes_none !=HPAGE_PMD_NR -1 ; +} +static inline bool file_thp_enabled (struct vm_area_struct *vma) {struct inode *inode;@@ -4188,7 +4196,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && +thp_underused_split_active ()) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page(folio, 1 << old_order, ttu_flags);mm/huge_memory.c +1
-1@@ -4497,7 +4506,7 @@ static bool thp_underused(struct folio *folio)int num_zero_pages =0 , num_filled_pages =0 ;int i; -if (khugepaged_max_ptes_none ==HPAGE_PMD_NR -1 ) +if (!thp_underused_split_active ())return false;if (folio_contain_hwpoisoned_page (folio))
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I tend to like (2), and maybe (3) on top. Opinions?
Coming back to this.
for (2), I have to export the KSM run state, which may be fine, but it
Right, you just need a simple helper.
ack. Ill add that peice back! thanks
Quoted 5 lines
Quoted 3 lines
introduces a race window. If a user disables KSM, a split occurs, and then re-enables it, the bug will present itself again.
Who cares?
Haha, okay, fair enough-- was just making sure before I resent.
Quoted 6 lines
Quoted 3 lines
Would it be better to just check if its
VM_MERGEABLE?Some user space unconditionally sets
VM_MERGEABLE, even if KSM is never enabled (IIRC QEMU, for example).
ack! I'll keep the is_ksm_running() check :)
Cheers,
-- Nico
Quoted 6 lines
--
Cheers,David
Lance Yang
reviewerQuoted 89 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.When KSM is enabled with THP=always, this causes two regressions:
1. use_zero_pages=1: KSM calls
try_to_merge_one_page()which triggerssplit_huge_page(). The split remaps all 512 zero-filled subpages to the shared zeropage at once, freeing the entire 2MB THP when KSM only intended to process a single 4KB page. This bypasses KSM'spages_to_scanrate limiting, causing ~1GB to be freed almost instantly.2. use_zero_pages=0: The same split side-effect occurs through the stable/unstable tree merge paths. Each
pages_to_scaniteration triggers an expensivesplit_huge_page()that silently frees 2MB, while the scanner wastes cycles on tree searches for zero-filled pages that were already freed as a side-effect.Fix this by restricting
TTU_USE_SHARED_ZEROPAGEto only the deferred split shrinker path (deferred_split_scan), which is the only caller that intentionally splits underused THPs to reclaim zero-filled subpages. Introducefolio_split_underused()as a dedicated entry point that passes is_underused_thp=true through__folio_split(), and use it fromdeferred_split_scan(). All other split callers (KSM, compaction, etc.) no longer get the zero-page remapping side-effect.Reviewers notes: this patch is one of two potential approaches. This patch turns off the zero-page freeing that has been done since the noted commit, in all the other callers, only leaving the underused shrinker to do such behavior. We can also take the opposite approach of with something like
split_huge_page_no_zeropage()and call this within KSM.Fixes b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp")Signed-off-by Nico Pache <[email protected]>---
include/linux/huge_mm.h| 2 +-mm/huge_memory.c| 17 ++++++++++++-----2 files changed, 13 insertions(+), 6 deletions(-)include/linux/huge_mm.h +1
-1@@ -378,7 +378,7 @@ int folio_check_splittable(struct folio *folio, unsigned int new_order, enum split_type split_type);int folio_split (struct folio *folio,unsigned int new_order,struct page *page,struct list_head *list); - +int folio_split_underused (struct folio *folio);static inline int split_huge_page_to_list_to_order (struct page *page,struct list_head *list,unsigned int new_order) {mm/huge_memory.c +4
-3@@ -4045,7 +4045,8 @@ static int __folio_freeze_and_split_unmapped(struct folio *folio, unsigned int n*/ static int __folio_split (struct folio *folio,unsigned int new_order,struct page *split_at,struct page *lock_at, -struct list_head *list,enum split_type split_type) +struct list_head *list,enum split_type split_type, +bool is_underused_thp) {XA_STATE (xas, &folio->mapping->i_pages, folio->index);struct folio *end_folio =folio_next (folio);@@ -4174,7 +4175,7 @@ static int __folio_split(struct folio *folio, unsigned int new_order,if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && is_underused_thp) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page (folio,1 << old_order, ttu_flags);@@ -4309,7 +4310,7 @@ int __split_huge_page_to_list_to_order(struct page *page, struct list_head *liststruct folio *folio =page_folio (page);return __folio_split (folio, new_order, &folio->page, page, list, -SPLIT_TYPE_UNIFORM ); +SPLIT_TYPE_UNIFORM , false); }/** @@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list)
mm/huge_memory.c int folio_split(struct folio *folio, unsigned int new_order, · line 4350
Quoted 10 lines
mm/huge_memory.c +7
-1{return __folio_split (folio, new_order, split_at, &folio->page, list, -SPLIT_TYPE_NON_UNIFORM ); +SPLIT_TYPE_NON_UNIFORM , false); +} + +int folio_split_underused (struct folio *folio) +{ +return __folio_split (folio,0 , &folio->page, &folio->page, +NULL ,SPLIT_TYPE_NON_UNIFORM , true);
IIUC, it should be SPLIT_TYPE_UNIFORM, not SPLIT_TYPE_NON_UNIFORM ...
deferred_split_scan() used split_folio(), so for the underused case it split the whole THP uniformly down to order-0 pages. The shared zeropage remapping happens later, via remove_migration_ptes(), after the split.
With SPLIT_TYPE_NON_UNIFORM and split_at == &folio->page, most of an order-9 THP can stays as larger folios.
Then try_to_map_unused_to_zeropage() rejects those folios:
if (PageCompound(page) || PageHWPoison(page)) return false;
So the underused shrinker would no longer remap/free many zero-filled subpages ...
Quoted 16 lines
mm/huge_memory.c +1
-1}/** @@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, }if (!folio_trylock (folio)) goto requeue; -if (!split_folio (folio)) { +if (!folio_split_underused (folio)) { did_split = true;if (underused)count_vm_event (THP_UNDERUSED_SPLIT_PAGE );Signature · 1 line
2.54.0
Quoted 89 lines · Lance Yang
Quoted 86 lines · Nico Pache
Since commit
b1f202060afe("mm: remap unused subpages to shared zeropage when splitting isolated thp"), splitting an anonymous THP remaps all zero-filled subpages to the shared zeropage viaTTU_USE_SHARED_ZEROPAGE. This flag is set unconditionally for every anonymous folio split, including splits triggered by KSM.When KSM is enabled with THP=always, this causes two regressions:
1. use_zero_pages=1: KSM calls
try_to_merge_one_page()which triggerssplit_huge_page(). The split remaps all 512 zero-filled subpages to the shared zeropage at once, freeing the entire 2MB THP when KSM only intended to process a single 4KB page. This bypasses KSM'spages_to_scanrate limiting, causing ~1GB to be freed almost instantly.2. use_zero_pages=0: The same split side-effect occurs through the stable/unstable tree merge paths. Each
pages_to_scaniteration triggers an expensivesplit_huge_page()that silently frees 2MB, while the scanner wastes cycles on tree searches for zero-filled pages that were already freed as a side-effect.Fix this by restricting
TTU_USE_SHARED_ZEROPAGEto only the deferred split shrinker path (deferred_split_scan), which is the only caller that intentionally splits underused THPs to reclaim zero-filled subpages. Introducefolio_split_underused()as a dedicated entry point that passes is_underused_thp=true through__folio_split(), and use it fromdeferred_split_scan(). All other split callers (KSM, compaction, etc.) no longer get the zero-page remapping side-effect.Reviewers notes: this patch is one of two potential approaches. This patch turns off the zero-page freeing that has been done since the noted commit, in all the other callers, only leaving the underused shrinker to do such behavior. We can also take the opposite approach of with something like
split_huge_page_no_zeropage()and call this within KSM.Fixes b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp")Signed-off-by Nico Pache <[email protected]>---
include/linux/huge_mm.h| 2 +-mm/huge_memory.c| 17 ++++++++++++-----2 files changed, 13 insertions(+), 6 deletions(-)include/linux/huge_mm.h +1
-1@@ -378,7 +378,7 @@ int folio_check_splittable(struct folio *folio, unsigned int new_order, enum split_type split_type);int folio_split (struct folio *folio,unsigned int new_order,struct page *page,struct list_head *list); - +int folio_split_underused (struct folio *folio);static inline int split_huge_page_to_list_to_order (struct page *page,struct list_head *list,unsigned int new_order) {mm/huge_memory.c +4
-3@@ -4045,7 +4045,8 @@ static int __folio_freeze_and_split_unmapped(struct folio *folio, unsigned int n*/ static int __folio_split (struct folio *folio,unsigned int new_order,struct page *split_at,struct page *lock_at, -struct list_head *list,enum split_type split_type) +struct list_head *list,enum split_type split_type, +bool is_underused_thp) {XA_STATE (xas, &folio->mapping->i_pages, folio->index);struct folio *end_folio =folio_next (folio);@@ -4174,7 +4175,7 @@ static int __folio_split(struct folio *folio, unsigned int new_order,if (nr_shmem_dropped)shmem_uncharge (mapping->host, nr_shmem_dropped); -if (!ret && is_anon && !folio_is_device_private (folio)) +if (!ret && is_anon && !folio_is_device_private (folio) && is_underused_thp) ttu_flags =TTU_USE_SHARED_ZEROPAGE ;remap_page (folio,1 << old_order, ttu_flags);@@ -4309,7 +4310,7 @@ int __split_huge_page_to_list_to_order(struct page *page, struct list_head *liststruct folio *folio =page_folio (page);return __folio_split (folio, new_order, &folio->page, page, list, -SPLIT_TYPE_UNIFORM ); +SPLIT_TYPE_UNIFORM , false); }
Quoted 15 lines
mm/huge_memory.c int folio_split(struct folio *folio, unsigned int new_order, · line 4350
Quoted 13 lines
mm/huge_memory.c +7
-1/** @@ -4340,7 +4341,13 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *split_at, struct list_head *list) { return __folio_split(folio, new_order, split_at, &folio->page, list, -SPLIT_TYPE_NON_UNIFORM ); +SPLIT_TYPE_NON_UNIFORM ,false ); +} + +int folio_split_underused (struct folio *folio) +{ +return __folio_split (folio,0 , &folio->page, &folio->page, +NULL ,SPLIT_TYPE_NON_UNIFORM , true);IIUC, it should be
SPLIT_TYPE_UNIFORM, notSPLIT_TYPE_NON_UNIFORM...
Thats what i had originally then convinced myself otherwise. Ill reverify before submitting again. Unless we just end up on a different solution like David suggested.
Thanks!
-- Nico
Quoted 33 lines
deferred_split_scan()usedsplit_folio(), so for the underused case it split the whole THP uniformly down to order-0 pages. The shared zeropage remapping happens later, viaremove_migration_ptes(), after the split.With
SPLIT_TYPE_NON_UNIFORMandsplit_at==&folio->page, most of an order-9 THP can stays as larger folios.Then
try_to_map_unused_to_zeropage()rejects those folios:if (PageCompound(page) || PageHWPoison(page)) return false;So the underused shrinker would no longer remap/free many zero-filled subpages ...
Quoted 16 lines
mm/huge_memory.c +1
-2}/** @@ -4559,7 +4566,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, }if (!folio_trylock (folio)) goto requeue; -if (!split_folio (folio)) { +if (!folio_split_underused (folio)) { did_split = true;if (underused)count_vm_event (THP_UNDERUSED_SPLIT_PAGE ); --2.54.0