mm: migrate: transfer large_rmappable flag in folio_migrate_flags()
Subsystems (2)
PATCH 1/1 mm: migrate: transfer large_rmappable flag in folio_migrate_flags()
folio_migrate_flags() transfers folio state from source to destination during migration, but does not transfer the large_rmappable flag.
Migration allocators like alloc_migration_target() and alloc_misplaced_dst_folio() use __folio_alloc() directly without wrapping the result in page_rmappable_folio(), so the destination folio never gets large_rmappable set.
This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue via deferred_split_folio() (which does not check large_rmappable), but when the folio is later freed, folio_unqueue_deferred_split() bails out early because large_rmappable is not set:
if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio))
return false;This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.
Fix this by transferring large_rmappable in folio_migrate_flags(), consistent with how all other folio flags are handled.
--- mm/migrate.c | 3 +++ 1 file changed, 3 insertions(+)
mm/migrate.c +3 -0
@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)folio_copy_owner (newfolio, folio);pgalloc_tag_swap (newfolio, folio); +if (folio_test_large_rmappable (folio)) +folio_set_large_rmappable (newfolio); +mem_cgroup_migrate (folio, newfolio); }EXPORT_SYMBOL (folio_migrate_flags);
Signature · 1 line
2.52.0
David Hildenbrand (Arm)
maintainerQuoted 34 lines · Usama Arif
folio_migrate_flags()transfers folio state from source to destination during migration, but does not transfer thelarge_rmappableflag.Migration allocators like
alloc_migration_target()andalloc_misplaced_dst_folio()use__folio_alloc()directly without wrapping the result inpage_rmappable_folio(), so the destination folio never getslarge_rmappableset.This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue viadeferred_split_folio()(which does not checklarge_rmappable), but when the folio is later freed,folio_unqueue_deferred_split()bails out early becauselarge_rmappableis not set:if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio)) return false;This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.
Fix this by transferring
large_rmappableinfolio_migrate_flags(), consistent with how all other folio flags are handled.Fixes dafff3f4c850 ("mm: split underused THPs")Signed-off-by Usama Arif <[email protected]>--- mm/migrate.c | 3 +++ 1 file changed, 3 insertions(+)mm/migrate.c +0
-0
mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854
Quoted 9 lines
mm/migrate.c +3
-0@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)folio_copy_owner (newfolio, folio);pgalloc_tag_swap (newfolio, folio); +if (folio_test_large_rmappable (folio)) +folio_set_large_rmappable (newfolio); +mem_cgroup_migrate (folio, newfolio); }EXPORT_SYMBOL (folio_migrate_flags);
compaction_alloc_noprof() does the page_rmappable_folio() at the end.
I'd assume that all migration allocation functions must take care of that.
It's a responsibility of the folio allocation code, not folio migration code.
Signature · 3 lines
Cheers, David
Zi Yan
reviewerQuoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif
folio_migrate_flags()transfers folio state from source to destination during migration, but does not transfer thelarge_rmappableflag.Migration allocators like
alloc_migration_target()andalloc_misplaced_dst_folio()use__folio_alloc()directly without wrapping the result inpage_rmappable_folio(), so the destination folio never getslarge_rmappableset.This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue viadeferred_split_folio()(which does not checklarge_rmappable), but when the folio is later freed,folio_unqueue_deferred_split()bails out early becauselarge_rmappableis not set:if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio)) return false;This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.
Fix this by transferring
large_rmappableinfolio_migrate_flags(), consistent with how all other folio flags are handled.Fixes dafff3f4c850 ("mm: split underused THPs")Signed-off-by Usama Arif <[email protected]>--- mm/migrate.c | 3 +++ 1 file changed, 3 insertions(+)mm/migrate.c +0
-0mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854
Quoted 9 lines
mm/migrate.c +3
-0@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)folio_copy_owner (newfolio, folio);pgalloc_tag_swap (newfolio, folio); +if (folio_test_large_rmappable (folio)) +folio_set_large_rmappable (newfolio); +mem_cgroup_migrate (folio, newfolio); }EXPORT_SYMBOL (folio_migrate_flags);
compaction_alloc_noprof()does thepage_rmappable_folio()at the end.
Quoted 4 lines
I'd assume that all migration allocation functions must take care of that.
It's a responsibility of the folio allocation code, not folio migration code.
I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's large_rmappable state?
--
Best Regards,
Yan, Zi
David Hildenbrand (Arm)
maintainerQuoted 49 lines · Zi Yan
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif
folio_migrate_flags()transfers folio state from source to destination during migration, but does not transfer thelarge_rmappableflag.Migration allocators like
alloc_migration_target()andalloc_misplaced_dst_folio()use__folio_alloc()directly without wrapping the result inpage_rmappable_folio(), so the destination folio never getslarge_rmappableset.This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue viadeferred_split_folio()(which does not checklarge_rmappable), but when the folio is later freed,folio_unqueue_deferred_split()bails out early becauselarge_rmappableis not set:if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio)) return false;This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.
Fix this by transferring
large_rmappableinfolio_migrate_flags(), consistent with how all other folio flags are handled.Fixes dafff3f4c850 ("mm: split underused THPs")Signed-off-by Usama Arif <[email protected]>--- mm/migrate.c | 3 +++ 1 file changed, 3 insertions(+)mm/migrate.c +0
-0mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854
Quoted 9 lines
mm/migrate.c +3
-0@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)folio_copy_owner (newfolio, folio);pgalloc_tag_swap (newfolio, folio); +if (folio_test_large_rmappable (folio)) +folio_set_large_rmappable (newfolio); +mem_cgroup_migrate (folio, newfolio); }EXPORT_SYMBOL (folio_migrate_flags);
compaction_alloc_noprof()does thepage_rmappable_folio()at the end.
Quoted 8 lines
Quoted 4 lines
I'd assume that all migration allocation functions must take care of that.
It's a responsibility of the folio allocation code, not folio migration code.
I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's
large_rmappablestate?
VM_WARN, sounds reasonable.
Signature · 3 lines
Cheers, David
Quoted 49 lines · Zi Yan
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif
folio_migrate_flags()transfers folio state from source to destination during migration, but does not transfer thelarge_rmappableflag.Migration allocators like
alloc_migration_target()andalloc_misplaced_dst_folio()use__folio_alloc()directly without wrapping the result inpage_rmappable_folio(), so the destination folio never getslarge_rmappableset.This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue viadeferred_split_folio()(which does not checklarge_rmappable), but when the folio is later freed,folio_unqueue_deferred_split()bails out early becauselarge_rmappableis not set:if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio)) return false;This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.
Fix this by transferring
large_rmappableinfolio_migrate_flags(), consistent with how all other folio flags are handled.Fixes dafff3f4c850 ("mm: split underused THPs")Signed-off-by Usama Arif <[email protected]>--- mm/migrate.c | 3 +++ 1 file changed, 3 insertions(+)mm/migrate.c +0
-0mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854
Quoted 9 lines
mm/migrate.c +3
-0@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)folio_copy_owner (newfolio, folio);pgalloc_tag_swap (newfolio, folio); +if (folio_test_large_rmappable (folio)) +folio_set_large_rmappable (newfolio); +mem_cgroup_migrate (folio, newfolio); }EXPORT_SYMBOL (folio_migrate_flags);
compaction_alloc_noprof()does thepage_rmappable_folio()at the end.
Quoted 9 lines
Quoted 4 lines
I'd assume that all migration allocation functions must take care of that.
It's a responsibility of the folio allocation code, not folio migration code.
I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's
large_rmappablestate?
hmm its done in compaction_alloc() and alloc_migration_target_by_mpol() but I dont see this being done in alloc_migration_target() and alloc_misplaced_dst_folio(). I think alloc_misplaced_dst_folio() is used by NUMA balancing and alloc_migration_target() by hotplug and memory failure?
I am not very familiar with migration code, so maybe I am missing where its being done in these paths?
Quoted 62 lines · Usama Arif
Quoted 49 lines · Zi Yan
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif
folio_migrate_flags()transfers folio state from source to destination during migration, but does not transfer thelarge_rmappableflag.Migration allocators like
alloc_migration_target()andalloc_misplaced_dst_folio()use__folio_alloc()directly without wrapping the result inpage_rmappable_folio(), so the destination folio never getslarge_rmappableset.This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue viadeferred_split_folio()(which does not checklarge_rmappable), but when the folio is later freed,folio_unqueue_deferred_split()bails out early becauselarge_rmappableis not set:if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio)) return false;This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.
Fix this by transferring
large_rmappableinfolio_migrate_flags(), consistent with how all other folio flags are handled.Fixes dafff3f4c850 ("mm: split underused THPs")Signed-off-by Usama Arif <[email protected]>--- mm/migrate.c | 3 +++ 1 file changed, 3 insertions(+)mm/migrate.c +0
-0mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854
Quoted 9 lines
mm/migrate.c +3
-0@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)folio_copy_owner (newfolio, folio);pgalloc_tag_swap (newfolio, folio); +if (folio_test_large_rmappable (folio)) +folio_set_large_rmappable (newfolio); +mem_cgroup_migrate (folio, newfolio); }EXPORT_SYMBOL (folio_migrate_flags);
compaction_alloc_noprof()does thepage_rmappable_folio()at the end.Quoted 9 lines
Quoted 4 lines
I'd assume that all migration allocation functions must take care of that.
It's a responsibility of the folio allocation code, not folio migration code.
I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's
large_rmappablestate?
Quoted 7 lines
hmm its done in
compaction_alloc()andalloc_migration_target_by_mpol()but I dont see this being done inalloc_migration_target()andalloc_misplaced_dst_folio(). I thinkalloc_misplaced_dst_folio()is used by NUMA balancing andalloc_migration_target()by hotplug and memory failure?I am not very familiar with migration code, so maybe I am missing where its being done in these paths?
ah no ignore me :) __folio_alloc_noprof should set it. Thanks and sorry for the noise!
David Hildenbrand (Arm)
maintainerQuoted 13 lines · Usama Arif
Quoted 10 lines · Zi Yan
Quoted 7 lines · David Hildenbrand (Arm)
compaction_alloc_noprof()does thepage_rmappable_folio()at the end.I'd assume that all migration allocation functions must take care of that.
It's a responsibility of the folio allocation code, not folio migration code.
Quoted 7 lines
Quoted 4 lines
I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's
large_rmappablestate?hmm its done in
compaction_alloc()andalloc_migration_target_by_mpol()but I dont see this being done inalloc_migration_target()andalloc_misplaced_dst_folio().
__folio_alloc_node() and friends should be calling page_rmappable_folio(). See __folio_alloc_noprof().
Quoted 5 lines
I think
alloc_misplaced_dst_folio()is used by NUMA balancing andalloc_migration_target()by hotplug and memory failure?I am not very familiar with migration code, so maybe I am missing where its being done in these paths?
Please double check. If a call is missing, we should fix it there.
Signature · 3 lines
Cheers, David