Skip to contentSkip to conversation

mm: migrate: transfer large_rmappable flag in folio_migrate_flags()

PATCH In review Usama Arif 3 participants 7 messages Mar 11

Subsystems (2)
lore
Share & tools
lore

Usama Arif

author
#0 lore raw

PATCH 1/1 mm: migrate: transfer large_rmappable flag in folio_migrate_flags()

folio_migrate_flags() transfers folio state from source to destination during migration, but does not transfer the large_rmappable flag.

Migration allocators like alloc_migration_target() and alloc_misplaced_dst_folio() use __folio_alloc() directly without wrapping the result in page_rmappable_folio(), so the destination folio never gets large_rmappable set.

This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue via deferred_split_folio() (which does not check large_rmappable), but when the folio is later freed, folio_unqueue_deferred_split() bails out early because large_rmappable is not set:

  if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio))
      return false;

This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.

Fix this by transferring large_rmappable in folio_migrate_flags(), consistent with how all other folio flags are handled.

Fixes dafff3f4c850 ("mm: split underused THPs")
---
 mm/migrate.c | 3 +++
 1 file changed, 3 insertions(+)
mm/migrate.c +3 -0
index 3380021fd3db..ee1c7bc851dd 100644
@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)
 	folio_copy_owner(newfolio, folio);
 	pgalloc_tag_swap(newfolio, folio);
 
+	if (folio_test_large_rmappable(folio))
+		folio_set_large_rmappable(newfolio);
+
 	mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(folio_migrate_flags);
Signature · 1 line
2.52.0
#1 lore raw
Quoted 34 lines · Usama Arif

folio_migrate_flags() transfers folio state from source to destination during migration, but does not transfer the large_rmappable flag.

Migration allocators like alloc_migration_target() and alloc_misplaced_dst_folio() use __folio_alloc() directly without wrapping the result in page_rmappable_folio(), so the destination folio never gets large_rmappable set.

This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue via deferred_split_folio() (which does not check large_rmappable), but when the folio is later freed, folio_unqueue_deferred_split() bails out early because large_rmappable is not set:

  if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio))
      return false;

This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.

Fix this by transferring large_rmappable in folio_migrate_flags(), consistent with how all other folio flags are handled.

Fixes dafff3f4c850 ("mm: split underused THPs")
---
 mm/migrate.c | 3 +++
 1 file changed, 3 insertions(+)
mm/migrate.c +0 -0
index 3380021fd3db..ee1c7bc851dd 100644

mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854

Quoted 9 lines
mm/migrate.c +3 -0
@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)
 	folio_copy_owner(newfolio, folio);
 	pgalloc_tag_swap(newfolio, folio);
 
+	if (folio_test_large_rmappable(folio))
+		folio_set_large_rmappable(newfolio);
+
 	mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(folio_migrate_flags);

compaction_alloc_noprof() does the page_rmappable_folio() at the end.

I'd assume that all migration allocation functions must take care of that.

It's a responsibility of the folio allocation code, not folio migration code.

Signature · 3 lines
Cheers,

David
reviewer
#2 lore raw
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif

folio_migrate_flags() transfers folio state from source to destination during migration, but does not transfer the large_rmappable flag.

Migration allocators like alloc_migration_target() and alloc_misplaced_dst_folio() use __folio_alloc() directly without wrapping the result in page_rmappable_folio(), so the destination folio never gets large_rmappable set.

This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue via deferred_split_folio() (which does not check large_rmappable), but when the folio is later freed, folio_unqueue_deferred_split() bails out early because large_rmappable is not set:

  if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio))
      return false;

This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.

Fix this by transferring large_rmappable in folio_migrate_flags(), consistent with how all other folio flags are handled.

Fixes dafff3f4c850 ("mm: split underused THPs")
---
 mm/migrate.c | 3 +++
 1 file changed, 3 insertions(+)
mm/migrate.c +0 -0
index 3380021fd3db..ee1c7bc851dd 100644

mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854

Quoted 9 lines
mm/migrate.c +3 -0
@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)
 	folio_copy_owner(newfolio, folio);
 	pgalloc_tag_swap(newfolio, folio);
 
+	if (folio_test_large_rmappable(folio))
+		folio_set_large_rmappable(newfolio);
+
 	mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(folio_migrate_flags);

compaction_alloc_noprof() does the page_rmappable_folio() at the end.

Quoted 4 lines

I'd assume that all migration allocation functions must take care of that.

It's a responsibility of the folio allocation code, not folio migration code.

I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's large_rmappable state?

--
Best Regards,
Yan, Zi

#3 lore raw
Quoted 49 lines · Zi Yan
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif

folio_migrate_flags() transfers folio state from source to destination during migration, but does not transfer the large_rmappable flag.

Migration allocators like alloc_migration_target() and alloc_misplaced_dst_folio() use __folio_alloc() directly without wrapping the result in page_rmappable_folio(), so the destination folio never gets large_rmappable set.

This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue via deferred_split_folio() (which does not check large_rmappable), but when the folio is later freed, folio_unqueue_deferred_split() bails out early because large_rmappable is not set:

  if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio))
      return false;

This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.

Fix this by transferring large_rmappable in folio_migrate_flags(), consistent with how all other folio flags are handled.

Fixes dafff3f4c850 ("mm: split underused THPs")
---
 mm/migrate.c | 3 +++
 1 file changed, 3 insertions(+)
mm/migrate.c +0 -0
index 3380021fd3db..ee1c7bc851dd 100644

mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854

Quoted 9 lines
mm/migrate.c +3 -0
@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)
 	folio_copy_owner(newfolio, folio);
 	pgalloc_tag_swap(newfolio, folio);
 
+	if (folio_test_large_rmappable(folio))
+		folio_set_large_rmappable(newfolio);
+
 	mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(folio_migrate_flags);

compaction_alloc_noprof() does the page_rmappable_folio() at the end.

Quoted 8 lines
Quoted 4 lines

I'd assume that all migration allocation functions must take care of that.

It's a responsibility of the folio allocation code, not folio migration code.

I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's large_rmappable state?

VM_WARN, sounds reasonable.

Signature · 3 lines
Cheers,

David
author
#4 lore raw
In reply to Zi Yan
Quoted 49 lines · Zi Yan
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif

folio_migrate_flags() transfers folio state from source to destination during migration, but does not transfer the large_rmappable flag.

Migration allocators like alloc_migration_target() and alloc_misplaced_dst_folio() use __folio_alloc() directly without wrapping the result in page_rmappable_folio(), so the destination folio never gets large_rmappable set.

This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue via deferred_split_folio() (which does not check large_rmappable), but when the folio is later freed, folio_unqueue_deferred_split() bails out early because large_rmappable is not set:

  if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio))
      return false;

This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.

Fix this by transferring large_rmappable in folio_migrate_flags(), consistent with how all other folio flags are handled.

Fixes dafff3f4c850 ("mm: split underused THPs")
---
 mm/migrate.c | 3 +++
 1 file changed, 3 insertions(+)
mm/migrate.c +0 -0
index 3380021fd3db..ee1c7bc851dd 100644

mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854

Quoted 9 lines
mm/migrate.c +3 -0
@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)
 	folio_copy_owner(newfolio, folio);
 	pgalloc_tag_swap(newfolio, folio);
 
+	if (folio_test_large_rmappable(folio))
+		folio_set_large_rmappable(newfolio);
+
 	mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(folio_migrate_flags);

compaction_alloc_noprof() does the page_rmappable_folio() at the end.

Quoted 9 lines
Quoted 4 lines

I'd assume that all migration allocation functions must take care of that.

It's a responsibility of the folio allocation code, not folio migration code.

I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's large_rmappable state?

hmm its done in compaction_alloc() and alloc_migration_target_by_mpol() but I dont see this being done in alloc_migration_target() and alloc_misplaced_dst_folio(). I think alloc_misplaced_dst_folio() is used by NUMA balancing and alloc_migration_target() by hotplug and memory failure?

I am not very familiar with migration code, so maybe I am missing where its being done in these paths?

author
#5 lore raw
Quoted 62 lines · Usama Arif
Quoted 49 lines · Zi Yan
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 34 lines · Usama Arif

folio_migrate_flags() transfers folio state from source to destination during migration, but does not transfer the large_rmappable flag.

Migration allocators like alloc_migration_target() and alloc_misplaced_dst_folio() use __folio_alloc() directly without wrapping the result in page_rmappable_folio(), so the destination folio never gets large_rmappable set.

This becomes a problem when a folio on the deferred split queue is
migrated: the destination folio can be added to the deferred split queue via deferred_split_folio() (which does not check large_rmappable), but when the folio is later freed, folio_unqueue_deferred_split() bails out early because large_rmappable is not set:

  if (folio_order(folio) <= 1 || !folio_test_large_rmappable(folio))
      return false;

This leaves a stale entry on the deferred split queue, leading to use-after-free when the shrinker walks the list.

Fix this by transferring large_rmappable in folio_migrate_flags(), consistent with how all other folio flags are handled.

Fixes dafff3f4c850 ("mm: split underused THPs")
---
 mm/migrate.c | 3 +++
 1 file changed, 3 insertions(+)
mm/migrate.c +0 -0
index 3380021fd3db..ee1c7bc851dd 100644

mm/migrate.c void folio_migrate_flags(struct folio *newfolio, struct folio *folio) · line 854

Quoted 9 lines
mm/migrate.c +3 -0
@@ -846,6 +846,9 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio)
 	folio_copy_owner(newfolio, folio);
 	pgalloc_tag_swap(newfolio, folio);
 
+	if (folio_test_large_rmappable(folio))
+		folio_set_large_rmappable(newfolio);
+
 	mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(folio_migrate_flags);

compaction_alloc_noprof() does the page_rmappable_folio() at the end.

Quoted 9 lines
Quoted 4 lines

I'd assume that all migration allocation functions must take care of that.

It's a responsibility of the folio allocation code, not folio migration code.

I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's large_rmappable state?

Quoted 7 lines

hmm its done in compaction_alloc() and alloc_migration_target_by_mpol() but I dont see this being done in alloc_migration_target() and alloc_misplaced_dst_folio(). I think alloc_misplaced_dst_folio() is used by NUMA balancing and alloc_migration_target() by hotplug and memory failure?

I am not very familiar with migration code, so maybe I am missing where its being done in these paths?

ah no ignore me :) __folio_alloc_noprof should set it. Thanks and sorry for the noise!

#6 lore raw
In reply to Usama Arif
Quoted 13 lines · Usama Arif
Quoted 10 lines · Zi Yan
Quoted 7 lines · David Hildenbrand (Arm)

compaction_alloc_noprof() does the page_rmappable_folio() at the end.

I'd assume that all migration allocation functions must take care of that.

It's a responsibility of the folio allocation code, not folio migration code.

Quoted 7 lines
Quoted 4 lines

I agree. The migration allocation function needs to give a folio or other types of page matching the original one. Do we want to turn this into a WARN to make sure newfolio matches folio's large_rmappable state?

hmm its done in compaction_alloc() and alloc_migration_target_by_mpol() but I dont see this being done in alloc_migration_target() and alloc_misplaced_dst_folio().

__folio_alloc_node() and friends should be calling page_rmappable_folio(). See __folio_alloc_noprof().

Quoted 5 lines

I think alloc_misplaced_dst_folio() is used by NUMA balancing and alloc_migration_target() by hotplug and memory failure?

I am not very familiar with migration code, so maybe I am missing where its being done in these paths?

Please double check. If a call is missing, we should fix it there.

Signature · 3 lines
Cheers,

David