Skip to contentSkip to conversation

mm/migrate_device: Clear stale mapping after freeing swapcache

PATCH Superseded Arvind Yadav 3 participants 4 messages Jul 27

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 mm/migrate_device: Clear stale mapping after freeing swapcache

__migrate_device_pages() reads the folio mapping before calling folio_free_swap(). When folio_free_swap() succeeds, the folio is removed from the swap cache, but the saved mapping still points to swap_space.

Passing the stale mapping to folio_migrate_mapping() makes it take the mapped-folio path after the swapcache reference has been dropped. This can cause an invalid swap_space lock access followed by a folio reference count BUG.

After a successful split, nr still contains the number of pages in the original large folio, although each resulting page is now a separate order-0 folio. Reset nr to 1 so each split folio is processed separately, including its own swapcache removal and mapping lookup.

Refresh the saved mapping after folio_free_swap() so the current folio state is used during migration.

v2:
- Refresh the mapping using folio_mapping(), as suggested by Zi Yan.

v3:
- Reset nr to 1 after a successful split so each resulting folio is processed independently, as suggested by Zi Yan.

Fixes df263d9a7dff ("mm/migrate_device: try to handle swapcache pages")
Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Gregory Price <[email protected]>
Cc Ying Huang <[email protected]>
Cc Alistair Popple <[email protected]>
---
 mm/migrate_device.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
mm/migrate_device.c +11 -0
index 554754eb26ff..74ac6651d4b3 100644
@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,
 							 MIGRATE_PFN_COMPOUND);
 					goto next;
 				}
+				/*
+				 * reset nr so that only first after-split folio
+				 * is processed below
+				 */
+				nr = 1;
 			} else if ((src_pfns[i] & MIGRATE_PFN_MIGRATE) &&
 				(dst_pfns[i] & MIGRATE_PFN_COMPOUND) &&
 				!(src_pfns[i] & MIGRATE_PFN_COMPOUND)) {
@@ -1204,6 +1209,12 @@ static void __migrate_device_pages(unsigned long *src_pfns,
 					src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
 					goto next;
 				}
+
+				/*
+				 * folio_free_swap() removed the folio from the swap
+				 * cache. Refresh the saved mapping before migration.
+				 */
+				mapping = folio_mapping(folio);
 			}
 		} else if (folio_is_zone_device(newfolio)) {
 			/*
Signature · 1 line
2.43.0
#1 lore raw
Quoted 46 lines · Arvind Yadav

__migrate_device_pages() reads the folio mapping before calling folio_free_swap(). When folio_free_swap() succeeds, the folio is removed from the swap cache, but the saved mapping still points to swap_space.

Passing the stale mapping to folio_migrate_mapping() makes it take the mapped-folio path after the swapcache reference has been dropped. This can cause an invalid swap_space lock access followed by a folio reference count BUG.

After a successful split, nr still contains the number of pages in the original large folio, although each resulting page is now a separate order-0 folio. Reset nr to 1 so each split folio is processed separately, including its own swapcache removal and mapping lookup.

Refresh the saved mapping after folio_free_swap() so the current folio state is used during migration.

v2:
- Refresh the mapping using folio_mapping(), as suggested by Zi Yan.

v3:
- Reset nr to 1 after a successful split so each resulting folio is processed independently, as suggested by Zi Yan.

Fixes df263d9a7dff ("mm/migrate_device: try to handle swapcache pages")
Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Gregory Price <[email protected]>
Cc Ying Huang <[email protected]>
Cc Alistair Popple <[email protected]>
---
 mm/migrate_device.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
mm/migrate_device.c +0 -0
index 554754eb26ff..74ac6651d4b3 100644

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1189

Quoted 8 lines
mm/migrate_device.c +5 -0
@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,
 							 MIGRATE_PFN_COMPOUND);
 					goto next;
 				}
+				/*
+				 * reset nr so that only first after-split folio
+				 * is processed below
+				 */
+				nr = 1;

We should also add a VM_WARN_ON_ONCE(folio_test_swapcache(folio))

Quoted 4 lines
mm/migrate_device.c +0 -0
 			} else if ((src_pfns[i] & MIGRATE_PFN_MIGRATE) &&
 				(dst_pfns[i] & MIGRATE_PFN_COMPOUND) &&
 				!(src_pfns[i] & MIGRATE_PFN_COMPOUND)) {

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1218

Quoted 10 lines
mm/migrate_device.c +6 -0
@@ -1204,6 +1209,12 @@ static void __migrate_device_pages(unsigned long *src_pfns,
 					src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
 					goto next;
 				}
+
+				/*
+				 * folio_free_swap() removed the folio from the swap
+				 * cache. Refresh the saved mapping before migration.
+				 */
+				mapping = folio_mapping(folio);
 			}

I wonder if we should change this instead to (sorry for the wrap around, I'll need to fix my mailer)

@@ -1242,6 +1232,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,

                for (j = 0; j < nr && i + j < npages; j++) {
                        folio = page_folio(migrate_pfn_to_page(src_pfns[i+j]));
                        newfolio = page_folio(migrate_pfn_to_page(dst_pfns[i+j]));
+                       /*
+                        * folio_free_swap() removed the folio from the swap
+                        * cache. Refresh the saved mapping before migration.
+                        */

+ mapping = folio_mapping(folio);

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1220

Quoted 2 lines
mm/migrate_device.c +0 -0
 		} else if (folio_is_zone_device(newfolio)) {
 			/*

Thanks,
Balbir

#2 lore raw
Quoted 42 lines · Balbir Singh
Quoted 41 lines · Arvind Yadav

__migrate_device_pages() reads the folio mapping before calling folio_free_swap(). When folio_free_swap() succeeds, the folio is removed from the swap cache, but the saved mapping still points to swap_space.

Passing the stale mapping to folio_migrate_mapping() makes it take the mapped-folio path after the swapcache reference has been dropped. This can cause an invalid swap_space lock access followed by a folio reference count BUG.

After a successful split, nr still contains the number of pages in the original large folio, although each resulting page is now a separate order-0 folio. Reset nr to 1 so each split folio is processed separately, including its own swapcache removal and mapping lookup.

Refresh the saved mapping after folio_free_swap() so the current folio state is used during migration.

v2:
- Refresh the mapping using folio_mapping(), as suggested by Zi Yan.

v3:
- Reset nr to 1 after a successful split so each resulting folio is processed independently, as suggested by Zi Yan.

Fixes df263d9a7dff ("mm/migrate_device: try to handle swapcache pages")
Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Gregory Price <[email protected]>
Cc Ying Huang <[email protected]>
Cc Alistair Popple <[email protected]>
---
  mm/migrate_device.c | 11 +++++++++++
  1 file changed, 11 insertions(+)
Quoted 14 lines

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1189

Quoted 13 lines
mm/migrate_device.c +5 -0
index 554754eb26ff..74ac6651d4b3 100644
@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,
  							 MIGRATE_PFN_COMPOUND);
  					goto next;
  				}
+				/*
+				 * reset nr so that only first after-split folio
+				 * is processed below
+				 */
+				nr = 1;

We should also add a VM_WARN_ON_ONCE(folio_test_swapcache(folio))

Agreed , but folio_test_swapcache()  is not valid immediately after the split, as the order-0 folios may still be in swapcache. I’ll use just before nr = 1

Quoted 19 lines
Quoted 5 lines
mm/migrate_device.c +0 -0
  			} else if ((src_pfns[i] & MIGRATE_PFN_MIGRATE) &&
  				(dst_pfns[i] & MIGRATE_PFN_COMPOUND) &&
  				!(src_pfns[i] & MIGRATE_PFN_COMPOUND)) {

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1218

Quoted 10 lines
mm/migrate_device.c +6 -0
@@ -1204,6 +1209,12 @@ static void __migrate_device_pages(unsigned long *src_pfns,
  					src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
  					goto next;
  				}
+
+				/*
+				 * folio_free_swap() removed the folio from the swap
+				 * cache. Refresh the saved mapping before migration.
+				 */
+				mapping = folio_mapping(folio);
  			}

I wonder if we should change this instead to (sorry for the wrap around, I'll need to fix my mailer)

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1240

Quoted 9 lines
mm/migrate_device.c +5 -0
@@ -1242,6 +1232,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,
                 for (j = 0; j < nr && i + j < npages; j++) {
                         folio = page_folio(migrate_pfn_to_page(src_pfns[i+j]));
                         newfolio = page_folio(migrate_pfn_to_page(dst_pfns[i+j]));
+                       /*
+                        * folio_free_swap() removed the folio from the swap
+                        * cache. Refresh the saved mapping before migration.
+                        */
+                       mapping = folio_mapping(folio);
 

Agreed, Moving the mapping lookup into the per-folio loop looks better; I’ll update it

Thanks,
Arvind

Quoted 5 lines
(quoted diff) +0 -0
 >   		} else if (folio_is_zone_device(newfolio)) {

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1219

Quoted 1 line
mm/migrate_device.c +0 -0
  			/*

Thanks,
Balbir

maintainer
#3 lore raw
Quoted 14 lines · Arvind
Quoted 4 lines
Quoted 4 lines
mm/migrate_device.c +0 -0
@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,
  							 MIGRATE_PFN_COMPOUND);
Quoted 8 lines

mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1189

Quoted 7 lines
mm/migrate_device.c +5 -0
  					goto next;
  				}
+				/*
+				 * reset nr so that only first after-split folio
+				 * is processed below
+				 */
+				nr = 1;

We should also add a VM_WARN_ON_ONCE(folio_test_swapcache(folio))

Quoted 3 lines

Agreed , but folio_test_swapcache()  is not valid immediately after the split, as the order-0 folios may still be in swapcache.
I’ll use just before nr = 1

AI review suggest there might be another bug in here:

	https://sashiko.dev/#/patchset/[email protected]