mm/migrate_device: Clear stale mapping after freeing swapcache
Subsystems (2)
PATCH v3 1/1 mm/migrate_device: Clear stale mapping after freeing swapcache
__migrate_device_pages() reads the folio mapping before calling folio_free_swap(). When folio_free_swap() succeeds, the folio is removed from the swap cache, but the saved mapping still points to swap_space.
Passing the stale mapping to folio_migrate_mapping() makes it take the mapped-folio path after the swapcache reference has been dropped. This can cause an invalid swap_space lock access followed by a folio reference count BUG.
After a successful split, nr still contains the number of pages in the original large folio, although each resulting page is now a separate order-0 folio. Reset nr to 1 so each split folio is processed separately, including its own swapcache removal and mapping lookup.
Refresh the saved mapping after folio_free_swap() so the current folio state is used during migration.
v2:
- Refresh the mapping using folio_mapping(), as suggested by Zi Yan.
v3:
- Reset nr to 1 after a successful split so each resulting folio is processed independently, as suggested by Zi Yan.
--- mm/migrate_device.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
mm/migrate_device.c +11 -0
@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns, MIGRATE_PFN_COMPOUND); goto next; } +/* +* reset nr so that only first after-split folio +* is processed below +*/ + nr =1 ; }else if ((src_pfns[i] &MIGRATE_PFN_MIGRATE ) && (dst_pfns[i] &MIGRATE_PFN_COMPOUND ) && !(src_pfns[i] &MIGRATE_PFN_COMPOUND )) {
@@ -1204,6 +1209,12 @@ static void __migrate_device_pages(unsigned long *src_pfns, src_pfns[i] &= ~MIGRATE_PFN_MIGRATE ; goto next; } + +/* +* folio_free_swap() removed the folio from the swap +* cache. Refresh the saved mapping before migration. +*/ + mapping =folio_mapping (folio); } }else if (folio_is_zone_device (newfolio)) {/*
Signature · 1 line
2.43.0
Quoted 46 lines · Arvind Yadav
__migrate_device_pages()reads the folio mapping before callingfolio_free_swap(). Whenfolio_free_swap()succeeds, the folio is removed from the swap cache, but the saved mapping still points toswap_space.Passing the stale mapping to
folio_migrate_mapping()makes it take the mapped-folio path after the swapcache reference has been dropped. This can cause an invalidswap_spacelock access followed by a folio reference count BUG.After a successful split, nr still contains the number of pages in the original large folio, although each resulting page is now a separate order-0 folio. Reset nr to 1 so each split folio is processed separately, including its own swapcache removal and mapping lookup.
Refresh the saved mapping after
folio_free_swap()so the current folio state is used during migration.v2:
- Refresh the mapping usingfolio_mapping(), as suggested by Zi Yan.v3:
- Reset nr to 1 after a successful split so each resulting folio is processed independently, as suggested by Zi Yan.Fixes df263d9a7dff ("mm/migrate_device: try to handle swapcache pages")Cc Andrew Morton <[email protected]>Cc David Hildenbrand <[email protected]>Cc Matthew Brost <[email protected]>Cc Joshua Hahn <[email protected]>Cc Rakie Kim <[email protected]>Cc Byungchul Park <[email protected]>Cc Gregory Price <[email protected]>Cc Ying Huang <[email protected]>Cc Alistair Popple <[email protected]>Reviewed-by Zi Yan <[email protected]>Reviewed-by Balbir Singh <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- mm/migrate_device.c | 11 +++++++++++ 1 file changed, 11 insertions(+)mm/migrate_device.c +0
-0
mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1189
Quoted 8 lines
mm/migrate_device.c +5
-0@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns, MIGRATE_PFN_COMPOUND); goto next; } +/* +* reset nr so that only first after-split folio +* is processed below +*/ + nr =1 ;
We should also add a VM_WARN_ON_ONCE(folio_test_swapcache(folio))
Quoted 4 lines
mm/migrate_device.c +0
-0}else if ((src_pfns[i] &MIGRATE_PFN_MIGRATE ) && (dst_pfns[i] &MIGRATE_PFN_COMPOUND ) && !(src_pfns[i] &MIGRATE_PFN_COMPOUND )) {
mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1218
Quoted 10 lines
mm/migrate_device.c +6
-0@@ -1204,6 +1209,12 @@ static void __migrate_device_pages(unsigned long *src_pfns, src_pfns[i] &= ~MIGRATE_PFN_MIGRATE ; goto next; } + +/* +* folio_free_swap() removed the folio from the swap +* cache. Refresh the saved mapping before migration. +*/ + mapping =folio_mapping (folio); }
I wonder if we should change this instead to (sorry for the wrap around, I'll need to fix my mailer)
@@ -1242,6 +1232,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,
for (j = 0; j < nr && i + j < npages; j++) {
folio = page_folio(migrate_pfn_to_page(src_pfns[i+j]));
newfolio = page_folio(migrate_pfn_to_page(dst_pfns[i+j]));
+ /*
+ * folio_free_swap() removed the folio from the swap
+ * cache. Refresh the saved mapping before migration.
+ */+ mapping = folio_mapping(folio);
mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1220
Quoted 2 lines
mm/migrate_device.c +0
-0}else if (folio_is_zone_device (newfolio)) {/*
Thanks,
Balbir
Quoted 42 lines · Balbir Singh
Quoted 41 lines · Arvind Yadav
__migrate_device_pages()reads the folio mapping before callingfolio_free_swap(). Whenfolio_free_swap()succeeds, the folio is removed from the swap cache, but the saved mapping still points toswap_space.Passing the stale mapping to
folio_migrate_mapping()makes it take the mapped-folio path after the swapcache reference has been dropped. This can cause an invalidswap_spacelock access followed by a folio reference count BUG.After a successful split, nr still contains the number of pages in the original large folio, although each resulting page is now a separate order-0 folio. Reset nr to 1 so each split folio is processed separately, including its own swapcache removal and mapping lookup.
Refresh the saved mapping after
folio_free_swap()so the current folio state is used during migration.v2:
- Refresh the mapping usingfolio_mapping(), as suggested by Zi Yan.v3:
- Reset nr to 1 after a successful split so each resulting folio is processed independently, as suggested by Zi Yan.Fixes df263d9a7dff ("mm/migrate_device: try to handle swapcache pages")Cc Andrew Morton <[email protected]>Cc David Hildenbrand <[email protected]>Cc Matthew Brost <[email protected]>Cc Joshua Hahn <[email protected]>Cc Rakie Kim <[email protected]>Cc Byungchul Park <[email protected]>Cc Gregory Price <[email protected]>Cc Ying Huang <[email protected]>Cc Alistair Popple <[email protected]>Reviewed-by Zi Yan <[email protected]>Reviewed-by Balbir Singh <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- mm/migrate_device.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
Quoted 14 lines
mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1189
Quoted 13 lines
mm/migrate_device.c +5
-0@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns, MIGRATE_PFN_COMPOUND); goto next; } +/* +* reset nr so that only first after-split folio +* is processed below +*/ + nr =1 ;We should also add a VM_WARN_ON_ONCE(folio_test_swapcache(folio))
Agreed , but folio_test_swapcache() is not valid immediately after the split, as the order-0 folios may still be in swapcache. I’ll use just before nr = 1
Quoted 19 lines
Quoted 5 lines
mm/migrate_device.c +0
-0}else if ((src_pfns[i] &MIGRATE_PFN_MIGRATE ) && (dst_pfns[i] &MIGRATE_PFN_COMPOUND ) && !(src_pfns[i] &MIGRATE_PFN_COMPOUND )) {mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1218
Quoted 10 lines
mm/migrate_device.c +6
-0@@ -1204,6 +1209,12 @@ static void __migrate_device_pages(unsigned long *src_pfns, src_pfns[i] &= ~MIGRATE_PFN_MIGRATE ; goto next; } + +/* +* folio_free_swap() removed the folio from the swap +* cache. Refresh the saved mapping before migration. +*/ + mapping =folio_mapping (folio); }I wonder if we should change this instead to (sorry for the wrap around, I'll need to fix my mailer)
mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1240
Quoted 9 lines
mm/migrate_device.c +5
-0@@ -1242,6 +1232,11 @@ static void __migrate_device_pages(unsigned long *src_pfns,for (j =0 ; j < nr && i + j < npages; j++) { folio =page_folio (migrate_pfn_to_page (src_pfns[i+j])); newfolio =page_folio (migrate_pfn_to_page (dst_pfns[i+j])); +/* +* folio_free_swap() removed the folio from the swap +* cache. Refresh the saved mapping before migration. +*/ + mapping =folio_mapping (folio);
Agreed, Moving the mapping lookup into the per-folio loop looks better; I’ll update it
Thanks,
Arvind
Quoted 5 lines
(quoted diff) +0
-0> } else if (folio_is_zone_device(newfolio)) {mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1219
Quoted 1 line
mm/migrate_device.c +0
-0/* Thanks,
Balbir
Andrew Morton
maintainerQuoted 14 lines · Arvind
Quoted 4 lines
Quoted 4 lines
mm/migrate_device.c +0
-0@@ -1182,6 +1182,11 @@ static void __migrate_device_pages(unsigned long *src_pfns, MIGRATE_PFN_COMPOUND);Quoted 8 lines
mm/migrate_device.c static void __migrate_device_pages(unsigned long *src_pfns, · line 1189
Quoted 7 lines
mm/migrate_device.c +5
-0goto next; } +/* +* reset nr so that only first after-split folio +* is processed below +*/ + nr =1 ;We should also add a VM_WARN_ON_ONCE(folio_test_swapcache(folio))
Quoted 3 lines
Agreed , but folio_test_swapcache() is not valid immediately after the split, as the order-0 folios may still be in swapcache.
I’ll use just before nr = 1
AI review suggest there might be another bug in here:
https://sashiko.dev/#/patchset/[email protected]