mm/migrate.c: Rework migration_entry_wait() to not take a pageref
Subsystems (3)
Alistair Popple
reviewerPATCH v2 1/1 mm/migrate.c: Rework migration_entry_wait() to not take a pageref
This fixes a FIXME in migrate_vma_check_page().
Before migrating a page migration code will take a reference and check there are no unexpected page references, failing the migration if there are. When a thread faults on a migration entry it will take a temporary reference to the page to wait for the page to become unlocked signifying the migration entry has been removed.
This reference is dropped just prior to waiting on the page lock, however the extra reference can cause migration failures so it is desirable to avoid taking it.
As migration code already has a reference to the migrating page an extra reference to wait on PG_locked is unnecessary so long as the reference can't be dropped whilst setting up the wait.
When faulting on a migration entry the ptl is taken to check the migration entry. Removing a migration entry also requires the ptl, and migration code won't drop its page reference until after the migration entry has been removed. Therefore retaining the ptl of a migration entry is sufficient to ensure the page has a reference. Reworking migration_entry_wait() to hold the ptl until the wait setup is complete means the extra page reference is no longer needed.
---
Trying to refactor some of folio_wait_bit_common() into common functions didn't seem to end up making things simpler so I've left it as a manual inline into migration_entry_wait_on_locked() for now.
The only change needed to folio_wait_bit_common() is a behaviour that replaces folio_put() with pte_unmap_unlock(). I think adding the extra two arguments (ptep and ptl) to folio_wait_bit_common() is a bit messy, however it's borderline and I could go either way so happy to do that if others disagree.
v2:
- Rebase to master with folios
- Avoid taking a pageref in pmd_migration_entry_wait() as well
--- include/linux/pagemap.h | 2 + mm/filemap.c | 85 +++++++++++++++++++++++++++++++++++++++++ mm/migrate.c | 33 ++-------------- 3 files changed, 91 insertions(+), 29 deletions(-)
include/linux/pagemap.h +2 -0
@@ -769,6 +769,8 @@ static inline int wait_on_page_locked_killable(struct page *page)return folio_wait_locked_killable (page_folio (page)); } +void migration_entry_wait_on_locked (struct folio *folio,pte_t *ptep, +spinlock_t *ptl);int put_and_wait_on_page_locked (struct page *page,int state);void wait_on_page_writeback (struct page *page);void folio_wait_writeback (struct folio *folio);
mm/filemap.c +85 -0
91 lines not on this page · raw message
mm/migrate.c +4 -29
@@ -305,15 +305,7 @@ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, page =pfn_swap_entry_to_page (entry); page =compound_head (page); -/* -* Once page cache replacement of page migration started, page_count -* is zero; but we must not call put_and_wait_on_page_locked() without -* a ref. Use get_page_unless_zero(), and just fault again if it fails. -*/ -if (!get_page_unless_zero (page)) - goto out; -pte_unmap_unlock (ptep, ptl); -put_and_wait_on_page_locked (page,TASK_UNINTERRUPTIBLE ); +migration_entry_wait_on_locked (page_folio (page), ptep, ptl);return ; out:pte_unmap_unlock (ptep, ptl);
@@ -344,10 +336,7 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd)if (!is_pmd_migration_entry (*pmd)) goto unlock; page =pfn_swap_entry_to_page (pmd_to_swp_entry (*pmd)); -if (!get_page_unless_zero (page)) - goto unlock; -spin_unlock (ptl); -put_and_wait_on_page_locked (page,TASK_UNINTERRUPTIBLE ); +migration_entry_wait_on_locked (page_folio (page),NULL , ptl);return ; unlock:spin_unlock (ptl);
@@ -2488,22 +2477,8 @@ static bool migrate_vma_check_page(struct page *page)return false;/* Page from ZONE_DEVICE have one extra reference */ -if (is_zone_device_page (page)) { -/* -* Private page can never be pin as they have no valid pte and -* GUP will fail for those. Yet if there is a pending migration -* a thread might try to wait on the pte migration entry and -* will bump the page reference count. Sadly there is no way to -* differentiate a regular pin from migration wait. Hence to -* avoid 2 racing thread trying to migrate back to CPU to enter -* infinite loop (one stopping migration because the other is -* waiting on pte migration entry). We always return true here. -* -* FIXME proper solution is to rework migration_entry_wait() so -* it does not need to take a reference on page. -*/ -return is_device_private_page (page); - } +if (is_zone_device_page (page)) + extra++;/* For file back page */ if (page_mapping (page))
Signature · 1 line
2.30.2