mm: folio conversion and refactoring in do_swap_page()
Subsystems (2)
v3 cover mm: folio conversion and refactoring in do_swap_page()
From: Hongfu Li <[email protected]>
Convert the device-private migration path in do_swap_page() to use folio APIs, then extract all non-swap entry handling into a new do_non_swap_page() helper to improve readability and reduce the size of the main swap fault function.
Hongfu Li (2):
mm: Use a folio in the softleaf_is_device_private path
mm: Extract non-swap entry handling to do_nonswap_softleaf()
mm/memory.c | 118 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 68 insertions(+), 50 deletions(-)
---
v3:
- Rename do_non_swap_page() to do_nonswap_softleaf().
- Update the commit subject and body of patch 2/2 accordingly.
- Drop a blank line after the device_private condition in do_nonswap_softleaf().
v2:
- Drop fault_folio, reuse the existing folio variable instead
- Refresh commit messages
- Update comment to use "folio" instead of "page" in the
device_private path
- Add Acked-by: David Hildenbrand
- Add patch 2/2 to extract do_non_swap_page() from do_swap_page()
- Link to v1: https://lore.kernel.org/all/[email protected]/
Signature · 1 line
2.54.0
PATCH v3 1/2 mm: Use a folio in the softleaf_is_device_private path
From: Hongfu Li <[email protected]>
Use the folio APIs in the device_private migration path of do_swap_page(), replacing four calls to compound_head() with one.
--- mm/memory.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
mm/memory.c +6 -5
@@ -4873,18 +4873,19 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto unlock;/* -* Get a page reference while we know the page can't be +* Get a folio reference while we know the folio can't be * freed. */ -if (trylock_page (vmf->page)) { + folio =page_folio (vmf->page); +if (folio_trylock (folio)) {struct dev_pagemap *pgmap; -get_page (vmf->page); +folio_get (folio);pte_unmap_unlock (vmf->pte, vmf->ptl); pgmap =page_pgmap (vmf->page); ret = pgmap->ops->migrate_to_ram (vmf); -unlock_page (vmf->page); -put_page (vmf->page); +folio_unlock (folio); +folio_put (folio); }else {pte_unmap (vmf->pte);softleaf_entry_wait_on_locked (entry, vmf->ptl);
Signature · 1 line
2.54.0
PATCH v3 2/2 mm: Extract non-swap entry handling to do_nonswap_softleaf()
In reply to Hongfu LiFrom: Hongfu Li <[email protected]>
Factor non-swap entry logic out of bloated do_swap_page() to simplify the main swap fault flow and improve readability.
The new do_nonswap_softleaf() handles migration, device private/exclusive, hwpoison, marker and bad PTE entries.
--- mm/memory.c | 119 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 68 insertions(+), 51 deletions(-)
mm/memory.c +68 -51
@@ -4817,6 +4817,73 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry, }while (--nr_pages); } +static vm_fault_t do_nonswap_softleaf (struct vm_fault *vmf) +{ +struct vm_area_struct *vma = vmf->vma; +struct folio *folio; +softleaf_t entry; +vm_fault_t ret =0 ; + + entry =softleaf_from_pte (vmf->orig_pte); +if (softleaf_is_migration (entry)) { +migration_entry_wait (vma->vm_mm, vmf->pmd, + vmf->address); + }else if (softleaf_is_device_exclusive (entry)) { + vmf->page =softleaf_to_page (entry); + ret =remove_device_exclusive_entry (vmf); + }else if (softleaf_is_device_private (entry)) { +if (vmf->flags &FAULT_FLAG_VMA_LOCK ) { +/* +* migrate_to_ram is not yet ready to operate +* under VMA lock. +*/ +vma_end_read (vma); +return VM_FAULT_RETRY ; + } + + vmf->page =softleaf_to_page (entry); + vmf->pte =pte_offset_map_lock (vma->vm_mm, vmf->pmd, + vmf->address, &vmf->ptl); +if (unlikely (!vmf->pte || + !pte_same (ptep_get (vmf->pte), + vmf->orig_pte))) + goto unlock; + +/* +* Get a folio reference while we know the folio can't be +* freed. +*/ + folio =page_folio (vmf->page); +if (folio_trylock (folio)) { +struct dev_pagemap *pgmap; + +folio_get (folio); +pte_unmap_unlock (vmf->pte, vmf->ptl); + pgmap =page_pgmap (vmf->page); + ret = pgmap->ops->migrate_to_ram (vmf); +folio_unlock (folio); +folio_put (folio); + }else { +pte_unmap (vmf->pte); +softleaf_entry_wait_on_locked (entry, vmf->ptl); + } + }else if (softleaf_is_hwpoison (entry)) { + ret =VM_FAULT_HWPOISON ; + }else if (softleaf_is_marker (entry)) { + ret =handle_pte_marker (vmf); + }else { +print_bad_pte (vma, vmf->address, vmf->orig_pte,NULL ); + ret =VM_FAULT_SIGBUS ; + } + +return ret; + +unlock: +if (vmf->pte) +pte_unmap_unlock (vmf->pte, vmf->ptl); +return ret; +} +/* * We enter with either the VMA lock or the mmap_lock held (see * FAULT_FLAG_VMA_LOCK), and pte mapped but not yet locked.
@@ -4847,57 +4914,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) entry =softleaf_from_pte (vmf->orig_pte);if (unlikely (!softleaf_is_swap (entry))) { -if (softleaf_is_migration (entry)) { -migration_entry_wait (vma->vm_mm, vmf->pmd, - vmf->address); - }else if (softleaf_is_device_exclusive (entry)) { - vmf->page =softleaf_to_page (entry); - ret =remove_device_exclusive_entry (vmf); - }else if (softleaf_is_device_private (entry)) { -if (vmf->flags &FAULT_FLAG_VMA_LOCK ) { -/* -* migrate_to_ram is not yet ready to operate -* under VMA lock. -*/ -vma_end_read (vma); - ret =VM_FAULT_RETRY ; - goto out; - } - - vmf->page =softleaf_to_page (entry); - vmf->pte =pte_offset_map_lock (vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); -if (unlikely (!vmf->pte || - !pte_same (ptep_get (vmf->pte), - vmf->orig_pte))) - goto unlock; - -/* -* Get a folio reference while we know the folio can't be -* freed. -*/ - folio =page_folio (vmf->page); -if (folio_trylock (folio)) { -struct dev_pagemap *pgmap; - -folio_get (folio); -pte_unmap_unlock (vmf->pte, vmf->ptl); - pgmap =page_pgmap (vmf->page); - ret = pgmap->ops->migrate_to_ram (vmf); -folio_unlock (folio); -folio_put (folio); - }else { -pte_unmap (vmf->pte); -softleaf_entry_wait_on_locked (entry, vmf->ptl); - } - }else if (softleaf_is_hwpoison (entry)) { - ret =VM_FAULT_HWPOISON ; - }else if (softleaf_is_marker (entry)) { - ret =handle_pte_marker (vmf); - }else { -print_bad_pte (vma, vmf->address, vmf->orig_pte,NULL ); - ret =VM_FAULT_SIGBUS ; - } + ret =do_nonswap_softleaf (vmf); goto out; }
Signature · 1 line
2.54.0
Quoted 4 lines
Convert the device-private migration path in
do_swap_page()to use folio APIs, then extract all non-swap entry handling into a newdo_non_swap_page()helper to improve readability and reduce the size of the main swap fault function.
Please disregard this email. There is something I got wrong. Sorry for the noise.