mm: fix address wraparound when scanning large folio PTEs
Subsystems (3)
David Hildenbrand (Arm)
maintainerPATCH v2 1/1 mm: fix address wraparound when scanning large folio PTEs
numa_rebuild_large_mapping() and do_swap_page() can be tricked into generating an address wraparound, crashing the kernel. All that's required is to map a tail page of a large folio close enough to the start/end of the address space to e.g., make
unsigned long addr_start = addr - (nr << PAGE_SHIFT);
to wrap around.
Let's fix it by providing a better helper, folio_scan_range(), that takes care of staying within VMA and page table boundaries while handling possible wraparounds.
There are some nasty details, like not being able to use pmd_addr_end() in this context. Add some helpful comments for these cases.
---
Changes in v2:
- s/folio_scan_pte_range/folio_pte_range/
- Add more comments + improve existing ones.
- Link to v1: https://patch.msgid.link/[email protected]
--- mm/memory.c | 50 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 36 insertions(+), 14 deletions(-)
mm/memory.c +36 -14
@@ -4933,6 +4933,34 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry, }while (--nr_pages); } +/* +* Get the address range in which a folio might be mapped, assuming its pages +* are mapped consecutively and the folio page at the given idx is mapped at +* the given address. +*/ +static void folio_pte_range (const struct folio *folio, +const struct vm_area_struct *vma,unsigned long addr, +unsigned long idx,unsigned long *start,unsigned long *end) +{ +/* Stay within the page table and VMA boundaries. */ +const unsigned long pmd_start = addr &PMD_MASK ; +const unsigned long pmd_end = pmd_start +PMD_SIZE ; +const unsigned long lower =max (pmd_start, vma->vm_start); +/* +* We cannot use pmd_addr_end(), as it does not clamp reliably. To +* avoid wraparounds, compute using the inclusive end. +*/ +const unsigned long upper =min (pmd_end -1 , vma->vm_end -1 ) +1 ; +const unsigned long offset = idx <<PAGE_SHIFT ; + +/* +* Avoid start/end addresses wrapping by not adding/subtracting more +* than the lower/upper limits would allow. +*/ + *start = addr -min (addr - lower, offset); + *end = addr +min (upper - addr,folio_size (folio) - offset); +} +/* * We enter with either the VMA lock or the mmap_lock held (see * FAULT_FLAG_VMA_LOCK), and pte mapped but not yet locked.
@@ -5121,16 +5149,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) address = vmf->address; ptep = vmf->pte;if (folio_test_large (folio) &&folio_test_swapcache (folio)) { -int nr =folio_nr_pages (folio); -unsigned long idx =folio_page_idx (folio, page); -unsigned long folio_start = address - idx *PAGE_SIZE ; -unsigned long folio_end = folio_start + nr *PAGE_SIZE ; +const int nr =folio_nr_pages (folio); +const unsigned long idx =folio_page_idx (folio, page); +unsigned long folio_start, folio_end;pte_t *folio_ptep;pte_t folio_pte; -if (unlikely (folio_start <max (address &PMD_MASK , vma->vm_start))) - goto check_folio; -if (unlikely (folio_end >pmd_addr_end (address, vma->vm_end))) +folio_pte_range (folio, vma, address, idx, &folio_start, + &folio_end); +if (unlikely ((folio_end - folio_start) >>PAGE_SHIFT != nr)) goto check_folio; folio_ptep = vmf->pte - idx;
@@ -6280,16 +6307,11 @@ static void numa_rebuild_large_mapping(struct vm_fault *vmf, struct vm_area_stru struct folio *folio, pte_t fault_pte, bool ignore_writable, bool pte_write_upgrade) { - int nr = pte_pfn(fault_pte) - folio_pfn(folio); + const unsigned long idx = pte_pfn(fault_pte) - folio_pfn(folio);unsigned long start, end, addr = vmf->address; -unsigned long addr_start = addr - (nr <<PAGE_SHIFT ); -unsigned long pt_start =ALIGN_DOWN (addr,PMD_SIZE );pte_t *start_ptep; -/* Stay within the VMA and within the page table. */ - start =max3 (addr_start, pt_start, vma->vm_start); - end =min3 (addr_start +folio_size (folio), pt_start +PMD_SIZE , - vma->vm_end); +folio_pte_range (folio, vma, addr, idx, &start, &end); start_ptep = vmf->pte - ((addr - start) >>PAGE_SHIFT );/* Restore all PTEs' mapping of the large folio */
---
base-commit: 634cbf5b794b734c78631b92e381cfc85f357bbe
change-id: 20261013-folio_pte_scan_range-c6ca19cc8e9d
--
Cheers,
David
Quoted 17 lines · David Hildenbrand (Arm)
numa_rebuild_large_mapping()anddo_swap_page()can be tricked into generating an address wraparound, crashing the kernel. All that's required is to map a tail page of a large folio close enough to the start/end of the address space to e.g., makeunsigned long addr_start = addr - (nr << PAGE_SHIFT);to wrap around.
Let's fix it by providing a better helper,
folio_scan_range(), that takes care of staying within VMA and page table boundaries while handling possible wraparounds.There are some nasty details, like not being able to use
pmd_addr_end()in this context. Add some helpful comments for these cases.
Quoted 9 lines
Reported-by Kyle Zeng <[email protected]>Fixes d2136d749d76 ("mm: support multi-size THP numa balancing")Fixes 508758960b8d ("mm: swap: entirely map large folios found in swapcache")Reviewed-by Lorenzo Stoakes (ARM) <[email protected]>Reviewed-by Kiryl Shutsemau (Meta) <[email protected]>Reviewed-by Baolin Wang <[email protected]>Reviewed-by Lance Yang <[email protected]>Signed-off-by David Hildenbrand (Arm) <[email protected]>