mm: do not install PMD mappings when handling a COW fault
Subsystems (2)
PATCH v3 1/1 mm: do not install PMD mappings when handling a COW fault
From: Zhang Yi <[email protected]>
When pinning a page with FOLL_LONGTERM in a CoW VMA and a PMD-aligned (2MB on x86) large folio follow_page_mask() failed to obtain a valid anonymous page, resulting in an infinite loop issue. The specific triggering process is as follows:
1. User call mmap with a 2MB size in MAP_PRIVATE mode for a file that has a 2MB large folio installed in the page cache.
addr = mmap(NULL, 2*1024*1024, PROT_READ, MAP_PRIVATE, file_fd, 0);
2. The kernel driver pass this mapped address to pin_user_pages_fast() in FOLL_LONGTERM mode.
pin_user_pages_fast(addr, 512, FOLL_LONGTERM, pages);
-> pin_user_pages_fast()
| gup_fast_fallback() | __gup_longterm_locked() | __get_user_pages_locked() | __get_user_pages() | follow_page_mask() | follow_p4d_mask() | follow_pud_mask() | follow_pmd_mask() //pmd_leaf(pmdval) is true because the | //huge PMD is installed. This is normal | //in the first round, but it shouldn't | //happen in the second round. | follow_huge_pmd() //require an anonymous page | return -EMLINK; | faultin_page() | handle_mm_fault() | wp_huge_pmd() //remove PMD and fall back to PTE | handle_pte_fault() | do_pte_missing() | do_fault() | do_read_fault() //FAULT_FLAG_WRITE is not set | finish_fault() | do_set_pmd() //install a huge PMD again, this is wrong!!! | do_wp_page() //create private anonymous pages
<- goto retry;
Due to an incorrectly large PMD set in do_read_fault(), follow_pmd_mask() always returns -EMLINK, causing an infinite loop.
David pointed out that we can preallocate a page table and remap the PMD to be mapped by a PTE table in wp_huge_pmd() in the future. But now we can avoid this issue by not installing PMD mappings when handling a COW and unshare fault in do_set_pmd().
---v2->v3:
- Update comments to clarify why we shouldn't install PMD mappings while doing CoW.
mm/memory.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
mm/memory.c +11 -0
@@ -5520,6 +5520,17 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct folio *folio, struct page *paif (!thp_vma_suitable_order (vma, haddr,PMD_ORDER ))return ret; +/* +* We're about to trigger a write or unshare fault on a CoW +* mapping, breaking the shared folio into private anonymous +* copies at PTE granularity. A PMD mapping would bind an +* entire PMD-sized range to the shared folio, defeating CoW. +* Fall back to direct PTE mapping. +*/ +if (is_cow_mapping (vma->vm_flags) && + (vmf->flags & (FAULT_FLAG_WRITE |FAULT_FLAG_UNSHARE ))) +return ret; +if (!is_pmd_order (folio_order (folio)))return ret; page = &folio->page;
Signature · 1 line
2.52.0
Thank you, that's what I was hoping to see.
Quoted 92 lines · On May 20, 2026, at 09:16, [email protected] wrote:
From: Zhang Yi <[email protected]>
When pinning a page with
FOLL_LONGTERMin a CoW VMA and a PMD-aligned (2MB on x86) large foliofollow_page_mask()failed to obtain a valid anonymous page, resulting in an infinite loop issue. The specific triggering process is as follows:1. User call mmap with a 2MB size in
MAP_PRIVATEmode for a file that has a 2MB large folio installed in the page cache.addr = mmap(NULL, 2*1024*1024, PROT_READ, MAP_PRIVATE, file_fd, 0);2. The kernel driver pass this mapped address to
pin_user_pages_fast()inFOLL_LONGTERMmode.pin_user_pages_fast(addr, 512, FOLL_LONGTERM, pages);->
pin_user_pages_fast()| gup_fast_fallback() | __gup_longterm_locked() | __get_user_pages_locked() | __get_user_pages() | follow_page_mask() | follow_p4d_mask() | follow_pud_mask() | follow_pmd_mask() //pmd_leaf(pmdval) is true because the | //huge PMD is installed. This is normal | //in the first round, but it shouldn't | //happen in the second round. | follow_huge_pmd() //require an anonymous page | return -EMLINK; | faultin_page() | handle_mm_fault() | wp_huge_pmd() //remove PMD and fall back to PTE | handle_pte_fault() | do_pte_missing() | do_fault() | do_read_fault() //FAULT_FLAG_WRITE is not set | finish_fault() | do_set_pmd() //install a huge PMD again, this is wrong!!! | do_wp_page() //create private anonymous pages<- goto retry;
Due to an incorrectly large PMD set in
do_read_fault(),follow_pmd_mask()always returns-EMLINK, causing an infinite loop.David pointed out that we can preallocate a page table and remap the PMD to be mapped by a PTE table in
wp_huge_pmd()in the future. But now we can avoid this issue by not installing PMD mappings when handling a COW and unshare fault indo_set_pmd().Fixes a7f226604170 ("mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared anonymous page")Reported-by Karol Wachowski <[email protected]>Suggested-by David Hildenbrand <[email protected]>Signed-off-by Zhang Yi <[email protected]>---
v2->v3:
- Update comments to clarify why we shouldn't install PMD mappings while doing CoW.
mm/memory.c| 11 +++++++++++1 file changed, 11 insertions(+)mm/memory.c +11
-0@@ -5520,6 +5520,17 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct folio *folio, struct page *paif (!thp_vma_suitable_order (vma, haddr,PMD_ORDER ))return ret; +/* +* We're about to trigger a write or unshare fault on a CoW +* mapping, breaking the shared folio into private anonymous +* copies at PTE granularity. A PMD mapping would bind an +* entire PMD-sized range to the shared folio, defeating CoW. +* Fall back to direct PTE mapping. +*/ +if (is_cow_mapping (vma->vm_flags) && + (vmf->flags & (FAULT_FLAG_WRITE |FAULT_FLAG_UNSHARE ))) +return ret; +if (!is_pmd_order (folio_order (folio)))return ret; page = &folio->page;Signature · 1 line
2.52.0
Quoted 51 lines · On Wed, May 20, 2026 at 11:16:24AM -0400, [email protected] wrote:
From: Zhang Yi <[email protected]>
When pinning a page with
FOLL_LONGTERMin a CoW VMA and a PMD-aligned (2MB on x86) large foliofollow_page_mask()failed to obtain a valid anonymous page, resulting in an infinite loop issue. The specific triggering process is as follows:1. User call mmap with a 2MB size in
MAP_PRIVATEmode for a file that has a 2MB large folio installed in the page cache.addr = mmap(NULL, 2*1024*1024, PROT_READ, MAP_PRIVATE, file_fd, 0);2. The kernel driver pass this mapped address to
pin_user_pages_fast()inFOLL_LONGTERMmode.pin_user_pages_fast(addr, 512, FOLL_LONGTERM, pages);->
pin_user_pages_fast()| gup_fast_fallback() | __gup_longterm_locked() | __get_user_pages_locked() | __get_user_pages() | follow_page_mask() | follow_p4d_mask() | follow_pud_mask() | follow_pmd_mask() //pmd_leaf(pmdval) is true because the | //huge PMD is installed. This is normal | //in the first round, but it shouldn't | //happen in the second round. | follow_huge_pmd() //require an anonymous page | return -EMLINK; | faultin_page() | handle_mm_fault() | wp_huge_pmd() //remove PMD and fall back to PTE | handle_pte_fault() | do_pte_missing() | do_fault() | do_read_fault() //FAULT_FLAG_WRITE is not set | finish_fault() | do_set_pmd() //install a huge PMD again, this is wrong!!! | do_wp_page() //create private anonymous pages<- goto retry;
Due to an incorrectly large PMD set in
do_read_fault(),follow_pmd_mask()always returns-EMLINK, causing an infinite loop.David pointed out that we can preallocate a page table and remap the PMD to be mapped by a PTE table in
wp_huge_pmd()in the future. But now we can avoid this issue by not installing PMD mappings when handling a COW and unshare fault indo_set_pmd().
Quoted 6 lines
Fixes a7f226604170 ("mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared anonymous page")Reported-by Karol Wachowski <[email protected]>Suggested-by David Hildenbrand <[email protected]>Signed-off-by Zhang Yi <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE Labs