Skip to contentSkip to conversation

mm: do not install PMD mappings when handling a COW fault

PATCH In review [email protected] 3 participants 3 messages May 20 – May 21

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 mm: do not install PMD mappings when handling a COW fault

From: Zhang Yi <[email protected]>

When pinning a page with FOLL_LONGTERM in a CoW VMA and a PMD-aligned (2MB on x86) large folio follow_page_mask() failed to obtain a valid anonymous page, resulting in an infinite loop issue. The specific triggering process is as follows:

1. User call mmap with a 2MB size in MAP_PRIVATE mode for a file that has a 2MB large folio installed in the page cache.

   addr = mmap(NULL, 2*1024*1024, PROT_READ, MAP_PRIVATE, file_fd, 0);

2. The kernel driver pass this mapped address to pin_user_pages_fast() in FOLL_LONGTERM mode.

   pin_user_pages_fast(addr, 512, FOLL_LONGTERM, pages);

-> pin_user_pages_fast()

  |   gup_fast_fallback()
  |    __gup_longterm_locked()
  |     __get_user_pages_locked()
  |      __get_user_pages()
  |       follow_page_mask()
  |        follow_p4d_mask()
  |         follow_pud_mask()
  |          follow_pmd_mask() //pmd_leaf(pmdval) is true because the
  |                            //huge PMD is installed. This is normal
  |                            //in the first round, but it shouldn't
  |                            //happen in the second round.
  |           follow_huge_pmd() //require an anonymous page
  |            return -EMLINK;
  |   faultin_page()
  |    handle_mm_fault()
  |     wp_huge_pmd() //remove PMD and fall back to PTE
  |     handle_pte_fault()
  |      do_pte_missing()
  |       do_fault()
  |        do_read_fault() //FAULT_FLAG_WRITE is not set
  |         finish_fault()
  |          do_set_pmd() //install a huge PMD again, this is wrong!!!
  |      do_wp_page() //create private anonymous pages

<- goto retry;

Due to an incorrectly large PMD set in do_read_fault(), follow_pmd_mask() always returns -EMLINK, causing an infinite loop.

David pointed out that we can preallocate a page table and remap the PMD to be mapped by a PTE table in wp_huge_pmd() in the future. But now we can avoid this issue by not installing PMD mappings when handling a COW and unshare fault in do_set_pmd().

Fixes a7f226604170 ("mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared anonymous page")

---
v2->v3:
- Update comments to clarify why we shouldn't install PMD mappings while doing CoW.

 mm/memory.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
mm/memory.c +11 -0
index ea6568571131..b1aed4f08224 100644
@@ -5520,6 +5520,17 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct folio *folio, struct page *pa
 	if (!thp_vma_suitable_order(vma, haddr, PMD_ORDER))
 		return ret;
 
+	/*
+	 * We're about to trigger a write or unshare fault on a CoW
+	 * mapping, breaking the shared folio into private anonymous
+	 * copies at PTE granularity.  A PMD mapping would bind an
+	 * entire PMD-sized range to the shared folio, defeating CoW.
+	 * Fall back to direct PTE mapping.
+	 */
+	if (is_cow_mapping(vma->vm_flags) &&
+	    (vmf->flags & (FAULT_FLAG_WRITE | FAULT_FLAG_UNSHARE)))
+		return ret;
+
 	if (!is_pmd_order(folio_order(folio)))
 		return ret;
 	page = &folio->page;
Signature · 1 line
2.52.0
Reviewed-by on v3 1/1
#1 lore raw

Thank you, that's what I was hoping to see.

Quoted 92 lines · On May 20, 2026, at 09:16, [email protected] wrote:

From: Zhang Yi <[email protected]>

When pinning a page with FOLL_LONGTERM in a CoW VMA and a PMD-aligned (2MB on x86) large folio follow_page_mask() failed to obtain a valid anonymous page, resulting in an infinite loop issue. The specific triggering process is as follows:

1. User call mmap with a 2MB size in MAP_PRIVATE mode for a file that has a 2MB large folio installed in the page cache.

  addr = mmap(NULL, 2*1024*1024, PROT_READ, MAP_PRIVATE, file_fd, 0);

2. The kernel driver pass this mapped address to pin_user_pages_fast() in FOLL_LONGTERM mode.

  pin_user_pages_fast(addr, 512, FOLL_LONGTERM, pages);

-> pin_user_pages_fast()

 |   gup_fast_fallback()
 |    __gup_longterm_locked()
 |     __get_user_pages_locked()
 |      __get_user_pages()
 |       follow_page_mask()
 |        follow_p4d_mask()
 |         follow_pud_mask()
 |          follow_pmd_mask() //pmd_leaf(pmdval) is true because the
 |                            //huge PMD is installed. This is normal
 |                            //in the first round, but it shouldn't
 |                            //happen in the second round.
 |           follow_huge_pmd() //require an anonymous page
 |            return -EMLINK;
 |   faultin_page()
 |    handle_mm_fault()
 |     wp_huge_pmd() //remove PMD and fall back to PTE
 |     handle_pte_fault()
 |      do_pte_missing()
 |       do_fault()
 |        do_read_fault() //FAULT_FLAG_WRITE is not set
 |         finish_fault()
 |          do_set_pmd() //install a huge PMD again, this is wrong!!!
 |      do_wp_page() //create private anonymous pages

<- goto retry;

Due to an incorrectly large PMD set in do_read_fault(), follow_pmd_mask() always returns -EMLINK, causing an infinite loop.

David pointed out that we can preallocate a page table and remap the PMD to be mapped by a PTE table in wp_huge_pmd() in the future. But now we can avoid this issue by not installing PMD mappings when handling a COW and unshare fault in do_set_pmd().

Fixes a7f226604170 ("mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared anonymous page")

---
v2->v3:
- Update comments to clarify why we shouldn't install PMD mappings while doing CoW.

mm/memory.c | 11 +++++++++++

1 file changed, 11 insertions(+)
mm/memory.c +11 -0
index ea6568571131..b1aed4f08224 100644
@@ -5520,6 +5520,17 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct folio *folio, struct page *pa
 if (!thp_vma_suitable_order(vma, haddr, PMD_ORDER))
 return ret;
 
+ /*
+ * We're about to trigger a write or unshare fault on a CoW
+ * mapping, breaking the shared folio into private anonymous
+ * copies at PTE granularity.  A PMD mapping would bind an
+ * entire PMD-sized range to the shared folio, defeating CoW.
+ * Fall back to direct PTE mapping.
+ */
+ if (is_cow_mapping(vma->vm_flags) &&
+    (vmf->flags & (FAULT_FLAG_WRITE | FAULT_FLAG_UNSHARE)))
+ return ret;
+
 if (!is_pmd_order(folio_order(folio)))
 return ret;
 page = &folio->page;
Signature · 1 line
2.52.0
Reviewed-by on v3 1/1
#2 lore raw
In reply to [email protected]
Quoted 51 lines · On Wed, May 20, 2026 at 11:16:24AM -0400, [email protected] wrote:

From: Zhang Yi <[email protected]>

When pinning a page with FOLL_LONGTERM in a CoW VMA and a PMD-aligned (2MB on x86) large folio follow_page_mask() failed to obtain a valid anonymous page, resulting in an infinite loop issue. The specific triggering process is as follows:

1. User call mmap with a 2MB size in MAP_PRIVATE mode for a file that has a 2MB large folio installed in the page cache.

   addr = mmap(NULL, 2*1024*1024, PROT_READ, MAP_PRIVATE, file_fd, 0);

2. The kernel driver pass this mapped address to pin_user_pages_fast() in FOLL_LONGTERM mode.

   pin_user_pages_fast(addr, 512, FOLL_LONGTERM, pages);

-> pin_user_pages_fast()

  |   gup_fast_fallback()
  |    __gup_longterm_locked()
  |     __get_user_pages_locked()
  |      __get_user_pages()
  |       follow_page_mask()
  |        follow_p4d_mask()
  |         follow_pud_mask()
  |          follow_pmd_mask() //pmd_leaf(pmdval) is true because the
  |                            //huge PMD is installed. This is normal
  |                            //in the first round, but it shouldn't
  |                            //happen in the second round.
  |           follow_huge_pmd() //require an anonymous page
  |            return -EMLINK;
  |   faultin_page()
  |    handle_mm_fault()
  |     wp_huge_pmd() //remove PMD and fall back to PTE
  |     handle_pte_fault()
  |      do_pte_missing()
  |       do_fault()
  |        do_read_fault() //FAULT_FLAG_WRITE is not set
  |         finish_fault()
  |          do_set_pmd() //install a huge PMD again, this is wrong!!!
  |      do_wp_page() //create private anonymous pages

<- goto retry;

Due to an incorrectly large PMD set in do_read_fault(), follow_pmd_mask() always returns -EMLINK, causing an infinite loop.

David pointed out that we can preallocate a page table and remap the PMD to be mapped by a PTE table in wp_huge_pmd() in the future. But now we can avoid this issue by not installing PMD mappings when handling a COW and unshare fault in do_set_pmd().

Quoted 6 lines
Fixes a7f226604170 ("mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared anonymous page")
Signature · 2 lines
Oscar Salvador
SUSE Labs