Skip to contentSkip to conversation

mm/huge_memory: add pmd folio to ds_queue in do_huge_zero_wp_pmd()

PATCH Superseded Wei Yang 7 participants 18 messages Oct 2, 2025 – Oct 14, 2025

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Wei Yang

author
#0 lore raw

PATCH v2 1/1 mm/huge_memory: add pmd folio to ds_queue in do_huge_zero_wp_pmd()

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

---
v2:
* add fix, cc stable and put description about the flow of current code
* move deferred_split_folio() into map_anon_folio_pmd()

---
 mm/huge_memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -1
index 1b81680b4225..f13de93637bf 100644
@@ -1232,6 +1232,7 @@ static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
 	count_vm_event(THP_FAULT_ALLOC);
 	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
 	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	deferred_split_folio(folio, false);
 }
 
 static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
@@ -1272,7 +1273,6 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
 		pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
 		map_anon_folio_pmd(folio, vmf->pmd, vma, haddr);
 		mm_inc_nr_ptes(vma->vm_mm);
-		deferred_split_folio(folio, false);
 		spin_unlock(vmf->ptl);
 	}
 
Signature · 1 line
2.34.1
author
#1 lore raw
Quoted 5 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Quoted 7 lines

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Since we move deferred_split_folio() into map_anon_folio_pmd(), I am thinking about whether we can consolidate the process in collapse_huge_page().

Use map_anon_folio_pmd() in collapse_huge_page(), but skip those statistic adjustment.

Quoted 7 lines
Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>
Signature · 2 lines
Wei Yang
Help you, Help me
reviewer
#2 lore raw
Quoted 14 lines · Wei Yang
Quoted 5 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Quoted 7 lines

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 5 lines

Since we move deferred_split_folio() into map_anon_folio_pmd(), I am thinking about whether we can consolidate the process in collapse_huge_page().

Use map_anon_folio_pmd() in collapse_huge_page(), but skip those statistic adjustment.

Yeah, that's a good idea :)

We could add a simple bool is_fault parameter to map_anon_folio_pmd() to control the statistics.

The fault paths would call it with true, and the collapse paths could then call it with false.

Something like this:

```
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 1b81680b4225..9924180a4a56 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1218,7 +1218,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma,
 }

 static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
-		struct vm_area_struct *vma, unsigned long haddr)
+		struct vm_area_struct *vma, unsigned long haddr, bool is_fault)
 {
 	pmd_t entry;

@@ -1228,10 +1228,15 @@ static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
 	folio_add_lru_vma(folio, vma);
 	set_pmd_at(vma->vm_mm, haddr, pmd, entry);
 	update_mmu_cache_pmd(vma, haddr, pmd);
-	add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
-	count_vm_event(THP_FAULT_ALLOC);
-	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
-	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+
+	if (is_fault) {
+		add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
+		count_vm_event(THP_FAULT_ALLOC);
+		count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
+		count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	}
+
+	deferred_split_folio(folio, false);
 }

 static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index d0957648db19..2eddd5a60e48 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1227,17 +1227,10 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,
 	__folio_mark_uptodate(folio);
 	pgtable = pmd_pgtable(_pmd);

-	_pmd = folio_mk_pmd(folio, vma->vm_page_prot);
-	_pmd = maybe_pmd_mkwrite(pmd_mkdirty(_pmd), vma);
-
 	spin_lock(pmd_ptl);
 	BUG_ON(!pmd_none(*pmd));
-	folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
-	folio_add_lru_vma(folio, vma);
 	pgtable_trans_huge_deposit(mm, pmd, pgtable);
-	set_pmd_at(mm, address, pmd, _pmd);
-	update_mmu_cache_pmd(vma, address, pmd);
-	deferred_split_folio(folio, false);
+	map_anon_folio_pmd(folio, pmd, vma, address, false);
 	spin_unlock(pmd_ptl);

 	folio = NULL;
```

Untested, though.

Quoted 9 lines
Quoted 8 lines
Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>
author
#3 lore raw
Quoted 93 lines · Lance Yang
Quoted 14 lines · Wei Yang
Quoted 5 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Quoted 7 lines

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 5 lines

Since we move deferred_split_folio() into map_anon_folio_pmd(), I am thinking about whether we can consolidate the process in collapse_huge_page().

Use map_anon_folio_pmd() in collapse_huge_page(), but skip those statistic adjustment.

Yeah, that's a good idea :)

We could add a simple bool is_fault parameter to map_anon_folio_pmd() to control the statistics.

The fault paths would call it with true, and the collapse paths could then call it with false.

Something like this:

```

mm/huge_memory.c +10 -5
index 1b81680b4225..9924180a4a56 100644
@@ -1218,7 +1218,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct
 vm_area_struct *vma,
 }
 
 static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
-		struct vm_area_struct *vma, unsigned long haddr)
+		struct vm_area_struct *vma, unsigned long haddr, bool is_fault)
 {
 	pmd_t entry;
@@ -1228,10 +1228,15 @@ static void map_anon_folio_pmd(struct folio *folio,
 pmd_t *pmd,
 	folio_add_lru_vma(folio, vma);
 	set_pmd_at(vma->vm_mm, haddr, pmd, entry);
 	update_mmu_cache_pmd(vma, haddr, pmd);
-	add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
-	count_vm_event(THP_FAULT_ALLOC);
-	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
-	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+
+	if (is_fault) {
+		add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
+		count_vm_event(THP_FAULT_ALLOC);
+		count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
+		count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	}
+
+	deferred_split_folio(folio, false);
 }
 

static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)

mm/khugepaged.c +1 -8
index d0957648db19..2eddd5a60e48 100644
@@ -1227,17 +1227,10 @@ static int collapse_huge_page(struct mm_struct *mm,
 unsigned long address,
 	__folio_mark_uptodate(folio);
 	pgtable = pmd_pgtable(_pmd);
 
-	_pmd = folio_mk_pmd(folio, vma->vm_page_prot);
-	_pmd = maybe_pmd_mkwrite(pmd_mkdirty(_pmd), vma);
-
 	spin_lock(pmd_ptl);
 	BUG_ON(!pmd_none(*pmd));
-	folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
-	folio_add_lru_vma(folio, vma);
 	pgtable_trans_huge_deposit(mm, pmd, pgtable);
-	set_pmd_at(mm, address, pmd, _pmd);
-	update_mmu_cache_pmd(vma, address, pmd);
-	deferred_split_folio(folio, false);
+	map_anon_folio_pmd(folio, pmd, vma, address, false);
 	spin_unlock(pmd_ptl);
 
	folio = NULL;
Quoted 4 lines

```

Untested, though.

This is the same as I thought.

Will prepare a patch for it.

Signature · 2 lines
Wei Yang
Help you, Help me
#4 lore raw
Quoted 96 lines · Wei Yang
Quoted 95 lines · Lance Yang
Quoted 14 lines · Wei Yang
Quoted 5 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Quoted 7 lines

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 5 lines

Since we move deferred_split_folio() into map_anon_folio_pmd(), I am thinking about whether we can consolidate the process in collapse_huge_page().

Use map_anon_folio_pmd() in collapse_huge_page(), but skip those statistic adjustment.

Yeah, that's a good idea :)

We could add a simple bool is_fault parameter to map_anon_folio_pmd() to control the statistics.

The fault paths would call it with true, and the collapse paths could then call it with false.

Something like this:

```

mm/huge_memory.c +10 -5
index 1b81680b4225..9924180a4a56 100644
@@ -1218,7 +1218,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct
 vm_area_struct *vma,
 }
 
 static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
-		struct vm_area_struct *vma, unsigned long haddr)
+		struct vm_area_struct *vma, unsigned long haddr, bool is_fault)
 {
 	pmd_t entry;
@@ -1228,10 +1228,15 @@ static void map_anon_folio_pmd(struct folio *folio,
 pmd_t *pmd,
 	folio_add_lru_vma(folio, vma);
 	set_pmd_at(vma->vm_mm, haddr, pmd, entry);
 	update_mmu_cache_pmd(vma, haddr, pmd);
-	add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
-	count_vm_event(THP_FAULT_ALLOC);
-	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
-	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+
+	if (is_fault) {
+		add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
+		count_vm_event(THP_FAULT_ALLOC);
+		count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
+		count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	}
+
+	deferred_split_folio(folio, false);
 }
 

static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)

mm/khugepaged.c +1 -8
index d0957648db19..2eddd5a60e48 100644
@@ -1227,17 +1227,10 @@ static int collapse_huge_page(struct mm_struct *mm,
 unsigned long address,
 	__folio_mark_uptodate(folio);
 	pgtable = pmd_pgtable(_pmd);
 
-	_pmd = folio_mk_pmd(folio, vma->vm_page_prot);
-	_pmd = maybe_pmd_mkwrite(pmd_mkdirty(_pmd), vma);
-
 	spin_lock(pmd_ptl);
 	BUG_ON(!pmd_none(*pmd));
-	folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
-	folio_add_lru_vma(folio, vma);
 	pgtable_trans_huge_deposit(mm, pmd, pgtable);
-	set_pmd_at(mm, address, pmd, _pmd);
-	update_mmu_cache_pmd(vma, address, pmd);
-	deferred_split_folio(folio, false);
+	map_anon_folio_pmd(folio, pmd, vma, address, false);
 	spin_unlock(pmd_ptl);
 
	folio = NULL;

```

Quoted 6 lines
Quoted 2 lines

Untested, though.

This is the same as I thought.

Will prepare a patch for it.

Let's do that as an add-on patch, though.

Signature · 3 lines
Cheers

David / dhildenb
reviewer
#5 lore raw
Quoted 100 lines · David Hildenbrand
Quoted 99 lines · Wei Yang
Quoted 98 lines · Lance Yang
Quoted 14 lines · Wei Yang
Quoted 5 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Quoted 7 lines

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 5 lines

Since we move deferred_split_folio() into map_anon_folio_pmd(), I am thinking about whether we can consolidate the process in collapse_huge_page().

Use map_anon_folio_pmd() in collapse_huge_page(), but skip those statistic adjustment.

Yeah, that's a good idea :)

We could add a simple bool is_fault parameter to map_anon_folio_pmd() to control the statistics.

The fault paths would call it with true, and the collapse paths could then call it with false.

Something like this:

```

mm/huge_memory.c +1 -1
index 1b81680b4225..9924180a4a56 100644
@@ -1218,7 +1218,7 @@ static struct folio 
 *vma_alloc_anon_folio_pmd(struct
 vm_area_struct *vma,
 }
 
 static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
-        struct vm_area_struct *vma, unsigned long haddr)
+        struct vm_area_struct *vma, unsigned long haddr, bool is_fault)
 {

    pmd_t entry;

mm/huge_memory.c +9 -4
@@ -1228,10 +1228,15 @@ static void map_anon_folio_pmd(struct folio 
 *folio,
 pmd_t *pmd,
     folio_add_lru_vma(folio, vma);
     set_pmd_at(vma->vm_mm, haddr, pmd, entry);
     update_mmu_cache_pmd(vma, haddr, pmd);
-    add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
-    count_vm_event(THP_FAULT_ALLOC);
-    count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
-    count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+
+    if (is_fault) {
+        add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
+        count_vm_event(THP_FAULT_ALLOC);
+        count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
+        count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+    }
+
+    deferred_split_folio(folio, false);
 }

static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)

mm/khugepaged.c +1 -8
index d0957648db19..2eddd5a60e48 100644
@@ -1227,17 +1227,10 @@ static int collapse_huge_page(struct 
 mm_struct *mm,
 unsigned long address,
     __folio_mark_uptodate(folio);
     pgtable = pmd_pgtable(_pmd);
 
-    _pmd = folio_mk_pmd(folio, vma->vm_page_prot);
-    _pmd = maybe_pmd_mkwrite(pmd_mkdirty(_pmd), vma);
-
     spin_lock(pmd_ptl);
     BUG_ON(!pmd_none(*pmd));
-    folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
-    folio_add_lru_vma(folio, vma);
     pgtable_trans_huge_deposit(mm, pmd, pgtable);
-    set_pmd_at(mm, address, pmd, _pmd);
-    update_mmu_cache_pmd(vma, address, pmd);
-    deferred_split_folio(folio, false);
+    map_anon_folio_pmd(folio, pmd, vma, address, false);
     spin_unlock(pmd_ptl);

    folio = NULL;
```

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Untested, though.

This is the same as I thought.

Will prepare a patch for it.

Let's do that as an add-on patch, though.

Yeah, let’s do that separately ;)

Acked-by on v2 1/1
#6 lore raw
In reply to Wei Yang
Quoted 12 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 6 lines
Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>
Signature · 3 lines
Cheers

David / dhildenb
reviewer Reviewed-by on v2 1/1
#7 lore raw
In reply to Wei Yang
Quoted 12 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 6 lines
Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

Cool. LGTM.

Quoted 30 lines

---
v2:
* add fix, cc stable and put description about the flow of current code
* move deferred_split_folio() into map_anon_folio_pmd()

---
  mm/huge_memory.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -1
index 1b81680b4225..f13de93637bf 100644
@@ -1232,6 +1232,7 @@ static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
  	count_vm_event(THP_FAULT_ALLOC);
  	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
  	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	deferred_split_folio(folio, false);
  }
 
  static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
@@ -1272,7 +1273,6 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
  		pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
  		map_anon_folio_pmd(folio, vmf->pmd, vma, haddr);
  		mm_inc_nr_ptes(vma->vm_mm);
-		deferred_split_folio(folio, false);
  		spin_unlock(vmf->ptl);
  	}
 
reviewer Reviewed-by on v2 1/1
#8 lore raw
In reply to Wei Yang
Quoted 12 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 8 lines
Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

---

Thanks!

reviewer
#9 lore raw
In reply to Wei Yang

Hey Wei,

Quoted 7 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory

IIRC, it was commit dafff3f4c850 ("mm: split underused THPs") that started unconditionally adding all new anon THPs to _deferred_list :)

Quoted 6 lines

in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")

Shouldn't this rather be the following?

Fixes dafff3f4c850 ("mm: split underused THPs")

Thanks,
Lance

Quoted 35 lines
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

---
v2:
* add fix, cc stable and put description about the flow of current code
* move deferred_split_folio() into map_anon_folio_pmd()

---
  mm/huge_memory.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -1
index 1b81680b4225..f13de93637bf 100644
@@ -1232,6 +1232,7 @@ static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
  	count_vm_event(THP_FAULT_ALLOC);
  	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
  	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	deferred_split_folio(folio, false);
  }
 
  static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
@@ -1272,7 +1273,6 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
  		pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
  		map_anon_folio_pmd(folio, vmf->pmd, vma, haddr);
  		mm_inc_nr_ptes(vma->vm_mm);
-		deferred_split_folio(folio, false);
  		spin_unlock(vmf->ptl);
  	}
 
reviewer
#10 lore raw
Quoted 16 lines · Lance Yang

Hey Wei,

Quoted 7 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory

IIRC, it was commit dafff3f4c850 ("mm: split underused THPs") that started unconditionally adding all new anon THPs to _deferred_list :)

Quoted 2 lines

in case of memory pressure even the pmd folio is under used.

Quoted 8 lines
Quoted 4 lines

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")

Shouldn't this rather be the following?

Fixes dafff3f4c850 ("mm: split underused THPs")

Yes, I agree. In this case, this patch looks more like an optimization for split underused THPs.

One observation on this change is that right after zero pmd wp, the deferred split queue could be scanned, the newly added pmd folio will split since it is all zero except one subpage. This means we probably should allocate a base folio for zero pmd wp and map the rest to zero page at the beginning if split underused THP is enabled to avoid this long trip. The downside is that user app cannot get a pmd folio if it is intended to write data into the entire folio.

Usama might be able to give some insight here.

Quoted 30 lines

Thanks,
Lance

Quoted 26 lines
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

---
v2:
* add fix, cc stable and put description about the flow of current code
* move deferred_split_folio() into map_anon_folio_pmd()

---
  mm/huge_memory.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -0
index 1b81680b4225..f13de93637bf 100644
@@ -1232,6 +1232,7 @@ static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
  	count_vm_event(THP_FAULT_ALLOC);
  	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
  	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	deferred_split_folio(folio, false);
  }
   static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
Quoted 8 lines

mm/huge_memory.c static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf) · line 1278

Quoted 8 lines
mm/huge_memory.c +0 -1
@@ -1272,7 +1273,6 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
  		pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
  		map_anon_folio_pmd(folio, vmf->pmd, vma, haddr);
  		mm_inc_nr_ptes(vma->vm_mm);
-		deferred_split_folio(folio, false);
  		spin_unlock(vmf->ptl);
  	}
 

Best Regards,
Yan, Zi

Acked-by
#11 lore raw
Quoted 38 lines · Zi Yan
Quoted 16 lines · Lance Yang

Hey Wei,

Quoted 7 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory

IIRC, it was commit dafff3f4c850 ("mm: split underused THPs") that started unconditionally adding all new anon THPs to _deferred_list :)

Quoted 2 lines

in case of memory pressure even the pmd folio is under used.

Quoted 8 lines
Quoted 4 lines

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")

Shouldn't this rather be the following?

Fixes dafff3f4c850 ("mm: split underused THPs")

Yes, I agree. In this case, this patch looks more like an optimization for split underused THPs.

One observation on this change is that right after zero pmd wp, the deferred split queue could be scanned, the newly added pmd folio will split since it is all zero except one subpage. This means we probably should allocate a base folio for zero pmd wp and map the rest to zero page at the beginning if split underused THP is enabled to avoid this long trip. The downside is that user app cannot get a pmd folio if it is intended to write data into the entire folio.

Quoted 2 lines

Usama might be able to give some insight here.

Thanks for CCing me Zi!

hmm I think the downside of not having PMD folio probably outweights the cost of splitting a zer-filled page?
ofcourse I dont have any numbers to back that up, but that would be my initial guess.

Also:

Quoted 43 lines
Quoted 31 lines

Thanks,
Lance

Quoted 26 lines
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

---
v2:
* add fix, cc stable and put description about the flow of current code
* move deferred_split_folio() into map_anon_folio_pmd()

---
  mm/huge_memory.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -0
index 1b81680b4225..f13de93637bf 100644
@@ -1232,6 +1232,7 @@ static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
  	count_vm_event(THP_FAULT_ALLOC);
  	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
  	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	deferred_split_folio(folio, false);
  }
   static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
Quoted 8 lines

mm/huge_memory.c static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf) · line 1278

Quoted 8 lines
mm/huge_memory.c +0 -1
@@ -1272,7 +1273,6 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
  		pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
  		map_anon_folio_pmd(folio, vmf->pmd, vma, haddr);
  		mm_inc_nr_ptes(vma->vm_mm);
-		deferred_split_folio(folio, false);
  		spin_unlock(vmf->ptl);
  	}
 

Best Regards,
Yan, Zi

reviewer
#12 lore raw
Quoted 39 lines · Usama Arif
Quoted 38 lines · Zi Yan
Quoted 16 lines · Lance Yang

Hey Wei,

Quoted 7 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory

IIRC, it was commit dafff3f4c850 ("mm: split underused THPs") that started unconditionally adding all new anon THPs to _deferred_list :)

Quoted 2 lines

in case of memory pressure even the pmd folio is under used.

Quoted 8 lines
Quoted 4 lines

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")

Shouldn't this rather be the following?

Fixes dafff3f4c850 ("mm: split underused THPs")

Yes, I agree. In this case, this patch looks more like an optimization for split underused THPs.

One observation on this change is that right after zero pmd wp, the deferred split queue could be scanned, the newly added pmd folio will split since it is all zero except one subpage. This means we probably should allocate a base folio for zero pmd wp and map the rest to zero page at the beginning if split underused THP is enabled to avoid this long trip. The downside is that user app cannot get a pmd folio if it is intended to write data into the entire folio.

Quoted 7 lines
Quoted 2 lines

Usama might be able to give some insight here.

Thanks for CCing me Zi!

hmm I think the downside of not having PMD folio probably outweights the cost of splitting a zer-filled page?

Yeah, I agree.

Quoted 50 lines

ofcourse I dont have any numbers to back that up, but that would be my initial guess.

Also:

Quoted 43 lines
Quoted 31 lines

Thanks,
Lance

Quoted 26 lines
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

---
v2:
* add fix, cc stable and put description about the flow of current code
* move deferred_split_folio() into map_anon_folio_pmd()

---
  mm/huge_memory.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -0
index 1b81680b4225..f13de93637bf 100644
@@ -1232,6 +1232,7 @@ static void map_anon_folio_pmd(struct folio *folio, pmd_t *pmd,
  	count_vm_event(THP_FAULT_ALLOC);
  	count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC);
  	count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
+	deferred_split_folio(folio, false);
  }
   static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
Quoted 8 lines

mm/huge_memory.c static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf) · line 1278

Quoted 8 lines
mm/huge_memory.c +0 -1
@@ -1272,7 +1273,6 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf)
  		pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
  		map_anon_folio_pmd(folio, vmf->pmd, vma, haddr);
  		mm_inc_nr_ptes(vma->vm_mm);
-		deferred_split_folio(folio, false);
  		spin_unlock(vmf->ptl);
  	}
 

Best Regards,
Yan, Zi

Best Regards,
Yan, Zi

author
#13 lore raw
In reply to Zi Yan
Quoted 30 lines · Zi Yan
Quoted 16 lines · Lance Yang

Hey Wei,

Quoted 7 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory

IIRC, it was commit dafff3f4c850 ("mm: split underused THPs") that started unconditionally adding all new anon THPs to _deferred_list :)

Quoted 2 lines

in case of memory pressure even the pmd folio is under used.

Quoted 8 lines
Quoted 4 lines

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")

Shouldn't this rather be the following?

Fixes dafff3f4c850 ("mm: split underused THPs")

Yes, I agree. In this case, this patch looks more like an optimization for split underused THPs.

Quoted 7 lines

One observation on this change is that right after zero pmd wp, the deferred split queue could be scanned, the newly added pmd folio will split since it is all zero except one subpage. This means we probably should allocate a base folio for zero pmd wp and map the rest to zero page at the beginning if split underused THP is enabled to avoid this long trip. The downside is that user app cannot get a pmd folio if it is intended to write data into the entire folio.

Thanks for raising this.

IMHO, we could face the similar situation in __do_huge_pmd_anonymous_page(). If my understanding is correct, the allocated folio is zeroed and we don't have idea how user would write data to it.

Since shrinker is active when memory is low, maybe vma_alloc_anon_folio_pmd() has told use current status of the memory. If it does get a pmd folio, we are probably having enough memory in the system.

Quoted 3 lines

Usama might be able to give some insight here.

Signature · 2 lines
Wei Yang
Help you, Help me
author
#14 lore raw
In reply to Lance Yang
Quoted 8 lines · Lance Yang

Hey Wei,

Quoted 5 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Quoted 6 lines
Quoted 2 lines

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory

IIRC, it was commit dafff3f4c850 ("mm: split underused THPs") that started unconditionally adding all new anon THPs to _deferred_list :)

Thanks for taking a look.

While at this time do_huge_zero_wp_pmd() is not introduced, how it fix a non-exist case? And how could it be backported? I am confused here.

Quoted 13 lines
Quoted 6 lines

in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")

Shouldn't this rather be the following?

Fixes dafff3f4c850 ("mm: split underused THPs")

Thanks,
Lance

Signature · 2 lines
Wei Yang
Help you, Help me
reviewer
#15 lore raw
Quoted 12 lines · Wei Yang
Quoted 11 lines · Lance Yang

Hey Wei,

Quoted 7 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory

Quoted 7 lines
Quoted 3 lines

IIRC, it was commit dafff3f4c850 ("mm: split underused THPs") that started unconditionally adding all new anon THPs to _deferred_list :)

Thanks for taking a look.

While at this time do_huge_zero_wp_pmd() is not introduced, how it fix a

Ah, I see. I was focused on the policy change ...

Quoted 1 line

non-exist case? And how could it be backported? I am confused here.

And, yes, 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") was merged later and it introduced the new do_huge_zero_wp_pmd() path without aligning with the policy ...

Thanks for clarifying!
Lance

Quoted 15 lines
Quoted 14 lines
Quoted 7 lines

in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")

Shouldn't this rather be the following?

Fixes dafff3f4c850 ("mm: split underused THPs")

Thanks,
Lance

reviewer
#16 lore raw
In reply to Wei Yang
Quoted 19 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>
Quoted 8 lines

---
v2:
* add fix, cc stable and put description about the flow of current code
* move deferred_split_folio() into map_anon_folio_pmd()

---
 mm/huge_memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

LGTM. Reviewed-by: Zi Yan <[email protected]>

Best Regards,
Yan, Zi

reviewer Reviewed-by on v2 1/1
#17 lore raw
In reply to Wei Yang
Quoted 12 lines · Wei Yang

We add pmd folio into ds_queue on the first page fault in __do_huge_pmd_anonymous_page(), so that we can split it in case of memory pressure. This should be the same for a pmd folio during wp page fault.

Commit 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault") miss to add it to ds_queue, which means system may not reclaim enough memory in case of memory pressure even the pmd folio is under used.

Move deferred_split_folio() into map_anon_folio_pmd() to make the pmd folio installation consistent.

Quoted 8 lines
Fixes 1ced09e0331f ("mm: allocate THP on hugezeropage wp-fault")
Cc David Hildenbrand <[email protected]>
Cc Lance Yang <[email protected]>
Cc Dev Jain <[email protected]>

---

Nice catch. LGTM.