mm/huge_memory: refactor zap_huge_pmd()
Subsystems (3)
Lorenzo Stoakes (Oracle)
reviewerv3 cover mm/huge_memory: refactor zap_huge_pmd()
The zap_huge_pmd() function is overly complicated, clean it up and also add an assert in the case that we encounter a buggy PMD entry that doesn't match expectations.
This is motivated by a bug discovered [0] where the PMD entry was none of:
* A non-DAX, PFN or mixed map.
* The huge zero folio
* A present PMD entry
* A softleaf entry
In zap_huge_pmd(), but due to the bug we manged to reach this code.
It is useful to explicitly call this out rather than have an arbitrary NULL pointer dereference happen, which also improves understanding of what's going on.
The series goes further to make use of vm_normal_folio_pmd() rather than implementing custom logic for retrieving the folio, and extends softleaf functionality to provide and use an equivalent softleaf function.
[0]:https://lore.kernel.org/all/[email protected]/
v3:
* Propagated tags, thanks everybody!
* Fixed const vma parameter in vma_is_special_huge() in 1/13 as per Sashiko.
* Renamed needs_deposit -> has_deposit as per Kiryl, better describing the situation as we're zapping deposited tables, not depositing them.
* Initialised has_deposit to arch_needs_pgtable_deposit(), and updated huge zero page case to account for that as per Kiryl.
* Dropped separated logic approach as per Baolin.
* Added 'No functional change intended.' caveats.
* Removed seemingly superfluous, inconsistent pot-folio_remove_rmap_pmd() mapcount sanity checks.
* De-duplicated tlb->mm's.
* Separated folio-specific logic into another function.
* Added softleaf_is_valid_pmd_entry(), pmd_to_softleaf_folio() functions.
* Add and use normal_or_softleaf_folio_pmd() to make use of vm_normal_folio_pmd() and pmd_to_softleaf_folio() for obtaining the folio.
* Add and use has_deposited_pgtable() to figure out deposits.
* Added a bunch of explanatory comments as per Baolin.
v2:
* Added tags thanks everybody!
* Fixed issue with returning false on bug case potentially looping forever as per Baolin.
* Fixed further issue in bug path in 5/8 with double pte unlock.
* Add patch to use vm_normal_folio_pmd() as per David.
https://lore.kernel.org/all/[email protected]/
v1:
https://lore.kernel.org/all/[email protected]/
Lorenzo Stoakes (Oracle) (13):
mm/huge_memory: simplify vma_is_specal_huge() mm/huge: avoid big else branch in zap_huge_pmd() mm/huge_memory: have zap_huge_pmd return a boolean, add kdoc mm/huge_memory: handle buggy PMD entry in zap_huge_pmd() mm/huge_memory: add a common exit path to zap_huge_pmd() mm/huge_memory: remove unnecessary VM_BUG_ON_PAGE() mm/huge_memory: deduplicate zap deposited table call mm/huge_memory: remove unnecessary sanity checks mm/huge_memory: use mm instead of tlb->mm mm/huge_memory: separate out the folio part of zap_huge_pmd()
mm: add softleaf_is_valid_pmd_entry(), pmd_to_softleaf_folio() mm/huge_memory: add and use normal_or_softleaf_folio_pmd() mm/huge_memory: add and use has_deposited_pgtable()
include/linux/huge_mm.h | 8 +- include/linux/leafops.h | 39 +++++++++- include/linux/mm.h | 16 ---- mm/huge_memory.c | 168 +++++++++++++++++++++++++--------------- 4 files changed, 143 insertions(+), 88 deletions(-)
--
2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 1/13 mm/huge_memory: simplify vma_is_specal_huge()
This function is confused - it overloads the term 'special' yet again, checks for DAX but in many cases the code explicitly excludes DAX before invoking the predicate.
It also unnecessarily checks for vma->vm_file - this has to be present for a driver to have set VMA_MIXEDMAP_BIT or VMA_PFNMAP_BIT.
In fact, a far simpler form of this is to reverse the DAX predicate and return false if DAX is set.
This makes sense from the point of view of 'special' as in vm_normal_page(), as DAX actually does potentially have retrievable folios.
Also there's no need to have this in mm.h so move it to huge_memory.c.
No functional change intended.
--- include/linux/huge_mm.h | 4 ++-- include/linux/mm.h | 16 ---------------- mm/huge_memory.c | 30 +++++++++++++++++++++++------- 3 files changed, 25 insertions(+), 25 deletions(-)
include/linux/huge_mm.h +2 -2
@@ -83,7 +83,7 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr;* file is never split and the MAX_PAGECACHE_ORDER limit does not apply to * it. Same to PFNMAPs where there's neither page* nor pagecache. */ -#define THP_ORDERS_ALL_SPECIAL \ +#define THP_ORDERS_ALL_SPECIAL_DAX \ (BIT(PMD_ORDER) | BIT(PUD_ORDER))#define THP_ORDERS_ALL_FILE_DEFAULT \ ((BIT(MAX_PAGECACHE_ORDER + 1) - 1) & ~BIT(0))
@@ -92,7 +92,7 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr;* Mask of all large folio orders supported for THP. */ #define THP_ORDERS_ALL \ - (THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_SPECIAL | THP_ORDERS_ALL_FILE_DEFAULT) + (THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_SPECIAL_DAX | THP_ORDERS_ALL_FILE_DEFAULT)enum tva_type {TVA_SMAPS ,/* Exposing "THPeligible:" in smaps. */
include/linux/mm.h +0 -16
@@ -5078,22 +5078,6 @@ long copy_folio_from_user(struct folio *dst_folio,const void __user *usr_src, bool allow_pagefault); -/** -* vma_is_special_huge - Are transhuge page-table entries considered special? -* @vma: Pointer to the struct vm_area_struct to consider -* -* Whether transhuge page-table entries are considered "special" following -* the definition in vm_normal_page(). -* -* Return: true if transhuge page-table entries should be considered special, -* false otherwise. -*/ -static inline bool vma_is_special_huge (const struct vm_area_struct *vma) -{ -return vma_is_dax (vma) || (vma->vm_file && - (vma->vm_flags & (VM_PFNMAP |VM_MIXEDMAP ))); -} -#endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ #if MAX_NUMNODES >1
mm/huge_memory.c +23 -7
@@ -103,6 +103,14 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)return !inode_is_open_for_write (inode) &&S_ISREG (inode->i_mode); } +/* If returns true, we are unable to access the VMA's folios. */ +static bool vma_is_special_huge (const struct vm_area_struct *vma) +{ +if (vma_is_dax (vma)) +return false; +return vma_test_any (vma,VMA_PFNMAP_BIT ,VMA_MIXEDMAP_BIT ); +} + unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, vm_flags_t vm_flags, enum tva_type type,
@@ -116,8 +124,8 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,/* Check the intersection of requested and supported orders. */ if (vma_is_anonymous (vma)) supported_orders =THP_ORDERS_ALL_ANON ; -else if (vma_is_special_huge (vma)) - supported_orders =THP_ORDERS_ALL_SPECIAL ; +else if (vma_is_dax (vma) ||vma_is_special_huge (vma)) + supported_orders =THP_ORDERS_ALL_SPECIAL_DAX ;else supported_orders =THP_ORDERS_ALL_FILE_DEFAULT ;
@@ -2338,7 +2346,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm);arch_check_zapped_pmd (vma, orig_pmd);tlb_remove_pmd_tlb_entry (tlb, pmd, addr); -if (!vma_is_dax (vma) &&vma_is_special_huge (vma)) { +if (vma_is_special_huge (vma)) {if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl);
@@ -2840,7 +2848,7 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, orig_pud =pudp_huge_get_and_clear_full (vma, addr, pud, tlb->fullmm);arch_check_zapped_pud (vma, orig_pud);tlb_remove_pud_tlb_entry (tlb, pud, addr); -if (!vma_is_dax (vma) &&vma_is_special_huge (vma)) { +if (vma_is_special_huge (vma)) {spin_unlock (ptl);/* No zero page support yet */ }else {
@@ -2991,7 +2999,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,*/ if (arch_needs_pgtable_deposit ())zap_deposited_table (mm, pmd); -if (!vma_is_dax (vma) &&vma_is_special_huge (vma)) +if (vma_is_special_huge (vma))return ;if (unlikely (pmd_is_migration_entry (old_pmd))) {const softleaf_t old_entry =softleaf_from_pmd (old_pmd);
@@ -4517,8 +4525,16 @@ static void split_huge_pages_all(void)static inline bool vma_not_suitable_for_thp_split (struct vm_area_struct *vma) { -return vma_is_special_huge (vma) || (vma->vm_flags &VM_IO ) || -is_vm_hugetlb_page (vma); +if (vma_is_dax (vma)) +return true; +if (vma_is_special_huge (vma)) +return true; +if (vma_test (vma,VMA_IO_BIT )) +return true; +if (is_vm_hugetlb_page (vma)) +return true; + +return false; } static int split_huge_pages_pid(int pid, unsigned long vaddr_start,
Signature · 1 line
2.53.0
Re: [PATCH v3 01/13] mm/huge_memory: simplify vma_is_specal_huge()
Quoted 15 lines · Lorenzo Stoakes (Oracle)
This function is confused - it overloads the term 'special' yet again, checks for DAX but in many cases the code explicitly excludes DAX before invoking the predicate.
It also unnecessarily checks for
vma->vm_file- this has to be present for a driver to have setVMA_MIXEDMAP_BITorVMA_PFNMAP_BIT.In fact, a far simpler form of this is to reverse the DAX predicate and return false if DAX is set.
This makes sense from the point of view of 'special' as in
vm_normal_page(), as DAX actually does potentially have retrievable folios.
Quoted 5 lines
Also there's no need to have this in
mm.hso move it tohuge_memory.c.No functional change intended.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 134 lines
--- include/linux/huge_mm.h | 4 ++-- include/linux/mm.h | 16 ---------------- mm/huge_memory.c | 30 +++++++++++++++++++++++------- 3 files changed, 25 insertions(+), 25 deletions(-)include/linux/huge_mm.h +2
-2@@ -83,7 +83,7 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr;* file is never split and the MAX_PAGECACHE_ORDER limit does not apply to * it. Same to PFNMAPs where there's neither page* nor pagecache. */ -#define THP_ORDERS_ALL_SPECIAL \ +#define THP_ORDERS_ALL_SPECIAL_DAX \ (BIT(PMD_ORDER) | BIT(PUD_ORDER))#define THP_ORDERS_ALL_FILE_DEFAULT \ ((BIT(MAX_PAGECACHE_ORDER + 1) - 1) & ~BIT(0))@@ -92,7 +92,7 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr;* Mask of all large folio orders supported for THP. */ #define THP_ORDERS_ALL \ - (THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_SPECIAL | THP_ORDERS_ALL_FILE_DEFAULT) + (THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_SPECIAL_DAX | THP_ORDERS_ALL_FILE_DEFAULT)enum tva_type {TVA_SMAPS ,/* Exposing "THPeligible:" in smaps. */ include/linux/mm.h +0
-16@@ -5078,22 +5078,6 @@ long copy_folio_from_user(struct folio *dst_folio,const void __user *usr_src, bool allow_pagefault); -/** -* vma_is_special_huge - Are transhuge page-table entries considered special? -* @vma: Pointer to the struct vm_area_struct to consider -* -* Whether transhuge page-table entries are considered "special" following -* the definition in vm_normal_page(). -* -* Return: true if transhuge page-table entries should be considered special, -* false otherwise. -*/ -static inline bool vma_is_special_huge (const struct vm_area_struct *vma) -{ -return vma_is_dax (vma) || (vma->vm_file && - (vma->vm_flags & (VM_PFNMAP |VM_MIXEDMAP ))); -} -#endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ #if MAX_NUMNODES >1 mm/huge_memory.c +23
-8@@ -103,6 +103,14 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)return !inode_is_open_for_write (inode) &&S_ISREG (inode->i_mode); } +/* If returns true, we are unable to access the VMA's folios. */ +static bool vma_is_special_huge (const struct vm_area_struct *vma) +{ +if (vma_is_dax (vma)) +return false; +return vma_test_any (vma,VMA_PFNMAP_BIT ,VMA_MIXEDMAP_BIT ); +} + unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, vm_flags_t vm_flags, enum tva_type type,@@ -116,8 +124,8 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,/* Check the intersection of requested and supported orders. */ if (vma_is_anonymous (vma)) supported_orders =THP_ORDERS_ALL_ANON ; -else if (vma_is_special_huge (vma)) - supported_orders =THP_ORDERS_ALL_SPECIAL ; +else if (vma_is_dax (vma) ||vma_is_special_huge (vma)) + supported_orders =THP_ORDERS_ALL_SPECIAL_DAX ;else supported_orders =THP_ORDERS_ALL_FILE_DEFAULT ;@@ -2338,7 +2346,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm);arch_check_zapped_pmd (vma, orig_pmd);tlb_remove_pmd_tlb_entry (tlb, pmd, addr); -if (!vma_is_dax (vma) &&vma_is_special_huge (vma)) { +if (vma_is_special_huge (vma)) {if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl);@@ -2840,7 +2848,7 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, orig_pud =pudp_huge_get_and_clear_full (vma, addr, pud, tlb->fullmm);arch_check_zapped_pud (vma, orig_pud);tlb_remove_pud_tlb_entry (tlb, pud, addr); -if (!vma_is_dax (vma) &&vma_is_special_huge (vma)) { +if (vma_is_special_huge (vma)) {spin_unlock (ptl);/* No zero page support yet */ }else {@@ -2991,7 +2999,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,*/ if (arch_needs_pgtable_deposit ())zap_deposited_table (mm, pmd); -if (!vma_is_dax (vma) &&vma_is_special_huge (vma)) +if (vma_is_special_huge (vma))return ;if (unlikely (pmd_is_migration_entry (old_pmd))) {const softleaf_t old_entry =softleaf_from_pmd (old_pmd);@@ -4517,8 +4525,16 @@ static void split_huge_pages_all(void)static inline bool vma_not_suitable_for_thp_split (struct vm_area_struct *vma) { -return vma_is_special_huge (vma) || (vma->vm_flags &VM_IO ) || -is_vm_hugetlb_page (vma); +if (vma_is_dax (vma)) +return true; +if (vma_is_special_huge (vma)) +return true; +if (vma_test (vma,VMA_IO_BIT )) +return true; +if (is_vm_hugetlb_page (vma)) +return true; + +return false; } static int split_huge_pages_pid(int pid, unsigned long vaddr_start, --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 2/13 mm/huge: avoid big else branch in zap_huge_pmd()
In reply to Lorenzo Stoakes (Oracle)We don't need to have an extra level of indentation, we can simply exit early in the first two branches.
No functional change intended.
--- mm/huge_memory.c | 87 +++++++++++++++++++++++++----------------------- 1 file changed, 45 insertions(+), 42 deletions(-)
mm/huge_memory.c +45 -42
@@ -2328,8 +2328,10 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)int zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr) { -pmd_t orig_pmd; +struct folio *folio =NULL ; +int flush_needed =1 ;spinlock_t *ptl; +pmd_t orig_pmd;tlb_change_page_size (tlb,HPAGE_PMD_SIZE );
@@ -2350,59 +2352,60 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); - }else if (is_huge_zero_pmd (orig_pmd)) { +return 1 ; + } +if (is_huge_zero_pmd (orig_pmd)) {if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); - }else { -struct folio *folio =NULL ; -int flush_needed =1 ; +return 1 ; + } -if (pmd_present (orig_pmd)) { -struct page *page =pmd_page (orig_pmd); +if (pmd_present (orig_pmd)) { +struct page *page =pmd_page (orig_pmd); - folio =page_folio (page); -folio_remove_rmap_pmd (folio, page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 ); -VM_BUG_ON_PAGE (!PageHead (page), page); - }else if (pmd_is_valid_softleaf (orig_pmd)) { -const softleaf_t entry =softleaf_from_pmd (orig_pmd); + folio =page_folio (page); +folio_remove_rmap_pmd (folio, page, vma); +WARN_ON_ONCE (folio_mapcount (folio) <0 ); +VM_BUG_ON_PAGE (!PageHead (page), page); + }else if (pmd_is_valid_softleaf (orig_pmd)) { +const softleaf_t entry =softleaf_from_pmd (orig_pmd); - folio =softleaf_to_folio (entry); - flush_needed =0 ; + folio =softleaf_to_folio (entry); + flush_needed =0 ; -if (!thp_migration_supported ()) -WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); - } +if (!thp_migration_supported ()) +WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); + } -if (folio_test_anon (folio)) { +if (folio_test_anon (folio)) { +zap_deposited_table (tlb->mm, pmd); +add_mm_counter (tlb->mm,MM_ANONPAGES , -HPAGE_PMD_NR ); + }else { +if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd); -add_mm_counter (tlb->mm,MM_ANONPAGES , -HPAGE_PMD_NR ); - }else { -if (arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd); -add_mm_counter (tlb->mm,mm_counter_file (folio), - -HPAGE_PMD_NR ); - -/* -* Use flush_needed to indicate whether the PMD entry -* is present, instead of checking pmd_present() again. -*/ -if (flush_needed &&pmd_young (orig_pmd) && -likely (vma_has_recency (vma))) -folio_mark_accessed (folio); - } +add_mm_counter (tlb->mm,mm_counter_file (folio), + -HPAGE_PMD_NR ); -if (folio_is_device_private (folio)) { -folio_remove_rmap_pmd (folio, &folio->page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 ); -folio_put (folio); - } +/* +* Use flush_needed to indicate whether the PMD entry +* is present, instead of checking pmd_present() again. +*/ +if (flush_needed &&pmd_young (orig_pmd) && +likely (vma_has_recency (vma))) +folio_mark_accessed (folio); + } -spin_unlock (ptl); -if (flush_needed) -tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); +if (folio_is_device_private (folio)) { +folio_remove_rmap_pmd (folio, &folio->page, vma); +WARN_ON_ONCE (folio_mapcount (folio) <0 ); +folio_put (folio); } + +spin_unlock (ptl); +if (flush_needed) +tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); +return 1 ; }
Signature · 1 line
2.53.0
Re: [PATCH v3 02/13] mm/huge: avoid big else branch in zap_huge_pmd()
Quoted 9 lines · Lorenzo Stoakes (Oracle)
We don't need to have an extra level of indentation, we can simply exit early in the first two branches.
No functional change intended.
Reviewed-by Baolin Wang <[email protected]>Acked-by Qi Zheng <[email protected]>Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 125 lines
--- mm/huge_memory.c | 87 +++++++++++++++++++++++++----------------------- 1 file changed, 45 insertions(+), 42 deletions(-)mm/huge_memory.c +45
-43@@ -2328,8 +2328,10 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)int zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr) { -pmd_t orig_pmd; +struct folio *folio =NULL ; +int flush_needed =1 ;spinlock_t *ptl; +pmd_t orig_pmd;tlb_change_page_size (tlb,HPAGE_PMD_SIZE );@@ -2350,59 +2352,60 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); - }else if (is_huge_zero_pmd (orig_pmd)) { +return 1 ; + } +if (is_huge_zero_pmd (orig_pmd)) {if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); - }else { -struct folio *folio =NULL ; -int flush_needed =1 ; +return 1 ; + } -if (pmd_present (orig_pmd)) { -struct page *page =pmd_page (orig_pmd); +if (pmd_present (orig_pmd)) { +struct page *page =pmd_page (orig_pmd); - folio =page_folio (page); -folio_remove_rmap_pmd (folio, page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 ); -VM_BUG_ON_PAGE (!PageHead (page), page); - }else if (pmd_is_valid_softleaf (orig_pmd)) { -const softleaf_t entry =softleaf_from_pmd (orig_pmd); + folio =page_folio (page); +folio_remove_rmap_pmd (folio, page, vma); +WARN_ON_ONCE (folio_mapcount (folio) <0 ); +VM_BUG_ON_PAGE (!PageHead (page), page); + }else if (pmd_is_valid_softleaf (orig_pmd)) { +const softleaf_t entry =softleaf_from_pmd (orig_pmd); - folio =softleaf_to_folio (entry); - flush_needed =0 ; + folio =softleaf_to_folio (entry); + flush_needed =0 ; -if (!thp_migration_supported ()) -WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); - } +if (!thp_migration_supported ()) +WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); + } -if (folio_test_anon (folio)) { +if (folio_test_anon (folio)) { +zap_deposited_table (tlb->mm, pmd); +add_mm_counter (tlb->mm,MM_ANONPAGES , -HPAGE_PMD_NR ); + }else { +if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd); -add_mm_counter (tlb->mm,MM_ANONPAGES , -HPAGE_PMD_NR ); - }else { -if (arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd); -add_mm_counter (tlb->mm,mm_counter_file (folio), - -HPAGE_PMD_NR ); - -/* -* Use flush_needed to indicate whether the PMD entry -* is present, instead of checking pmd_present() again. -*/ -if (flush_needed &&pmd_young (orig_pmd) && -likely (vma_has_recency (vma))) -folio_mark_accessed (folio); - } +add_mm_counter (tlb->mm,mm_counter_file (folio), + -HPAGE_PMD_NR ); -if (folio_is_device_private (folio)) { -folio_remove_rmap_pmd (folio, &folio->page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 ); -folio_put (folio); - } +/* +* Use flush_needed to indicate whether the PMD entry +* is present, instead of checking pmd_present() again. +*/ +if (flush_needed &&pmd_young (orig_pmd) && +likely (vma_has_recency (vma))) +folio_mark_accessed (folio); + } -spin_unlock (ptl); -if (flush_needed) -tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); +if (folio_is_device_private (folio)) { +folio_remove_rmap_pmd (folio, &folio->page, vma); +WARN_ON_ONCE (folio_mapcount (folio) <0 ); +folio_put (folio); } + +spin_unlock (ptl); +if (flush_needed) +tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); +return 1 ; } --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 3/13 mm/huge_memory: have zap_huge_pmd return a boolean, add kdoc
In reply to Lorenzo Stoakes (Oracle)There's no need to use the ancient approach of returning an integer here, just return a boolean.
Also update flush_needed to be a boolean, similarly.
Also add a kdoc comment describing the function.
No functional change intended.
--- include/linux/huge_mm.h | 4 ++-- mm/huge_memory.c | 23 ++++++++++++++++------- 2 files changed, 18 insertions(+), 9 deletions(-)
include/linux/huge_mm.h +2 -2
@@ -27,8 +27,8 @@ static inline void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud)vm_fault_t do_huge_pmd_wp_page (struct vm_fault *vmf);bool madvise_free_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr,unsigned long next); -int zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd, -unsigned long addr); +bool zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd, +unsigned long addr);int zap_huge_pud (struct mmu_gather *tlb,struct vm_area_struct *vma,pud_t *pud,unsigned long addr); bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
mm/huge_memory.c +16 -7
@@ -2325,11 +2325,20 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)mm_dec_nr_ptes (mm); } -int zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma, +/** +* zap_huge_pmd - Zap a huge THP which is of PMD size. +* @tlb: The MMU gather TLB state associated with the operation. +* @vma: The VMA containing the range to zap. +* @pmd: A pointer to the leaf PMD entry. +* @addr: The virtual address for the range to zap. +* +* Returns: %true on success, %false otherwise. +*/ +bool zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr) {struct folio *folio =NULL ; -int flush_needed =1 ; +bool flush_needed = true;spinlock_t *ptl;pmd_t orig_pmd;
@@ -2337,7 +2346,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, ptl =__pmd_trans_huge_lock (pmd, vma);if (!ptl) -return 0 ; +return false;/* * For architectures like ppc64 we look at deposited pgtable * when calling pmdp_huge_get_and_clear. So do the
@@ -2352,13 +2361,13 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); -return 1 ; +return true; }if (is_huge_zero_pmd (orig_pmd)) {if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); -return 1 ; +return true; }if (pmd_present (orig_pmd)) {
@@ -2372,7 +2381,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,const softleaf_t entry =softleaf_from_pmd (orig_pmd); folio =softleaf_to_folio (entry); - flush_needed =0 ; + flush_needed = false;if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" );
@@ -2406,7 +2415,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (flush_needed)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); -return 1 ; +return true; }#ifndef pmd_move_must_withdraw
Signature · 1 line
2.53.0
Re: [PATCH v3 03/13] mm/huge_memory: have zap_huge_pmd return a boolean, add kdoc
Quoted 6 lines · Lorenzo Stoakes (Oracle)
There's no need to use the ancient approach of returning an integer here, just return a boolean.
Also update
flush_neededto be a boolean, similarly.
Quoted 7 lines
Also add a kdoc comment describing the function.
No functional change intended.
Reviewed-by Baolin Wang <[email protected]>Acked-by Qi Zheng <[email protected]>Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 93 lines
--- include/linux/huge_mm.h | 4 ++-- mm/huge_memory.c | 23 ++++++++++++++++------- 2 files changed, 18 insertions(+), 9 deletions(-)include/linux/huge_mm.h +2
-2@@ -27,8 +27,8 @@ static inline void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud)vm_fault_t do_huge_pmd_wp_page (struct vm_fault *vmf);bool madvise_free_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr,unsigned long next); -int zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd, -unsigned long addr); +bool zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd, +unsigned long addr);int zap_huge_pud (struct mmu_gather *tlb,struct vm_area_struct *vma,pud_t *pud,unsigned long addr); bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,mm/huge_memory.c +16
-8@@ -2325,11 +2325,20 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)mm_dec_nr_ptes (mm); } -int zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma, +/** +* zap_huge_pmd - Zap a huge THP which is of PMD size. +* @tlb: The MMU gather TLB state associated with the operation. +* @vma: The VMA containing the range to zap. +* @pmd: A pointer to the leaf PMD entry. +* @addr: The virtual address for the range to zap. +* +* Returns: %true on success, %false otherwise. +*/ +bool zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr) {struct folio *folio =NULL ; -int flush_needed =1 ; +bool flush_needed = true;spinlock_t *ptl;pmd_t orig_pmd;@@ -2337,7 +2346,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, ptl =__pmd_trans_huge_lock (pmd, vma);if (!ptl) -return 0 ; +return false;/* * For architectures like ppc64 we look at deposited pgtable * when calling pmdp_huge_get_and_clear. So do the @@ -2352,13 +2361,13 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); -return 1 ; +return true; }if (is_huge_zero_pmd (orig_pmd)) {if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd);spin_unlock (ptl); -return 1 ; +return true; }if (pmd_present (orig_pmd)) {@@ -2372,7 +2381,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,const softleaf_t entry =softleaf_from_pmd (orig_pmd); folio =softleaf_to_folio (entry); - flush_needed =0 ; + flush_needed = false;if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" );@@ -2406,7 +2415,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (flush_needed)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); -return 1 ; +return true; }#ifndef pmd_move_must_withdraw --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 4/13 mm/huge_memory: handle buggy PMD entry in zap_huge_pmd()
In reply to Lorenzo Stoakes (Oracle)A recent bug I analysed managed to, through a bug in the userfaultfd implementation, reach an invalid point in the zap_huge_pmd() code where the PMD was none of:
- A non-DAX, PFN or mixed map.
- The huge zero folio
- A present PMD entry
- A softleaf entry
The code at this point calls folio_test_anon() on a known-NULL folio. Having logic like this explicitly NULL dereference in the code is hard to understand, and makes debugging potentially more difficult.
Add an else branch to handle this case and WARN().
No functional change intended.
--- mm/huge_memory.c | 4 ++++ 1 file changed, 4 insertions(+)
mm/huge_memory.c +4 -0
@@ -2385,6 +2385,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); + }else { +WARN_ON_ONCE (true); +spin_unlock (ptl); +return true; }if (folio_test_anon (folio)) {
Signature · 1 line
2.53.0
Re: [PATCH v3 04/13] mm/huge_memory: handle buggy PMD entry in zap_huge_pmd()
Quoted 14 lines · Lorenzo Stoakes (Oracle)
A recent bug I analysed managed to, through a bug in the userfaultfd implementation, reach an invalid point in the
zap_huge_pmd()code where the PMD was none of:- A non-DAX, PFN or mixed map.
- The huge zero folio
- A present PMD entry
- A softleaf entryThe code at this point calls
folio_test_anon()on a known-NULL folio. Having logic like this explicitly NULL dereference in the code is hard to understand, and makes debugging potentially more difficult.
Quoted 7 lines
Add an else branch to handle this case and
WARN().No functional change intended.
Overall LGTM, just a question below.
Quoted 9 lines
--- mm/huge_memory.c | 4 ++++ 1 file changed, 4 insertions(+)mm/huge_memory.c +0
-0
mm/huge_memory.c bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 2391
Quoted 7 lines
mm/huge_memory.c +4
-0@@ -2385,6 +2385,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); + }else { +WARN_ON_ONCE (true); +spin_unlock (ptl); +return true;
Apologies if this was already discussed in earlier versions but why do we return "true" for this case which would be interpreted as "success"? Perhaps because we still managed to do tlb_remove_pmd_tlb_entry()?
Quoted 6 lines
mm/huge_memory.c +0
-1}if (folio_test_anon (folio)) { --2.53.0
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 04/13] mm/huge_memory: handle buggy PMD entry in zap_huge_pmd()
Quoted 20 lines · Suren Baghdasaryan
Quoted 18 lines · Lorenzo Stoakes (Oracle)
A recent bug I analysed managed to, through a bug in the userfaultfd implementation, reach an invalid point in the
zap_huge_pmd()code where the PMD was none of:- A non-DAX, PFN or mixed map.
- The huge zero folio
- A present PMD entry
- A softleaf entryThe code at this point calls
folio_test_anon()on a known-NULL folio. Having logic like this explicitly NULL dereference in the code is hard to understand, and makes debugging potentially more difficult.Add an else branch to handle this case and
WARN().No functional change intended.
Quoted 7 lines
Quoted 3 lines
Overall LGTM, just a question below.
Reviewed-by Suren Baghdasaryan <[email protected]>
Thanks!
Quoted 11 lines
Quoted 11 lines
--- mm/huge_memory.c | 4 ++++ 1 file changed, 4 insertions(+)mm/huge_memory.c +0
-0@@ -2385,6 +2385,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
Quoted 11 lines
mm/huge_memory.c bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 2391
Quoted 6 lines
mm/huge_memory.c +4
-0if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); + }else { +WARN_ON_ONCE (true); +spin_unlock (ptl); +return true;Apologies if this was already discussed in earlier versions but why do we return "true" for this case which would be interpreted as "success"? Perhaps because we still managed to do
tlb_remove_pmd_tlb_entry()?
If we return false it can result in a potential loop in the caller I believe. Basically the caller won't handle this case properly. It was raised on review, previously I was returning false :)
Quoted 7 lines
Quoted 7 lines
mm/huge_memory.c +0
-1}if (folio_test_anon (folio)) { --2.53.0
Cheers, Lorenzo
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 5/13 mm/huge_memory: add a common exit path to zap_huge_pmd()
In reply to Lorenzo Stoakes (Oracle)Other than when we acquire the PTL, we always need to unlock the PTL, and optionally need to flush on exit.
The code is currently very duplicated in this respect, so default flush_needed to false, set it true in the case in which it's required, then share the same logic for all exit paths.
This also makes flush_needed make more sense as a function-scope value (we don't need to flush for the PFN map/mixed map, zero huge, error cases for instance).
--- mm/huge_memory.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)
mm/huge_memory.c +6 -9
@@ -2338,7 +2338,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) {struct folio *folio =NULL ; -bool flush_needed = true; +bool flush_needed = false;spinlock_t *ptl;pmd_t orig_pmd;
@@ -2360,19 +2360,18 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (vma_is_special_huge (vma)) {if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd); -spin_unlock (ptl); -return true; + goto out; }if (is_huge_zero_pmd (orig_pmd)) {if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd); -spin_unlock (ptl); -return true; + goto out; }if (pmd_present (orig_pmd)) {struct page *page =pmd_page (orig_pmd); + flush_needed = true; folio =page_folio (page);folio_remove_rmap_pmd (folio, page, vma);WARN_ON_ONCE (folio_mapcount (folio) <0 );
@@ -2381,14 +2380,12 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,const softleaf_t entry =softleaf_from_pmd (orig_pmd); folio =softleaf_to_folio (entry); - flush_needed = false;if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); }else {WARN_ON_ONCE (true); -spin_unlock (ptl); -return true; + goto out; }if (folio_test_anon (folio)) {
@@ -2415,10 +2412,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,folio_put (folio); } +out:spin_unlock (ptl);if (flush_needed)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); -return true; }
Signature · 1 line
2.53.0
Re: [PATCH v3 05/13] mm/huge_memory: add a common exit path to zap_huge_pmd()
Quoted 8 lines · Lorenzo Stoakes (Oracle)
Other than when we acquire the PTL, we always need to unlock the PTL, and optionally need to flush on exit.
The code is currently very duplicated in this respect, so default
flush_neededto false, set it true in the case in which it's required, then share the same logic for all exit paths.
Quoted 6 lines
This also makes
flush_neededmake more sense as a function-scope value (we don't need to flush for the PFN map/mixed map, zero huge, error cases for instance).Reviewed-by Baolin Wang <[email protected]>Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 71 lines
--- mm/huge_memory.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)mm/huge_memory.c +6
-10@@ -2338,7 +2338,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) {struct folio *folio =NULL ; -bool flush_needed = true; +bool flush_needed = false;spinlock_t *ptl;pmd_t orig_pmd;@@ -2360,19 +2360,18 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (vma_is_special_huge (vma)) {if (arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd); -spin_unlock (ptl); -return true; + goto out; }if (is_huge_zero_pmd (orig_pmd)) {if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ())zap_deposited_table (tlb->mm, pmd); -spin_unlock (ptl); -return true; + goto out; }if (pmd_present (orig_pmd)) {struct page *page =pmd_page (orig_pmd); + flush_needed = true; folio =page_folio (page);folio_remove_rmap_pmd (folio, page, vma);WARN_ON_ONCE (folio_mapcount (folio) <0 );@@ -2381,14 +2380,12 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,const softleaf_t entry =softleaf_from_pmd (orig_pmd); folio =softleaf_to_folio (entry); - flush_needed = false;if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); }else {WARN_ON_ONCE (true); -spin_unlock (ptl); -return true; + goto out; }if (folio_test_anon (folio)) {@@ -2415,10 +2412,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,folio_put (folio); } +out:spin_unlock (ptl);if (flush_needed)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); -return true; } --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 6/13 mm/huge_memory: remove unnecessary VM_BUG_ON_PAGE()
In reply to Lorenzo Stoakes (Oracle)This has been around since the beginnings of the THP implementation. I think we can safely assume that, if we have a THP folio, it will have a head page.
--- mm/huge_memory.c | 1 - 1 file changed, 1 deletion(-)
mm/huge_memory.c +0 -1
@@ -2375,7 +2375,6 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, folio =page_folio (page);folio_remove_rmap_pmd (folio, page, vma);WARN_ON_ONCE (folio_mapcount (folio) <0 ); -VM_BUG_ON_PAGE (!PageHead (page), page); }else if (pmd_is_valid_softleaf (orig_pmd)) {const softleaf_t entry =softleaf_from_pmd (orig_pmd);
Signature · 1 line
2.53.0
Re: [PATCH v3 06/13] mm/huge_memory: remove unnecessary VM_BUG_ON_PAGE()
Quoted 7 lines · Lorenzo Stoakes (Oracle)
This has been around since the beginnings of the THP implementation. I think we can safely assume that, if we have a THP folio, it will have a head page.
Reviewed-by Baolin Wang <[email protected]>Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 19 lines
--- mm/huge_memory.c | 1 - 1 file changed, 1 deletion(-)mm/huge_memory.c +0
-2@@ -2375,7 +2375,6 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, folio =page_folio (page);folio_remove_rmap_pmd (folio, page, vma);WARN_ON_ONCE (folio_mapcount (folio) <0 ); -VM_BUG_ON_PAGE (!PageHead (page), page); }else if (pmd_is_valid_softleaf (orig_pmd)) {const softleaf_t entry =softleaf_from_pmd (orig_pmd); --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 7/13 mm/huge_memory: deduplicate zap deposited table call
In reply to Lorenzo Stoakes (Oracle)Rather than having separate logic for each case determining whether to zap the deposited table, simply track this via a boolean.
We default this to whether the architecture requires it, and update it as required elsewhere.
--- mm/huge_memory.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-)
mm/huge_memory.c +10 -12
@@ -2337,6 +2337,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)bool zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr) { +bool has_deposit =arch_needs_pgtable_deposit ();struct folio *folio =NULL ;bool flush_needed = false;spinlock_t *ptl;
@@ -2357,23 +2358,19 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm);arch_check_zapped_pmd (vma, orig_pmd);tlb_remove_pmd_tlb_entry (tlb, pmd, addr); -if (vma_is_special_huge (vma)) { -if (arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd); +if (vma_is_special_huge (vma)) goto out; - }if (is_huge_zero_pmd (orig_pmd)) { -if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd); +if (!vma_is_dax (vma)) + has_deposit = true; goto out; }if (pmd_present (orig_pmd)) { -struct page *page =pmd_page (orig_pmd); + folio =pmd_folio (orig_pmd); flush_needed = true; - folio =page_folio (page); -folio_remove_rmap_pmd (folio, page, vma); +folio_remove_rmap_pmd (folio, &folio->page, vma);WARN_ON_ONCE (folio_mapcount (folio) <0 ); }else if (pmd_is_valid_softleaf (orig_pmd)) {const softleaf_t entry =softleaf_from_pmd (orig_pmd);
@@ -2388,11 +2385,9 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, }if (folio_test_anon (folio)) { -zap_deposited_table (tlb->mm, pmd); + has_deposit = true;add_mm_counter (tlb->mm,MM_ANONPAGES , -HPAGE_PMD_NR ); }else { -if (arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd);add_mm_counter (tlb->mm,mm_counter_file (folio), -HPAGE_PMD_NR );
@@ -2412,6 +2407,9 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, } out: +if (has_deposit) +zap_deposited_table (tlb->mm, pmd); +spin_unlock (ptl);if (flush_needed)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE );
Signature · 1 line
2.53.0
Re: [PATCH v3 07/13] mm/huge_memory: deduplicate zap deposited table call
Quoted 8 lines · Lorenzo Stoakes (Oracle)
Rather than having separate logic for each case determining whether to zap the deposited table, simply track this via a boolean.
We default this to whether the architecture requires it, and update it as required elsewhere.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>---
Thanks. The logic is much clearer now.
Re: [PATCH v3 07/13] mm/huge_memory: deduplicate zap deposited table call
In reply to Lorenzo Stoakes (Oracle)Quoted 8 lines · Lorenzo Stoakes (Oracle)
Rather than having separate logic for each case determining whether to zap the deposited table, simply track this via a boolean.
We default this to whether the architecture requires it, and update it as required elsewhere.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 71 lines
--- mm/huge_memory.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-)mm/huge_memory.c +10
-13@@ -2337,6 +2337,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)bool zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr) { +bool has_deposit =arch_needs_pgtable_deposit ();struct folio *folio =NULL ;bool flush_needed = false;spinlock_t *ptl;@@ -2357,23 +2358,19 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm);arch_check_zapped_pmd (vma, orig_pmd);tlb_remove_pmd_tlb_entry (tlb, pmd, addr); -if (vma_is_special_huge (vma)) { -if (arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd); +if (vma_is_special_huge (vma)) goto out; - }if (is_huge_zero_pmd (orig_pmd)) { -if (!vma_is_dax (vma) ||arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd); +if (!vma_is_dax (vma)) + has_deposit = true; goto out; }if (pmd_present (orig_pmd)) { -struct page *page =pmd_page (orig_pmd); + folio =pmd_folio (orig_pmd); flush_needed = true; - folio =page_folio (page); -folio_remove_rmap_pmd (folio, page, vma); +folio_remove_rmap_pmd (folio, &folio->page, vma);WARN_ON_ONCE (folio_mapcount (folio) <0 ); }else if (pmd_is_valid_softleaf (orig_pmd)) {const softleaf_t entry =softleaf_from_pmd (orig_pmd);@@ -2388,11 +2385,9 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, }if (folio_test_anon (folio)) { -zap_deposited_table (tlb->mm, pmd); + has_deposit = true;add_mm_counter (tlb->mm,MM_ANONPAGES , -HPAGE_PMD_NR ); }else { -if (arch_needs_pgtable_deposit ()) -zap_deposited_table (tlb->mm, pmd);add_mm_counter (tlb->mm,mm_counter_file (folio), -HPAGE_PMD_NR );@@ -2412,6 +2407,9 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, } out: +if (has_deposit) +zap_deposited_table (tlb->mm, pmd); +spin_unlock (ptl);if (flush_needed)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE ); --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 8/13 mm/huge_memory: remove unnecessary sanity checks
In reply to Lorenzo Stoakes (Oracle)These checks have been in place since 2014, I think we can safely assume that we are in a place where we don't need these as runtime checks.
In addition there are 4 other invocations of folio_remove_rmap_pmd(), none of which make this assertion.
If we need to add this assertion, it should be in folio_remove_rmap_pmd(), and as a VM_WARN_ON_ONCE(), however these seem superfluous so just remove them.
--- mm/huge_memory.c | 2 -- 1 file changed, 2 deletions(-)
mm/huge_memory.c +0 -2
@@ -2371,7 +2371,6 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, flush_needed = true;folio_remove_rmap_pmd (folio, &folio->page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 ); }else if (pmd_is_valid_softleaf (orig_pmd)) {const softleaf_t entry =softleaf_from_pmd (orig_pmd);
@@ -2402,7 +2401,6 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (folio_is_device_private (folio)) {folio_remove_rmap_pmd (folio, &folio->page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 );folio_put (folio); }
Signature · 1 line
2.53.0
Re: [PATCH v3 08/13] mm/huge_memory: remove unnecessary sanity checks
Quoted 4 lines · Lorenzo Stoakes (Oracle)
These checks have been in place since 2014, I think we can safely assume that we are in a place where we don't need these as runtime checks.
Quoted 8 lines
In addition there are 4 other invocations of
folio_remove_rmap_pmd(), none of which make this assertion.If we need to add this assertion, it should be in
folio_remove_rmap_pmd(), and as aVM_WARN_ON_ONCE(), however these seem superfluous so just remove them.Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 27 lines
--- mm/huge_memory.c | 2 -- 1 file changed, 2 deletions(-)mm/huge_memory.c +0
-3@@ -2371,7 +2371,6 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, flush_needed = true;folio_remove_rmap_pmd (folio, &folio->page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 ); }else if (pmd_is_valid_softleaf (orig_pmd)) {const softleaf_t entry =softleaf_from_pmd (orig_pmd);@@ -2402,7 +2401,6 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (folio_is_device_private (folio)) {folio_remove_rmap_pmd (folio, &folio->page, vma); -WARN_ON_ONCE (folio_mapcount (folio) <0 );folio_put (folio); } --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 9/13 mm/huge_memory: use mm instead of tlb->mm
In reply to Lorenzo Stoakes (Oracle)Reduce the repetition, and lay the ground for further refactorings by keeping this variable separate.
--- mm/huge_memory.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
mm/huge_memory.c +4 -3
@@ -2338,6 +2338,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { bool has_deposit = arch_needs_pgtable_deposit(); +struct mm_struct *mm = tlb->mm;struct folio *folio =NULL ;bool flush_needed = false;spinlock_t *ptl;
@@ -2385,9 +2386,9 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (folio_test_anon (folio)) { has_deposit = true; -add_mm_counter (tlb->mm,MM_ANONPAGES , -HPAGE_PMD_NR ); +add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); }else { -add_mm_counter (tlb->mm,mm_counter_file (folio), +add_mm_counter (mm,mm_counter_file (folio), -HPAGE_PMD_NR );/*
@@ -2406,7 +2407,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, out:if (has_deposit) -zap_deposited_table (tlb->mm, pmd); +zap_deposited_table (mm, pmd);spin_unlock (ptl);if (flush_needed)
Signature · 1 line
2.53.0
Re: [PATCH v3 09/13] mm/huge_memory: use mm instead of tlb->mm
Quoted 5 lines · Lorenzo Stoakes (Oracle)
Reduce the repetition, and lay the ground for further refactorings by keeping this variable separate.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>---
LGTM.
Re: [PATCH v3 09/13] mm/huge_memory: use mm instead of tlb->mm
Quoted 4 lines · Baolin Wang
On 3/21/26 2:07 AM, Lorenzo Stoakes (Oracle) wrote:
Quoted 8 lines
Quoted 5 lines
Reduce the repetition, and lay the ground for further refactorings by keeping this variable separate.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>---
LGTM.
Reviewed-by Baolin Wang <[email protected]>
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 10/13 mm/huge_memory: separate out the folio part of zap_huge_pmd()
In reply to Lorenzo Stoakes (Oracle)Place the part of the logic that manipulates counters and possibly updates the accessed bit of the folio into its own function to make zap_huge_pmd() more readable.
Also rename flush_needed to is_present as we only require a flush for present entries.
Additionally add comments as to why we're doing what we're doing with respect to softleaf entries.
This also lays the ground for further refactoring.
--- mm/huge_memory.c | 61 +++++++++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 26 deletions(-)
mm/huge_memory.c +35 -26
@@ -2325,6 +2325,37 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)mm_dec_nr_ptes (mm); } +static void zap_huge_pmd_folio (struct mm_struct *mm,struct vm_area_struct *vma, +pmd_t pmdval,struct folio *folio,bool is_present, +bool *has_deposit) +{ +const bool is_device_private =folio_is_device_private (folio); + +/* Present and device private folios are rmappable. */ +if (is_present || is_device_private) +folio_remove_rmap_pmd (folio, &folio->page, vma); + +if (folio_test_anon (folio)) { + *has_deposit = true; +add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); + }else { +add_mm_counter (mm,mm_counter_file (folio), + -HPAGE_PMD_NR ); + +/* +* Use flush_needed to indicate whether the PMD entry +* is present, instead of checking pmd_present() again. +*/ +if (is_present &&pmd_young (pmdval) && +likely (vma_has_recency (vma))) +folio_mark_accessed (folio); + } + +/* Device private folios are pinned. */ +if (is_device_private) +folio_put (folio); +} +/** * zap_huge_pmd - Zap a huge THP which is of PMD size. * @tlb: The MMU gather TLB state associated with the operation.
@@ -2340,7 +2371,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,bool has_deposit =arch_needs_pgtable_deposit ();struct mm_struct *mm = tlb->mm;struct folio *folio =NULL ; -bool flush_needed = false; +bool is_present = false;spinlock_t *ptl;pmd_t orig_pmd;
@@ -2369,14 +2400,11 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,if (pmd_present (orig_pmd)) { folio =pmd_folio (orig_pmd); - - flush_needed = true; -folio_remove_rmap_pmd (folio, &folio->page, vma); + is_present = true; }else if (pmd_is_valid_softleaf (orig_pmd)) {const softleaf_t entry =softleaf_from_pmd (orig_pmd); folio =softleaf_to_folio (entry); -if (!thp_migration_supported ())WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); }else {
@@ -2384,33 +2412,14 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out; } -if (folio_test_anon (folio)) { - has_deposit = true; -add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); - }else { -add_mm_counter (mm,mm_counter_file (folio), - -HPAGE_PMD_NR ); - -/* -* Use flush_needed to indicate whether the PMD entry -* is present, instead of checking pmd_present() again. -*/ -if (flush_needed &&pmd_young (orig_pmd) && -likely (vma_has_recency (vma))) -folio_mark_accessed (folio); - } - -if (folio_is_device_private (folio)) { -folio_remove_rmap_pmd (folio, &folio->page, vma); -folio_put (folio); - } +zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, &has_deposit); out:if (has_deposit)zap_deposited_table (mm, pmd);spin_unlock (ptl); -if (flush_needed) +if (is_present)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE );return true; }
Signature · 1 line
2.53.0
Re: [PATCH v3 10/13] mm/huge_memory: separate out the folio part of zap_huge_pmd()
Quoted 39 lines · Lorenzo Stoakes (Oracle)
Place the part of the logic that manipulates counters and possibly updates the accessed bit of the folio into its own function to make
zap_huge_pmd()more readable.Also rename
flush_neededtois_presentas we only require a flush for present entries.Additionally add comments as to why we're doing what we're doing with respect to softleaf entries.
This also lays the ground for further refactoring.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>--- mm/huge_memory.c | 61 +++++++++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 26 deletions(-)mm/huge_memory.c +14
-0@@ -2325,6 +2325,37 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)mm_dec_nr_ptes (mm); } +static void zap_huge_pmd_folio (struct mm_struct *mm,struct vm_area_struct *vma, +pmd_t pmdval,struct folio *folio,bool is_present, +bool *has_deposit) +{ +const bool is_device_private =folio_is_device_private (folio); + +/* Present and device private folios are rmappable. */ +if (is_present || is_device_private) +folio_remove_rmap_pmd (folio, &folio->page, vma); + +if (folio_test_anon (folio)) { + *has_deposit = true; +add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); + }else {
mm/huge_memory.c static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) · line 2351
Quoted 10 lines
mm/huge_memory.c +10
-0+add_mm_counter (mm,mm_counter_file (folio), + -HPAGE_PMD_NR ); + +/* +* Use flush_needed to indicate whether the PMD entry +* is present, instead of checking pmd_present() again. +*/ +if (is_present &&pmd_young (pmdval) && +likely (vma_has_recency (vma))) +folio_mark_accessed (folio);
Nit: these comments were added by me to explain why 'flush_needed' was used:). Since it has been renamed to the more readable 'is_present', these comments are now redundant and can be removed.
With that,
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 10/13] mm/huge_memory: separate out the folio part of zap_huge_pmd()
Quoted 47 lines · Baolin Wang
Quoted 44 lines · Lorenzo Stoakes (Oracle)
Place the part of the logic that manipulates counters and possibly updates the accessed bit of the folio into its own function to make
zap_huge_pmd()more readable.Also rename
flush_neededtois_presentas we only require a flush for present entries.Additionally add comments as to why we're doing what we're doing with respect to softleaf entries.
This also lays the ground for further refactoring.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>--- mm/huge_memory.c | 61 +++++++++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 26 deletions(-)mm/huge_memory.c +20
-0@@ -2325,6 +2325,37 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)mm_dec_nr_ptes (mm); } +static void zap_huge_pmd_folio (struct mm_struct *mm,struct vm_area_struct *vma, +pmd_t pmdval,struct folio *folio,bool is_present, +bool *has_deposit) +{ +const bool is_device_private =folio_is_device_private (folio); + +/* Present and device private folios are rmappable. */ +if (is_present || is_device_private) +folio_remove_rmap_pmd (folio, &folio->page, vma); + +if (folio_test_anon (folio)) { + *has_deposit = true; +add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); + }else { +add_mm_counter (mm,mm_counter_file (folio), + -HPAGE_PMD_NR ); + +/* +* Use flush_needed to indicate whether the PMD entry +* is present, instead of checking pmd_present() again.
Quoted 8 lines
mm/huge_memory.c static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) · line 2351
Quoted 4 lines
mm/huge_memory.c +4
-0+*/ +if (is_present &&pmd_young (pmdval) && +likely (vma_has_recency (vma))) +folio_mark_accessed (folio);Nit: these comments were added by me to explain why '
flush_needed' was used:). Since it has been renamed to the more readable 'is_present', these comments are now redundant and can be removed.
Ack, I think it's _probably_ ok to leave that as a later commit removes it anyway, if that works for you?
Quoted 3 lines
With that,
Reviewed-by Baolin Wang <[email protected]>
Thanks!
Cheers, Lorenzo
Baolin Wang
reviewerRe: [PATCH v3 10/13] mm/huge_memory: separate out the folio part of zap_huge_pmd()
Quoted 57 lines · Lorenzo Stoakes (Oracle)
Quoted 47 lines · Baolin Wang
Quoted 44 lines · Lorenzo Stoakes (Oracle)
Place the part of the logic that manipulates counters and possibly updates the accessed bit of the folio into its own function to make
zap_huge_pmd()more readable.Also rename
flush_neededtois_presentas we only require a flush for present entries.Additionally add comments as to why we're doing what we're doing with respect to softleaf entries.
This also lays the ground for further refactoring.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>--- mm/huge_memory.c | 61 +++++++++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 26 deletions(-)mm/huge_memory.c +20
-0@@ -2325,6 +2325,37 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)mm_dec_nr_ptes (mm); } +static void zap_huge_pmd_folio (struct mm_struct *mm,struct vm_area_struct *vma, +pmd_t pmdval,struct folio *folio,bool is_present, +bool *has_deposit) +{ +const bool is_device_private =folio_is_device_private (folio); + +/* Present and device private folios are rmappable. */ +if (is_present || is_device_private) +folio_remove_rmap_pmd (folio, &folio->page, vma); + +if (folio_test_anon (folio)) { + *has_deposit = true; +add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); + }else { +add_mm_counter (mm,mm_counter_file (folio), + -HPAGE_PMD_NR ); + +/* +* Use flush_needed to indicate whether the PMD entry +* is present, instead of checking pmd_present() again. Quoted 8 lines
mm/huge_memory.c static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) · line 2351
Quoted 4 lines
mm/huge_memory.c +4
-0+*/ +if (is_present &&pmd_young (pmdval) && +likely (vma_has_recency (vma))) +folio_mark_accessed (folio);Nit: these comments were added by me to explain why '
flush_needed' was used:). Since it has been renamed to the more readable 'is_present', these comments are now redundant and can be removed.
Quoted 2 lines
Ack, I think it's _probably_ ok to leave that as a later commit removes it anyway, if that works for you?
Ah, I saw you removed them in the following patch. Looks fine to me. Thanks.
Re: [PATCH v3 10/13] mm/huge_memory: separate out the folio part of zap_huge_pmd()
In reply to Baolin WangQuoted 58 lines · Baolin Wang
Quoted 39 lines · Lorenzo Stoakes (Oracle)
Place the part of the logic that manipulates counters and possibly updates the accessed bit of the folio into its own function to make
zap_huge_pmd()more readable.Also rename
flush_neededtois_presentas we only require a flush for present entries.Additionally add comments as to why we're doing what we're doing with respect to softleaf entries.
This also lays the ground for further refactoring.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>--- mm/huge_memory.c | 61 +++++++++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 26 deletions(-)mm/huge_memory.c +14
-0@@ -2325,6 +2325,37 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)mm_dec_nr_ptes (mm); } +static void zap_huge_pmd_folio (struct mm_struct *mm,struct vm_area_struct *vma, +pmd_t pmdval,struct folio *folio,bool is_present, +bool *has_deposit) +{ +const bool is_device_private =folio_is_device_private (folio); + +/* Present and device private folios are rmappable. */ +if (is_present || is_device_private) +folio_remove_rmap_pmd (folio, &folio->page, vma); + +if (folio_test_anon (folio)) { + *has_deposit = true; +add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); + }else {mm/huge_memory.c static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) · line 2351
Quoted 10 lines
mm/huge_memory.c +10
-0+add_mm_counter (mm,mm_counter_file (folio), + -HPAGE_PMD_NR ); + +/* +* Use flush_needed to indicate whether the PMD entry +* is present, instead of checking pmd_present() again. +*/ +if (is_present &&pmd_young (pmdval) && +likely (vma_has_recency (vma))) +folio_mark_accessed (folio);Nit: these comments were added by me to explain why '
flush_needed' was used:). Since it has been renamed to the more readable 'is_present', these comments are now redundant and can be removed.
Quoted 2 lines
With that,
Reviewed-by Baolin Wang <[email protected]>
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 11/13 mm: add softleaf_is_valid_pmd_entry(), pmd_to_softleaf_folio()
In reply to Lorenzo Stoakes (Oracle)Separate pmd_is_valid_softleaf() into separate components, then use the pmd_is_valid_softleaf() predicate to implement pmd_to_softleaf_folio().
This returns the folio associated with a softleaf entry at PMD level. It expects this to be valid for a PMD entry.
If CONFIG_DEBUG_VM is set, then assert on this being an invalid entry, and either way return NULL in this case.
This lays the ground for further refactorings.
--- include/linux/leafops.h | 39 +++++++++++++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 4 deletions(-)
include/linux/leafops.h +35 -4
@@ -603,7 +603,20 @@ static inline bool pmd_is_migration_entry(pmd_t pmd) }/** -* pmd_is_valid_softleaf() - Is this PMD entry a valid leaf entry? +* softleaf_is_valid_pmd_entry() - Is the specified softleaf entry obtained from +* a PMD one that we support at PMD level? +* @entry: Entry to check. +* Returns: true if the softleaf entry is valid at PMD, otherwise false. +*/ +static inline bool softleaf_is_valid_pmd_entry (softleaf_t entry) +{ +/* Only device private, migration entries valid for PMD. */ +return softleaf_is_device_private (entry) || +softleaf_is_migration (entry); +} + +/** +* pmd_is_valid_softleaf() - Is this PMD entry a valid softleaf entry? * @pmd: PMD entry. * * PMD leaf entries are valid only if they are device private or migration
@@ -616,9 +629,27 @@ static inline bool pmd_is_valid_softleaf(pmd_t pmd) {const softleaf_t entry =softleaf_from_pmd (pmd); -/* Only device private, migration entries valid for PMD. */ -return softleaf_is_device_private (entry) || -softleaf_is_migration (entry); +return softleaf_is_valid_pmd_entry (entry); +} + +/** +* pmd_to_softleaf_folio() - Convert the PMD entry to a folio. +* @pmd: PMD entry. +* +* The PMD entry is expected to be a valid PMD softleaf entry. +* +* Returns: the folio the softleaf entry references if this is a valid softleaf +* entry, otherwise NULL. +*/ +static inline struct folio *pmd_to_softleaf_folio (pmd_t pmd) +{ +const softleaf_t entry =softleaf_from_pmd (pmd); + +if (!softleaf_is_valid_pmd_entry (entry)) { +VM_WARN_ON_ONCE (true); +return NULL ; + } +return softleaf_to_folio (entry); }#endif /* CONFIG_MMU */
Signature · 1 line
2.53.0
Re: [PATCH v3 11/13] mm: add softleaf_is_valid_pmd_entry(), pmd_to_softleaf_folio()
Quoted 3 lines · Lorenzo Stoakes (Oracle)
Separate
pmd_is_valid_softleaf()into separate components, then use thepmd_is_valid_softleaf()predicate to implementpmd_to_softleaf_folio().
nit: I strongly suspect we'll find out why you need pmd_to_softleaf_folio() in one of the later patches. Not a big deal but it might be better to introduce it in the patch where you use it. Anyway, this is minor and not worth refactoring.
Quoted 10 lines
This returns the folio associated with a softleaf entry at PMD level. It expects this to be valid for a PMD entry.
If
CONFIG_DEBUG_VMis set, then assert on this being an invalid entry, and either way return NULL in this case.This lays the ground for further refactorings.
Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
Quoted 64 lines
--- include/linux/leafops.h | 39 +++++++++++++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 4 deletions(-)include/linux/leafops.h +35
-5@@ -603,7 +603,20 @@ static inline bool pmd_is_migration_entry(pmd_t pmd) }/** -* pmd_is_valid_softleaf() - Is this PMD entry a valid leaf entry? +* softleaf_is_valid_pmd_entry() - Is the specified softleaf entry obtained from +* a PMD one that we support at PMD level? +* @entry: Entry to check. +* Returns: true if the softleaf entry is valid at PMD, otherwise false. +*/ +static inline bool softleaf_is_valid_pmd_entry (softleaf_t entry) +{ +/* Only device private, migration entries valid for PMD. */ +return softleaf_is_device_private (entry) || +softleaf_is_migration (entry); +} + +/** +* pmd_is_valid_softleaf() - Is this PMD entry a valid softleaf entry? * @pmd: PMD entry. * * PMD leaf entries are valid only if they are device private or migration @@ -616,9 +629,27 @@ static inline bool pmd_is_valid_softleaf(pmd_t pmd) {const softleaf_t entry =softleaf_from_pmd (pmd); -/* Only device private, migration entries valid for PMD. */ -return softleaf_is_device_private (entry) || -softleaf_is_migration (entry); +return softleaf_is_valid_pmd_entry (entry); +} + +/** +* pmd_to_softleaf_folio() - Convert the PMD entry to a folio. +* @pmd: PMD entry. +* +* The PMD entry is expected to be a valid PMD softleaf entry. +* +* Returns: the folio the softleaf entry references if this is a valid softleaf +* entry, otherwise NULL. +*/ +static inline struct folio *pmd_to_softleaf_folio (pmd_t pmd) +{ +const softleaf_t entry =softleaf_from_pmd (pmd); + +if (!softleaf_is_valid_pmd_entry (entry)) { +VM_WARN_ON_ONCE (true); +return NULL ; + } +return softleaf_to_folio (entry); }#endif /* CONFIG_MMU */ --2.53.0
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 12/13 mm/huge_memory: add and use normal_or_softleaf_folio_pmd()
In reply to Lorenzo Stoakes (Oracle)Now we have pmd_to_softleaf_folio() available to us which also raises a CONFIG_DEBUG_VM warning if unexpectedly an invalid softleaf entry, we can now abstract folio handling altogether.
vm_normal_folio() deals with the huge zero page (which is present), as well as PFN map/mixed map mappings in both cases returning NULL.
Otherwise, we try to obtain the softleaf folio.
This makes the logic far easier to comprehend and has it use the standard vm_normal_folio_pmd() path for decoding of present entries.
Finally, we have to update the flushing logic to only do so if a folio is established.
This patch also makes the 'is_present' value more accurate - because PFN map, mixed map and zero huge pages are present, just not present and 'normal'.
--- mm/huge_memory.c | 47 +++++++++++++++++++---------------------------- 1 file changed, 19 insertions(+), 28 deletions(-)
mm/huge_memory.c +19 -28
@@ -2342,10 +2342,6 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,add_mm_counter (mm,mm_counter_file (folio), -HPAGE_PMD_NR ); -/* -* Use flush_needed to indicate whether the PMD entry -* is present, instead of checking pmd_present() again. -*/ if (is_present &&pmd_young (pmdval) &&likely (vma_has_recency (vma)))folio_mark_accessed (folio);
@@ -2356,6 +2352,17 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,folio_put (folio); } +static struct folio *normal_or_softleaf_folio_pmd (struct vm_area_struct *vma, +unsigned long addr,pmd_t pmdval,bool is_present) +{ +if (is_present) +return vm_normal_folio_pmd (vma, addr, pmdval); + +if (!thp_migration_supported ()) +WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); +return pmd_to_softleaf_folio (pmdval); +} +/** * zap_huge_pmd - Zap a huge THP which is of PMD size. * @tlb: The MMU gather TLB state associated with the operation.
@@ -2390,36 +2397,20 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm);arch_check_zapped_pmd (vma, orig_pmd);tlb_remove_pmd_tlb_entry (tlb, pmd, addr); -if (vma_is_special_huge (vma)) - goto out; -if (is_huge_zero_pmd (orig_pmd)) { -if (!vma_is_dax (vma)) - has_deposit = true; - goto out; - } -if (pmd_present (orig_pmd)) { - folio =pmd_folio (orig_pmd); - is_present = true; - }else if (pmd_is_valid_softleaf (orig_pmd)) { -const softleaf_t entry =softleaf_from_pmd (orig_pmd); + is_present =pmd_present (orig_pmd); + folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present); +if (folio) +zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, + &has_deposit); +else if (is_huge_zero_pmd (orig_pmd)) + has_deposit = !vma_is_dax (vma); - folio =softleaf_to_folio (entry); -if (!thp_migration_supported ()) -WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); - }else { -WARN_ON_ONCE (true); - goto out; - } - -zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, &has_deposit); - -out:if (has_deposit)zap_deposited_table (mm, pmd);spin_unlock (ptl); -if (is_present) +if (is_present && folio)tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE );return true; }
Signature · 1 line
2.53.0
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 12/13] mm/huge_memory: add and use normal_or_softleaf_folio_pmd()
Hi Andrew,
Just a quick fix-patch for 12/13 to avoid a bisection hazard, as per Sashiko. The next patch eliminates this case in any case.
Thanks, Lorenzo
----8<----
From 7c5c0a66517efba0e675e707f931cf8a3f1a4d87 Mon Sep 17 00:00:00 2001
From: "Lorenzo Stoakes (Oracle)" <[email protected]>
Date: Mon, 23 Mar 2026 11:22:32 +0000
Subject: [PATCH] fix
--- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/huge_memory.c +1 -1
@@ -2404,7 +2404,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, &has_deposit); else if (is_huge_zero_pmd(orig_pmd)) - has_deposit = !vma_is_dax (vma); + has_deposit = has_deposit || !vma_is_dax (vma);if (has_deposit)zap_deposited_table (mm, pmd);
--
2.53.0
Re: [PATCH v3 12/13] mm/huge_memory: add and use normal_or_softleaf_folio_pmd()
In reply to Lorenzo Stoakes (Oracle)Quoted 13 lines · Lorenzo Stoakes (Oracle)
Now we have
pmd_to_softleaf_folio()available to us which also raises aCONFIG_DEBUG_VMwarning if unexpectedly an invalid softleaf entry, we can now abstract folio handling altogether.
vm_normal_folio()deals with the huge zero page (which is present), as well as PFN map/mixed map mappings in both cases returning NULL.Otherwise, we try to obtain the softleaf folio.
This makes the logic far easier to comprehend and has it use the standard
vm_normal_folio_pmd()path for decoding of present entries.
Quoted 8 lines
Finally, we have to update the flushing logic to only do so if a folio is established.
This patch also makes the '
is_present' value more accurate - because PFN map, mixed map and zero huge pages are present, just not present and 'normal'.Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>
One clarifying question below but I think I know the answer, just want to double-check.
Quoted 45 lines
--- mm/huge_memory.c | 47 +++++++++++++++++++---------------------------- 1 file changed, 19 insertions(+), 28 deletions(-)mm/huge_memory.c +11
-8@@ -2342,10 +2342,6 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,add_mm_counter (mm,mm_counter_file (folio), -HPAGE_PMD_NR ); -/* -* Use flush_needed to indicate whether the PMD entry -* is present, instead of checking pmd_present() again. -*/ if (is_present &&pmd_young (pmdval) &&likely (vma_has_recency (vma)))folio_mark_accessed (folio);@@ -2356,6 +2352,17 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,folio_put (folio); } +static struct folio *normal_or_softleaf_folio_pmd (struct vm_area_struct *vma, +unsigned long addr,pmd_t pmdval,bool is_present) +{ +if (is_present) +return vm_normal_folio_pmd (vma, addr, pmdval); + +if (!thp_migration_supported ()) +WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); +return pmd_to_softleaf_folio (pmdval); +} +/** * zap_huge_pmd - Zap a huge THP which is of PMD size. * @tlb: The MMU gather TLB state associated with the operation. @@ -2390,36 +2397,20 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm);arch_check_zapped_pmd (vma, orig_pmd);tlb_remove_pmd_tlb_entry (tlb, pmd, addr); -if (vma_is_special_huge (vma)) - goto out; -if (is_huge_zero_pmd (orig_pmd)) { -if (!vma_is_dax (vma))
mm/huge_memory.c bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 2401
Quoted 10 lines
mm/huge_memory.c +1
-8- has_deposit = true; - goto out; - } -if (pmd_present (orig_pmd)) { - folio =pmd_folio (orig_pmd); - is_present = true; - }else if (pmd_is_valid_softleaf (orig_pmd)) { -const softleaf_t entry =softleaf_from_pmd (orig_pmd); + is_present =pmd_present (orig_pmd);
nit: With this you don't need to initialize is_present anymore when you define it.
Quoted 14 lines
mm/huge_memory.c +6
-7+ folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present); +if (folio) +zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, + &has_deposit); +else if (is_huge_zero_pmd (orig_pmd)) + has_deposit = !vma_is_dax (vma); - folio =softleaf_to_folio (entry); -if (!thp_migration_supported ()) -WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); - }else { -WARN_ON_ONCE (true); - goto out; - }
mm/huge_memory.c bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 2413
Quoted 10 lines
mm/huge_memory.c +1
-5- -zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, &has_deposit); - -out:if (has_deposit)zap_deposited_table (mm, pmd);spin_unlock (ptl); -if (is_present) +if (is_present && folio)
In which case would you have a valid folio and !is_present? Is that the softleaf case?
Quoted 6 lines
mm/huge_memory.c +0
-1tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE );return true; } --2.53.0
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 12/13] mm/huge_memory: add and use normal_or_softleaf_folio_pmd()
Quoted 18 lines · Suren Baghdasaryan
Quoted 16 lines · Lorenzo Stoakes (Oracle)
Now we have
pmd_to_softleaf_folio()available to us which also raises aCONFIG_DEBUG_VMwarning if unexpectedly an invalid softleaf entry, we can now abstract folio handling altogether.
vm_normal_folio()deals with the huge zero page (which is present), as well as PFN map/mixed map mappings in both cases returning NULL.Otherwise, we try to obtain the softleaf folio.
This makes the logic far easier to comprehend and has it use the standard
vm_normal_folio_pmd()path for decoding of present entries.Finally, we have to update the flushing logic to only do so if a folio is established.
Quoted 7 lines
Quoted 5 lines
This patch also makes the '
is_present' value more accurate - because PFN map, mixed map and zero huge pages are present, just not present and 'normal'.Signed-off-by Lorenzo Stoakes (Oracle) <[email protected]>Reviewed-by Suren Baghdasaryan <[email protected]>
Thanks!
Quoted 53 lines
One clarifying question below but I think I know the answer, just want to double-check.
Quoted 49 lines
--- mm/huge_memory.c | 47 +++++++++++++++++++---------------------------- 1 file changed, 19 insertions(+), 28 deletions(-)mm/huge_memory.c +11
-11@@ -2342,10 +2342,6 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,add_mm_counter (mm,mm_counter_file (folio), -HPAGE_PMD_NR ); -/* -* Use flush_needed to indicate whether the PMD entry -* is present, instead of checking pmd_present() again. -*/ if (is_present &&pmd_young (pmdval) &&likely (vma_has_recency (vma)))folio_mark_accessed (folio);@@ -2356,6 +2352,17 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,folio_put (folio); } +static struct folio *normal_or_softleaf_folio_pmd (struct vm_area_struct *vma, +unsigned long addr,pmd_t pmdval,bool is_present) +{ +if (is_present) +return vm_normal_folio_pmd (vma, addr, pmdval); + +if (!thp_migration_supported ()) +WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); +return pmd_to_softleaf_folio (pmdval); +} +/** * zap_huge_pmd - Zap a huge THP which is of PMD size. * @tlb: The MMU gather TLB state associated with the operation. @@ -2390,36 +2397,20 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm);arch_check_zapped_pmd (vma, orig_pmd);tlb_remove_pmd_tlb_entry (tlb, pmd, addr); -if (vma_is_special_huge (vma)) - goto out; -if (is_huge_zero_pmd (orig_pmd)) { -if (!vma_is_dax (vma)) - has_deposit = true; - goto out; - }
Quoted 9 lines
mm/huge_memory.c bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 2401
Quoted 6 lines
mm/huge_memory.c +1
-5-if (pmd_present (orig_pmd)) { - folio =pmd_folio (orig_pmd); - is_present = true; - }else if (pmd_is_valid_softleaf (orig_pmd)) { -const softleaf_t entry =softleaf_from_pmd (orig_pmd); + is_present =pmd_present (orig_pmd);nit: With this you don't need to initialize
is_presentanymore when you define it.
Ah I think that's sorted out later.
Quoted 22 lines
Quoted 22 lines
mm/huge_memory.c +6
-11+ folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present); +if (folio) +zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, + &has_deposit); +else if (is_huge_zero_pmd (orig_pmd)) + has_deposit = !vma_is_dax (vma); - folio =softleaf_to_folio (entry); -if (!thp_migration_supported ()) -WARN_ONCE (1 ,"Non present huge pmd without pmd migration enabled!" ); - }else { -WARN_ON_ONCE (true); - goto out; - } - -zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, &has_deposit); - -out:if (has_deposit)zap_deposited_table (mm, pmd);
Quoted 6 lines
mm/huge_memory.c bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 2413
Quoted 3 lines
mm/huge_memory.c +1
-1spin_unlock (ptl); -if (is_present) +if (is_present && folio)In which case would you have a valid folio and !is_present? Is that the softleaf case?
Yes, as pmd_present() will return false for a softleaf entry. This is an easy way of figuring that out, kind of expressed a bit differently 'it's present so maybe we have to do a TLB operation, but make sure we have a folio to operate on'.
Quoted 7 lines
Quoted 7 lines
mm/huge_memory.c +0
-1tlb_remove_page_size (tlb, &folio->page,HPAGE_PMD_SIZE );return true; } --2.53.0
Cheers, Lorenzo
Lorenzo Stoakes (Oracle)
reviewerPATCH v3 13/13 mm/huge_memory: add and use has_deposited_pgtable()
In reply to Lorenzo Stoakes (Oracle)Rather than thread has_deposited through zap_huge_pmd(), make things clearer by adding has_deposited_pgtable() with comments describing why in each case.
--- mm/huge_memory.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-)
mm/huge_memory.c +24 -9
@@ -2326,8 +2326,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) }static void zap_huge_pmd_folio (struct mm_struct *mm,struct vm_area_struct *vma, -pmd_t pmdval,struct folio *folio,bool is_present, -bool *has_deposit) +pmd_t pmdval,struct folio *folio,bool is_present) {const bool is_device_private =folio_is_device_private (folio);
@@ -2336,7 +2335,6 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,folio_remove_rmap_pmd (folio, &folio->page, vma);if (folio_test_anon (folio)) { - *has_deposit = true;add_mm_counter (mm,MM_ANONPAGES , -HPAGE_PMD_NR ); }else { add_mm_counter(mm, mm_counter_file(folio),
@@ -2363,6 +2361,27 @@ static struct folio *normal_or_softleaf_folio_pmd(struct vm_area_struct *vma,return pmd_to_softleaf_folio (pmdval); } +static bool has_deposited_pgtable (struct vm_area_struct *vma,pmd_t pmdval, +struct folio *folio) +{ +/* Some architectures require unconditional depositing. */ +if (arch_needs_pgtable_deposit ()) +return true; + +/* +* Huge zero always deposited except for DAX which handles itself, see +* set_huge_zero_folio(). +*/ +if (is_huge_zero_pmd (pmdval)) +return !vma_is_dax (vma); + +/* +* Otherwise, only anonymous folios are deposited, see +* __do_huge_pmd_anonymous_page(). +*/ +return folio &&folio_test_anon (folio); +} +/** * zap_huge_pmd - Zap a huge THP which is of PMD size. * @tlb: The MMU gather TLB state associated with the operation.
@@ -2375,7 +2394,6 @@ static struct folio *normal_or_softleaf_folio_pmd(struct vm_area_struct *vma,bool zap_huge_pmd (struct mmu_gather *tlb,struct vm_area_struct *vma,pmd_t *pmd,unsigned long addr) { -bool has_deposit =arch_needs_pgtable_deposit ();struct mm_struct *mm = tlb->mm;struct folio *folio =NULL ;bool is_present = false;
@@ -2401,12 +2419,9 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, is_present =pmd_present (orig_pmd); folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present);if (folio) -zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present, - &has_deposit); -else if (is_huge_zero_pmd (orig_pmd)) - has_deposit = !vma_is_dax (vma); +zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present); -if (has_deposit) +if (has_deposited_pgtable (vma, orig_pmd, folio))zap_deposited_table (mm, pmd);spin_unlock (ptl);
Signature · 1 line
2.53.0
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Hi Andrew,
Could you apply the below fix-patch to resolve an issue with us performing folio_put() on a folio before checking it again to see if a table was deposited, as per Sashiko.
This patch resolves the issue by storing whether or not this is the case in a has_deposit local variable (as used previously) before invoking zap_huge_pmd_folio(), then using this boolean to determine whether or not to zap any deposited table.
Thanks, Lorenzo
----8<----
From 009f8abba834b49f8285b03a680dbd04d953a528 Mon Sep 17 00:00:00 2001
From: "Lorenzo Stoakes (Oracle)" <[email protected]>
Date: Mon, 23 Mar 2026 11:42:01 +0000
Subject: [PATCH] fix
--- mm/huge_memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
mm/huge_memory.c +3 -2
@@ -2397,6 +2397,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,struct mm_struct *mm = tlb->mm;struct folio *folio =NULL ;bool is_present = false; +bool has_deposit;spinlock_t *ptl;pmd_t orig_pmd;
@@ -2420,8 +2421,8 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present);if (folio)zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present); - -if (has_deposited_pgtable (vma, orig_pmd, folio)) + has_deposit =has_deposited_pgtable (vma, orig_pmd, folio); +if (has_deposit)zap_deposited_table (mm, pmd);spin_unlock (ptl);
--
2.53.0
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 10 lines · Lorenzo Stoakes (Oracle)
Hi Andrew,
Could you apply the below fix-patch to resolve an issue with us performing
folio_put()on a folio before checking it again to see if a table was deposited, as per Sashiko.This patch resolves the issue by storing whether or not this is the case in a
has_depositlocal variable (as used previously) before invokingzap_huge_pmd_folio(), then using this boolean to determine whether or not to zap any deposited table.
Oops this is wrong...!
Please apply the below instead :)
Thanks, Lorenzo
----8<----
From e6d58747d00dd954c605201e97f8b769b2ba8cf8 Mon Sep 17 00:00:00 2001
From: "Lorenzo Stoakes (Oracle)" <[email protected]>
Date: Mon, 23 Mar 2026 11:42:01 +0000
Subject: [PATCH] fix
--- mm/huge_memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
mm/huge_memory.c +3 -2
@@ -2397,6 +2397,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,struct mm_struct *mm = tlb->mm;struct folio *folio =NULL ;bool is_present = false; +bool has_deposit;spinlock_t *ptl;pmd_t orig_pmd;
@@ -2418,10 +2419,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, is_present =pmd_present (orig_pmd); folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present); + has_deposit =has_deposited_pgtable (vma, orig_pmd, folio);if (folio)zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present); - -if (has_deposited_pgtable (vma, orig_pmd, folio)) +if (has_deposit)zap_deposited_table (mm, pmd);spin_unlock (ptl);
--
2.53.0
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 13 lines · Lorenzo Stoakes (Oracle)
Quoted 10 lines · Lorenzo Stoakes (Oracle)
Hi Andrew,
Could you apply the below fix-patch to resolve an issue with us performing
folio_put()on a folio before checking it again to see if a table was deposited, as per Sashiko.This patch resolves the issue by storing whether or not this is the case in a
has_depositlocal variable (as used previously) before invokingzap_huge_pmd_folio(), then using this boolean to determine whether or not to zap any deposited table.
Quoted 5 lines
Oops this is wrong...!
Please apply the below instead :)
Thanks, Lorenzo
With the fix LGTM. Overall very nice cleanup making the code much more readable. Thanks Lorenzo!
Quoted 39 lines
----8<----
Frome6d58747d00dd954c605201e97f8b769b2ba8cf8Mon Sep 17 00:00:00 2001
From: "Lorenzo Stoakes (Oracle)" <[email protected]>
Date: Mon, 23 Mar 2026 11:42:01 +0000
Subject: [PATCH] fixSigned-off-by Lorenzo Stoakes (Oracle) <[email protected]>--- mm/huge_memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)mm/huge_memory.c +3
-3@@ -2397,6 +2397,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,struct mm_struct *mm = tlb->mm;struct folio *folio =NULL ;bool is_present = false; +bool has_deposit;spinlock_t *ptl;pmd_t orig_pmd;@@ -2418,10 +2419,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, is_present =pmd_present (orig_pmd); folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present); + has_deposit =has_deposited_pgtable (vma, orig_pmd, folio);if (folio)zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present); - -if (has_deposited_pgtable (vma, orig_pmd, folio)) +if (has_deposit)zap_deposited_table (mm, pmd);spin_unlock (ptl); --2.53.0
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 14 lines · Suren Baghdasaryan
Quoted 13 lines · Lorenzo Stoakes (Oracle)
Quoted 10 lines · Lorenzo Stoakes (Oracle)
Hi Andrew,
Could you apply the below fix-patch to resolve an issue with us performing
folio_put()on a folio before checking it again to see if a table was deposited, as per Sashiko.This patch resolves the issue by storing whether or not this is the case in a
has_depositlocal variable (as used previously) before invokingzap_huge_pmd_folio(), then using this boolean to determine whether or not to zap any deposited table.
Quoted 8 lines
Quoted 5 lines
Oops this is wrong...!
Please apply the below instead :)
Thanks, Lorenzo
With the fix LGTM. Overall very nice cleanup making the code much more readable. Thanks Lorenzo!
Thanks :)
Quoted 2 lines
Reviewed-by Suren Baghdasaryan <[email protected]>
Thanks for the review!
Cheers, Lorenzo
Quoted 40 lines
Quoted 40 lines
----8<----
Frome6d58747d00dd954c605201e97f8b769b2ba8cf8Mon Sep 17 00:00:00 2001
From: "Lorenzo Stoakes (Oracle)" <[email protected]>
Date: Mon, 23 Mar 2026 11:42:01 +0000
Subject: [PATCH] fixSigned-off-by Lorenzo Stoakes (Oracle) <[email protected]>--- mm/huge_memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)mm/huge_memory.c +3
-3@@ -2397,6 +2397,7 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,struct mm_struct *mm = tlb->mm;struct folio *folio =NULL ;bool is_present = false; +bool has_deposit;spinlock_t *ptl;pmd_t orig_pmd;@@ -2418,10 +2419,10 @@ bool zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, is_present =pmd_present (orig_pmd); folio =normal_or_softleaf_folio_pmd (vma, addr, orig_pmd, is_present); + has_deposit =has_deposited_pgtable (vma, orig_pmd, folio);if (folio)zap_huge_pmd_folio (mm, vma, orig_pmd, folio, is_present); - -if (has_deposited_pgtable (vma, orig_pmd, folio)) +if (has_deposit)zap_deposited_table (mm, pmd);spin_unlock (ptl); --2.53.0
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
In reply to Lorenzo Stoakes (Oracle)Quoted 10 lines · Lorenzo Stoakes (Oracle)
mm/huge_memory.c +10
-0+static bool has_deposited_pgtable (struct vm_area_struct *vma,pmd_t pmdval, +struct folio *folio) +{ +/* Some architectures require unconditional depositing. */ +if (arch_needs_pgtable_deposit ()) +return true; + +/* +* Huge zero always deposited except for DAX which handles itself, see +* set_huge_zero_folio().
mm/huge_memory.c static struct folio *normal_or_softleaf_folio_pmd(struct vm_area_struct *vma, · line 2383
Quoted 10 lines
mm/huge_memory.c +10
-0+*/ +if (is_huge_zero_pmd (pmdval)) +return !vma_is_dax (vma); + +/* +* Otherwise, only anonymous folios are deposited, see +* __do_huge_pmd_anonymous_page(). +*/ +return folio &&folio_test_anon (folio); +}
Hi Lorenzo,
I just wanted to mention a potential intersection with my upcoming v4 of the "mm: add huge pfnmap support for remap_pfn_range()" series [1].
To safely support PMD splitting on partial unmaps, my series makes VM_PFNMAP huge pages allocate and deposit page tables. However, they neither have an underlying struct folio nor are anonymous.
Given the has_deposited_pgtable() helper, do you have any suggestions on what would be the cleanest way to integrate the VM_PFNMAP condition into it?
[1]
https://lore.kernel.org/linux-mm/[email protected]/
Signature · 1 line
Yin Tirui
Lorenzo Stoakes (Oracle)
reviewerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 30 lines · Yin Tirui
Quoted 10 lines · Lorenzo Stoakes (Oracle)
mm/huge_memory.c +10
-0+static bool has_deposited_pgtable (struct vm_area_struct *vma,pmd_t pmdval, +struct folio *folio) +{ +/* Some architectures require unconditional depositing. */ +if (arch_needs_pgtable_deposit ()) +return true; + +/* +* Huge zero always deposited except for DAX which handles itself, see +* set_huge_zero_folio(). mm/huge_memory.c static struct folio *normal_or_softleaf_folio_pmd(struct vm_area_struct *vma, · line 2383
Quoted 10 lines
mm/huge_memory.c +10
-0+*/ +if (is_huge_zero_pmd (pmdval)) +return !vma_is_dax (vma); + +/* +* Otherwise, only anonymous folios are deposited, see +* __do_huge_pmd_anonymous_page(). +*/ +return folio &&folio_test_anon (folio); +}Hi Lorenzo,
I just wanted to mention a potential intersection with my upcoming v4 of the "mm: add huge pfnmap support for
remap_pfn_range()" series [1].
Quoted 6 lines
To safely support PMD splitting on partial unmaps, my series makes
VM_PFNMAPhuge pages allocate and deposit page tables. However, they neither have an underlying struct folio nor are anonymous.Given the
has_deposited_pgtable()helper, do you have any suggestions on what would be the cleanest way to integrate theVM_PFNMAPcondition into it?
I mean you would have needed to handle this case in any event, since this change is strictly an equivalent reworking of zap_huge_pmd().
But it seems that doing so has clarified the requirements somewhat here :)
I haven't had a look at that series yet (please cc this email if you weren't already, I do filter a lot of stuff due to how much mail I get daily)
So if this is a PMD leaf entry it will be present and PFN map, so I'd have thought simply adding:
/* Huge PFN map must deposit, as cannot refault. */ if (vma_test(vma, VMA_PFNMAP_BIT)) return true;
Would suffice?
By the way, I am wondering if the prot bits are correctly preserved on page table deposit, as this is key for pfn map (e.g. if the range is uncached, for instance). That's something to check and ensure is correct.
I _suspect_ they will be, as we have pretty well established mechanisms for that (propagate vma->vm_page_prot etc.) but definitely worth making sure.
Quoted 7 lines
[1]
https://lore.kernel.org/linux-mm/[email protected]/--
Yin Tirui
Cheers, Lorenzo
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 8 lines · Lorenzo Stoakes (Oracle)
I mean you would have needed to handle this case in any event, since this change is strictly an equivalent reworking of
zap_huge_pmd().But it seems that doing so has clarified the requirements somewhat here :)
I haven't had a look at that series yet (please cc this email if you weren't already, I do filter a lot of stuff due to how much mail I get daily)
Hi Lorenzo,
Thanks for the quick reply. I will definitely CC you on the v4 series.
Quoted 9 lines
So if this is a PMD leaf entry it will be present and PFN map, so I'd have thought simply adding:
/* Huge PFN map must deposit, as cannot refault. */ if (vma_test(vma, VMA_PFNMAP_BIT)) return true;Would suffice?
Here is the dilemma:
Currently, VFIO uses vmf_insert_pfn_pmd() to create huge pfnmaps on page faults. This sets VM_PFNMAP in vfio_pci_core_mmap(), but it does not deposit a pgtable (unless arch_needs_pgtable_deposit() is true).
To resolve this,
Option A: Force VFIO (vmf_insert_pfn_pmd) to also deposit pgtables. This unifies the VM_PFNMAP lifecycle. However, since VFIO can refault, depositing pgtables here incurs unnecessary memory overhead.
Option B: Introduce a new VMA flag set during remap_pfn_range(), which we can explicitly check in has_deposited_pgtable().
Option C: Check vma->vm_ops->fault (and huge_fault). We would only deposit pgtables for mappings without fault handlers. However, this is fragile because a driver might still register a .fault() handler that simply returns VM_FAULT_SIGBUS.
Do you have a preference among these, or perhaps another idea?
Quoted 8 lines
By the way, I am wondering if the prot bits are correctly preserved on page table deposit, as this is key for pfn map (e.g. if the range is uncached, for instance). That's something to check and ensure is correct.
I _suspect_ they will be, as we have pretty well established mechanisms for that (propagate
vma->vm_page_protetc.) but definitely worth making sure.
Yes, they are correctly preserved!
During a PMD split in __split_huge_pmd_locked(), we populate the deposited pgtable like this:
entry = pfn_pte(pmd_pfn(old_pmd), pmd_pgprot(old_pmd));
set_ptes(mm, haddr, pte, entry, HPAGE_PMD_NR);The newly refactored pmd_pgprot() correctly extracts the exact protection bits (including crucial cache modes like UC/WC for device memory) from the huge PMD, strips the hardware-specific huge bit, and returns a pure PTE-level pgprot_t.
Quoted 9 lines
Quoted 7 lines
[1]
https://lore.kernel.org/linux-mm/[email protected]/--
Yin TiruiCheers, Lorenzo
Signature · 1 line
Yin Tirui
Lorenzo Stoakes
reviewerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 7 lines · Yin Tirui
Quoted 4 lines · Lorenzo Stoakes (Oracle)
I mean you would have needed to handle this case in any event, since this change is strictly an equivalent reworking of
zap_huge_pmd().
Quoted 8 lines
Quoted 4 lines
But it seems that doing so has clarified the requirements somewhat here :)
I haven't had a look at that series yet (please cc this email if you weren't already, I do filter a lot of stuff due to how much mail I get daily)
Hi Lorenzo,
Thanks for the quick reply. I will definitely CC you on the v4 series.
Thanks.
Quoted 9 lines
Quoted 9 lines
So if this is a PMD leaf entry it will be present and PFN map, so I'd have thought simply adding:
/* Huge PFN map must deposit, as cannot refault. */ if (vma_test(vma, VMA_PFNMAP_BIT)) return true;
Quoted 7 lines
Quoted 1 line
Would suffice?
Here is the dilemma:
Currently, VFIO uses
vmf_insert_pfn_pmd()to create huge pfnmaps on page faults. This setsVM_PFNMAPinvfio_pci_core_mmap(), but it does not deposit a pgtable (unlessarch_needs_pgtable_deposit()is true).
Hmmm... it's only the VFIO and hyperv drivers using this.
Wouldn't we generally want a deposited huge page here now we're allowing huge PFN maps?
Or are this _special cases_ where we have a PMD-sized entry but are not necessarily wanting to treat it as THP?
This is a real wrinkle in this whole series no?
David - any thoughts?
Quoted 6 lines
To resolve this,
Option A: Force VFIO (
vmf_insert_pfn_pmd) to also deposit pgtables. This unifies theVM_PFNMAPlifecycle. However, since VFIO can refault, depositing pgtables here incurs unnecessary memory overhead.
How can VFIO refault as a PFN mapping? Does it intentionally sometimes clear PTE entries to effect a refault, and implement a custom fault handler?
I guess having a fault handler makes it refaultable...
I mean obviously that then contradicts the suggested comment above :)
That seems to me to cast a bit of a question over the whole series - having PMD mappings that are _sometimes_ THP and _sometimes_ not is weird (TM).
And it'd suck to add - yet another very specific check - to determine if we do, in fact, assume THP for a PMD sized PFN map.
Quoted 3 lines
Option B: Introduce a new VMA flag set during
remap_pfn_range(), which we can explicitly check inhas_deposited_pgtable().
Yeah would rather not, that feels like a hack.
Quoted 5 lines
Option C: Check
vma->vm_ops->fault(andhuge_fault). We would only deposit pgtables for mappings without fault handlers. However, this is fragile because a driver might still register a .fault() handler that simply returnsVM_FAULT_SIGBUS.
I mean again this is yet another check (TM). But probably the most preferable I think.
Wouldn't a driver doing that be being somewhat redundant? E.g. in do_fault();
if (!vma->vm_ops->fault) {
vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
vmf->address, &vmf->ptl);
if (unlikely(!vmf->pte))
ret = VM_FAULT_SIGBUS;And so can expect maybe some more redundancy if they also happen to map PMD-sized ranges? :)
And the only two callers of vmf_insert_pfn_pmd() - hyperv and VFIO both implement actual fault handlers anyway.
So I think this is fine?
Quoted 17 lines
Do you have a preference among these, or perhaps another idea?
Quoted 8 lines
By the way, I am wondering if the prot bits are correctly preserved on page table deposit, as this is key for pfn map (e.g. if the range is uncached, for instance). That's something to check and ensure is correct.
I _suspect_ they will be, as we have pretty well established mechanisms for that (propagate
vma->vm_page_protetc.) but definitely worth making sure.Yes, they are correctly preserved!
During a PMD split in
__split_huge_pmd_locked(), we populate the deposited pgtable like this:
Quoted 7 lines
entry = pfn_pte(pmd_pfn(old_pmd), pmd_pgprot(old_pmd)); set_ptes(mm, haddr, pte, entry, HPAGE_PMD_NR);The newly refactored
pmd_pgprot()correctly extracts the exact protection bits (including crucial cache modes like UC/WC for device memory) from the huge PMD, strips the hardware-specific huge bit, and returns a pure PTE-levelpgprot_t.
OK good :)
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
--
Yin Tirui
Cheers, Lorenzo
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Hi Lorenzo and David,
Sorry for the late reply.
Quoted 61 lines · Lorenzo Stoakes
Quoted 7 lines · Yin Tirui
Quoted 4 lines · Lorenzo Stoakes (Oracle)
I mean you would have needed to handle this case in any event, since this change is strictly an equivalent reworking of
zap_huge_pmd().Quoted 8 lines
Quoted 4 lines
But it seems that doing so has clarified the requirements somewhat here :)
I haven't had a look at that series yet (please cc this email if you weren't already, I do filter a lot of stuff due to how much mail I get daily)
Hi Lorenzo,
Thanks for the quick reply. I will definitely CC you on the v4 series.
Thanks.
Quoted 9 lines
Quoted 9 lines
So if this is a PMD leaf entry it will be present and PFN map, so I'd have thought simply adding:
/* Huge PFN map must deposit, as cannot refault. */ if (vma_test(vma, VMA_PFNMAP_BIT)) return true;Quoted 7 lines
Quoted 1 line
Would suffice?
Here is the dilemma:
Currently, VFIO uses
vmf_insert_pfn_pmd()to create huge pfnmaps on page faults. This setsVM_PFNMAPinvfio_pci_core_mmap(), but it does not deposit a pgtable (unlessarch_needs_pgtable_deposit()is true).Hmmm... it's only the VFIO and hyperv drivers using this.
Wouldn't we generally want a deposited huge page here now we're allowing huge PFN maps?
Or are this
_specialcases_where we have a PMD-sized entry but are not necessarily wanting to treat it as THP?This is a real wrinkle in this whole series no?
David - any thoughts?
Quoted 6 lines
To resolve this,
Option A: Force VFIO (
vmf_insert_pfn_pmd) to also deposit pgtables. This unifies theVM_PFNMAPlifecycle. However, since VFIO can refault, depositing pgtables here incurs unnecessary memory overhead.How can VFIO refault as a PFN mapping? Does it intentionally sometimes clear PTE entries to effect a refault, and implement a custom fault handler?
I guess having a fault handler makes it refaultable...
Quoted 7 lines
I mean obviously that then contradicts the suggested comment above :)
That seems to me to cast a bit of a question over the whole series - having PMD mappings that are _sometimes_ THP and _sometimes_ not is weird (TM).
And it'd suck to add - yet another very specific check - to determine if we do, in fact, assume THP for a PMD sized PFN map.
Yes, exactly. VFIO and Hyper-V rely on their custom .fault handlers to dynamically build mappings. In contrast, remap_pfn_range() establishes static pre-mappings.
Quoted 6 lines
Quoted 4 lines
Option B: Introduce a new VMA flag set during
remap_pfn_range(), which we can explicitly check inhas_deposited_pgtable().Yeah would rather not, that feels like a hack.
Agreed.
Quoted 18 lines
Quoted 6 lines
Option C: Check
vma->vm_ops->fault(andhuge_fault). We would only deposit pgtables for mappings without fault handlers. However, this is fragile because a driver might still register a .fault() handler that simply returnsVM_FAULT_SIGBUS.I mean again this is yet another check (TM). But probably the most preferable I think.
Wouldn't a driver doing that be being somewhat redundant? E.g. in
do_fault();if (!vma->vm_ops->fault) { vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address, &vmf->ptl); if (unlikely(!vmf->pte)) ret = VM_FAULT_SIGBUS;
Quoted 8 lines
And so can expect maybe some more redundancy if they also happen to map PMD-sized ranges? :)
And the only two callers of
vmf_insert_pfn_pmd()- hyperv and VFIO both implement actual fault handlers anyway.So I think this is fine?
I agree.
David, since Lorenzo also asked for your thoughts on the overall design aspect ("sometimes THP and sometimes not"), what is your opinion on this? Should we proceed with checking !vma->vm_ops->fault to differentiate the deposit behavior for huge PFNMAPs?
Quoted 37 lines
Quoted 17 lines
Do you have a preference among these, or perhaps another idea?
Quoted 8 lines
By the way, I am wondering if the prot bits are correctly preserved on page table deposit, as this is key for pfn map (e.g. if the range is uncached, for instance). That's something to check and ensure is correct.
I _suspect_ they will be, as we have pretty well established mechanisms for that (propagate
vma->vm_page_protetc.) but definitely worth making sure.Yes, they are correctly preserved!
During a PMD split in
__split_huge_pmd_locked(), we populate the deposited pgtable like this:Quoted 7 lines
entry = pfn_pte(pmd_pfn(old_pmd), pmd_pgprot(old_pmd)); set_ptes(mm, haddr, pte, entry, HPAGE_PMD_NR);The newly refactored
pmd_pgprot()correctly extracts the exact protection bits (including crucial cache modes like UC/WC for device memory) from the huge PMD, strips the hardware-specific huge bit, and returns a pure PTE-levelpgprot_t.OK good :)
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
--
Yin TiruiCheers, Lorenzo
Signature · 1 line
Yin Tirui
David Hildenbrand (Arm)
maintainerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 34 lines · Yin Tirui
Hi Lorenzo and David,
Sorry for the late reply.
Quoted 29 lines · Lorenzo Stoakes
Quoted 6 lines · Yin Tirui
Hi Lorenzo,
Thanks for the quick reply. I will definitely CC you on the v4 series.
Thanks.
Quoted 7 lines
Here is the dilemma:
Currently, VFIO uses
vmf_insert_pfn_pmd()to create huge pfnmaps on page faults. This setsVM_PFNMAPinvfio_pci_core_mmap(), but it does not deposit a pgtable (unlessarch_needs_pgtable_deposit()is true).Hmmm... it's only the VFIO and hyperv drivers using this.
Wouldn't we generally want a deposited huge page here now we're allowing huge PFN maps?
Or are this
_specialcases_where we have a PMD-sized entry but are not necessarily wanting to treat it as THP?This is a real wrinkle in this whole series no?
David - any thoughts?
Sorry, catching up with that now.
Quoted 62 lines
Quoted 14 lines
Quoted 7 lines
To resolve this,
Option A: Force VFIO (
vmf_insert_pfn_pmd) to also deposit pgtables. This unifies theVM_PFNMAPlifecycle. However, since VFIO can refault, depositing pgtables here incurs unnecessary memory overhead.How can VFIO refault as a PFN mapping? Does it intentionally sometimes clear PTE entries to effect a refault, and implement a custom fault handler?
I guess having a fault handler makes it refaultable...
Quoted 7 lines
I mean obviously that then contradicts the suggested comment above :)
That seems to me to cast a bit of a question over the whole series - having PMD mappings that are _sometimes_ THP and _sometimes_ not is weird (TM).
And it'd suck to add - yet another very specific check - to determine if we do, in fact, assume THP for a PMD sized PFN map.
Yes, exactly. VFIO and Hyper-V rely on their custom
.faulthandlers to dynamically build mappings. In contrast,remap_pfn_range()establishes static pre-mappings.Quoted 6 lines
Quoted 4 lines
Option B: Introduce a new VMA flag set during
remap_pfn_range(), which we can explicitly check inhas_deposited_pgtable().Yeah would rather not, that feels like a hack.
Agreed.
Quoted 18 lines
Quoted 6 lines
Option C: Check
vma->vm_ops->fault(andhuge_fault). We would only deposit pgtables for mappings without fault handlers. However, this is fragile because a driver might still register a .fault() handler that simply returnsVM_FAULT_SIGBUS.I mean again this is yet another check (TM). But probably the most preferable I think.
Wouldn't a driver doing that be being somewhat redundant? E.g. in
do_fault();if (!vma->vm_ops->fault) { vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address, &vmf->ptl); if (unlikely(!vmf->pte)) ret = VM_FAULT_SIGBUS;Quoted 8 lines
And so can expect maybe some more redundancy if they also happen to map PMD-sized ranges? :)
And the only two callers of
vmf_insert_pfn_pmd()- hyperv and VFIO both implement actual fault handlers anyway.So I think this is fine?
Quoted 6 lines
I agree.
David, since Lorenzo also asked for your thoughts on the overall design aspect ("sometimes THP and sometimes not"), what is your opinion on this? Should we proceed with checking
!vma->vm_ops->faultto differentiate the deposit behavior for huge PFNMAPs?
I mean, we need some indication to know also during folio splitting whether we can just discard the PMD, as we can refault it later, or whether we really have to install a PTE table.
What if someone used remap_pfn_range() on some part of the VMA, and faults on another part?
Doesn't really work.
Do we have users of remap_pfn_range() that have ->fault set? If not, we should probably just disallow this combination.
Then we know for sure whether something was installed through remap_pfn_range() or through a fault handler.
Signature · 3 lines
Cheers, David
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 5 lines · David Hildenbrand (Arm)
I mean, we need some indication to know also during folio splitting whether we can just discard the PMD, as we can refault it later, or whether we really have to install a PTE table.
Quoted 7 lines
What if someone used
remap_pfn_range()on some part of the VMA, and faults on another part?Doesn't really work.
Do we have users of
remap_pfn_range()that have ->fault set? If not, we should probably just disallow this combination.
I did a quick tree-wide grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.fault\s*=" arch/powerpc/platforms/book3s/vas-api.c drivers/infiniband/hw/hfi1/file_ops.c drivers/uio/uio.c drivers/vfio/pci/vfio_pci_core.c fs/proc/vmcore.c security/selinux/selinuxfs.c
It turns out there are two users of this "hybrid" approach in the kernel:
1. fs/proc/vmcore.c: It pre-maps via remap_pfn_range() but registers mmap_vmcore_fault().
2. arch/powerpc/platforms/book3s/vas-api.c: It pre-maps via remap_pfn_range(), but registers vas_mmap_fault().
Quoted 4 lines
Then we know for sure whether something was installed through
remap_pfn_range()or through a fault handler.
How would you suggest we proceed here?
Signature · 1 line
Yin Tirui
David Hildenbrand (Arm)
maintainerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 30 lines · Yin Tirui
Quoted 5 lines · David Hildenbrand (Arm)
I mean, we need some indication to know also during folio splitting whether we can just discard the PMD, as we can refault it later, or whether we really have to install a PTE table.
Quoted 7 lines
What if someone used
remap_pfn_range()on some part of the VMA, and faults on another part?Doesn't really work.
Do we have users of
remap_pfn_range()that have ->fault set? If not, we should probably just disallow this combination.I did a quick tree-wide grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.fault\s*=" arch/powerpc/platforms/book3s/vas-api.c drivers/infiniband/hw/hfi1/file_ops.c drivers/uio/uio.c drivers/vfio/pci/vfio_pci_core.c fs/proc/vmcore.c security/selinux/selinuxfs.cIt turns out there are two users of this "hybrid" approach in the kernel:
1.fs/proc/vmcore.c: It pre-maps viaremap_pfn_range()but registersmmap_vmcore_fault().
2.arch/powerpc/platforms/book3s/vas-api.c: It pre-maps viaremap_pfn_range(), but registersvas_mmap_fault().Quoted 1 line
Quoted 5 lines
Quoted 3 lines
Then we know for sure whether something was installed through
remap_pfn_range()or through a fault handler.How would you suggest we proceed here?
How about we populate PMDs in remap_pfn_range() only if !fault?
Then, if we have !fault, we know that the PMD is from remap_pfn_range() and has a disposed page table.
Would that work?
Signature · 3 lines
Cheers, David
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Hi David,
Quoted 31 lines · David Hildenbrand (Arm)
Quoted 30 lines · Yin Tirui
Quoted 5 lines · David Hildenbrand (Arm)
I mean, we need some indication to know also during folio splitting whether we can just discard the PMD, as we can refault it later, or whether we really have to install a PTE table.
Quoted 7 lines
What if someone used
remap_pfn_range()on some part of the VMA, and faults on another part?Doesn't really work.
Do we have users of
remap_pfn_range()that have ->fault set? If not, we should probably just disallow this combination.I did a quick tree-wide grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.fault\s*=" arch/powerpc/platforms/book3s/vas-api.c drivers/infiniband/hw/hfi1/file_ops.c drivers/uio/uio.c drivers/vfio/pci/vfio_pci_core.c fs/proc/vmcore.c security/selinux/selinuxfs.cIt turns out there are two users of this "hybrid" approach in the kernel:
1.fs/proc/vmcore.c: It pre-maps viaremap_pfn_range()but registersmmap_vmcore_fault().
2.arch/powerpc/platforms/book3s/vas-api.c: It pre-maps viaremap_pfn_range(), but registersvas_mmap_fault().Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Then we know for sure whether something was installed through
remap_pfn_range()or through a fault handler.How would you suggest we proceed here?
How about we populate PMDs in
remap_pfn_range()only if !fault?
Doing this would at most prevent VMAs with a ->fault() handler from getting huge mappings, which seems to have little negative impact.
But wait, dynamic huge mappings are actually created through ->huge_fault().
I did a quick grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.huge_fault\s*=" drivers/vfio/pci/vfio_pci_core.c
This is a false positive. There is no case in the kernel that mixes remap_pfn_range() and ->huge_fault() on the same VMA.
What if we use !huge_fault instead, disallowing remap_pfn_range() from populating PMDs if ->huge_fault() is provided?
Then, when we encounter a huge PMD, we know for sure whether it was installed through remap_pfn_range() (needs a deposited pgtable) or ->huge_fault() (no deposit needed, can be refaulted).
Quoted 6 lines
Then, if we have !fault, we know that the PMD is from
remap_pfn_range()and has a disposed page table.Would that work?
So for Lorenzo's has_deposited_pgtable() helper, we could simply use:
/* Huge PFN map without a huge_fault handler must deposit */ if (vma_test(vma, VMA_PFNMAP_BIT)) return !vma->vm_ops || !vma->vm_ops->huge_fault;
By the way, while auditing this, I noticed that drivers/gpu/drm/drm_gem_shmem_helper.c calls vmf_insert_pfn_pmd() directly from its normal ->fault() handler instead of implementing ->huge_fault().
If we adopt the !huge_fault check above, this DRM driver would be wrongly classified as needing a deposit. It seems that DRM driver needs a minor refactoring to properly use ->huge_fault() to keep the MM semantics clean.
Signature · 1 line
Yin Tirui
David Hildenbrand (Arm)
maintainerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 21 lines · Yin Tirui
Hi David,
Quoted 18 lines · David Hildenbrand (Arm)
Quoted 17 lines · Yin Tirui
I did a quick tree-wide grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.fault\s*=" arch/powerpc/platforms/book3s/vas-api.c drivers/infiniband/hw/hfi1/file_ops.c drivers/uio/uio.c drivers/vfio/pci/vfio_pci_core.c fs/proc/vmcore.c security/selinux/selinuxfs.cIt turns out there are two users of this "hybrid" approach in the kernel:
1.fs/proc/vmcore.c: It pre-maps viaremap_pfn_range()but registersmmap_vmcore_fault().
2.arch/powerpc/platforms/book3s/vas-api.c: It pre-maps viaremap_pfn_range(), but registersvas_mmap_fault().
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
How would you suggest we proceed here?
How about we populate PMDs in
remap_pfn_range()only if !fault?Doing this would at most prevent VMAs with a ->fault() handler from getting huge mappings, which seems to have little negative impact.
But wait, dynamic huge mappings are actually created through ->huge_fault().
If my memory serves me right, also fault() can nowadays install PMD mappings.
For example, shmem only implements ->fault through shmem_fault()
finish_fault() after __do_fault() takes care of that (mapping through a PMD if possible).
Quoted 10 lines
I did a quick grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.huge_fault\s*=" drivers/vfio/pci/vfio_pci_core.cThis is a false positive. There is no case in the kernel that mixes
remap_pfn_range()and ->huge_fault() on the same VMA.What if we use !huge_fault instead, disallowing
remap_pfn_range()from populating PMDs if ->huge_fault() is provided?
I think we should just disallow any PMD mappings if we either have ->fault or ->huge_fault.
I would assume that ->huge_fault implies >fault, but let's rather be save than sorry.
Quoted 9 lines
Then, when we encounter a huge PMD, we know for sure whether it was installed through
remap_pfn_range()(needs a deposited pgtable) or ->huge_fault() (no deposit needed, can be refaulted).Quoted 4 lines
Then, if we have !fault, we know that the PMD is from
remap_pfn_range()and has a disposed page table.
Quoted 8 lines
Quoted 2 lines
Would that work?
So for Lorenzo's
has_deposited_pgtable()helper, we could simply use:/* Huge PFN map without a huge_fault handler must deposit */ if (vma_test(vma, VMA_PFNMAP_BIT)) return !vma->vm_ops || !vma->vm_ops->huge_fault;
As mentioned above, also considering vma->vm_ops->fault;
Quoted 10 lines
By the way, while auditing this, I noticed that
drivers/gpu/drm/drm_gem_shmem_helper.ccallsvmf_insert_pfn_pmd()directly from its normal ->fault() handler instead of implementing ->huge_fault().
If we adopt the!huge_faultcheck above, this DRM driver would be wrongly classified as needing a deposit. It seems that DRM driver needs a minor refactoring to properly use ->huge_fault() to keep the MM semantics clean.
No, it's doing something that's allowed. If we call ->fault and there is not PTE table, it may insert a PMD.
Signature · 3 lines
Cheers, David
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Hi David,
Quoted 26 lines · David Hildenbrand (Arm)
Quoted 25 lines · Yin Tirui
Hi David,
Quoted 17 lines · David Hildenbrand (Arm)
Quoted 16 lines · Yin Tirui
I did a quick tree-wide grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.fault\s*=" arch/powerpc/platforms/book3s/vas-api.c drivers/infiniband/hw/hfi1/file_ops.c drivers/uio/uio.c drivers/vfio/pci/vfio_pci_core.c fs/proc/vmcore.c security/selinux/selinuxfs.cIt turns out there are two users of this "hybrid" approach in the kernel:
1.fs/proc/vmcore.c: It pre-maps viaremap_pfn_range()but registersmmap_vmcore_fault().
2.arch/powerpc/platforms/book3s/vas-api.c: It pre-maps viaremap_pfn_range(), but registersvas_mmap_fault().Quoted 2 lines
Quoted 1 line
How would you suggest we proceed here?
How about we populate PMDs in
remap_pfn_range()only if !fault?Doing this would at most prevent VMAs with a ->fault() handler from getting huge mappings, which seems to have little negative impact.
Quoted 8 lines
Quoted 1 line
But wait, dynamic huge mappings are actually created through ->huge_fault().
If my memory serves me right, also
fault()can nowadays install PMD mappings.For example, shmem only implements ->fault through
shmem_fault()
finish_fault()after__do_fault()takes care of that (mapping through a PMD if possible).
Ah, thanks for correcting my blind spot!
Quoted 7 lines
Quoted 7 lines
I did a quick grep:
$ git grep -l "remap_pfn_range" | xargs grep -l "\.huge_fault\s*=" drivers/vfio/pci/vfio_pci_core.cThis is a false positive. There is no case in the kernel that mixes
remap_pfn_range()and ->huge_fault() on the same VMA.
Quoted 7 lines
Quoted 2 lines
What if we use !huge_fault instead, disallowing
remap_pfn_range()from populating PMDs if ->huge_fault() is provided?I think we should just disallow any PMD mappings if we either have ->fault or ->huge_fault.
I would assume that ->huge_fault implies >fault, but let's rather be save than sorry.
Agreed. I think I have a clear idea of how to handle this now.
Quoted 7 lines
Quoted 7 lines
Then, when we encounter a huge PMD, we know for sure whether it was installed through
remap_pfn_range()(needs a deposited pgtable) or ->huge_fault() (no deposit needed, can be refaulted).Quoted 3 lines
Then, if we have !fault, we know that the PMD is from
remap_pfn_range()and has a disposed page table.
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
Would that work?
So for Lorenzo's
has_deposited_pgtable()helper, we could simply use:/* Huge PFN map without a huge_fault handler must deposit */ if (vma_test(vma, VMA_PFNMAP_BIT)) return !vma->vm_ops || !vma->vm_ops->huge_fault;As mentioned above, also considering
vma->vm_ops->fault;
Will do.
Quoted 10 lines
Quoted 10 lines
By the way, while auditing this, I noticed that
drivers/gpu/drm/drm_gem_shmem_helper.ccallsvmf_insert_pfn_pmd()directly from its normal ->fault() handler instead of implementing ->huge_fault().
If we adopt the!huge_faultcheck above, this DRM driver would be wrongly classified as needing a deposit. It seems that DRM driver needs a minor refactoring to properly use ->huge_fault() to keep the MM semantics clean.
Quoted 2 lines
No, it's doing something that's allowed. If we call ->fault and there is not PTE table, it may insert a PMD.
Thanks for your clarification.
Signature · 1 line
Yin Tirui
David Hildenbrand (Arm)
maintainerRe: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Then, when we encounter a huge PMD, we know for sure whether it was installed through
remap_pfn_range()(needs a deposited pgtable) or ->huge_fault() (no deposit needed, can be refaulted).So for Lorenzo's
has_deposited_pgtable()helper, we could simply use:
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
/* Huge PFN map without a
huge_faulthandler must deposit */ if (vma_test(vma, VMA_PFNMAP_BIT))
return !vma->vm_ops || !vma->vm_ops->huge_fault;As mentioned above, also considering
vma->vm_ops->fault;Will do.
Please make sure to wrap such a check in a well named and good documented helper function :)
Signature · 3 lines
Cheers, David
Re: [PATCH v3 13/13] mm/huge_memory: add and use has_deposited_pgtable()
Quoted 10 lines · David Hildenbrand (Arm)
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Then, when we encounter a huge PMD, we know for sure whether it was installed through
remap_pfn_range()(needs a deposited pgtable) or ->huge_fault() (no deposit needed, can be refaulted).So for Lorenzo's
has_deposited_pgtable()helper, we could simply use:/* Huge PFN map without a
huge_faulthandler must deposit */ if (vma_test(vma, VMA_PFNMAP_BIT))
return !vma->vm_ops || !vma->vm_ops->huge_fault;
Quoted 10 lines
Quoted 4 lines
Quoted 1 line
As mentioned above, also considering
vma->vm_ops->fault;Will do.
Please make sure to wrap such a check in a well named and good documented helper function :)
Sure! Will introduce a helper for this. Thanks!
Signature · 1 line
Yin Tirui
Andrew Morton
maintainerQuoted 13 lines · On Fri, 20 Mar 2026 18:14:50 +0000 "Lorenzo Stoakes (Oracle)" <[email protected]>…
The
zap_huge_pmd()function is overly complicated, clean it up and also add an assert in the case that we encounter a buggy PMD entry that doesn't match expectations.This is motivated by a bug discovered [0] where the PMD entry was none of:
* A non-DAX, PFN or mixed map.
* The huge zero folio
* A present PMD entry
* A softleaf entryIn
zap_huge_pmd(), but due to the bug we manged to reach this code.
Quoted 8 lines
It is useful to explicitly call this out rather than have an arbitrary NULL pointer dereference happen, which also improves understanding of what's going on.
The series goes further to make use of
vm_normal_folio_pmd()rather than implementing custom logic for retrieving the folio, and extends softleaf functionality to provide and use an equivalent softleaf function.
Thanks, I updated mm-unstable to this version.
Quoted 12 lines
v3:
* Propagated tags, thanks everybody!
* Fixed const vma parameter invma_is_special_huge()in 1/13 as per Sashiko.
* Renamedneeds_deposit->has_depositas per Kiryl, better describing the situation as we're zapping deposited tables, not depositing them.
* Initialisedhas_deposittoarch_needs_pgtable_deposit(), and updated huge zero page case to account for that as per Kiryl.
* Dropped separated logic approach as per Baolin.
* Added 'No functional change intended.' caveats.
* Removed seemingly superfluous, inconsistent pot-folio_remove_rmap_pmd() mapcount sanity checks.
Quoted 8 lines
* De-duplicated tlb->mm's.
* Separated folio-specific logic into another function.
* Addedsoftleaf_is_valid_pmd_entry(),pmd_to_softleaf_folio()functions.
* Add and usenormal_or_softleaf_folio_pmd()to make use ofvm_normal_folio_pmd()andpmd_to_softleaf_folio()for obtaining the folio.
* Add and usehas_deposited_pgtable()to figure out deposits.
* Added a bunch of explanatory comments as per Baolin.
Here's how v3 altered mm.git:
include/linux/leafops.h | 39 +++++++++++- mm/huge_memory.c | 115 ++++++++++++++++++++++---------------- 2 files changed, 102 insertions(+), 52 deletions(-)
include/linux/leafops.h renamed from include/linux/leafops.h~b +35 -4
@@ -603,7 +603,20 @@ static inline bool pmd_is_migration_entr }/** -* pmd_is_valid_softleaf() - Is this PMD entry a valid leaf entry? +* softleaf_is_valid_pmd_entry() - Is the specified softleaf entry obtained from +* a PMD one that we support at PMD level? +* @entry: Entry to check. +* Returns: true if the softleaf entry is valid at PMD, otherwise false. +*/ +static inline bool softleaf_is_valid_pmd_entry (softleaf_t entry) +{ +/* Only device private, migration entries valid for PMD. */ +return softleaf_is_device_private (entry) || +softleaf_is_migration (entry); +} + +/** +* pmd_is_valid_softleaf() - Is this PMD entry a valid softleaf entry? * @pmd: PMD entry. * * PMD leaf entries are valid only if they are device private or migration
@@ -616,9 +629,27 @@ static inline bool pmd_is_valid_softleaf {const softleaf_t entry =softleaf_from_pmd (pmd); -/* Only device private, migration entries valid for PMD. */ -return softleaf_is_device_private (entry) || -softleaf_is_migration (entry); +return softleaf_is_valid_pmd_entry (entry); +} + +/** +* pmd_to_softleaf_folio() - Convert the PMD entry to a folio. +* @pmd: PMD entry. +* +* The PMD entry is expected to be a valid PMD softleaf entry. +* +* Returns: the folio the softleaf entry references if this is a valid softleaf +* entry, otherwise NULL. +*/ +static inline struct folio *pmd_to_softleaf_folio (pmd_t pmd) +{ +const softleaf_t entry =softleaf_from_pmd (pmd); + +if (!softleaf_is_valid_pmd_entry (entry)) { +VM_WARN_ON_ONCE (true); +return NULL ; + } +return softleaf_to_folio (entry); }#endif /* CONFIG_MMU */
mm/huge_memory.c renamed from mm/huge_memory.c~b +67 -48
146 lines not on this page · raw message
_
Lorenzo Stoakes (Oracle)
reviewerI really don't want to have to reply to every bit of noise generated by Sashiko, but for the sake of it, the stuff I'm not changing:
4/13
Quoted 1 line
Are other PMD handlers vulnerable to this same userfaultfd bug?
That's completely out of scope for the series. It's not a regression, it's a suggestion for future work and so shouldn't be labelled as such by it, instead of listing a 'high' priority regression.
7/13
Quoted 2 lines
Is it possible for the error path immediately preceding this block to trigger a NULL pointer dereference?
You're already at the point where something has gone horribly wrong and a bug in the kernel renders it unstable. It's not really worth trying to avoid every possible bad outcome here in case of kernel bugs, that's not practical and not worth the maintenance load to put such things in.
Also if I add this, it breaks further refactorings for the sake of defending against a specific class of kernel bug - that's not worthwhile.
10/13
Quoted 2 lines
Could evaluating
folio_is_device_private(folio)cause issues if the PMD contains a migration entry rather than a device private entry?
If this is even possible (I don't think so), that was an existing issue. This is a refactoring, it'd not be appropriate for me to fundamentally change behaviour at the same time.
Quoted 2 lines
This isn't a bug, but the comment still refers to
flush_needed, which was renamed tois_presentin this patch.
Baolin already raised, and I don't think it really matters to leave that comment in there as it's removed in a later commit, and a comment isn't a bisection hazard :)
11/13
Quoted 2 lines
This isn't a bug, but the kernel-doc for
pmd_is_valid_softleaf()states that it asserts the validity of the entry, while the function strictly
This is a turn of phrase...! Anybody wondering can go read the function.
Quoted 4 lines
returns a boolean without triggering any warnings or bugs. Would it be better to update this comment to reflect the actual behavior, especially now that an actual assertion has been added to the neighboring
pmd_to_softleaf_folio()function?
I think the CONFIG_DEBUG_VM assert itself is pretty good documentation of there being a CONFIG_DEBUG_VM assert honestly. Should the kdoc comments list the code too?
Quoted 9 lines
Could this warning be written to evaluate the condition directly?
if (VM_WARN_ON_ONCE(!softleaf_is_valid_pmd_entry(entry))) { return NULL; }When
VM_WARN_ON_ONCE(true)is placed inside an if block, the kernel's warning machinery stringifies and prints "true" as the failing condition in the backtrace, which makes debugging more difficult. Wrapping the actual condition inside the warning macro ensures the specific violated constraint is visible in the console output.
This is a silly comment anyway, you can figure out why the thing failed very easily and this is a common pattern in the kernel.
But this is also a hallucination, VM_WARN_ON_ONCE() is defined as:
#define VM_WARN_ON_ONCE(cond) (void)WARN_ON_ONCE(cond)
So you know, that won't work, and even if I did it's a silly and pedantic comment. Plus you don't use {} for single line branches...
12/13:
While the comment about deposit was valid, the comment:
Quoted 3 lines
For non-DAX special VMAs, this also forces
has_depositto true even if the architecture does not need a deposit, potentially attempting to free a non-existent deposit.
Is another hallucination.:
else if (is_huge_zero_pmd(orig_pmd)) has_deposit = !vma_is_dax(vma);
This is the line it's discussing. So we're explicitly gating on is_huge_zero_pmd(). It's not any 'special' VMA.
And in the original code:
} else if (is_huge_zero_pmd(orig_pmd)) {
if (!vma_is_dax(vma) || arch_needs_pgtable_deposit())
zap_deposited_table(tlb->mm, pmd);
...
}With the fix-patch applied this is 'has_deposit = has_deposit || !vma_is_dax()' where has_deposit is initialised with arch_needs_pgtable_deposit(), so the logic matches.
--
Dealing with the above has taken a lot of time that I would rather spend doing other things.
AI can asymmetrically drown us with this kind of stuff, radically increasing workload.
This continues to be my primary concern with the application of AI, and the only acceptable use of it will be in cases where we are able to filter things well enough to avoid wasting people's time like this.
As I've said before, burnout continues to be a major hazard that is simply being ignored in mm, and I don't think that's a healthy or good thing.
Let's please be considerate as to the opinions of those actually doing the work.
Thanks, Lorenzo