mm: centralize+fix comments about compound_mapcount() in new sync_with_folio_pmd_zap()
Subsystems (3)
David Hildenbrand (Arm)
maintainerPATCH 1/1 mm: centralize+fix comments about compound_mapcount() in new sync_with_folio_pmd_zap()
We still mention compound_mapcount() in two comments.
Instead of simply referring to the folio mapcount in both places, let's factor out the odd-looking PTL sync into sync_with_folio_pmd_zap(), and add centralized documentation why this is required.
--- mm/internal.h | 19 +++++++++++++++++++ mm/memory.c | 8 +------- mm/page_vma_mapped.c | 11 ++--------- 3 files changed, 22 insertions(+), 16 deletions(-)
mm/internal.h +19 -0
@@ -516,6 +516,25 @@ void free_pgtables(struct mmu_gather *tlb, struct unmap_desc *desc);void pmd_install (struct mm_struct *mm,pmd_t *pmd,pgtable_t *pte); +/** +* sync_with_folio_pmd_zap - sync with concurrent zapping of a folio PMD +* @mm: The mm_struct. +* @pmdp: Pointer to the pmd that was found to be pmd_none(). +* +* When we stumble over a pmd_none() without holding the PTL while unmapping a +* folio that could have been mapped at that PMD, it could be that concurrent +* zapping of the PMD is not complete yet. While the PMD might be pmd_none() +* already, the folio might still appear to be mapped (folio_mapped()). +* +* Wait for concurrent zapping to complete by grabbing the PTL. +*/ +static inline void sync_with_folio_pmd_zap (struct mm_struct *mm,pmd_t *pmdp) +{ +spinlock_t *ptl =pmd_lock (mm, pmdp); + +spin_unlock (ptl); +} +struct zap_details ; void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
mm/memory.c +1 -7
@@ -2006,13 +2006,7 @@ static inline unsigned long zap_pmd_range(struct mmu_gather *tlb, }else if (details && details->single_folio &&folio_test_pmd_mappable (details->single_folio) && next - addr ==HPAGE_PMD_SIZE &&pmd_none (*pmd)) { -spinlock_t *ptl =pmd_lock (tlb->mm, pmd); -/* -* Take and drop THP pmd lock so that we cannot return -* prematurely, while zap_huge_pmd() has cleared *pmd, -* but not yet decremented compound_mapcount(). -*/ -spin_unlock (ptl); +sync_with_folio_pmd_zap (tlb->mm, pmd); }if (pmd_none (*pmd)) { addr = next;
mm/page_vma_mapped.c +2 -9
@@ -269,11 +269,6 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)spin_unlock (pvmw->ptl); pvmw->ptl =NULL ; }else if (!pmd_present (pmde)) { -/* -* If PVMW_SYNC, take and drop THP pmd lock so that we -* cannot return prematurely, while zap_huge_pmd() has -* cleared *pmd but not decremented compound_mapcount(). -*/ const softleaf_t entry =softleaf_from_pmd (pmde);if (softleaf_is_device_private (entry)) {
@@ -284,11 +279,9 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)if ((pvmw->flags &PVMW_SYNC ) &&thp_vma_suitable_order (vma, pvmw->address,PMD_ORDER ) && - (pvmw->nr_pages >=HPAGE_PMD_NR )) { -spinlock_t *ptl =pmd_lock (mm, pvmw->pmd); + (pvmw->nr_pages >=HPAGE_PMD_NR )) +sync_with_folio_pmd_zap (mm, pvmw->pmd); -spin_unlock (ptl); - }step_forward (pvmw,PMD_SIZE );continue ; }
Signature · 1 line
2.43.0
Quoted 14 lines · David Hildenbrand (Arm)
void
pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte);+/**
+ *sync_with_folio_pmd_zap- sync with concurrent zapping of a folio PMD
+ * @mm: Themm_struct.
+ * @pmdp: Pointer to the pmd that was found to bepmd_none().
+ *
+ * When we stumble over apmd_none()without holding the PTL while unmapping a
+ * folio that could have been mapped at that PMD, it could be that concurrent
+ * zapping of the PMD is not complete yet. While the PMD might bepmd_none()
+ * already, the folio might still appear to be mapped (folio_mapped()).
+ *
+ * Wait for concurrent zapping to complete by grabbing the PTL.
+ */
I like this. The one thing we've lost is the name of the function which does the zapping, which I think was a helpful detail. Perhaps not to someone who's deep in "how page tables work", but I wouldn't know where to look for the counterpart to this. So how about:
Option A:
+ * When we stumble over a pmd_none() without holding the PTL while
+ * unmapping a folio that could have been mapped at that PMD,
+ * zap_huge_pmd() may not be complete yet. While the PMD might be pmd_none()
+ * already, the folio might still appear to be mapped (folio_mapped()).
Option B:
+ * When we find a pmd_none() while unmapping a folio without holding
+ * the PTL, zap_huge_pmd() may have cleared the PMD but not yet
+ * modified the folio to indicate that it's unmapped.
(for both options, I'm just changing that one paragraph; the paragraph starting "Wait", I would leave unchanged)
Quoted 63 lines
mm/internal.h +7
-0+static inline void sync_with_folio_pmd_zap (struct mm_struct *mm,pmd_t *pmdp) +{ +spinlock_t *ptl =pmd_lock (mm, pmdp); + +spin_unlock (ptl); +} +struct zap_details ; void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma,mm/memory.c +1
-7@@ -2006,13 +2006,7 @@ static inline unsigned long zap_pmd_range(struct mmu_gather *tlb, }else if (details && details->single_folio &&folio_test_pmd_mappable (details->single_folio) && next - addr ==HPAGE_PMD_SIZE &&pmd_none (*pmd)) { -spinlock_t *ptl =pmd_lock (tlb->mm, pmd); -/* -* Take and drop THP pmd lock so that we cannot return -* prematurely, while zap_huge_pmd() has cleared *pmd, -* but not yet decremented compound_mapcount(). -*/ -spin_unlock (ptl); +sync_with_folio_pmd_zap (tlb->mm, pmd); }if (pmd_none (*pmd)) { addr = next;mm/page_vma_mapped.c +2
-9@@ -269,11 +269,6 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)spin_unlock (pvmw->ptl); pvmw->ptl =NULL ; }else if (!pmd_present (pmde)) { -/* -* If PVMW_SYNC, take and drop THP pmd lock so that we -* cannot return prematurely, while zap_huge_pmd() has -* cleared *pmd but not decremented compound_mapcount(). -*/ const softleaf_t entry =softleaf_from_pmd (pmde);if (softleaf_is_device_private (entry)) {@@ -284,11 +279,9 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)if ((pvmw->flags &PVMW_SYNC ) &&thp_vma_suitable_order (vma, pvmw->address,PMD_ORDER ) && - (pvmw->nr_pages >=HPAGE_PMD_NR )) { -spinlock_t *ptl =pmd_lock (mm, pvmw->pmd); + (pvmw->nr_pages >=HPAGE_PMD_NR )) +sync_with_folio_pmd_zap (mm, pvmw->pmd); -spin_unlock (ptl); - }step_forward (pvmw,PMD_SIZE );continue ; }Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerQuoted 16 lines · Matthew Wilcox
Quoted 14 lines · David Hildenbrand (Arm)
void
pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte);+/**
+ *sync_with_folio_pmd_zap- sync with concurrent zapping of a folio PMD
+ * @mm: Themm_struct.
+ * @pmdp: Pointer to the pmd that was found to bepmd_none().
+ *
+ * When we stumble over apmd_none()without holding the PTL while unmapping a
+ * folio that could have been mapped at that PMD, it could be that concurrent
+ * zapping of the PMD is not complete yet. While the PMD might bepmd_none()
+ * already, the folio might still appear to be mapped (folio_mapped()).
+ *
+ * Wait for concurrent zapping to complete by grabbing the PTL.
+ */
Quoted 2 lines
I like this. The one thing we've lost is the name of the function which does the zapping, which I think was a helpful detail.
I dropped it as we seem to have a talent to rename functions but to forget about updating comments.
This very patch is an example of that :)
But well, I used folio_mapped() ... so ...
Quoted 10 lines
Perhaps not to
someone who's deep in "how page tables work", but I wouldn't know where to look for the counterpart to this. So how about:Option A:
+ * When we stumble over apmd_none()without holding the PTL while
+ * unmapping a folio that could have been mapped at that PMD,
+ *zap_huge_pmd()may not be complete yet. While the PMD might bepmd_none()
+ * already, the folio might still appear to be mapped (folio_mapped()).
Quoted 4 lines
Option B:
+ * When we find apmd_none()while unmapping a folio without holding
+ * the PTL,zap_huge_pmd()may have cleared the PMD but not yet
+ * modified the folio to indicate that it's unmapped.
Let's do a slightly longer B:
* When we find a pmd_none() while unmapping a folio without holding
* the PTL, zap_huge_pmd() may have cleared the PMD but not yet
* modified the folio to indicate that it's unmapped. Skipping the PMD
* without synchronization could make folio unmapping code assume
* that unmapping failed.
Good? If so, Andrew can you squash or should I resend?
Thanks for the review!
Signature · 3 lines
Cheers, David