Skip to contentSkip to conversation

hugetlbfs: move lock assertions after early returns in huge_pmd_unshare()

PATCH Superseded Deepanshu Kartikey 2 participants 3 messages Oct 14, 2025

Subsystems (2)

v6 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

PATCH v6 1/1 hugetlbfs: move lock assertions after early returns in huge_pmd_unshare()

When hugetlb_vmdelete_list() processes VMAs during truncate operations, it may encounter VMAs where huge_pmd_unshare() is called without the required shareable lock. This triggers an assertion failure in hugetlb_vma_assert_locked().

The previous fix in commit dd83609b8898 ("hugetlbfs: skip VMAs without shareable locks in hugetlb_vmdelete_list") skipped entire VMAs without shareable locks to avoid the assertion. However, this prevented pages from being unmapped and freed, causing a regression in fallocate(PUNCH_HOLE) operations where pages were not freed immediately, as reported by Mark Brown.

A subsequent fix in commit 06e8ca1b3dca ("hugetlbfs: check for shareable lock before calling huge_pmd_unshare()") addressed this by checking __vma_shareable_lock() in the caller before calling huge_pmd_unshare(). However, a cleaner approach is to move the lock assertions in huge_pmd_unshare() itself to after the early return checks. The assertions are only needed when actual PMD unsharing work will be performed. If the function returns early because sz != PMD_SIZE or the PMD is not shared, no locks are required.

This patch removes the check added in commit 06e8ca1b3dca ("hugetlbfs:
check for shareable lock before calling huge_pmd_unshare()") and instead moves the assertions inside huge_pmd_unshare(), keeping all the logic within the function itself.

---
Changes in v6:
- Remove __vma_shareable_lock() check from __unmap_hugepage_range() that was added in v4 (commit 06e8ca1b3dca)
- Move lock assertions after early returns in huge_pmd_unshare()
- Complete implementation of David's cleaner approach

Changes in v5:
- Incomplete: only moved assertions, forgot to remove v4 check

Changes in v4:
- Check __vma_shareable_lock() in __unmap_hugepage_range() before calling huge_pmd_unshare() per Oscar's suggestion

Changes in v3:
- Add ZAP_FLAG_NO_UNSHARE to skip only PMD unsharing

Changes in v2:
- Skip entire VMAs without shareable locks (caused PUNCH_HOLE regression)

Changes in v1:
- Initial fix attempt

---
 mm/hugetlb.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
mm/hugetlb.c +3 -4
index 85b2dac79d25..0455119716ec 100644
@@ -5885,7 +5885,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
 		}
 
 		ptl = huge_pte_lock(h, mm, ptep);
-		if (__vma_shareable_lock(vma) && huge_pmd_unshare(mm, vma, address, ptep)) {
+		if (huge_pmd_unshare(mm, vma, address, ptep)) {
 			spin_unlock(ptl);
 			tlb_flush_pmd_range(tlb, address & PUD_MASK, PUD_SIZE);
 			force_flush = true;
@@ -7614,13 +7614,12 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma,
 	p4d_t *p4d = p4d_offset(pgd, addr);
 	pud_t *pud = pud_offset(p4d, addr);
 
-	i_mmap_assert_write_locked(vma->vm_file->f_mapping);
-	hugetlb_vma_assert_locked(vma);
 	if (sz != PMD_SIZE)
 		return 0;
 	if (!ptdesc_pmd_is_shared(virt_to_ptdesc(ptep)))
 		return 0;
-
+	i_mmap_assert_write_locked(vma->vm_file->f_mapping);
+	hugetlb_vma_assert_locked(vma);
 	pud_clear(pud);
 	/*
 	 * Once our caller drops the rmap lock, some other process might be
Signature · 1 line
2.34.1
#1 lore raw
Quoted 83 lines · Deepanshu Kartikey

When hugetlb_vmdelete_list() processes VMAs during truncate operations, it may encounter VMAs where huge_pmd_unshare() is called without the required shareable lock. This triggers an assertion failure in hugetlb_vma_assert_locked().

The previous fix in commit dd83609b8898 ("hugetlbfs: skip VMAs without shareable locks in hugetlb_vmdelete_list") skipped entire VMAs without shareable locks to avoid the assertion. However, this prevented pages from being unmapped and freed, causing a regression in fallocate(PUNCH_HOLE) operations where pages were not freed immediately, as reported by Mark Brown.

A subsequent fix in commit 06e8ca1b3dca ("hugetlbfs: check for shareable lock before calling huge_pmd_unshare()") addressed this by checking __vma_shareable_lock() in the caller before calling huge_pmd_unshare(). However, a cleaner approach is to move the lock assertions in huge_pmd_unshare() itself to after the early return checks. The assertions are only needed when actual PMD unsharing work will be performed. If the function returns early because sz != PMD_SIZE or the PMD is not shared, no locks are required.

This patch removes the check added in commit 06e8ca1b3dca ("hugetlbfs:
check for shareable lock before calling huge_pmd_unshare()") and instead moves the assertions inside huge_pmd_unshare(), keeping all the logic within the function itself.

---
Changes in v6:
- Remove __vma_shareable_lock() check from __unmap_hugepage_range() that was added in v4 (commit 06e8ca1b3dca)
- Move lock assertions after early returns in huge_pmd_unshare()
- Complete implementation of David's cleaner approach

Changes in v5:
- Incomplete: only moved assertions, forgot to remove v4 check

Changes in v4:
- Check __vma_shareable_lock() in __unmap_hugepage_range() before calling huge_pmd_unshare() per Oscar's suggestion

Changes in v3:
- Add ZAP_FLAG_NO_UNSHARE to skip only PMD unsharing

Changes in v2:
- Skip entire VMAs without shareable locks (caused PUNCH_HOLE regression)

Changes in v1:
- Initial fix attempt

---
  mm/hugetlb.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
mm/hugetlb.c +1 -3
index 85b2dac79d25..0455119716ec 100644
@@ -5885,7 +5885,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
  		}
 
  		ptl = huge_pte_lock(h, mm, ptep);
-		if (__vma_shareable_lock(vma) && huge_pmd_unshare(mm, vma, address, ptep)) {
+		if (huge_pmd_unshare(mm, vma, address, ptep)) {
  			spin_unlock(ptl);
  			tlb_flush_pmd_range(tlb, address & PUD_MASK, PUD_SIZE);
  			force_flush = true;
@@ -7614,13 +7614,12 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma,
  	p4d_t *p4d = p4d_offset(pgd, addr);
  	pud_t *pud = pud_offset(p4d, addr);
 
-	i_mmap_assert_write_locked(vma->vm_file->f_mapping);
-	hugetlb_vma_assert_locked(vma);

mm/hugetlb.c int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, · line 7625

Quoted 10 lines
mm/hugetlb.c +2 -1
  	if (sz != PMD_SIZE)
  		return 0;
  	if (!ptdesc_pmd_is_shared(virt_to_ptdesc(ptep)))
  		return 0;
-
+	i_mmap_assert_write_locked(vma->vm_file->f_mapping);
+	hugetlb_vma_assert_locked(vma);
  	pud_clear(pud);
  	/*
  	 * Once our caller drops the rmap lock, some other process might be

Can you resend based on mm/mm-stable please? Looks like this one is based on v5.

Signature · 3 lines
Cheers

David / dhildenb
#2 lore raw

Hi David,

I've sent v8 rebased on mm/mm-stable as you requested. It includes both the skip code revert and the assertion movement.

Thanks,
Deepanshu