Skip to contentSkip to conversation

mm-mprotect-push-mmu-notifier-to-puds.patch removed from -mm tree

merged mm-stable Andrew Morton 1 participant 1 message Sep 2, 2024

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: mm/mprotect: push mmu notifier to PUDs
has been removed from the -mm tree. Its filename was
mm-mprotect-push-mmu-notifier-to-puds.patch

This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: Peter Xu <[email protected]>
Subject: mm/mprotect: push mmu notifier to PUDs
Date: Mon, 12 Aug 2024 14:12:20 -0400

mprotect() does mmu notifiers in PMD levels. It's there since 2014 of commit a5338093bfb4 ("mm: move mmu notifier call from change_protection to change_pmd_range").

At that time, the issue was that NUMA balancing can be applied on a huge range of VM memory, even if nothing was populated. The notification can be avoided in this case if no valid pmd detected, which includes either THP or a PTE pgtable page.

Now to pave way for PUD handling, this isn't enough. We need to generate mmu notifications even on PUD entries properly. mprotect() is currently broken on PUD (e.g., one can easily trigger kernel error with dax 1G mappings already), this is the start to fix it.

To fix that, this patch proposes to push such notifications to the PUD layers.

There is risk on regressing the problem Rik wanted to resolve before, but I think it shouldn't really happen, and I still chose this solution because of a few reasons:

1) Consider a large VM that should definitely contain more than GBs of memory, it's highly likely that PUDs are also none. In this case there will have no regression.

2) KVM has evolved a lot over the years to get rid of rmap walks, which might be the major cause of the previous soft-lockup. At least TDP MMU already got rid of rmap as long as not nested (which should be the major use case, IIUC), then the TDP MMU pgtable walker will simply see empty VM pgtable (e.g. EPT on x86), the invalidation of a full empty region in most cases could be pretty fast now, comparing to 2014.

3) KVM has explicit code paths now to even give way for mmu notifiers just like this one, e.g. in commit d02c357e5bfa ("KVM: x86/mmu: Retry fault before acquiring mmu_lock if mapping is changing"). It'll also avoid contentions that may also contribute to a soft-lockup.

4) Stick with PMD layer simply don't work when PUD is there... We need one way or another to fix PUD mappings on mprotect().

Pushing it to PUD should be the safest approach as of now, e.g. there's yet no sign of huge P4D coming on any known archs.

Cc Sean Christopherson <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc David Rientjes <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Christophe Leroy <[email protected]>
Cc Dan Williams <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Dave Jiang <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc "Edgecombe, Rick P" <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Kirill A. Shutemov <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Michael Ellerman <[email protected]>
Cc Nicholas Piggin <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vlastimil Babka <[email protected]>

---

 mm/mprotect.c |   32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)
mm/mprotect.c renamed from mm/mprotect.c~mm-mprotect-push-mmu-notifier-to-puds +16 -16
@@ -363,9 +363,6 @@ static inline long change_pmd_range(stru
 	unsigned long next;
 	long pages = 0;
 	unsigned long nr_huge_updates = 0;
-	struct mmu_notifier_range range;
-
-	range.start = 0;
 
 	pmd = pmd_offset(pud, addr);
 	do {
@@ -383,14 +380,6 @@ again:
 		if (pmd_none(*pmd))
 			goto next;
 
-		/* invoke the mmu notifier if the pmd is populated */
-		if (!range.start) {
-			mmu_notifier_range_init(&range,
-				MMU_NOTIFY_PROTECTION_VMA, 0,
-				vma->vm_mm, addr, end);
-			mmu_notifier_invalidate_range_start(&range);
-		}
-
 		_pmd = pmdp_get_lockless(pmd);
 		if (is_swap_pmd(_pmd) || pmd_trans_huge(_pmd) || pmd_devmap(_pmd)) {
 			if ((next - addr != HPAGE_PMD_SIZE) ||
@@ -431,9 +420,6 @@ next:
 		cond_resched();
 	} while (pmd++, addr = next, addr != end);
 
-	if (range.start)
-		mmu_notifier_invalidate_range_end(&range);
-
 	if (nr_huge_updates)
 		count_vm_numa_events(NUMA_HUGE_PTE_UPDATES, nr_huge_updates);
 	return pages;
@@ -443,22 +429,36 @@ static inline long change_pud_range(stru
 		struct vm_area_struct *vma, p4d_t *p4d, unsigned long addr,
 		unsigned long end, pgprot_t newprot, unsigned long cp_flags)
 {
+	struct mmu_notifier_range range;
 	pud_t *pud;
 	unsigned long next;
 	long pages = 0, ret;
 
+	range.start = 0;
+
 	pud = pud_offset(p4d, addr);
 	do {
 		next = pud_addr_end(addr, end);
 		ret = change_prepare(vma, pud, pmd, addr, cp_flags);
-		if (ret)
-			return ret;
+		if (ret) {
+			pages = ret;
+			break;
+		}
 		if (pud_none_or_clear_bad(pud))
 			continue;
+		if (!range.start) {
+			mmu_notifier_range_init(&range,
+						MMU_NOTIFY_PROTECTION_VMA, 0,
+						vma->vm_mm, addr, end);
+			mmu_notifier_invalidate_range_start(&range);
+		}
 		pages += change_pmd_range(tlb, vma, pud, addr, next, newprot,
 					  cp_flags);
 	} while (pud++, addr = next, addr != end);
 
+	if (range.start)
+		mmu_notifier_invalidate_range_end(&range);
+
 	return pages;
 }
 

_

Patches currently in -mm which might be from [email protected] are

mm-introduce-arch_supports_huge_pfnmap-and-special-bits-to-pmd-pud.patch mm-drop-is_huge_zero_pud.patch mm-mark-special-bits-for-huge-pfn-mappings-when-inject.patch mm-allow-thp-orders-for-pfnmaps.patch mm-gup-detect-huge-pfnmap-entries-in-gup-fast.patch mm-pagewalk-check-pfnmap-for-folio_walk_start.patch mm-pagewalk-check-pfnmap-for-folio_walk_start-fix.patch mm-fork-accept-huge-pfnmap-entries.patch mm-always-define-pxx_pgprot.patch mm-new-follow_pfnmap-api.patch kvm-use-follow_pfnmap-api.patch s390-pci_mmio-use-follow_pfnmap-api.patch mm-x86-pat-use-the-new-follow_pfnmap-api.patch vfio-use-the-new-follow_pfnmap-api.patch acrn-use-the-new-follow_pfnmap-api.patch mm-access_process_vm-use-the-new-follow_pfnmap-api.patch mm-remove-follow_pte.patch mm-x86-support-large-pfn-mappings.patch mm-arm64-support-large-pfn-mappings.patch