Skip to contentSkip to conversation

mm-mremap-hold-the-rmap-lock-in-write-mode-when-moving-page-table-entries.patch removed from -mm tree

merged [email protected] 1 participant 1 message Jul 9, 2021 Stable branch

lore
Share & tools
lore

[email protected]

author tree notice Applied
#0 lore raw

The patch titled
Subject: mm/mremap: hold the rmap lock in write mode when moving page table entries.
has been removed from the -mm tree. Its filename was
mm-mremap-hold-the-rmap-lock-in-write-mode-when-moving-page-table-entries.patch

This patch was dropped because it was merged into mainline or a subsystem tree

------------------------------------------------------
From: "Aneesh Kumar K.V" <[email protected]>
Subject: mm/mremap: hold the rmap lock in write mode when moving page table entries.

To avoid a race between rmap walk and mremap, mremap does take_rmap_locks(). The lock was taken to ensure that rmap walk don't miss a page table entry due to PTE moves via move_pagetables(). The kernel does further optimization of this lock such that if we are going to find the newly added vma after the old vma, the rmap lock is not taken. This is because rmap walk would find the vmas in the same order and if we don't find the page table attached to older vma we would find it with the new vma which we would iterate later.

As explained in commit eb66ae030829 ("mremap: properly flush TLB before releasing the page") mremap is special in that it doesn't take ownership of the page. The optimized version for PUD/PMD aligned mremap also doesn't hold the ptl lock. This can result in stale TLB entries as show below.

This patch updates the rmap locking requirement in mremap to handle the race condition explained below with optimized mremap::

Optmized PMD move

CPU 1 CPU 2 CPU 3

mremap(old_addr, new_addr) page_shrinker/try_to_unmap_one

mmap_write_lock_killable()

addr = old_addr lock(pte_ptl)
lock(pmd_ptl)
pmd = *old_pmd
pmd_clear(old_pmd) flush_tlb_range(old_addr)

*new_pmd = pmd
*new_addr = 10; and fills TLB with new addr and old pfn

unlock(pmd_ptl)
ptep_clear_flush()
old pfn is free.
Stale TLB entry

Optimized PUD move also suffers from a similar race. Both the above race condition can be fixed if we force mremap path to take rmap lock.

Fixes 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
Fixes c49dd3401802 ("mm: speedup mremap on 1GB or larger regions")
Cc Christophe Leroy <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Kalesh Singh <[email protected]>
Cc Kirill A. Shutemov <[email protected]>
Cc Michael Ellerman <[email protected]>
Cc Nicholas Piggin <[email protected]>
Cc Stephen Rothwell <[email protected]>

---

 mm/mremap.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/mremap.c renamed from mm/mremap.c~mm-mremap-hold-the-rmap-lock-in-write-mode-when-moving-page-table-entries +2 -2
@@ -504,7 +504,7 @@ unsigned long move_page_tables(struct vm
 		} else if (IS_ENABLED(CONFIG_HAVE_MOVE_PUD) && extent == PUD_SIZE) {
 
 			if (move_pgt_entry(NORMAL_PUD, vma, old_addr, new_addr,
-					   old_pud, new_pud, need_rmap_locks))
+					   old_pud, new_pud, true))
 				continue;
 		}
 
@@ -531,7 +531,7 @@ unsigned long move_page_tables(struct vm
 			 * moving at the PMD level if possible.
 			 */
 			if (move_pgt_entry(NORMAL_PMD, vma, old_addr, new_addr,
-					   old_pmd, new_pmd, need_rmap_locks))
+					   old_pmd, new_pmd, true))
 				continue;
 		}
 

_

Patches currently in -mm which might be from [email protected] are