Skip to contentSkip to conversation

mm-khugepaged-fix-gup-fast-interaction-by-sending-ipi.patch removed from -mm tree

merged mm-hotfixes-stable Andrew Morton 1 participant 1 message Nov 30, 2022 Stable branch

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: mm/khugepaged: fix GUP-fast interaction by sending IPI
has been removed from the -mm tree. Its filename was
mm-khugepaged-fix-gup-fast-interaction-by-sending-ipi.patch

This patch was dropped because it was merged into the mm-hotfixes-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: Jann Horn <[email protected]>
Subject: mm/khugepaged: fix GUP-fast interaction by sending IPI
Date: Fri, 25 Nov 2022 22:37:13 +0100

Since commit 70cbc3cc78a99 ("mm: gup: fix the fast GUP race against THP collapse"), the lockless_pages_from_mm() fastpath rechecks the pmd_t to ensure that the page table was not removed by khugepaged in between.

However, lockless_pages_from_mm() still requires that the page table is not concurrently freed. Fix it by sending IPIs (if the architecture uses semi-RCU-style page table freeing) before freeing/reusing page tables.

---

 include/asm-generic/tlb.h |    4 ++++
 mm/khugepaged.c           |    2 ++
 mm/mmu_gather.c           |    4 +---
 3 files changed, 7 insertions(+), 3 deletions(-)
include/asm-generic/tlb.h renamed from include/asm-generic/tlb.h~mm-khugepaged-fix-gup-fast-interaction-by-sending-ipi +4 -0
@@ -222,12 +222,16 @@ extern void tlb_remove_table(struct mmu_
 #define tlb_needs_table_invalidate() (true)
 #endif
 
+void tlb_remove_table_sync_one(void);
+
 #else
 
 #ifdef tlb_needs_table_invalidate
 #error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE
 #endif
 
+static inline void tlb_remove_table_sync_one(void) { }
+
 #endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */
 
 
mm/khugepaged.c renamed from mm/khugepaged.c~mm-khugepaged-fix-gup-fast-interaction-by-sending-ipi +2 -0
@@ -1051,6 +1051,7 @@ static int collapse_huge_page(struct mm_
 	_pmd = pmdp_collapse_flush(vma, address, pmd);
 	spin_unlock(pmd_ptl);
 	mmu_notifier_invalidate_range_end(&range);
+	tlb_remove_table_sync_one();
 
 	spin_lock(pte_ptl);
 	result =  __collapse_huge_page_isolate(vma, address, pte, cc,
@@ -1410,6 +1411,7 @@ static void collapse_and_free_pmd(struct
 		lockdep_assert_held_write(&vma->anon_vma->root->rwsem);
 
 	pmd = pmdp_collapse_flush(vma, addr, pmdp);
+	tlb_remove_table_sync_one();
 	mm_dec_nr_ptes(mm);
 	page_table_check_pte_clear_range(mm, addr, pmd);
 	pte_free(mm, pmd_pgtable(pmd));
mm/mmu_gather.c renamed from mm/mmu_gather.c~mm-khugepaged-fix-gup-fast-interaction-by-sending-ipi +1 -3
@@ -153,7 +153,7 @@ static void tlb_remove_table_smp_sync(vo
 	/* Simply deliver the interrupt */
 }
 
-static void tlb_remove_table_sync_one(void)
+void tlb_remove_table_sync_one(void)
 {
 	/*
 	 * This isn't an RCU grace period and hence the page-tables cannot be
@@ -177,8 +177,6 @@ static void tlb_remove_table_free(struct
 
 #else /* !CONFIG_MMU_GATHER_RCU_TABLE_FREE */
 
-static void tlb_remove_table_sync_one(void) { }
-
 static void tlb_remove_table_free(struct mmu_table_batch *batch)
 {
 	__tlb_remove_table_free(batch);

_

Patches currently in -mm which might be from [email protected] are