Skip to contentSkip to conversation

mm-memory-optimize-fork-with-pte-mapped-thp.patch removed from -mm tree

merged mm-stable Andrew Morton 1 participant 1 message Feb 22, 2024

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: mm/memory: optimize fork() with PTE-mapped THP
has been removed from the -mm tree. Its filename was
mm-memory-optimize-fork-with-pte-mapped-thp.patch

This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: David Hildenbrand <[email protected]>
Subject: mm/memory: optimize fork() with PTE-mapped THP
Date: Mon, 29 Jan 2024 13:46:47 +0100

Let's implement PTE batching when consecutive (present) PTEs map consecutive pages of the same large folio, and all other PTE bits besides the PFNs are equal.

We will optimize folio_pte_batch() separately, to ignore selected PTE bits. This patch is based on work by Ryan Roberts.

Use __always_inline for __copy_present_ptes() and keep the handling for single PTEs completely separate from the multi-PTE case: we really want the compiler to optimize for the single-PTE case with small folios, to not degrade performance.

Note that PTE batching will never exceed a single page table and will always stay within VMA boundaries.

Further, processing PTE-mapped THP that maybe pinned and have PageAnonExclusive set on at least one subpage should work as expected, but there is room for improvement: We will repeatedly (1) detect a PTE batch
(2) detect that we have to copy a page (3) fall back and allocate a single page to copy a single page. For now we won't care as pinned pages are a corner case, and we should rather look into maintaining only a single PageAnonExclusive bit for large folios.

Cc Albert Ou <[email protected]>
Cc Alexander Gordeev <[email protected]>
Cc Alexandre Ghiti <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Christian Borntraeger <[email protected]>
Cc Christophe Leroy <[email protected]>
Cc David S. Miller <[email protected]>
Cc Dinh Nguyen <[email protected]>
Cc Gerald Schaefer <[email protected]>
Cc Heiko Carstens <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Michael Ellerman <[email protected]>
Cc Naveen N. Rao <[email protected]>
Cc Nicholas Piggin <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc Paul Walmsley <[email protected]>
Cc Russell King (Oracle) <[email protected]>
Cc Sven Schnelle <[email protected]>
Cc Vasily Gorbik <[email protected]>
Cc Will Deacon <[email protected]>

---

 include/linux/pgtable.h |   31 ++++++++++
 mm/memory.c             |  112 +++++++++++++++++++++++++++++++-------
 2 files changed, 124 insertions(+), 19 deletions(-)
include/linux/pgtable.h renamed from include/linux/pgtable.h~mm-memory-optimize-fork-with-pte-mapped-thp +31 -0
@@ -650,6 +650,37 @@ static inline void ptep_set_wrprotect(st
 }
 #endif
 
+#ifndef wrprotect_ptes
+/**
+ * wrprotect_ptes - Write-protect PTEs that map consecutive pages of the same
+ *		    folio.
+ * @mm: Address space the pages are mapped into.
+ * @addr: Address the first page is mapped at.
+ * @ptep: Page table pointer for the first entry.
+ * @nr: Number of entries to write-protect.
+ *
+ * May be overridden by the architecture; otherwise, implemented as a simple
+ * loop over ptep_set_wrprotect().
+ *
+ * Note that PTE bits in the PTE range besides the PFN can differ. For example,
+ * some PTEs might be write-protected.
+ *
+ * Context: The caller holds the page table lock.  The PTEs map consecutive
+ * pages that belong to the same folio.  The PTEs are all in the same PMD.
+ */
+static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,
+		pte_t *ptep, unsigned int nr)
+{
+	for (;;) {
+		ptep_set_wrprotect(mm, addr, ptep);
+		if (--nr == 0)
+			break;
+		ptep++;
+		addr += PAGE_SIZE;
+	}
+}
+#endif
+
 /*
  * On some architectures hardware does not set page access bit when accessing
  * memory page, it is responsibility of software setting this bit. It brings
mm/memory.c renamed from mm/memory.c~mm-memory-optimize-fork-with-pte-mapped-thp +93 -19

190 lines not on this page · raw message

_

Patches currently in -mm which might be from [email protected] are

mm-memory-factor-out-zapping-of-present-pte-into-zap_present_pte.patch mm-memory-handle-page-case-in-zap_present_pte-separately.patch mm-memory-further-separate-anon-and-pagecache-folio-handling-in-zap_present_pte.patch mm-memory-factor-out-zapping-folio-pte-into-zap_present_folio_pte.patch mm-mmu_gather-pass-delay_rmap-instead-of-encoded-page-to-__tlb_remove_page_size.patch mm-mmu_gather-define-encoded_page_flag_delay_rmap.patch mm-mmu_gather-add-tlb_remove_tlb_entries.patch mm-mmu_gather-add-__tlb_remove_folio_pages.patch mm-mmu_gather-improve-cond_resched-handling-with-large-folios-and-expensive-page-freeing.patch mm-memory-optimize-unmap-zap-with-pte-mapped-thp.patch