mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags.patch removed from -mm tree
The quilt patch titled
Subject: mm: split folio_pte_batch() into folio_pte_batch() and folio_pte_batch_flags()
has been removed from the -mm tree. Its filename was
mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags.patch
This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
------------------------------------------------------
From: David Hildenbrand <[email protected]>
Subject: mm: split folio_pte_batch() into folio_pte_batch() and folio_pte_batch_flags()
Date: Wed, 2 Jul 2025 12:49:25 +0200
Many users (including upcoming ones) don't really need the flags etc, and can live with the possible overhead of a function call.
So let's provide a basic, non-inlined folio_pte_batch(), to avoid code bloat while still providing a variant that optimizes out all flag checks at runtime. folio_pte_batch_flags() will get inlined into folio_pte_batch(), optimizing out any conditionals that depend on input flags.
folio_pte_batch() will behave like folio_pte_batch_flags() when no flags are specified. It's okay to add new users of folio_pte_batch_flags(), but using folio_pte_batch() if applicable is preferred.
So, before this change, folio_pte_batch() was inlined into the C file optimized by propagating constants within the resulting object file.
With this change, we now also have a folio_pte_batch() that is optimized by propagating all constants. But instead of having one instance per object file, we have a single shared one.
In zap_present_ptes(), where we care about performance, the compiler already seem to generate a call to a common inlined folio_pte_batch() variant, shared with fork() code. So calling the new non-inlined variant should not make a difference.
While at it, drop the "addr" parameter that is unused.
---
mm/internal.h | 11 ++++++++--- mm/madvise.c | 4 ++-- mm/memory.c | 8 +++----- mm/mempolicy.c | 3 +-- mm/mlock.c | 3 +-- mm/mremap.c | 3 +-- mm/rmap.c | 3 +-- mm/util.c | 29 +++++++++++++++++++++++++++++ 8 files changed, 46 insertions(+), 18 deletions(-)
mm/internal.h renamed from mm/internal.h~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +8 -3
@@ -218,9 +218,8 @@ static inline pte_t __pte_batch_clear_ig }/** -* folio_pte_batch - detect a PTE batch for a large folio +* folio_pte_batch_flags - detect a PTE batch for a large folio * @folio: The large folio to detect a PTE batch for. -* @addr: The user virtual address the first page is mapped at. * @ptep: Page table pointer for the first entry. * @pte: Page table entry for the first page. * @max_nr: The maximum number of table entries to consider.
@@ -243,9 +242,12 @@ static inline pte_t __pte_batch_clear_ig* must be limited by the caller so scanning cannot exceed a single VMA and * a single page table. * +* This function will be inlined to optimize based on the input parameters; +* consider using folio_pte_batch() instead if applicable. +* * Return: the number of table entries in the batch. */ -static inline unsigned int folio_pte_batch (struct folio *folio,unsigned long addr, +static inline unsigned int folio_pte_batch_flags (struct folio *folio,pte_t *ptep,pte_t pte,unsigned int max_nr,fpb_t flags,bool *any_writable,bool *any_young,bool *any_dirty) {
@@ -293,6 +295,9 @@ static inline unsigned int folio_pte_batreturn min (nr, max_nr); } +unsigned int folio_pte_batch (struct folio *folio,pte_t *ptep,pte_t pte, +unsigned int max_nr); +/** * pte_move_swp_offset - Move the swap entry offset field of a swap pte * forward or backward by delta
mm/madvise.c renamed from mm/madvise.c~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +2 -2
@@ -348,8 +348,8 @@ static inline int madvise_folio_pte_batc {int max_nr = (end - addr) /PAGE_SIZE ; -return folio_pte_batch (folio, addr, ptep, pte, max_nr,0 ,NULL , - any_young, any_dirty); +return folio_pte_batch_flags (folio, ptep, pte, max_nr,0 ,NULL , + any_young, any_dirty); } static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
mm/memory.c renamed from mm/memory.c~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +3 -5
@@ -995,8 +995,8 @@ copy_present_ptes(struct vm_area_structif (vma_soft_dirty_enabled (src_vma)) flags |=FPB_RESPECT_SOFT_DIRTY ; - nr =folio_pte_batch (folio, addr, src_pte, pte, max_nr, flags, - &any_writable,NULL ,NULL ); + nr =folio_pte_batch_flags (folio, src_pte, pte, max_nr, flags, + &any_writable,NULL ,NULL );folio_ref_add (folio, nr);if (folio_test_anon (folio)) { if (unlikely(folio_try_dup_anon_rmap_ptes(folio, page,
@@ -1564,9 +1564,7 @@ static inline int zap_present_ptes(struc* by keeping the batching logic separate. */ if (unlikely (folio_test_large (folio) && max_nr !=1 )) { - nr =folio_pte_batch (folio, addr, pte, ptent, max_nr,0 , -NULL ,NULL ,NULL ); - + nr =folio_pte_batch (folio, pte, ptent, max_nr);zap_present_folio_ptes (tlb, vma, folio, page, pte, ptent, nr, addr, details, rss, force_flush, force_break, any_skipped);
mm/mempolicy.c renamed from mm/mempolicy.c~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +1 -2
@@ -711,8 +711,7 @@ static int queue_folios_pte_range(pmd_tif (!folio ||folio_is_zone_device (folio))continue ;if (folio_test_large (folio) && max_nr !=1 ) - nr =folio_pte_batch (folio, addr, pte, ptent, - max_nr,0 ,NULL ,NULL ,NULL ); + nr =folio_pte_batch (folio, pte, ptent, max_nr);/* * vm_normal_folio() filters out zero pages, but there might * still be reserved folios to skip, perhaps in a VDSO.
mm/mlock.c renamed from mm/mlock.c~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +1 -2
@@ -313,8 +313,7 @@ static inline unsigned int folio_mlock_sif (!folio_test_large (folio))return 1 ; -return folio_pte_batch (folio, addr, pte, ptent, count,0 ,NULL , -NULL ,NULL ); +return folio_pte_batch (folio, pte, ptent, count); } static inline bool allow_mlock_munlock(struct folio *folio,
mm/mremap.c renamed from mm/mremap.c~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +1 -2
@@ -182,8 +182,7 @@ static int mremap_folio_pte_batch(structif (!folio || !folio_test_large (folio))return 1 ; -return folio_pte_batch (folio, addr, ptep, pte, max_nr,0 ,NULL , -NULL ,NULL ); +return folio_pte_batch (folio, ptep, pte, max_nr); } static int move_ptes(struct pagetable_move_control *pmc,
mm/rmap.c renamed from mm/rmap.c~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +1 -2
@@ -1868,8 +1868,7 @@ static inline unsigned int folio_unmap_pif (pte_unused (pte))return 1 ; -return folio_pte_batch (folio, addr, pvmw->pte, pte, max_nr,0 , -NULL ,NULL ,NULL ); +return folio_pte_batch (folio, pvmw->pte, pte, max_nr); }/*
mm/util.c renamed from mm/util.c~mm-split-folio_pte_batch-into-folio_pte_batch-and-folio_pte_batch_flags +29 -0
@@ -1171,3 +1171,32 @@ int compat_vma_mmap_prepare(struct filereturn 0 ; }EXPORT_SYMBOL (compat_vma_mmap_prepare); + +#ifdef CONFIG_MMU +/** +* folio_pte_batch - detect a PTE batch for a large folio +* @folio: The large folio to detect a PTE batch for. +* @ptep: Page table pointer for the first entry. +* @pte: Page table entry for the first page. +* @max_nr: The maximum number of table entries to consider. +* +* This is a simplified variant of folio_pte_batch_flags(). +* +* Detect a PTE batch: consecutive (present) PTEs that map consecutive +* pages of the same large folio in a single VMA and a single page table. +* +* All PTEs inside a PTE batch have the same PTE bits set, excluding the PFN, +* the accessed bit, writable bit, dirt-bit and soft-dirty bit. +* +* ptep must map any page of the folio. max_nr must be at least one and +* must be limited by the caller so scanning cannot exceed a single VMA and +* a single page table. +* +* Return: the number of table entries in the batch. +*/ +unsigned int folio_pte_batch (struct folio *folio,pte_t *ptep,pte_t pte, +unsigned int max_nr) +{ +return folio_pte_batch_flags (folio, ptep, pte, max_nr,0 ,NULL ,NULL ,NULL ); +} +#endif /* CONFIG_MMU */
_
Patches currently in -mm which might be from [email protected] are
mm-memory-introduce-is_huge_zero_pfn-and-use-it-in-vm_normal_page_pmd.patch