Skip to content

synchronously scan and reclaim empty user PTE pages

Qi Zheng linux-mm lkml oe-lkp 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest7 patches 9 repliesIn review

Changes in v1 from the cover letter

- replace [RFC PATCH 1/7] with a separate serise (already merge into mm-unstable):
https://lore.kernel.org/lkml/[email protected]/
(suggested by David Hildenbrand)
- squash [RFC PATCH 2/7] into [RFC PATCH 4/7]
(suggested by David Hildenbrand)
- change to scan and reclaim empty user PTE pages in zap_pte_range() (suggested by David Hildenbrand)
- sent a separate RFC patch to track the tlb flushing issue, and remove that part form this series ([RFC PATCH 3/7] and [RFC PATCH 6/7]).

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7mm: khugepaged: retract_page_tables() use pte_offset_map_lock()mm/khugepaged.c+8 -12 replies
2/7mm: make zap_pte_range() handle full within-PMD rangemm/memory.c+6 -22 replies
3/7mm: zap_install_uffd_wp_if_needed: return whether uffd-wp pte has been re-installedinclude/linux/mm_inline.h mm/memory.c+19 -70 replies
4/7mm: zap_present_ptes: return whether the PTE page is unreclaimablemm/memory.c+15 -100 replies
5/7mm: pgtable: try to reclaim empty PTE page in madvise(MADV_DONTNEED)include/linux/mm.h mm/Kconfig mm/Makefile +4 more+156 -85 replies
6/7x86: mm: free page table pages by RCU instead of semi RCUarch/x86/include/asm/tlb.h arch/x86/kernel/paravirt.c arch/x86/mm/pgtable.c +1 more+43 -20 replies
7/7x86: select ARCH_SUPPORTS_PT_RECLAIM if X86_64arch/x86/Kconfig+1 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH v1 0/7] synchronously scan and reclaim empty user PTE pages

Changes in v1:
- replace [RFC PATCH 1/7] with a separate serise (already merge into mm-unstable):
https://lore.kernel.org/lkml/[email protected]/
(suggested by David Hildenbrand)
- squash [RFC PATCH 2/7] into [RFC PATCH 4/7]
(suggested by David Hildenbrand)
- change to scan and reclaim empty user PTE pages in zap_pte_range() (suggested by David Hildenbrand)
- sent a separate RFC patch to track the tlb flushing issue, and remove that part form this series ([RFC PATCH 3/7] and [RFC PATCH 6/7]).

- add [PATCH v1 1/7] into this series
- drop RFC tag
- rebase onto the next-20241011

Changes in RFC v2:
- fix compilation errors in [RFC PATCH 5/7] and [RFC PATCH 7/7] reproted by kernel test robot
- use pte_offset_map_nolock() + pmd_same() instead of check_pmd_still_valid() in retract_page_tables() (in [RFC PATCH 4/7])
- rebase onto the next-20240805

Hi all,

Previously, we tried to use a completely asynchronous method to reclaim empty user PTE pages [1]. After discussing with David Hildenbrand, we decided to implement synchronous reclaimation in the case of madvise(MADV_DONTNEED) as the first step.

So this series aims to synchronously free the empty PTE pages in madvise(MADV_DONTNEED) case. We will detect and free empty PTE pages in zap_pte_range(), and will add zap_details.reclaim_pt to exclude cases other than madvise(MADV_DONTNEED).

In zap_pte_range(), mmu_gather is used to perform batch tlb flushing and page freeing operations. Therefore, if we want to free the empty PTE page in this path, the most natural way is to add it to mmu_gather as well. Now, if CONFIG_MMU_GATHER_RCU_TABLE_FREE is selected, mmu_gather will free page table pages by semi RCU:

- batch table freeing: asynchronous free by RCU
- single table freeing: IPI + synchronous free

But this is not enough to free the empty PTE page table pages in paths other that munmap and exit_mmap path, because IPI cannot be synchronized with rcu_read_lock() in pte_offset_map{_lock}(). So we should let single table also be freed by RCU like batch table freeing.

As a first step, we supported this feature on x86_64 and selectd the newly introduced CONFIG_ARCH_SUPPORTS_PT_RECLAIM.

For other cases such as madvise(MADV_FREE), consider scanning and freeing empty PTE pages asynchronously in the future.

This series is based on next-20241011 (which contains the series [2]).

Comments and suggestions are welcome!

Thanks,
Qi

[1]. https://lore.kernel.org/lkml/[email protected]/
[2]. https://lore.kernel.org/lkml/[email protected]/

Qi Zheng (7):
mm: khugepaged: retract_page_tables() use pte_offset_map_lock()
mm: make zap_pte_range() handle full within-PMD range
mm: zap_install_uffd_wp_if_needed: return whether uffd-wp pte has been
re-installed
mm: zap_present_ptes: return whether the PTE page is unreclaimable
mm: pgtable: try to reclaim empty PTE page in madvise(MADV_DONTNEED)
x86: mm: free page table pages by RCU instead of semi RCU
x86: select ARCH_SUPPORTS_PT_RECLAIM if X86_64

 arch/x86/Kconfig           |  1 +
 arch/x86/include/asm/tlb.h | 19 ++++++++
 arch/x86/kernel/paravirt.c |  7 +++
 arch/x86/mm/pgtable.c      | 10 +++-
 include/linux/mm.h         |  1 +
 include/linux/mm_inline.h  | 11 +++--
 mm/Kconfig                 | 14 ++++++
 mm/Makefile                |  1 +
 mm/internal.h              | 29 ++++++++++++
 mm/khugepaged.c            |  9 +++-
 mm/madvise.c               |  4 +-
 mm/memory.c                | 95 +++++++++++++++++++++++++++++---------
 mm/mmu_gather.c            |  9 +++-
 mm/pt_reclaim.c            | 68 +++++++++++++++++++++++++++
 14 files changed, 248 insertions(+), 30 deletions(-)
 create mode 100644 mm/pt_reclaim.c
Signature · 1 line
2.20.1