Skip to content

mm: free retracted page table by RCU

Hugh Dickins linux-mm linux-s390 linuxppc-dev lkml sparclinux 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest13 patches 14 repliesIn review 12 of 19 applied
  2. v212 patches 33 repliesSuperseded
  3. v112 patches 47 repliesSuperseded

Patches in v3 12 of 19 applied

#Subject and filesSizeRepliesApplied
1/13mm/pgtable: add rcu_read_lock() and rcu_read_unlock()sinclude/linux/pgtable.h mm/pgtable-generic.c+4 -40 replies
2/13mm/pgtable: add PAE safety to __pte_offset_map()include/linux/pgtable.h mm/pgtable-generic.c+33 -00 repliesApplied
3/13arm: adjust_pte() use pte_offset_map_nolock()arch/arm/mm/fault-armv.c+1 -20 repliesApplied
4/13powerpc: assert_pte_locked() use pte_offset_map_nolock()arch/powerpc/mm/pgtable.c+6 -104 repliesApplied
4/13powerpc: assert_pte_locked() use pte_offset_map_nolock(): fixarch/powerpc/mm/pgtable.c+8 -00 replies
5/13powerpc: add pte_free_defer() for pgtables sharing pagearch/powerpc/include/asm/pgalloc.h arch/powerpc/mm/pgtable-frag.c+30 -30 repliesApplied
6/13sparc: add pte_free_defer() for pte_t *pgtable_tarch/sparc/include/asm/pgalloc_64.h arch/sparc/mm/init_64.c+20 -00 repliesApplied
7/13s390: add pte_free_defer() for pgtables sharing pagearch/s390/include/asm/pgalloc.h arch/s390/mm/pgalloc.c+72 -122 repliesApplied
7/13s390: add pte_free_defer() for pgtables sharing page: fixarch/s390/mm/pgalloc.c+1 -10 replies
8/13mm/pgtable: add pte_free_defer() for pgtable as pageinclude/linux/mm_types.h include/linux/pgtable.h mm/pgtable-generic.c+26 -00 repliesApplied
9/13mm/khugepaged: retract_page_tables() without mmap or vma lockmm/khugepaged.c+75 -1090 repliesApplied
10/13mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock()mm/khugepaged.c+77 -958 repliesApplied
10/13mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock(): fixmm/khugepaged.c+1 -10 replies
10/13mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock(): fix2mm/khugepaged.c+1 -10 replies
11/13mm/khugepaged: delete khugepaged_collapse_pte_mapped_thps()mm/khugepaged.c+16 -1090 repliesApplied
11/13mm/khugepaged: delete khugepaged_collapse_pte_mapped_thps(): fixmm/khugepaged.c+3 -50 replies
12/13mm: delete mmap_write_trylock() and vma_try_start_write()include/linux/mm.h include/linux/mmap_lock.h+0 -270 replies
12/13mm: delete mmap_write_trylock() and vma_try_start_write()include/linux/mm.h include/linux/mmap_lock.h+0 -270 repliesApplied
13/13mm/pgtable: notes on pte_offset_map[_lock]()mm/pgtable-generic.c+44 -00 repliesApplied

Reviews trailers per patch

5 of 19 patches have a Reviewed-by or Acked-by.

PatchGerald SchaeferAlexander GordeevQi Zheng
1/13
2/13
3/13
4/13
4/13
5/13
6/13
7/13R-b T-b A-b
7/13R-b T-b A-b
8/13
9/13
10/13R-b
10/13R-b
10/13R-b
11/13
11/13
12/13
12/13
13/13

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v3 thread

Cover letter [PATCH v3 00/13] mm: free retracted page table by RCU

Here is v3 of the series of patches to mm (and a few architectures), based on v6.5-rc1 which includes the preceding two series (thank you!): in which khugepaged takes advantage of pte_offset_map[_lock]() allowing for pmd transitions. Differences from v1 and v2 are noted patch by patch below.

This replaces the v2 "mm: free retracted page table by RCU"
https://lore.kernel.org/linux-mm/[email protected]/
series of 12 posted on 2023-06-20.

What is it all about? Some mmap_lock avoidance i.e. latency reduction. Initially just for the case of collapsing shmem or file pages to THPs:
the usefulness of MADV_COLLAPSE on shmem is being limited by that mmap_write_lock it currently requires.

Likely to be relied upon later in other contexts e.g. freeing of empty page tables (but that's not work I'm doing). mmap_write_lock avoidance when collapsing to anon THPs? Perhaps, but again that's not work I've done: a quick attempt was not as easy as the shmem/file case.

These changes (though of course not these exact patches) have been in Google's data centre kernel for three years now: we do rely upon them.

Based on v6.5-rc1; and almost good on current mm-unstable or current linux-next - just one patch conflicts, the 12/13: I'll reply to that one with its mm-unstable or linux-next equivalent (vma_assert_locked() has been added next to where vma_try_start_write() is being removed).

01/13 mm/pgtable: add rcu_read_lock() and rcu_read_unlock()s
v3: same as v1
02/13 mm/pgtable: add PAE safety to __pte_offset_map()
v3: same as v2
v2: rename to pmdp_get_lockless_start/end() per Matthew;
so use inlines without _irq_save(flags) macro oddity; add pmdp_get_lockless_sync() for use later in 09/13.
03/13 arm: adjust_pte() use pte_offset_map_nolock()
v3: same as v1
04/13 powerpc: assert_pte_locked() use pte_offset_map_nolock()
v3: same as v1
05/13 powerpc: add pte_free_defer() for pgtables sharing page
v3: much simpler version, following suggestion by Jason
v2: fix rcu_head usage to cope with concurrent deferrals;
add para to commit message explaining rcu_head issue.
06/13 sparc: add pte_free_defer() for pte_t *pgtable_t
v3: same as v2
v2: use page_address() instead of less common page_to_virt();
add para to commit message explaining simple conversion; changed title since sparc64 pgtables do not share page.
07/13 s390: add pte_free_defer() for pgtables sharing page
v3: much simpler version, following suggestion by Gerald
v2: complete rewrite, integrated with s390's existing pgtable
management; temporarily using a global mm_pgtable_list_lock, to be restored to per-mm spinlock in a later followup patch.
08/13 mm/pgtable: add pte_free_defer() for pgtable as page
v3: same as v2
v2: add comment on rcu_head to "Page table pages", per JannH
09/13 mm/khugepaged: retract_page_tables() without mmap or vma lock
v3: same as v2
v2: repeat checks under ptl because UFFD, per PeterX and JannH;
bring back mmu_notifier calls for PMD, per JannH and Jason; pmdp_get_lockless_sync() to issue missing interrupt if PAE.
10/13 mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock()
v3: updated to using ptent instead of *pte
v2: first check VMA, in case page tables torn down, per JannH;
pmdp_get_lockless_sync() to issue missing interrupt if PAE; moved mmu_notifier after step 1, reworked final goto labels.
11/13 mm/khugepaged: delete khugepaged_collapse_pte_mapped_thps()
v3: rediffed
v2: same as v1
12/13 mm: delete mmap_write_trylock() and vma_try_start_write()
v3: rediffed (different diff needed for mm-unstable or linux-next)
v2: same as v1
13/13 mm/pgtable: notes on pte_offset_map[_lock]()
v3: new: JannH asked for more helpful comment, this is my attempt;
could be moved to be the first in the series.

 arch/arm/mm/fault-armv.c            |   3 +-
 arch/powerpc/include/asm/pgalloc.h  |   4 +
 arch/powerpc/mm/pgtable-frag.c      |  29 +-
 arch/powerpc/mm/pgtable.c           |  16 +-
 arch/s390/include/asm/pgalloc.h     |   4 +
 arch/s390/mm/pgalloc.c              |  80 ++++-
 arch/sparc/include/asm/pgalloc_64.h |   4 +
 arch/sparc/mm/init_64.c             |  16 +
 include/linux/mm.h                  |  17 --
 include/linux/mm_types.h            |   4 +
 include/linux/mmap_lock.h           |  10 -
 include/linux/pgtable.h             |  10 +-
 mm/khugepaged.c                     | 481 +++++++++++-------------------
 mm/pgtable-generic.c                |  97 +++++-
 14 files changed, 404 insertions(+), 371 deletions(-)

Hugh