mm: free retracted page table by RCU
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest13 patches 14 repliesIn review 12 of 19 applied
- v212 patches 33 repliesSuperseded
- v112 patches 47 repliesSuperseded
Patches in v3 12 of 19 applied
Reviews trailers per patch
5 of 19 patches have a Reviewed-by or Acked-by.
| Patch | Gerald Schaefer | Alexander Gordeev | Qi Zheng |
|---|---|---|---|
| 1/13 | |||
| 2/13 | |||
| 3/13 | |||
| 4/13 | |||
| 4/13 | |||
| 5/13 | |||
| 6/13 | |||
| 7/13 | R-b | T-b A-b | |
| 7/13 | R-b | T-b A-b | |
| 8/13 | |||
| 9/13 | |||
| 10/13 | R-b | ||
| 10/13 | R-b | ||
| 10/13 | R-b | ||
| 11/13 | |||
| 11/13 | |||
| 12/13 | |||
| 12/13 | |||
| 13/13 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [PATCH v3 00/13] mm: free retracted page table by RCU
Here is v3 of the series of patches to mm (and a few architectures), based on v6.5-rc1 which includes the preceding two series (thank you!): in which khugepaged takes advantage of pte_offset_map[_lock]() allowing for pmd transitions. Differences from v1 and v2 are noted patch by patch below.
This replaces the v2 "mm: free retracted page table by RCU"
https://lore.kernel.org/linux-mm/[email protected]/
series of 12 posted on 2023-06-20.
What is it all about? Some mmap_lock avoidance i.e. latency reduction. Initially just for the case of collapsing shmem or file pages to THPs:
the usefulness of MADV_COLLAPSE on shmem is being limited by that mmap_write_lock it currently requires.
Likely to be relied upon later in other contexts e.g. freeing of empty page tables (but that's not work I'm doing). mmap_write_lock avoidance when collapsing to anon THPs? Perhaps, but again that's not work I've done: a quick attempt was not as easy as the shmem/file case.
These changes (though of course not these exact patches) have been in Google's data centre kernel for three years now: we do rely upon them.
Based on v6.5-rc1; and almost good on current mm-unstable or current linux-next - just one patch conflicts, the 12/13: I'll reply to that one with its mm-unstable or linux-next equivalent (vma_assert_locked() has been added next to where vma_try_start_write() is being removed).
01/13 mm/pgtable: add rcu_read_lock() and rcu_read_unlock()s
v3: same as v1
02/13 mm/pgtable: add PAE safety to __pte_offset_map()
v3: same as v2
v2: rename to pmdp_get_lockless_start/end() per Matthew;
so use inlines without _irq_save(flags) macro oddity; add pmdp_get_lockless_sync() for use later in 09/13.
03/13 arm: adjust_pte() use pte_offset_map_nolock()
v3: same as v1
04/13 powerpc: assert_pte_locked() use pte_offset_map_nolock()
v3: same as v1
05/13 powerpc: add pte_free_defer() for pgtables sharing page
v3: much simpler version, following suggestion by Jason
v2: fix rcu_head usage to cope with concurrent deferrals;
add para to commit message explaining rcu_head issue.
06/13 sparc: add pte_free_defer() for pte_t *pgtable_t
v3: same as v2
v2: use page_address() instead of less common page_to_virt();
add para to commit message explaining simple conversion; changed title since sparc64 pgtables do not share page.
07/13 s390: add pte_free_defer() for pgtables sharing page
v3: much simpler version, following suggestion by Gerald
v2: complete rewrite, integrated with s390's existing pgtable
management; temporarily using a global mm_pgtable_list_lock, to be restored to per-mm spinlock in a later followup patch.
08/13 mm/pgtable: add pte_free_defer() for pgtable as page
v3: same as v2
v2: add comment on rcu_head to "Page table pages", per JannH
09/13 mm/khugepaged: retract_page_tables() without mmap or vma lock
v3: same as v2
v2: repeat checks under ptl because UFFD, per PeterX and JannH;
bring back mmu_notifier calls for PMD, per JannH and Jason; pmdp_get_lockless_sync() to issue missing interrupt if PAE.
10/13 mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock()
v3: updated to using ptent instead of *pte
v2: first check VMA, in case page tables torn down, per JannH;
pmdp_get_lockless_sync() to issue missing interrupt if PAE; moved mmu_notifier after step 1, reworked final goto labels.
11/13 mm/khugepaged: delete khugepaged_collapse_pte_mapped_thps()
v3: rediffed
v2: same as v1
12/13 mm: delete mmap_write_trylock() and vma_try_start_write()
v3: rediffed (different diff needed for mm-unstable or linux-next)
v2: same as v1
13/13 mm/pgtable: notes on pte_offset_map[_lock]()
v3: new: JannH asked for more helpful comment, this is my attempt;
could be moved to be the first in the series.
arch/arm/mm/fault-armv.c | 3 +- arch/powerpc/include/asm/pgalloc.h | 4 + arch/powerpc/mm/pgtable-frag.c | 29 +- arch/powerpc/mm/pgtable.c | 16 +- arch/s390/include/asm/pgalloc.h | 4 + arch/s390/mm/pgalloc.c | 80 ++++- arch/sparc/include/asm/pgalloc_64.h | 4 + arch/sparc/mm/init_64.c | 16 + include/linux/mm.h | 17 -- include/linux/mm_types.h | 4 + include/linux/mmap_lock.h | 10 - include/linux/pgtable.h | 10 +- mm/khugepaged.c | 481 +++++++++++------------------- mm/pgtable-generic.c | 97 +++++- 14 files changed, 404 insertions(+), 371 deletions(-)
Hugh