Skip to content

Simplify special kernel page table handling

Kevin Brodsky linux-arch linux-efi linux-mm linux-parisc linux-riscv linux-s390 linux-um linuxppc-dev lkml loongarch sparclinux 1 version In review

Get series with b4
b4 am [email protected] change-id 20260611-remove_pgtable_cdtor-3b123938f8cd

Versions newest first

  1. v1 Latest22 patches 28 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/22mm: drop unused __mm_flags_set_mask_bits_word()include/linux/mm_types.h+0 -121 reply
2/22mm: move mm_flags helpers to mm_types.hinclude/linux/mm.h include/linux/mm_types.h+30 -321 reply
3/22mm: introduce MMF_KERNEL flag and set it for init_mminclude/linux/mm_types.h mm/init-mm.c+11 -013 replies
4/22mm: use mm_is_kernel() in generic page table codeinclude/asm-generic/pgalloc.h include/linux/mm.h mm/memory.c +2 more+22 -223 replies
5/22arm64: mm: use mm_is_kernel() for kernel mm checksarch/arm64/include/asm/pgalloc.h arch/arm64/mm/contpte.c arch/arm64/mm/hugetlbpage.c+7 -70 replies
6/22loongarch: mm: use mm_is_kernel() in switch_mm_irqs_off()arch/loongarch/include/asm/mmu_context.h+1 -10 replies
7/22parisc: mm: use mm_is_kernel() for kernel mm checksarch/parisc/include/asm/mmu_context.h arch/parisc/include/asm/pgalloc.h arch/parisc/include/asm/tlbflush.h+3 -30 replies
8/22powerpc: mm: use mm_is_kernel() for kernel mm checksarch/powerpc/include/asm/pgalloc.h arch/powerpc/mm/book3s64/pgtable.c arch/powerpc/mm/mmu_context.c +1 more+4 -40 replies
9/22s390: mm: use mm_is_kernel() for kernel mm checksarch/s390/include/asm/mmu_context.h arch/s390/include/asm/pgtable.h arch/s390/mm/pgalloc.c+6 -60 replies
10/22sparc: mm: use mm_is_kernel() for kernel mm checksarch/sparc/include/asm/mmu_context_64.h arch/sparc/include/asm/pgtable_64.h arch/sparc/mm/init_64.c +1 more+5 -70 replies
11/22um: mm: use mm_is_kernel() in TLB syncarch/um/kernel/tlb.c+1 -10 replies
12/22x86/mm: use mm_is_kernel() for kernel mm checksarch/x86/include/asm/tlbflush.h arch/x86/mm/pgtable.c arch/x86/mm/tlb.c+10 -91 reply
13/22mm: account page table pages when allocatedinclude/linux/mm.h+7 -20 replies
14/22mm: set page table page type when allocatedinclude/linux/mm.h+2 -154 replies
15/22mm: only initialise pt_share_count for user pgtablesinclude/linux/mm.h+6 -32 replies
16/22efi: mark efi_mm as a kernel mmdrivers/firmware/efi/efi.c+1 -00 replies
17/22mm: pagewalk: drop redundant address check for kernel mm walksmm/pagewalk.c+2 -51 reply
18/22arm64: mm: drop explicit mm_is_efi() check in contptearch/arm64/mm/contpte.c+9 -170 replies
19/22x86/tboot: mark tboot_mm as a kernel mmarch/x86/kernel/tboot.c+2 -21 reply
20/22arm64: mm: drop ctor/dtor calls for kernel page tablesarch/arm64/mm/mmu.c+23 -590 replies
21/22arm: mm: drop ctor call for kernel page tablesarch/arm/mm/mmu.c+1 -20 replies
22/22riscv: mm: drop ctor/dtor calls for kernel page tablesarch/riscv/mm/init.c+14 -490 replies

Reviews trailers per patch

3 of 22 patches have a Reviewed-by or Acked-by.

PatchDavid Hildenbrand (Arm)
1/22A-b
2/22A-b
3/22
4/22
5/22
6/22
7/22
8/22
9/22
10/22
11/22
12/22
13/22
14/22
15/22A-b
16/22
17/22
18/22
19/22
20/22
21/22
22/22

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 00/22] Simplify special kernel page table handling

Page table constructors and destructors were originally introduced specifically for user PTE pages. Over time, their use became more widespread, and starting from v6.16 they are called at all levels and for both user and kernel page tables [1]. Or so I thought. Turns out, we still have various special kernel page tables where the ctor/dtor are not called; vmemmap page tables is one such case.

Looking closer, we do not actually need per-level ctors for kernel page tables. In fact we do not need ctors for them at all, because nothing is specific to kernel page tables in the ctor/dtor and nowadays they should be allocated with pagetable_alloc().

Here's the plan: all the generic page table handling moves to pagetable_{alloc,free}(). Whatever is left is specific to user page tables; the ctor/dtor are only called for user page tables at the required levels (PTE/PMD), and renamed to make that clear.

Doing this will allow us to remove a good amount of special casing, but there's quite a bit of churn to get there. Here's what this series does:

1. Introduce the concept of "kernel mm" and replace most comparisons with &init_mm with a flag check (patch 1-12); no functional change so far.

2. Move the page table accounting/type setting to pagetable_{alloc,free} (patch 13-15). Kernel mm's do not need to call the ctor/dtor any more.

3. Mark special mm's (efi_mm, tboot_mm on x86) as kernel mm's (patch 16-19). They now abide by the same rules as init_mm, which means in particular that split page table locks are no longer used.

4. Remove the ctor/dtor calls (now no-ops) for special kernel page tables on arm/arm64/riscv (patch 20-22).

[1] https://lore.kernel.org/all/[email protected]/

---
I stopped there for part 1 to keep it digestible. Part 2 will then ensure that the ctor/dtor are only called for user PTE/PMD pages and rename them accordingly. This is comparatively simpler but also involves a lot of churn.

This series was suggested in reaction to [2]; removing dtor calls entirely for kernel page tables systematically solves such issues.

Worth noting that some comparisons against &init_mm have been left untouched, as they are truly specific to init_mm. Reviewers are encouraged to take a look at the remaining cases:

  $ git grep -nE '([=!]= *&init_mm|&init_mm *[=!]=)'
  Documentation/mm/active_mm.rst:78:      if (current->mm == &init_mm)
  Documentation/translations/zh_CN/mm/active_mm.rst:76:   if (current->mm == &init_mm)
  arch/arm/include/asm/mmu_context.h:144: if (mm != &init_mm)
  arch/arm64/include/asm/mmu_context.h:100:       if (mm != &init_mm && !system_uses_ttbr0_pan())
  arch/arm64/include/asm/mmu_context.h:210:       if (mm == &init_mm)
  arch/arm64/include/asm/mmu_context.h:242:       if (next == &init_mm) {
  arch/arm64/mm/fault.c:121:      if (mm == &init_mm)
  arch/arm64/mm/fault.c:139:              if (mm == &init_mm) {
  arch/arm64/mm/fault.c:154:               mm == &init_mm ? "swapper" : "user", PAGE_SIZE / SZ_1K,
  arch/arm64/mm/mmu.c:997:        BUG_ON(mm == &init_mm);
  arch/powerpc/xmon/xmon.c:3314:  if (mm == &init_mm)
  arch/riscv/mm/fault.c:42:                mm == &init_mm ? (u64)__pa_symbol(mm->pgd) : virt_to_phys(mm->pgd));
  kernel/cpu.c:917:       WARN_ON(mm != &init_mm);
  kernel/fork.c:725:      BUG_ON(mm == &init_mm);
  kernel/sched/core.c:8390:       if (mm != &init_mm) {
  mm/ptdump.c:182:        if (mm != &init_mm)
  mm/ptdump.c:191:        if (mm != &init_mm)

Testing:

* Built all modified architectures
* arm64
- mm kselftests (FVP)
- Hot adding/removing memory + memory-hotplug kselftests (QEMU)
- Reading /sys/kernel/debug/{efi,kernel}_page_tables
- Reading from /sys/firmware/efi/efivars/
*riscv
- memory-hotplug kselftests (QEMU)
* x86
- mm kselftests (lkvm)
- tboot page table setup using a fake tboot shared page (QEMU)
- CONFIG_EFI_PGT_DUMP=y (QEMU)

Patches based on mm-unstable.

[2] https://lore.kernel.org/all/[email protected]/

---

To: [email protected]
Cc: Andrew Morton <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: Lorenzo Stoakes <[email protected]>
Cc: "Liam R. Howlett" <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Cc: Mike Rapoport <[email protected]>
Cc: Suren Baghdasaryan <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Pasha Tatashin <[email protected]>

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc "James E.J. Bottomley" <[email protected]>
Cc Helge Deller <[email protected]>
Cc Madhavan Srinivasan <[email protected]>
Cc Michael Ellerman <[email protected]>
Cc Paul Walmsley <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc Albert Ou <[email protected]>
Cc Heiko Carstens <[email protected]>
Cc Vasily Gorbik <[email protected]>
Cc Alexander Gordeev <[email protected]>
Cc Gerald Schaefer <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Andreas Larsson <[email protected]>
Cc Richard Weinberger <[email protected]>
Cc Anton Ivanov <[email protected]>
Cc Johannes Berg <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Ning Sun <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Ilias Apalodimas <[email protected]>
Cc Vishal Moola <[email protected]>
Cc Alistair Popple <[email protected]>
Cc "Matthew Wilcox (Oracle)" <[email protected]>

---
Kevin Brodsky (22):
mm: drop unused __mm_flags_set_mask_bits_word()
mm: move mm_flags helpers to mm_types.h
mm: introduce MMF_KERNEL flag and set it for init_mm
mm: use mm_is_kernel() in generic page table code
arm64: mm: use mm_is_kernel() for kernel mm checks
loongarch: mm: use mm_is_kernel() in switch_mm_irqs_off()
parisc: mm: use mm_is_kernel() for kernel mm checks
powerpc: mm: use mm_is_kernel() for kernel mm checks
s390: mm: use mm_is_kernel() for kernel mm checks
sparc: mm: use mm_is_kernel() for kernel mm checks
um: mm: use mm_is_kernel() in TLB sync
x86/mm: use mm_is_kernel() for kernel mm checks
mm: account page table pages when allocated
mm: set page table page type when allocated
mm: only initialise pt_share_count for user pgtables
efi: mark efi_mm as a kernel mm
mm: pagewalk: drop redundant address check for kernel mm walks
arm64: mm: drop explicit mm_is_efi() check in contpte x86/tboot: mark tboot_mm as a kernel mm
arm64: mm: drop ctor/dtor calls for kernel page tables
arm: mm: drop ctor call for kernel page tables
riscv: mm: drop ctor/dtor calls for kernel page tables

 arch/arm/mm/mmu.c                        |  3 +-
 arch/arm64/include/asm/pgalloc.h         | 10 ++--
 arch/arm64/mm/contpte.c                  | 26 ++++------
 arch/arm64/mm/hugetlbpage.c              |  2 +-
 arch/arm64/mm/mmu.c                      | 82 +++++++++-----------------------
 arch/loongarch/include/asm/mmu_context.h |  2 +-
 arch/parisc/include/asm/mmu_context.h    |  2 +-
 arch/parisc/include/asm/pgalloc.h        |  2 +-
 arch/parisc/include/asm/tlbflush.h       |  2 +-
 arch/powerpc/include/asm/pgalloc.h       |  2 +-
 arch/powerpc/mm/book3s64/pgtable.c       |  2 +-
 arch/powerpc/mm/mmu_context.c            |  2 +-
 arch/powerpc/mm/pgtable.c                |  2 +-
 arch/riscv/mm/init.c                     | 63 ++++++------------------
 arch/s390/include/asm/mmu_context.h      |  2 +-
 arch/s390/include/asm/pgtable.h          |  6 +--
 arch/s390/mm/pgalloc.c                   |  4 +-
 arch/sparc/include/asm/mmu_context_64.h  |  2 +-
 arch/sparc/include/asm/pgtable_64.h      |  6 +--
 arch/sparc/mm/init_64.c                  |  2 +-
 arch/sparc/mm/tlb.c                      |  2 +-
 arch/um/kernel/tlb.c                     |  2 +-
 arch/x86/include/asm/tlbflush.h          |  8 ++--
 arch/x86/kernel/tboot.c                  |  4 +-
 arch/x86/mm/pgtable.c                    |  2 +-
 arch/x86/mm/tlb.c                        |  9 ++--
 drivers/firmware/efi/efi.c               |  1 +
 include/asm-generic/pgalloc.h            | 16 +++----
 include/linux/mm.h                       | 69 +++++++--------------------
 include/linux/mm_types.h                 | 52 +++++++++++++++-----
 mm/init-mm.c                             |  1 +
 mm/memory.c                              |  6 +--
 mm/page_table_check.c                    | 14 +++---
 mm/pagewalk.c                            |  7 +--
 34 files changed, 163 insertions(+), 254 deletions(-)

---
base-commit: a531f0cf705af3374388eafe4115f651c625d8db
change-id: 20260611-remove_pgtable_cdtor-3b123938f8cd