Speed up boot with faster linear map creation
Status from this message: Applied to arm64 (for-next/mm), thanks!
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest3 patches 5 repliesApplied
- v24 patches 34 repliesSuperseded
- v13 patches 10 repliesSuperseded
Patches in v3 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | arm64: mm: Don't remap pgtables per-cont(pte|pmd) blockarch/ | +14 | 1 reply | |
| 2/3 | arm64: mm: Batch dsb and isb when populating pgtablesarch/ arch/ | +16 | 0 replies | |
| 3/3 | arm64: mm: Don't remap pgtables for allocate vs populatearch/ arch/ | +37 | 0 replies |
Reviews trailers per patch
None of the 3 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Itaru Kitayama | Eric Chanudet | Mark Rutland | Ard Biesheuvel |
|---|---|---|---|---|
| Series | T-b | R-b | R-b | |
| 1/3 | T-b Reviewied-by | T-b | ||
| 2/3 | T-b | T-b | ||
| 3/3 | T-b | T-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.
Cover letter [PATCH v3 0/3] Speed up boot with faster linear map creation
Hi All,
It turns out that creating the linear map can take a significant proportion of the total boot time, especially when rodata=full. And most of the time is spent waiting on superfluous tlb invalidation and memory barriers. This series reworks the kernel pgtable generation code to significantly reduce the number of those TLBIs, ISBs and DSBs. See each patch for details.
The below shows the execution time of map_mem() across a couple of different systems with different RAM configurations. We measure after applying each patch and show the improvement relative to base (v6.9-rc2):
| Apple M2 VM | Ampere Altra| Ampere Altra| Ampere Altra
| VM, 16G | VM, 64G | VM, 256G | Metal, 512G
---------------|-------------|-------------|-------------|-------------
| ms (%) | ms (%) | ms (%) | ms (%)
---------------|-------------|-------------|-------------|-------------
base | 168 (0%) | 2198 (0%) | 8644 (0%) | 17447 (0%)
no-cont-remap | 78 (-53%) | 435 (-80%) | 1723 (-80%) | 3779 (-78%)
batch-barriers | 11 (-93%) | 161 (-93%) | 656 (-92%) | 1654 (-91%)
no-alloc-remap | 10 (-94%) | 104 (-95%) | 438 (-95%) | 1223 (-93%)This series applies on top of v6.9-rc2. All mm selftests pass. I've compile and boot tested various PAGE_SIZE and VA size configs.
---
Changes since v2 [2]
====================
- Independently increment ptep/pmdp in alloc_init_cont_[pte|pmd]() rather than return the incremented value from int_[pte|pmd]() (per Mark)
- Removed explicit barriers from alloc_init_cont_pte() and instead rely on the barriers in pte_clear_fixmap() (per Mark)
- Significantly simplify approach to avoiding fixmap during alloc (patch 3 reworked) (per Mark)
- Dropped patch 4 - not possible with simplified patch 3 and improvement (~2%) didn't warrant complexity (per Mark)
Changes since v1 [1]
====================
- Added Tested-by tags (thanks to Eric and Itaru)
- Renamed ___set_pte() -> __set_pte_nosync() (per Ard)
- Reordered patches (biggest impact & least controversial first)
- Reordered alloc/map/unmap functions in mmu.c to aid reader
- pte_clear() -> __pte_clear() in clear_fixmap_nosync()
- Reverted generic p4d_index() which caused x86 build error. Replaced with unconditional p4d_index() define under arm64.
[1] https://lore.kernel.org/linux-arm-kernel/[email protected]/
[2] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Thanks,
Ryan
Ryan Roberts (3):
arm64: mm: Don't remap pgtables per-cont(pte|pmd) block
arm64: mm: Batch dsb and isb when populating pgtables
arm64: mm: Don't remap pgtables for allocate vs populate
arch/arm64/include/asm/pgtable.h | 9 ++- arch/arm64/mm/mmu.c | 101 +++++++++++++++++-------------- 2 files changed, 65 insertions(+), 45 deletions(-)
--
2.25.1