RFC riscv: mm: Introduce lazy tlb flush
Get series with b4
b4 am [email protected]Versions newest first
- v2 RFC Latest9 patches 0 repliesNew
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/9 | riscv: Introduce RISCV_LAZY_TLB_FLUSH configarch/ | +12 | 0 replies | |
| 2/9 | riscv: mm: Apply a threshold to the number of active ASIDs on each CPUarch/ arch/ arch/ | +94 | 0 replies | |
| 3/9 | riscv: mm: Grab mm_count to avoid mm getting releasedarch/ arch/ | +51 | 0 replies | |
| 4/9 | fork: Add arch override for do_shoot_lazy_tlb()kernel/ | +4 | 0 replies | |
| 5/9 | riscv: mm: Introduce arch_do_shoot_lazy_tlbarch/ arch/ arch/ +1 more | +64 | 0 replies | |
| 6/9 | riscv: mm: Introduce percpu TLB Flush queuearch/ arch/ arch/ | +90 | 0 replies | |
| 7/9 | riscv: mm: Defer the TLB Flush to switch_mmarch/ | +28 | 0 replies | |
| 8/9 | riscv: mm: Clear mm_cpumask during local_flush_tlb_all_asid()arch/ | +19 | 0 replies | |
| 9/9 | riscv: mm: Clear mm_cpumask during local_flush_tlb_all()arch/ arch/ arch/ | +36 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.
Cover letter [RFC PATCH v2 0/9] riscv: mm: Introduce lazy tlb flush
This patch series introduces a lazy tlb flush mechanism for riscv. This mechanism is based on two insights:
1) Since each CPU has limited TLB entries, there exist limited active ASIDs in each CPU's TLB at the same time. When a mm has not been used for enough long time (or, after enough switch_mm times), we can assume its TLB entries are all evicted out. Then we can clear current CPU in its mm_cpumask so that next time when the memory mapping of this mm is modified, no IPI will be sent to current CPU.
2) When memory mapping of a mm is modified, instead of sending IPI to all CPUs recorded in its mm_cpumask, we check whether each target CPU is using this mm right now. If not, we just store the TLB Flush information in target CPU's percpu buffer, avoiding the IPI. Next time when the target CPU switch_mm to this mm, it can check the percpu buffer and perform TLB Flush itself, without IPI involvement either.
Using this mechanism, we significantly reduced the number of IPI due to TLB Flush:
* ltp - mmapstress01
Before: ~108k
After: ~17k
* ltp - hackbench
Before: ~385k
After: ~2k
Thanks Guo Ren for his advice on memory access latency test via lmbench. We are unable to test it now due to lack of real machines. We will supply this test and adjust our mechanism according to it as soon as possible.
Xu Lu (9):
riscv: Introduce RISCV_LAZY_TLB_FLUSH config
riscv: mm: Apply a threshold to the number of active ASIDs on each CPU
riscv: mm: Grab mm_count to avoid mm getting released
fork: Add arch override for do_shoot_lazy_tlb()
riscv: mm: Introduce arch_do_shoot_lazy_tlb
riscv: mm: Introduce percpu TLB Flush queue
riscv: mm: Defer the TLB Flush to switch_mm
riscv: mm: Clear mm_cpumask during local_flush_tlb_all_asid()
riscv: mm: Clear mm_cpumask during local_flush_tlb_all()
arch/riscv/Kconfig | 12 ++ arch/riscv/include/asm/mmu.h | 4 + arch/riscv/include/asm/mmu_context.h | 5 + arch/riscv/include/asm/tlbflush.h | 63 ++++++ arch/riscv/mm/context.c | 24 ++- arch/riscv/mm/tlbflush.c | 302 +++++++++++++++++++++++++-- kernel/fork.c | 6 +- 7 files changed, 394 insertions(+), 22 deletions(-)
Signature · 1 line
2.20.1