Skip to content

x86/mm/tlb: make lazy TLB mode even lazier

Rik van Riel lkml 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest7 patches 7 repliesIn review
  2. v17 patches 3 repliesSuperseded

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7x86/mm/tlb: Always use lazy TLB modearch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c+1 -302 replies
2/7x86/mm/tlb: Restructure switch_mm_irqs_off()arch/x86/mm/tlb.c+33 -311 reply
3/7smp: use __cpumask_set_cpu in on_each_cpu_condkernel/smp.c+1 -10 replies
4/7smp,cpumask: introduce on_each_cpu_cond_maskinclude/linux/smp.h kernel/smp.c kernel/up.c+28 -70 replies
5/7Add freed_tables argument to flush_tlb_mm_rangearch/x86/include/asm/tlb.h arch/x86/include/asm/tlbflush.h arch/x86/kernel/ldt.c +2 more+11 -80 replies
6/7Add freed_tables element to flush_tlb_infoarch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c+2 -00 replies
7/7x86/mm/tlb: Make lazy TLB mode lazierarch/x86/mm/tlb.c+58 -91 reply

Reviews trailers per patch

3 of 7 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchSong LiuDave HansenAndy LutomirskiPeter Zijlstra (Intel)
SeriesA-b
1/7T-b A-b
2/7A-b
3/7R-b
4/7
5/7
6/7
7/7T-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v2 thread

Cover letter [PATCH v2 0/7] x86/mm/tlb: make lazy TLB mode even lazier

Linus asked me to come up with a smaller patch set to get the benefits of lazy TLB mode, so I spent some time trying out various permutations of the code, with a few workloads that do lots of context switches, and also happen to have a fair number of TLB flushes a second.

Both of the workloads tested are memcache style workloads, running on two socket systems. One of the workloads has around 300,000 context switches a second, and around 19,000 TLB flushes.

The first patch in the series, of always using lazy TLB mode, reduces CPU use around 1% on both Haswell and Broadwell systems.

The rest of the series reduces the number of TLB flush IPIs by about 1,500 a second, resulting in a 0.2% reduction in CPU use, on top of the 1% seen by just enabling lazy TLB mode.

These are the low hanging fruits in the context switch code.

The big thing remaining is the reference count overhead of the lazy TLB mm_struct, but getting rid of that is rather a lot of code for a small performance gain. Not quite what Linus asked for :)

This v2 is "identical" to the version I posted yesterday, except this one is actually against current -tip (not sure what went wrong before), with a number of relevant patches on top:
- tip x86/core

	012e77a903d ("x86/nmi: Fix NMI uaccess race against CR3 switching")
- arm64 tlb/asm-generic (entire branch)
- peterz queue mm/tlb
	12b2b80ec6f4 ("x86/mm: Page size aware flush_tlb_mm_range()")