Skip to content

x86,tlb,mm: more lazy TLB cleanups & optimizations

Rik van Riel lkml 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest11 patches 5 repliesIn review
  2. v110 patches 16 repliesSuperseded

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11x86,tlb: clarify memory barrier in switch_mm_irqs_offarch/x86/mm/tlb.c+5 -20 replies
2/11smp: use __cpumask_set_cpu in on_each_cpu_condkernel/smp.c+1 -10 replies
3/11smp,cpumask: introduce on_each_cpu_cond_maskinclude/linux/smp.h kernel/smp.c kernel/up.c+28 -70 replies
4/11x86,mm: use on_each_cpu_cond for TLB flushesarch/x86/mm/tlb.c+15 -600 replies
5/11mm,tlb: turn dummy defines into inline functionsinclude/asm-generic/tlb.h+7 -20 replies
6/11mm,x86: skip cr4 and ldt reload when mm stays the samearch/x86/mm/tlb.c+4 -20 replies
7/11x86,mm: remove leave_mm cpu argumentarch/x86/include/asm/mmu.h arch/x86/mm/tlb.c arch/x86/xen/mmu_pv.c +1 more+4 -40 replies
8/11arch,mm: add config variable to skip lazy TLB mm refcountingarch/Kconfig+4 -00 replies
9/11mm,x86: shoot down lazy TLB references at exit_mmap timearch/x86/Kconfig arch/x86/include/asm/mmu_context.h arch/x86/include/asm/tlbflush.h +1 more+19 -00 replies
10/11x86,tlb: really leave mm on shootdownarch/x86/mm/tlb.c+4 -00 replies
11/11mm,sched: conditionally skip lazy TLB mm refcountingarch/x86/mm/tlb.c fs/exec.c include/linux/sched/mm.h +2 more+64 -185 replies

Reviews trailers per patch

4 of 11 patches have a Reviewed-by or Acked-by.

PatchAndy LutomirskiPeter Zijlstra (Intel)
1/11R-b
2/11R-b
3/11
4/11
5/11
6/11A-b
7/11
8/11
9/11
10/11
11/11A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 0/11] x86,tlb,mm: more lazy TLB cleanups & optimizations

This patch series implements the cleanups suggested by Peter and Andy, removes lazy TLB mm refcounting on x86, and shows how other architectures could implement that same optimization.

The previous patch series already seems to have removed most of the cache line contention I was seeing at context switch time, so CPU use of the memcache and memcache-like workloads has not changed measurably with this patch series.

However, the memory bandwidth used by the memcache system has been reduced by about 1%, to serve the same number of queries per second.

This happens on two socket Haswell and Broadwell systems. Maybe on larger systems (4 or 8 socket) one might also see a measurable drop in the amount of CPU time used, with workloads where the previous patch series does not remove all cache line contention on the mm.

This is against the latest -tip tree, and seems to be stable (on top of another tree) with workloads that do over a million context switches a second.

V2 of the series uses Peter's logic flow in context_switch(), and is a step in the direction of completely getting rid of ->active_mm.

Read the cover letter in its thread