skip redundant TLB sync IPIs

From: Lance Yang <[email protected]>

Hi all,

When unsharing hugetlb PMD page tables or collapsing pages in khugepaged, we send two IPIs: one for TLB invalidation, and another to synchronize with concurrent GUP-fast walkers. However, if the TLB flush already reaches all CPUs, the second IPI is redundant. GUP-fast runs with IRQs disabled, so when the TLB flush IPI completes, any concurrent GUP-fast must have finished.

This series introduces a way for architectures to indicate their TLB flush already provides full synchronization, allowing the redundant IPI to be skipped. For now, the optimization is implemented for x86 first and applied to all page table operations that free or unshare tables.

David Hildenbrand did the initial implementation. I built on his work and relied on off-list discussions to push it further — thanks a lot David!

RFC -> v1:
- Use a callback function in pv_mmu_ops instead of comparing function pointers (per David)
- Embed the check directly in tlb_remove_table_sync_one() instead of requiring every caller to check explicitly (per David)
- Move tlb_table_flush_implies_ipi_broadcast() outside of CONFIG_MMU_GATHER_RCU_TABLE_FREE to fix build error on architectures that don't enable this config.
https://lore.kernel.org/oe-kbuild-all/[email protected]/
- https://lore.kernel.org/linux-mm/[email protected]/

Lance Yang (3):
mm/tlb: allow architectures to skip redundant TLB sync IPIs x86/mm: implement redundant IPI elimination for page table operations
mm: embed TLB flush IPI check in tlb_remove_table_sync_one()

 arch/x86/include/asm/paravirt_types.h |  6 ++++++
 arch/x86/include/asm/tlb.h            | 19 ++++++++++++++++++-
 arch/x86/kernel/paravirt.c            | 10 ++++++++++
 include/asm-generic/tlb.h             | 14 ++++++++++++++
 mm/mmu_gather.c                       |  4 ++++
 5 files changed, 52 insertions(+), 1 deletion(-)

2.49.0