skip redundant sync IPIs when TLB flush sent them

PATCH Superseded Lance Yang 3 participants 7 messages Mar 4 – Mar 6

Subsystems (8)

Memory management x86 KVM Generic include/asm header files +4 more subsystems

lore

Share & tools

lore

Lance Yang

author

2026-03-04 02:11 UTC #0 lore raw

v6 cover skip redundant sync IPIs when TLB flush sent them

Hi all,

When page table operations require synchronization with software/lockless walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the TLB (tlb->freed_tables or tlb->unshared_tables).

On architectures where the TLB flush already sends IPIs to all target CPUs, the subsequent sync IPI broadcast is redundant. This is not only costly on large systems where it disrupts all CPUs even for single-process page table operations, but has also been reported to hurt RT workloads[1].

This series introduces tlb_table_flush_implies_ipi_broadcast() to check if the prior TLB flush already provided the necessary synchronization. When true, the sync calls can early-return.

A few cases rely on this synchronization:

1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse of the PMD table for other purposes in the last remaining user after unsharing.

2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing and (possibly) freeing the page table / re-depositing it.

Two-step plan as David suggested[4]:

Step 1 (this series): Skip redundant sync when we're 100% certain the TLB flush sent IPIs. INVLPGB is excluded because when supported, we cannot guarantee IPIs were sent, keeping it clean and simple.

Step 2 (future work): Send targeted IPIs only to CPUs actually doing software/lockless page table walks, benefiting all architectures.

Regarding Step 2, it obviously only applies to setups where Step 1 does not
apply: like x86 with INVLPGB or arm64. Step 2 work is ongoing; early attempts showed ~3% GUP-fast overhead. Reducing the overhead requires more work and tuning; it will be submitted separately once ready.

David Hildenbrand did the initial implementation. I built on his work and relied on off-list discussions to push it further - thanks a lot David!

[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
[4] https://lore.kernel.org/linux-mm/[email protected]/

v5 -> v6:
- Use static_branch to eliminating the branch overhead (per Peter)
- https://lore.kernel.org/linux-mm/[email protected]/

v4 -> v5:
- Drop per-CPU tracking (active_lockless_pt_walk_mm) from this series; defer to Step 2 as it adds ~3% GUP-fast overhead
- Keep pv_ops property false for PV backends like KVM: preempted vCPUs cannot be assumed safe（per Sean)
https://lore.kernel.org/linux-mm/[email protected]/
- https://lore.kernel.org/linux-mm/[email protected]/

v3 -> v4:
- Rework based on David's two-step direction and per-CPU idea:
1) Targeted IPIs: per-CPU variable when entering/leaving lockless page table walk; tlb_remove_table_sync_mm() IPIs only those CPUs.
2) On x86, pv_mmu_ops property set at init to skip the extra sync when flush_tlb_multi() already sends IPIs.
https://lore.kernel.org/linux-mm/[email protected]/
- https://lore.kernel.org/linux-mm/[email protected]/

v2 -> v3:
- Complete rewrite: use dynamic IPI tracking instead of static checks (per Dave Hansen, thanks!)
- Track IPIs via mmu_gather: native_flush_tlb_multi() sets flag when actually sending IPIs
- Motivation for skipping redundant IPIs explained by David:
https://lore.kernel.org/linux-mm/[email protected]/
- https://lore.kernel.org/linux-mm/[email protected]/

v1 -> v2:
- Fix cover letter encoding to resolve send-email issues. Apologies for any email flood caused by the failed send attempts :(

RFC -> v1:
- Use a callback function in pv_mmu_ops instead of comparing function pointers (per David)
- Embed the check directly in tlb_remove_table_sync_one() instead of requiring every caller to check explicitly (per David)
- Move tlb_table_flush_implies_ipi_broadcast() outside of CONFIG_MMU_GATHER_RCU_TABLE_FREE to fix build error on architectures that don't enable this config.
https://lore.kernel.org/oe-kbuild-all/[email protected]/
- https://lore.kernel.org/linux-mm/[email protected]/

Lance Yang (2):
mm/mmu_gather: prepare to skip redundant sync IPIs x86/tlb: skip redundant sync IPIs for native TLB flush

 arch/x86/include/asm/paravirt_types.h |  5 +++++
 arch/x86/include/asm/smp.h            |  3 +++
 arch/x86/include/asm/tlb.h            | 16 +++++++++++++++-
 arch/x86/include/asm/tlbflush.h       |  3 +++
 arch/x86/kernel/paravirt.c            | 20 ++++++++++++++++++++
 arch/x86/kernel/smpboot.c             |  1 +
 arch/x86/mm/tlb.c                     | 14 ++++++++++++++
 include/asm-generic/tlb.h             | 17 +++++++++++++++++
 mm/mmu_gather.c                       | 15 +++++++++++++++
 9 files changed, 93 insertions(+), 1 deletion(-)

Signature · 1 line

2.49.0

Lance Yang

author

2026-03-04 02:11 UTC #1 lore raw

PATCH v6 1/2 mm/mmu_gather: prepare to skip redundant sync IPIs

From: Lance Yang <[email protected]>

When page table operations require synchronization with software/lockless walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the TLB (tlb->freed_tables or tlb->unshared_tables).

Introduce tlb_table_flush_implies_ipi_broadcast() to check if the prior TLB flush already provided the necessary synchronization. When true, the sync calls can early-return.

A few cases rely on this synchronization:

1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse of the PMD table for other purposes in the last remaining user after unsharing.

2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing and (possibly) freeing the page table / re-depositing it.

Currently always returns false (no behavior change). The follow-up patch will enable the optimization for x86.

[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/

Signed-off-by Lance Yang <[email protected]>

---
 include/asm-generic/tlb.h | 17 +++++++++++++++++
 mm/mmu_gather.c           | 15 +++++++++++++++
 2 files changed, 32 insertions(+)

include/asm-generic/tlb.h +17 -0

index bdcc2778ac64..cb41cc6a0024 100644
@@ -240,6 +240,23 @@ static inline void tlb_remove_table(struct mmu_gather *tlb, void *table)
 }
 #endif /* CONFIG_MMU_GATHER_TABLE_FREE */
 
+/**
+ * tlb_table_flush_implies_ipi_broadcast - does TLB flush imply IPI sync
+ *
+ * When page table operations require synchronization with software/lockless
+ * walkers, they flush the TLB (tlb->freed_tables or tlb->unshared_tables)
+ * then call tlb_remove_table_sync_{one,rcu}(). If the flush already sent
+ * IPIs to all CPUs, the sync call is redundant.
+ *
+ * Returns false by default. Architectures can override by defining this.
+ */
+#ifndef tlb_table_flush_implies_ipi_broadcast
+static inline bool tlb_table_flush_implies_ipi_broadcast(void)
+{
+	return false;
+}
+#endif
+
 #ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE
 /*
  * This allows an architecture that does not use the linux page-tables for

mm/mmu_gather.c +15 -0

index 3985d856de7f..37a6a711c37e 100644
@@ -283,6 +283,14 @@ void tlb_remove_table_sync_one(void)
 	 * It is however sufficient for software page-table walkers that rely on
 	 * IRQ disabling.
 	 */
+
+	/*
+	 * Skip IPI if the preceding TLB flush already synchronized with
+	 * all CPUs that could be doing software/lockless page table walks.
+	 */
+	if (tlb_table_flush_implies_ipi_broadcast())
+		return;
+
 	smp_call_function(tlb_remove_table_smp_sync, NULL, 1);
 }

@@ -312,6 +320,13 @@ static void tlb_remove_table_free(struct mmu_table_batch *batch)
  */
 void tlb_remove_table_sync_rcu(void)
 {
+	/*
+	 * Skip RCU wait if the preceding TLB flush already synchronized
+	 * with all CPUs that could be doing software/lockless page table walks.
+	 */
+	if (tlb_table_flush_implies_ipi_broadcast())
+		return;
+
 	synchronize_rcu();
 }

Signature · 1 line

2.49.0

Lance Yang

author

2026-03-04 02:11 UTC #2 lore raw

PATCH v6 2/2 x86/tlb: skip redundant sync IPIs for native TLB flush

In reply to Lance Yang

From: Lance Yang <[email protected]>

Enable the optimization introduced in the previous patch for x86.

Add pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast to track whether flush_tlb_multi() sends real IPIs. Initialize it once in native_pv_tlb_init() during boot.

On CONFIG_PARAVIRT systems, tlb_table_flush_implies_ipi_broadcast() reads the pv_ops property. On non-PARAVIRT, it directly checks for INVLPGB.

PV backends (KVM, Xen, Hyper-V) typically have their own implementations and don't call native_flush_tlb_multi() directly, so they cannot be trusted to provide the IPI guarantees we need. They keep the property false.

Two-step plan as David suggested[1]:

Step 1 (this patch): Skip redundant sync when we're 100% certain the TLB flush sent IPIs. INVLPGB is excluded because when supported, we cannot guarantee IPIs were sent, keeping it clean and simple.

Step 2 (future work): Send targeted IPIs only to CPUs actually doing software/lockless page table walks, benefiting all architectures.

Regarding Step 2, it obviously only applies to setups where Step 1 does not apply: like x86 with INVLPGB or arm64.

[1] https://lore.kernel.org/linux-mm/[email protected]/

Signed-off-by Lance Yang <[email protected]>

---
 arch/x86/include/asm/paravirt_types.h |  5 +++++
 arch/x86/include/asm/smp.h            |  3 +++
 arch/x86/include/asm/tlb.h            | 16 +++++++++++++++-
 arch/x86/include/asm/tlbflush.h       |  3 +++
 arch/x86/kernel/paravirt.c            | 20 ++++++++++++++++++++
 arch/x86/kernel/smpboot.c             |  1 +
 arch/x86/mm/tlb.c                     | 14 ++++++++++++++
 7 files changed, 61 insertions(+), 1 deletion(-)

arch/x86/include/asm/paravirt_types.h +5 -0

index 9bcf6bce88f6..ec01268f2e3e 100644
@@ -112,6 +112,11 @@ struct pv_mmu_ops {
 	void (*flush_tlb_multi)(const struct cpumask *cpus,
 				const struct flush_tlb_info *info);
 
+	/*
+	 * True if flush_tlb_multi() sends real IPIs to all target CPUs.
+	 */
+	bool flush_tlb_multi_implies_ipi_broadcast;
+
 	/* Hook for intercepting the destruction of an mm_struct. */
 	void (*exit_mmap)(struct mm_struct *mm);
 	void (*notify_page_enc_status_changed)(unsigned long pfn, int npages, bool enc);

arch/x86/include/asm/smp.h +3 -0

index 84951572ab81..ef1fe0cc4c73 100644
@@ -105,6 +105,9 @@ void native_smp_prepare_boot_cpu(void);
 void smp_prepare_cpus_common(void);
 void native_smp_prepare_cpus(unsigned int max_cpus);
 void native_smp_cpus_done(unsigned int max_cpus);
+
+void __init native_pv_tlb_init(void);
+
 int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
 int native_kick_ap(unsigned int cpu, struct task_struct *tidle);
 int native_cpu_disable(void);

arch/x86/include/asm/tlb.h +15 -1

index 866ea78ba156..532578c5a2e7 100644
@@ -5,10 +5,19 @@
 #define tlb_flush tlb_flush
 static inline void tlb_flush(struct mmu_gather *tlb);
 
+#define tlb_table_flush_implies_ipi_broadcast tlb_table_flush_implies_ipi_broadcast
+static inline bool tlb_table_flush_implies_ipi_broadcast(void);
+
 #include <asm-generic/tlb.h>
 #include <linux/kernel.h>
 #include <vdso/bits.h>
 #include <vdso/page.h>
+#include <asm/paravirt.h>
+
+static inline bool tlb_table_flush_implies_ipi_broadcast(void)
+{
+	return static_branch_likely(&tlb_ipi_broadcast_key);
+}
 
 static inline void tlb_flush(struct mmu_gather *tlb)
 {

@@ -20,7 +29,12 @@ static inline void tlb_flush(struct mmu_gather *tlb)
 		end = tlb->end;
 	}
 
-	flush_tlb_mm_range(tlb->mm, start, end, stride_shift, tlb->freed_tables);
+	/*
+	 * Pass both freed_tables and unshared_tables so that lazy-TLB CPUs
+	 * also receive IPIs during unsharing page tables.
+	 */
+	flush_tlb_mm_range(tlb->mm, start, end, stride_shift,
+			   tlb->freed_tables || tlb->unshared_tables);
 }
 
 static inline void invlpg(unsigned long addr)

arch/x86/include/asm/tlbflush.h +3 -0

index 5a3cdc439e38..a1b5efef3b90 100644
@@ -5,6 +5,7 @@
 #include <linux/mm_types.h>
 #include <linux/mmu_notifier.h>
 #include <linux/sched.h>
+#include <linux/jump_label.h>
 
 #include <asm/barrier.h>
 #include <asm/processor.h>

@@ -18,6 +19,8 @@
 
 DECLARE_PER_CPU(u64, tlbstate_untag_mask);
 
+DECLARE_STATIC_KEY_FALSE(tlb_ipi_broadcast_key);
+
 void __flush_tlb_all(void);
 
 #define TLB_FLUSH_ALL	-1UL

arch/x86/kernel/paravirt.c +20 -0

index a6ed52cae003..c8decadf16e0 100644
@@ -154,6 +154,7 @@ struct paravirt_patch_template pv_ops = {
 	.mmu.flush_tlb_kernel	= native_flush_tlb_global,
 	.mmu.flush_tlb_one_user	= native_flush_tlb_one_user,
 	.mmu.flush_tlb_multi	= native_flush_tlb_multi,
+	.mmu.flush_tlb_multi_implies_ipi_broadcast = false,
 
 	.mmu.exit_mmap		= paravirt_nop,
 	.mmu.notify_page_enc_status_changed	= paravirt_nop,

@@ -221,3 +222,22 @@ NOKPROBE_SYMBOL(native_load_idt);
 
 EXPORT_SYMBOL(pv_ops);
 EXPORT_SYMBOL_GPL(pv_info);
+
+void __init native_pv_tlb_init(void)
+{
+	/*
+	 * If PV backend already set the property, respect it.
+	 * Otherwise, check if native TLB flush sends real IPIs to all target
+	 * CPUs (i.e., not using INVLPGB broadcast invalidation).
+	 */
+	if (pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast) {
+		static_branch_enable(&tlb_ipi_broadcast_key);
+		return;
+	}
+
+	if (pv_ops.mmu.flush_tlb_multi == native_flush_tlb_multi &&
+	    !cpu_feature_enabled(X86_FEATURE_INVLPGB)) {
+		pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast = true;
+		static_branch_enable(&tlb_ipi_broadcast_key);
+	}
+}

arch/x86/kernel/smpboot.c +1 -0

index 5cd6950ab672..3cdb04162843 100644
@@ -1167,6 +1167,7 @@ void __init native_smp_prepare_boot_cpu(void)
 		switch_gdt_and_percpu_base(me);
 
 	native_pv_lock_init();
+	native_pv_tlb_init();
 }
 
 void __init native_smp_cpus_done(unsigned int max_cpus)

arch/x86/mm/tlb.c +14 -0

index 621e09d049cb..7b1acfb97782 100644
@@ -26,6 +26,8 @@
 
 #include "mm_internal.h"
 
+DEFINE_STATIC_KEY_FALSE(tlb_ipi_broadcast_key);
+
 #ifdef CONFIG_PARAVIRT
 # define STATIC_NOPV
 #else

@@ -1834,3 +1836,15 @@ static int __init create_tlb_single_page_flush_ceiling(void)
 	return 0;
 }
 late_initcall(create_tlb_single_page_flush_ceiling);
+
+#ifndef CONFIG_PARAVIRT
+void __init native_pv_tlb_init(void)
+{
+	/*
+	 * For non-PARAVIRT builds, check if native TLB flush sends real IPIs
+	 * (i.e., not using INVLPGB broadcast invalidation).
+	 */
+	if (!cpu_feature_enabled(X86_FEATURE_INVLPGB))
+		static_branch_enable(&tlb_ipi_broadcast_key);
+}
+#endif

Signature · 1 line

2.49.0

Dave Hansen

2026-03-04 17:59 UTC #3 lore raw

Re: [PATCH v6 2/2] x86/tlb: skip redundant sync IPIs for native TLB flush

On 3/3/26 18:10, Lance Yang wrote:
...

Quoted 6 lines

arch/x86/kernel/paravirt.c +6 -0

+	if (pv_ops.mmu.flush_tlb_multi == native_flush_tlb_multi &&
+	    !cpu_feature_enabled(X86_FEATURE_INVLPGB)) {
+		pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast = true;
+		static_branch_enable(&tlb_ipi_broadcast_key);
+	}
+}

...

arch/x86/mm/tlb.c static int __init create_tlb_single_page_flush_ceiling(void) · line 1850

Quoted 11 lines

arch/x86/mm/tlb.c +11 -0

+#ifndef CONFIG_PARAVIRT
+void __init native_pv_tlb_init(void)
+{
+	/*
+	 * For non-PARAVIRT builds, check if native TLB flush sends real IPIs
+	 * (i.e., not using INVLPGB broadcast invalidation).
+	 */
+	if (!cpu_feature_enabled(X86_FEATURE_INVLPGB))
+		static_branch_enable(&tlb_ipi_broadcast_key);
+}
+#endif

I really despise duplicated logic. The X86_FEATURE_INVLPGB check is small, but it is duplicated. You're also setting the static branch in a *bunch* of different places.

Can this be arranged so that the PV code just tells the core code that it is compatible with flush_tlb_multi_implies_ipi_broadcast?

void __init bool is_pv_ok(void)

{
	/* This check is super sketchy an unexplained: */
	if (pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast)
		return true;

	if (pv_ops.mmu.flush_tlb_multi != native_flush_tlb_multi)
		return false;

	pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast = true;

	return true;
}

void __init tlb_init(void)

{
	if (!is_pv_ok())
		return;

	if (cpu_feature_enabled(X86_FEATURE_INVLPGB))
		return;
		
	static_branch_enable(&tlb_ipi_broadcast_key);
}

Isn't that like a billion times more readable? It has one X86_FEATURE_INVLPGB check and one static_branch_enable() point and no #ifdeffery other than defining a stub is_pv_ok().

BTW, why is there even an early return for the case where flush_tlb_multi_implies_ipi_broadcast is already set? Isn't this decision made once on the boot CPU and then never touched again? Do any PV instances actually set the bit?

Lance Yang

author

2026-03-05 09:16 UTC #4 lore raw

Re: [PATCH v6 2/2] x86/tlb: skip redundant sync IPIs for native TLB flush

Hi Dave,

Thanks for taking time to review!

Quoted 9 lines · Dave Hansen

On 3/3/26 18:10, Lance Yang wrote:
...

Quoted 6 lines

(quoted diff) +6 -0

+	if (pv_ops.mmu.flush_tlb_multi == native_flush_tlb_multi &&
+	    !cpu_feature_enabled(X86_FEATURE_INVLPGB)) {
+		pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast = true;
+		static_branch_enable(&tlb_ipi_broadcast_key);
+	}
+}

...

Quoted 15 lines

Review
Quoted 11 lines
(quoted diff) +11 -0
+#ifndef CONFIG_PARAVIRT
+void __init native_pv_tlb_init(void)
+{
+	/*
+	 * For non-PARAVIRT builds, check if native TLB flush sends real IPIs
+	 * (i.e., not using INVLPGB broadcast invalidation).
+	 */
+	if (!cpu_feature_enabled(X86_FEATURE_INVLPGB))
+		static_branch_enable(&tlb_ipi_broadcast_key);
+}
+#endif
I really despise duplicated logic. The X86_FEATURE_INVLPGB check is small, but it is duplicated. You're also setting the static branch in a *bunch* of different places.

Sorry for the mess :(

Quoted 2 lines

Can this be arranged so that the PV code just tells the core code that it is compatible with flush_tlb_multi_implies_ipi_broadcast?

Yeah, much much better and cleaner!

Quoted 25 lines

void __init bool is_pv_ok(void)

{
	/* This check is super sketchy an unexplained: */
	if (pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast)
		return true;

	if (pv_ops.mmu.flush_tlb_multi != native_flush_tlb_multi)
		return false;

	pv_ops.mmu.flush_tlb_multi_implies_ipi_broadcast = true;

	return true;
}

void __init tlb_init(void)

{
	if (!is_pv_ok())
		return;

	if (cpu_feature_enabled(X86_FEATURE_INVLPGB))
		return;
		
	static_branch_enable(&tlb_ipi_broadcast_key);
}

Quoted 3 lines

Isn't that like a billion times more readable? It has one X86_FEATURE_INVLPGB check and one static_branch_enable() point and no #ifdeffery other than defining a stub is_pv_ok().

Yep, absolutely ;) Will rework it for v7 as you suggested.

Quoted 4 lines

BTW, why is there even an early return for the case where flush_tlb_multi_implies_ipi_broadcast is already set? Isn't this decision made once on the boot CPU and then never touched again? Do any PV instances actually set the bit?

Good point. No PV backend sets it today, so we don't need that check or even the property. I'll drop them. If a PV backend ever needs to indicate it sends real IPIs, we can add the property back then.

Does the following look reasonable?

---8<---

arch/x86/include/asm/tlb.h +14 -1

index 866ea78ba156..74292abe8852 100644
@@ -5,11 +5,19 @@
 #define tlb_flush tlb_flush
 static inline void tlb_flush(struct mmu_gather *tlb);
 
+#define tlb_table_flush_implies_ipi_broadcast tlb_table_flush_implies_ipi_broadcast
+static inline bool tlb_table_flush_implies_ipi_broadcast(void);
+
 #include <asm-generic/tlb.h>
 #include <linux/kernel.h>
 #include <vdso/bits.h>
 #include <vdso/page.h>
 
+static inline bool tlb_table_flush_implies_ipi_broadcast(void)
+{
+	return static_branch_likely(&tlb_ipi_broadcast_key);
+}
+
 static inline void tlb_flush(struct mmu_gather *tlb)
 {
 	unsigned long start = 0UL, end = TLB_FLUSH_ALL;

@@ -20,7 +28,12 @@ static inline void tlb_flush(struct mmu_gather *tlb)
 		end = tlb->end;
 	}
 
-	flush_tlb_mm_range(tlb->mm, start, end, stride_shift, tlb->freed_tables);
+	/*
+	 * Pass both freed_tables and unshared_tables so that lazy-TLB CPUs
+	 * also receive IPIs during unsharing page tables.
+	 */
+	flush_tlb_mm_range(tlb->mm, start, end, stride_shift,
+			   tlb->freed_tables || tlb->unshared_tables);
 }
 
 static inline void invlpg(unsigned long addr)

arch/x86/include/asm/tlbflush.h +27 -0

index 5a3cdc439e38..d086454eb760 100644
@@ -5,6 +5,7 @@
 #include <linux/mm_types.h>
 #include <linux/mmu_notifier.h>
 #include <linux/sched.h>
+#include <linux/jump_label.h>
 
 #include <asm/barrier.h>
 #include <asm/processor.h>

@@ -15,9 +16,35 @@
 #include <asm/pti.h>
 #include <asm/processor-flags.h>
 #include <asm/pgtable.h>
+#include <asm/paravirt.h>
 
 DECLARE_PER_CPU(u64, tlbstate_untag_mask);
 
+DECLARE_STATIC_KEY_FALSE(tlb_ipi_broadcast_key);
+
+#ifdef CONFIG_PARAVIRT
+static inline bool __init pv_tlb_is_native(void)
+{
+	return pv_ops.mmu.flush_tlb_multi == native_flush_tlb_multi;
+}
+#else
+static inline bool __init pv_tlb_is_native(void)
+{
+	return true;
+}
+#endif
+
+static inline void __init native_pv_tlb_init(void)
+{
+	if (!pv_tlb_is_native())
+		return;
+
+	if (cpu_feature_enabled(X86_FEATURE_INVLPGB))
+		return;
+
+	static_branch_enable(&tlb_ipi_broadcast_key);
+}
+
 void __flush_tlb_all(void);
 
 #define TLB_FLUSH_ALL	-1UL

arch/x86/kernel/smpboot.c +1 -0

index 5cd6950ab672..3cdb04162843 100644
@@ -1167,6 +1167,7 @@ void __init native_smp_prepare_boot_cpu(void)
 		switch_gdt_and_percpu_base(me);
 
 	native_pv_lock_init();
+	native_pv_tlb_init();
 }
 
 void __init native_smp_cpus_done(unsigned int max_cpus)

arch/x86/mm/tlb.c +2 -0

index 621e09d049cb..43e60cca38f1 100644
@@ -26,6 +26,8 @@
 
 #include "mm_internal.h"
 
+DEFINE_STATIC_KEY_FALSE(tlb_ipi_broadcast_key);
+
 #ifdef CONFIG_PARAVIRT
 # define STATIC_NOPV
 #else

---

Thanks,
Lance

kernel test robot

build bot

2026-03-06 14:36 UTC #5 lore raw

Re: [PATCH v6 2/2] x86/tlb: skip redundant sync IPIs for native TLB flush

In reply to Lance Yang

kernel test robot noticed the following build warnings:

Hi Lance,

kernel test robot noticed the following build warnings:

[auto build test WARNING on akpm-mm/mm-everything]
[also build test WARNING on next-20260305]
[cannot apply to tip/x86/core arnd-asm-generic/master peterz-queue/sched/core tip/x86/mm linus/master v7.0-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Lance-Yang/mm-mmu_gather-prepare-to-skip-redundant-sync-IPIs/20260304-101422
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20260304021046.18550-3-lance.yang%40linux.dev
patch subject: [PATCH v6 2/2] x86/tlb: skip redundant sync IPIs for native TLB flush
config: x86_64-allnoconfig (https://download.01.org/0day-ci/archive/20260306/[email protected]/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260306/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> arch/x86/mm/tlb.c:1841:13: warning: no previous prototype for function 'native_pv_tlb_init' [-Wmissing-prototypes]
    1841 | void __init native_pv_tlb_init(void)
         |             ^
   arch/x86/mm/tlb.c:1841:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
    1841 | void __init native_pv_tlb_init(void)
         | ^
         | static

1 warning generated.

vim +/native_pv_tlb_init +1841 arch/x86/mm/tlb.c

1839
1840 #ifndef CONFIG_PARAVIRT

> 1841	void __init native_pv_tlb_init(void)

Signature · 2 lines

0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

kernel test robot

build bot

2026-03-06 15:42 UTC #6 lore raw

Re: [PATCH v6 2/2] x86/tlb: skip redundant sync IPIs for native TLB flush

In reply to Lance Yang

kernel test robot noticed the following build warnings:

Hi Lance,

kernel test robot noticed the following build warnings:

url:    https://github.com/intel-lab-lkp/linux/commits/Lance-Yang/mm-mmu_gather-prepare-to-skip-redundant-sync-IPIs/20260304-101422
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20260304021046.18550-3-lance.yang%40linux.dev
patch subject: [PATCH v6 2/2] x86/tlb: skip redundant sync IPIs for native TLB flush
config: i386-allnoconfig (https://download.01.org/0day-ci/archive/20260306/[email protected]/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260306/[email protected]/reproduce)

All warnings (new ones prefixed by >>):

>> arch/x86/mm/tlb.c:1841:13: warning: no previous prototype for 'native_pv_tlb_init' [-Wmissing-prototypes]
    1841 | void __init native_pv_tlb_init(void)
         |             ^~~~~~~~~~~~~~~~~~

vim +/native_pv_tlb_init +1841 arch/x86/mm/tlb.c

1839
1840 #ifndef CONFIG_PARAVIRT

> 1841	void __init native_pv_tlb_init(void)

Signature · 2 lines

0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki