x86/CPU/AMD: Fix AMD TLBI Erratum #1718
Subsystems (2)
PATCH 1/1 x86/CPU/AMD: Fix AMD TLBI Erratum #1718
From: "Borislav Petkov (AMD)" <[email protected]>
The processor may fail to invalidate a TLB entry while executing an INVLPGB/TLBSYNC instruction pair.
Detect the erratum and apply a software workaround when a microcode fix is not present.
--- arch/x86/include/asm/cpufeatures.h | 3 +++ arch/x86/include/asm/tlb.h | 6 ++++++ arch/x86/kernel/cpu/amd.c | 4 ++++ 3 files changed, 13 insertions(+)
arch/x86/include/asm/cpufeatures.h +3 -0
@@ -477,6 +477,8 @@#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0)/* No Nested Data Breakpoints */ #define X86_FEATURE_WRMSR_XX_BASE_NS (20*32+ 1)/* WRMSR to {FS,GS,KERNEL_GS}_BASE is non-serializing */ #define X86_FEATURE_LFENCE_RDTSC (20*32+ 2)/* LFENCE always serializing / synchronizes RDTSC */ + +#define X86_FEATURE_TLBSYNC_SFW_NO (20*32+ 4)/* TLBI Erratum #1718 fixed in microcode */ #define X86_FEATURE_VERW_CLEAR (20*32+ 5)/* The memory form of VERW mitigates TSA */ #define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6)/* Null Selector Clears Base */
@@ -587,4 +589,5 @@#define X86_BUG_TSA X86_BUG( 1*32+ 9)/* "tsa" CPU is affected by Transient Scheduler Attacks */ #define X86_BUG_VMSCAPE X86_BUG( 1*32+10)/* "vmscape" CPU is affected by VMSCAPE attacks from guests */ #define X86_BUG_SEAMRET_INVD_VMCS X86_BUG( 1*32+11)/* "seamret_invd_vmcs" SEAMRET from P-SEAMLDR clears the current VMCS */ +#define X86_BUG_TLBI X86_BUG( 1*32+12)/* "tlbi" AMD TLBI Erratum #1718 */ #endif /* _ASM_X86_CPUFEATURES_H */
arch/x86/include/asm/tlb.h +6 -0
@@ -110,6 +110,12 @@ static inline void __tlbsync(void)/* TLBSYNC: supported in binutils >= 0.36. */ asmvolatile (".byte 0x0f, 0x01, 0xff" :::"memory" ); + +if (!static_cpu_has_bug (X86_BUG_TLBI )) +return ; + +/* TLBSYNC: supported in binutils >= 0.36. */ + asm volatile(".byte 0x0f, 0x01, 0xff" ::: "memory"); }#else /* Some compilers (I'm looking at you clang!) simply can't do DCE */
arch/x86/kernel/cpu/amd.c +4 -0
@@ -1049,6 +1049,10 @@ static void init_amd_zen5(struct cpuinfo_x86 *c)msr_clear_bit (MSR_AMD64_CPUID_FN_7 ,18 );pr_emerg_once ("RDSEED32 is broken. Disabling the corresponding CPUID bit.\n" ); } + +if (cpu_has (c,X86_FEATURE_INVLPGB ) && + !cpu_has (c,X86_FEATURE_TLBSYNC_SFW_NO )) +set_cpu_bug (c,X86_BUG_TLBI ); }static void init_amd (struct cpuinfo_x86 *c)
Signature · 1 line
2.53.0
Quoted 5 lines · Borislav Petkov
From: "Borislav Petkov (AMD)" <[email protected]>
The processor may fail to invalidate a TLB entry while executing an INVLPGB/TLBSYNC instruction pair.
Quoted 8 lines
Detect the erratum and apply a software workaround when a microcode fix
is not present.Fixes 82378c6c2f43 ("x86/mm: Use INVLPGB for kernel TLB flushes")Signed-off-by Borislav Petkov (AMD) <[email protected]>Cc <[email protected]>
This same change fell out of my testing with a reproducer earlier this year, so I know it works, at least for the test case I have.
I am surprised that it was good enough to be a real workaround, but I'll take it ;)
Signature · 1 line
All Rights Reversed.
Borislav Petkov
maintainerQuoted 6 lines · Rik van Riel
This same change fell out of my testing with a reproducer earlier this year, so I know it works, at least for the test case I have.
I am surprised that it was good enough to be a real workaround, but I'll take it ;)
Me too, but it really is - there's a good reason why :)
But the most proper workaround is for everyone to update microcode anyway - it is released too, so we don't even need the sw fallback.
Quoted 2 lines
Tested-by Rik van Riel <[email protected]>Reviewed-by Rik van Riel <[email protected]>
Thanks, my plan is to queue it next week so that we have it in 7.3 and it percolates to stable.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette+cc Rik FYI!
Quoted 4 lines · Borislav Petkov
From: "Borislav Petkov (AMD)" <[email protected]>
The processor may fail to invalidate a TLB entry while executing an INVLPGB/TLBSYNC instruction pair.
Hi,
I've repro'd the issue on a threadripper - a Ryzen Threadripper 9980X (family 0x1a model 0x08 stepping 1) on microcode 0x0b008121 (the latest I believe).
It failed Matt's repro in seconds on 7.2.7 (round 3 on the first run, round 1 on the second, same stale-offset signature Rik described).
It doesn't seem there's a microcode fix for the threadripper yet.
I confirmed tlbi=ipi fixed it, 20 rounds with no failures.
And I built and ran a kernel with this patch applied (and tlbi=ipi removed from the kernel command line) and this patch fixes it it too - 40 rounds with no failures.
So:
Hopefully some useful datapoints also - + I hope there'll be a firmware fix for the threadripper too :)
--
Cheers, Lorenzo
Borislav Petkov
maintainerQuoted 2 lines · Lorenzo Stoakes (ARM)
Hopefully some useful datapoints also - + I hope there'll be a firmware fix for the threadripper too :)
Yeah, working on it. We'll see.
Thx for the datapoints - very useful.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteThe following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 87db7795cbaaf636ae22ae031f4906db7bb77803 Gitweb: https://git.kernel.org/tip/87db7795cbaaf636ae22ae031f4906db7bb77803 Author: Borislav Petkov (AMD) <[email protected]> AuthorDate: Fri, 02 Oct 2026 14:16:17 -07:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Mon, 05 Oct 2026 10:29:08 -07:00
x86/CPU/AMD: Fix AMD TLBI Erratum #1718
The processor may fail to invalidate a TLB entry while executing an INVLPGB/TLBSYNC instruction pair.
Detect the erratum and apply a software workaround when a microcode fix is not present.
--- arch/x86/include/asm/cpufeatures.h | 3 +++ arch/x86/include/asm/tlb.h | 6 ++++++ arch/x86/kernel/cpu/amd.c | 4 ++++ 3 files changed, 13 insertions(+)
arch/x86/include/asm/cpufeatures.h +3 -0
@@ -477,6 +477,8 @@#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0)/* No Nested Data Breakpoints */ #define X86_FEATURE_WRMSR_XX_BASE_NS (20*32+ 1)/* WRMSR to {FS,GS,KERNEL_GS}_BASE is non-serializing */ #define X86_FEATURE_LFENCE_RDTSC (20*32+ 2)/* LFENCE always serializing / synchronizes RDTSC */ + +#define X86_FEATURE_TLBSYNC_SFW_NO (20*32+ 4)/* TLBI Erratum #1718 fixed in microcode */ #define X86_FEATURE_VERW_CLEAR (20*32+ 5)/* The memory form of VERW mitigates TSA */ #define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6)/* Null Selector Clears Base */
@@ -587,4 +589,5 @@#define X86_BUG_TSA X86_BUG( 1*32+ 9)/* "tsa" CPU is affected by Transient Scheduler Attacks */ #define X86_BUG_VMSCAPE X86_BUG( 1*32+10)/* "vmscape" CPU is affected by VMSCAPE attacks from guests */ #define X86_BUG_SEAMRET_INVD_VMCS X86_BUG( 1*32+11)/* "seamret_invd_vmcs" SEAMRET from P-SEAMLDR clears the current VMCS */ +#define X86_BUG_TLBI X86_BUG( 1*32+12)/* "tlbi" AMD TLBI Erratum #1718 */ #endif /* _ASM_X86_CPUFEATURES_H */
arch/x86/include/asm/tlb.h +6 -0
@@ -110,6 +110,12 @@ static inline void __tlbsync(void)/* TLBSYNC: supported in binutils >= 0.36. */ asmvolatile (".byte 0x0f, 0x01, 0xff" :::"memory" ); + +if (!static_cpu_has_bug (X86_BUG_TLBI )) +return ; + +/* TLBSYNC: supported in binutils >= 0.36. */ + asm volatile(".byte 0x0f, 0x01, 0xff" ::: "memory"); }#else /* Some compilers (I'm looking at you clang!) simply can't do DCE */
arch/x86/kernel/cpu/amd.c +4 -0
@@ -1049,6 +1049,10 @@ static void init_amd_zen5(struct cpuinfo_x86 *c)msr_clear_bit (MSR_AMD64_CPUID_FN_7 ,18 );pr_emerg_once ("RDSEED32 is broken. Disabling the corresponding CPUID bit.\n" ); } + +if (cpu_has (c,X86_FEATURE_INVLPGB ) && + !cpu_has (c,X86_FEATURE_TLBSYNC_SFW_NO )) +set_cpu_bug (c,X86_BUG_TLBI ); }static void init_amd (struct cpuinfo_x86 *c)