Skip to contentSkip to conversation

KVM: x86: Expose ARCH_CAP_FB_CLEAR when invulnerable to MDS

PATCH In review Jon Kohler 2 participants 3 messages Apr 1, 2025 – Apr 2, 2025

Subsystems (3)
lore
Share & tools
lore

Jon Kohler

author
#0 lore raw

PATCH 1/1 KVM: x86: Expose ARCH_CAP_FB_CLEAR when invulnerable to MDS

Expose FB_CLEAR in arch_capabilities for certain MDS-invulnerable cases to support live migration from older hardware (e.g., Cascade Lake, Ice Lake) to newer hardware (e.g., Sapphire Rapids or higher). This ensures compatibility when user space has previously configured vCPUs to see FB_CLEAR (ARCH_CAPABILITIES Bit 17).

Newer hardware sets the following bits but does not set FB_CLEAR, which can prevent user space from configuring a matching setup:
ARCH_CAP_MDS_NO
ARCH_CAP_TAA_NO
ARCH_CAP_PSDP_NO
ARCH_CAP_FBSDP_NO
ARCH_CAP_SBDR_SSDP_NO

This change has minimal impact, as these bit combinations already mark the host as MMIO immune (via arch_cap_mmio_immune()) and set disable_fb_clear in vmx_update_fb_clear_dis(), resulting in no additional overhead.

Cc Emanuele Giuseppe Esposito <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Pawan Gupta <[email protected]>
---
 arch/x86/kvm/x86.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/x86/kvm/x86.c +14 -0
index c841817a914a..2a4337aa78cd 100644
@@ -1641,6 +1641,20 @@ static u64 kvm_get_arch_capabilities(void)
 	if (!boot_cpu_has_bug(X86_BUG_GDS) || gds_ucode_mitigated())
 		data |= ARCH_CAP_GDS_NO;
 
+	/*
+	 * User space might set FB_CLEAR when starting a vCPU on a system
+	 * that does not enumerate FB_CLEAR but is also invulnerable to
+	 * other various MDS related bugs. To allow live migration from
+	 * hosts that do implement FB_CLEAR, leave it enabled.
+	 */
+	if ((data & ARCH_CAP_MDS_NO) &&
+	    (data & ARCH_CAP_TAA_NO) &&
+	    (data & ARCH_CAP_PSDP_NO) &&
+	    (data & ARCH_CAP_FBSDP_NO) &&
+	    (data & ARCH_CAP_SBDR_SSDP_NO)) {
+		data |= ARCH_CAP_FB_CLEAR;
+	}
+
 	return data;
 }
 
Signature · 1 line
2.43.0
#1 lore raw
Quoted 8 lines · Jon Kohler

Expose FB_CLEAR in arch_capabilities for certain MDS-invulnerable cases to support live migration from older hardware (e.g., Cascade Lake, Ice Lake) to newer hardware (e.g., Sapphire Rapids or higher). This ensures compatibility when user space has previously configured vCPUs to see FB_CLEAR (ARCH_CAPABILITIES Bit 17).

Newer hardware sets the following bits but does not set FB_CLEAR, which can prevent user space from configuring a matching setup:

I looked at this again right after PUCK, and KVM does NOT actually prevent userspace from matching the original, pre-SPR configuration. KVM effectively treats ARCH_CAPABILITIES like a CPUID leaf, and lets userspace shove in any value. I.e. userspace can still migrate+stuff FB_CLEAR irrespective of hardware support, and thus there is no need for KVM to lie to userspace.

So in effect, this is a userspace problem where it's being too aggressive in its sanity checks.

FWIW, even if KVM did reject unsupported ARCH_CAPABILITIES bits, I would still say this is userspace's problem to solve. E.g. by using MSR filtering to intercept and emulate RDMSR(ARCH_CAPABILITIES) in userspace.

Quoted 48 lines

ARCH_CAP_MDS_NO
ARCH_CAP_TAA_NO
ARCH_CAP_PSDP_NO
ARCH_CAP_FBSDP_NO
ARCH_CAP_SBDR_SSDP_NO

This change has minimal impact, as these bit combinations already mark the host as MMIO immune (via arch_cap_mmio_immune()) and set disable_fb_clear in vmx_update_fb_clear_dis(), resulting in no additional overhead.

Cc Emanuele Giuseppe Esposito <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Pawan Gupta <[email protected]>
---
 arch/x86/kvm/x86.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/x86/kvm/x86.c +14 -0
index c841817a914a..2a4337aa78cd 100644
@@ -1641,6 +1641,20 @@ static u64 kvm_get_arch_capabilities(void)
 	if (!boot_cpu_has_bug(X86_BUG_GDS) || gds_ucode_mitigated())
 		data |= ARCH_CAP_GDS_NO;
 
+	/*
+	 * User space might set FB_CLEAR when starting a vCPU on a system
+	 * that does not enumerate FB_CLEAR but is also invulnerable to
+	 * other various MDS related bugs. To allow live migration from
+	 * hosts that do implement FB_CLEAR, leave it enabled.
+	 */
+	if ((data & ARCH_CAP_MDS_NO) &&
+	    (data & ARCH_CAP_TAA_NO) &&
+	    (data & ARCH_CAP_PSDP_NO) &&
+	    (data & ARCH_CAP_FBSDP_NO) &&
+	    (data & ARCH_CAP_SBDR_SSDP_NO)) {
+		data |= ARCH_CAP_FB_CLEAR;
+	}
+
 	return data;
 }
 
Signature · 1 line
2.43.0
author
#2 lore raw
Quoted 25 lines · Sean Christopherson

!-------------------------------------------------------------------|
CAUTION: External Email

|-------------------------------------------------------------------!

Quoted 8 lines · Jon Kohler

Expose FB_CLEAR in arch_capabilities for certain MDS-invulnerable cases to support live migration from older hardware (e.g., Cascade Lake, Ice Lake) to newer hardware (e.g., Sapphire Rapids or higher). This ensures compatibility when user space has previously configured vCPUs to see FB_CLEAR (ARCH_CAPABILITIES Bit 17).

Newer hardware sets the following bits but does not set FB_CLEAR, which can prevent user space from configuring a matching setup:

I looked at this again right after PUCK, and KVM does NOT actually prevent userspace from matching the original, pre-SPR configuration. KVM effectively treats ARCH_CAPABILITIES like a CPUID leaf, and lets userspace shove in any value. I.e. userspace can still migrate+stuff FB_CLEAR irrespective of hardware support, and thus there is no need for KVM to lie to userspace.

Quoted 6 lines

So in effect, this is a userspace problem where it's being too aggressive in its sanity checks.

FWIW, even if KVM did reject unsupported ARCH_CAPABILITIES bits, I would still say this is userspace's problem to solve. E.g. by using MSR filtering to intercept and emulate RDMSR(ARCH_CAPABILITIES) in userspace.

Thanks, Sean, I appreciate it. I’ll see what sort of trouble I can get in on the user space side of the house with qemu to see if there is a clean way to special case this.

Cheers, Jon

Quoted 49 lines
Quoted 49 lines

ARCH_CAP_MDS_NO
ARCH_CAP_TAA_NO
ARCH_CAP_PSDP_NO
ARCH_CAP_FBSDP_NO
ARCH_CAP_SBDR_SSDP_NO

This change has minimal impact, as these bit combinations already mark the host as MMIO immune (via arch_cap_mmio_immune()) and set disable_fb_clear in vmx_update_fb_clear_dis(), resulting in no additional overhead.

Cc Emanuele Giuseppe Esposito <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Pawan Gupta <[email protected]>

---
arch/x86/kvm/x86.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)
arch/x86/kvm/x86.c +14 -0
index c841817a914a..2a4337aa78cd 100644
@@ -1641,6 +1641,20 @@ static u64 kvm_get_arch_capabilities(void)
 if (!boot_cpu_has_bug(X86_BUG_GDS) || gds_ucode_mitigated())
 data |= ARCH_CAP_GDS_NO;
 
+ /*
+ * User space might set FB_CLEAR when starting a vCPU on a system
+ * that does not enumerate FB_CLEAR but is also invulnerable to
+ * other various MDS related bugs. To allow live migration from
+ * hosts that do implement FB_CLEAR, leave it enabled.
+ */
+ if ((data & ARCH_CAP_MDS_NO) &&
+    (data & ARCH_CAP_TAA_NO) &&
+    (data & ARCH_CAP_PSDP_NO) &&
+    (data & ARCH_CAP_FBSDP_NO) &&
+    (data & ARCH_CAP_SBDR_SSDP_NO)) {
+ data |= ARCH_CAP_FB_CLEAR;
+ }
+
 return data;
 }
 
Signature · 1 line
2.43.0