Skip to contentSkip to conversation

KVM: x86/SVM: Fixes for AMD ERAPS virtualization

PATCH Superseded Jim Mattson 2 participants 5 messages Jul 16

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover KVM: x86/SVM: Fixes for AMD ERAPS virtualization

This series fixes three issues in KVM's virtualization of Enhanced Return Address Predictor Security (ERAPS):

Patch 1 moves ALLOW_LARGER_RAP configuration from init_vmcb() to svm_vcpu_after_set_cpuid(). On initial vCPU creation, init_vmcb() runs before guest CPUID is configured via KVM_SET_CPUID2, leaving the BSP without ALLOW_LARGER_RAP. Configuring ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid() handles all vCPUs in one place.

Patch 2 moves ERAPS dirty tracking into svm_flush_tlb_asid(). AMD ERAPS architecturally requires clearing the Return Address Predictor (RAP/RSB) on all implicit TLB invalidations. Centralizing dirty tracking in svm_flush_tlb_asid() ensures that local/current ASID flushes (e.g. same-value CR3 reloads under shadow paging, CR4.PAE/SMEP toggles, single-context INVPCID) correctly mark ERAPS dirty and clear the RAP on the next VMRUN.

Patch 3 adds a KVM_REQ_TLB_FLUSH_GUEST request to kvm_mtrr_set_msr() when MTRR MSRs are written via WRMSR. Per AMD APM and Intel SDM, MTRR writes are implicit TLB invalidations. Requesting a guest TLB flush restores x86 architectural compliance and ensures ERAPS RAP clearing on MTRR writes.

Jim Mattson (3):
KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()
KVM: SVM: Dirty ERAPS register on all ASID TLB flushes
KVM: x86: Flush guest TLB on MTRR MSR writes

 arch/x86/kvm/mtrr.c    |  1 +
 arch/x86/kvm/svm/svm.c | 19 ++++++++-----------
 2 files changed, 9 insertions(+), 11 deletions(-)
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#1 lore raw

PATCH 1/3 KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()

On initial vCPU creation, init_vmcb() is called before userspace configures guest CPUID via KVM_SET_CPUID2. Consequently, ERAP_CONTROL_ALLOW_LARGER_RAP is never set during initial vCPU creation. If an AP later receives an INIT signal after CPUID has been set, init_vmcb() would set the bit for that AP, but the BSP would be left with ALLOW_LARGER_RAP clear.

Move the ALLOW_LARGER_RAP configuration from init_vmcb() into svm_vcpu_after_set_cpuid() to dynamically set or clear the bit in vmcb01's erap_ctl whenever guest CPUID changes.

Remove the now redundaant code to set ALLOW_LARGER_RAP inside init_vmcb().

Fixes db5e82496492 ("KVM: SVM: Virtualize and advertise support for ERAPS")
Assisted-by Gemini:Gemini-Next
---
 arch/x86/kvm/svm/svm.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/svm.c +5 -3
index 4d2bacd00ec4..b76c6a76ce37 100644
@@ -1243,9 +1243,6 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
 		svm_clr_intercept(svm, INTERCEPT_PAUSE);
 	}
 
-	if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
-		svm->vmcb->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP;
-
 	if (enable_apicv && irqchip_in_kernel(vcpu->kvm))
 		avic_init_vmcb(svm, vmcb);
 
@@ -4736,6 +4733,11 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	if (guest_cpuid_is_intel_compatible(vcpu))
 		guest_cpu_cap_clear(vcpu, X86_FEATURE_V_VMSAVE_VMLOAD);
 
+	if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
+		svm->vmcb01.ptr->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP;
+	else
+		svm->vmcb01.ptr->control.erap_ctl &= ~ERAP_CONTROL_ALLOW_LARGER_RAP;
+
 	if (is_sev_guest(vcpu))
 		sev_vcpu_after_set_cpuid(svm);
 }
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#2 lore raw

Re: [PATCH 1/3] KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The ERAP_CONTROL_ALLOW_LARGER_RAP bit in the VMCB is permanently lost if the vCPU is reset due to…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The ERAP_CONTROL_ALLOW_LARGER_RAP bit in the VMCB is permanently lost if the vCPU is reset due to a triple-fault (SHUTDOWN intercept). --

--- Patch [1]: [PATCH 1/3] KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid() --- commit 7fe0a6077a4e214f36b5f568e5752ff88dbf1752
Author: Jim Mattson <[email protected]>

KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()

This commit moves the configuration of the ALLOW_LARGER_RAP bit from init_vmcb() to svm_vcpu_after_set_cpuid() so that it is properly set when userspace configures the guest CPUID. It removes the previous initialization logic from init_vmcb().

arch/x86/kvm/svm/svm.c static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) · line 1248

Quoted 11 lines
arch/x86/kvm/svm/svm.c +0 -3
index ef69a51ab27f9..11e023abc7ab9 100644
@@ -1243,9 +1243,6 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
 		svm_clr_intercept(svm, INTERCEPT_PAUSE);
 	}
 
-	if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
-		svm->vmcb->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP;
-

[Severity: High]
Does removing this initialization cause the ALLOW_LARGER_RAP bit to be permanently lost after a triple-fault?

When a guest triggers a triple fault, KVM intercepts the shutdown and wipes the VMCB:

arch/x86/kvm/svm/svm.c:shutdown_interception() {
    ...
    clear_page(svm->vmcb);
    kvm_vcpu_reset(vcpu, true);
    ...
}

kvm_vcpu_reset() then calls init_vmcb() to reinitialize the VMCB. Since the ALLOW_LARGER_RAP setup is removed from init_vmcb(), and userspace typically does not call KVM_SET_CPUID2 during a soft VM reset, will the ERAPS capability remain disabled indefinitely after the reset?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

PATCH 2/3 KVM: SVM: Dirty ERAPS register on all ASID TLB flushes

In reply to Jim Mattson

Per the AMD APM, ERAPS clears the Return Address Predictor (RAP/RSB) on all implicit TLB invalidations (e.g. modifying certain CR[04] bits).

Move kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS) into svm_flush_tlb_asid() so that any ASID-level TLB flush marks ERAPS dirty. Centralizing the ERAPS dirty call in svm_flush_tlb_asid() ensures that all ASID flushes (flush_tlb_current, flush_tlb_all, flush_tlb_guest) properly set ERAP_CONTROL_CLEAR_RAP on the next VMRUN.

Remove the now redundant svm_flush_tlb_guest() wrapper.

Fixes db5e82496492 ("KVM: SVM: Virtualize and advertise support for ERAPS")
Assisted-by Gemini:Gemini-Next
---
 arch/x86/kvm/svm/svm.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/svm.c +3 -8
index b76c6a76ce37..2514c6516fa1 100644
@@ -4168,6 +4168,8 @@ static void svm_flush_tlb_asid(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
+	kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS);
+
 	/*
 	 * Unlike VMX, SVM doesn't provide a way to flush only NPT TLB entries.
 	 * A TLB flush for the current ASID flushes both "host" and "guest" TLB
@@ -4226,13 +4228,6 @@ static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva)
 	invlpga(gva, svm->vmcb->control.asid);
 }
 
-static void svm_flush_tlb_guest(struct kvm_vcpu *vcpu)
-{
-	kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS);
-
-	svm_flush_tlb_asid(vcpu);
-}
-
 static inline void sync_cr8_to_lapic(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
@@ -5380,7 +5375,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = {
 	.flush_tlb_all = svm_flush_tlb_all,
 	.flush_tlb_current = svm_flush_tlb_current,
 	.flush_tlb_gva = svm_flush_tlb_gva,
-	.flush_tlb_guest = svm_flush_tlb_guest,
+	.flush_tlb_guest = svm_flush_tlb_asid,
 
 	.vcpu_pre_run = svm_vcpu_pre_run,
 	.vcpu_run = svm_vcpu_run,
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#4 lore raw

PATCH 3/3 KVM: x86: Flush guest TLB on MTRR MSR writes

In reply to Jim Mattson

Per both the AMD APM and the Intel SDM, writing to an MTRR with WRMSR is an implicit TLB invalidation that invalidates all TLB entries (including global entries). [The APM uses the word, "update," which could be construed to mean "modify," but it is unclear.]

Previously, kvm_mtrr_set_msr() stored the updated MTRR state without requesting a TLB flush. This broke x86 architectural compliance by leaving existing TLB entries in hardware. Additionally, on AMD CPUs supporting ERAPS, omitting the TLB flush request meant VCPU_REG_ERAPS was never dirtied on MTRR writes, leaving the Return Address Predictor (RAP/RSB) uncleared.

Issue a KVM_REQ_TLB_FLUSH_GUEST request on every MTRR MSR write to match x86 architectural semantics.

Fixes 9ba075a664df ("KVM: MTRR support")
Assisted-by Gemini:Gemini-Next
---
 arch/x86/kvm/mtrr.c | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/mtrr.c +1 -0
index 6f74e2b27c1e..ca5e5c7f7022 100644
@@ -105,6 +105,7 @@ int kvm_mtrr_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 		return 1;
 
 	*mtrr = data;
+	kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
 	return 0;
 }
 
Signature · 1 line
2.55.0.229.g6434b31f56-goog