KVM: x86/SVM: Fixes for AMD ERAPS virtualization
cover KVM: x86/SVM: Fixes for AMD ERAPS virtualization
This series fixes three issues in KVM's virtualization of Enhanced Return Address Predictor Security (ERAPS):
Patch 1 moves ALLOW_LARGER_RAP configuration from init_vmcb() to svm_vcpu_after_set_cpuid(). On initial vCPU creation, init_vmcb() runs before guest CPUID is configured via KVM_SET_CPUID2, leaving the BSP without ALLOW_LARGER_RAP. Configuring ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid() handles all vCPUs in one place.
Patch 2 moves ERAPS dirty tracking into svm_flush_tlb_asid(). AMD ERAPS architecturally requires clearing the Return Address Predictor (RAP/RSB) on all implicit TLB invalidations. Centralizing dirty tracking in svm_flush_tlb_asid() ensures that local/current ASID flushes (e.g. same-value CR3 reloads under shadow paging, CR4.PAE/SMEP toggles, single-context INVPCID) correctly mark ERAPS dirty and clear the RAP on the next VMRUN.
Patch 3 adds a KVM_REQ_TLB_FLUSH_GUEST request to kvm_mtrr_set_msr() when MTRR MSRs are written via WRMSR. Per AMD APM and Intel SDM, MTRR writes are implicit TLB invalidations. Requesting a guest TLB flush restores x86 architectural compliance and ensures ERAPS RAP clearing on MTRR writes.
Jim Mattson (3):
KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()
KVM: SVM: Dirty ERAPS register on all ASID TLB flushes
KVM: x86: Flush guest TLB on MTRR MSR writes
arch/x86/kvm/mtrr.c | 1 + arch/x86/kvm/svm/svm.c | 19 ++++++++----------- 2 files changed, 9 insertions(+), 11 deletions(-)
Signature · 1 line
2.55.0.229.g6434b31f56-goog
PATCH 1/3 KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()
On initial vCPU creation, init_vmcb() is called before userspace configures guest CPUID via KVM_SET_CPUID2. Consequently, ERAP_CONTROL_ALLOW_LARGER_RAP is never set during initial vCPU creation. If an AP later receives an INIT signal after CPUID has been set, init_vmcb() would set the bit for that AP, but the BSP would be left with ALLOW_LARGER_RAP clear.
Move the ALLOW_LARGER_RAP configuration from init_vmcb() into svm_vcpu_after_set_cpuid() to dynamically set or clear the bit in vmcb01's erap_ctl whenever guest CPUID changes.
Remove the now redundaant code to set ALLOW_LARGER_RAP inside init_vmcb().
--- arch/x86/kvm/svm/svm.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/svm.c +5 -3
@@ -1243,9 +1243,6 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)svm_clr_intercept (svm,INTERCEPT_PAUSE ); } -if (guest_cpu_cap_has (vcpu,X86_FEATURE_ERAPS )) - svm->vmcb->control.erap_ctl |=ERAP_CONTROL_ALLOW_LARGER_RAP ; -if (enable_apicv &&irqchip_in_kernel (vcpu->kvm))avic_init_vmcb (svm, vmcb);
@@ -4736,6 +4733,11 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)if (guest_cpuid_is_intel_compatible (vcpu))guest_cpu_cap_clear (vcpu,X86_FEATURE_V_VMSAVE_VMLOAD ); +if (guest_cpu_cap_has (vcpu,X86_FEATURE_ERAPS )) + svm->vmcb01.ptr->control.erap_ctl |=ERAP_CONTROL_ALLOW_LARGER_RAP ; +else + svm->vmcb01.ptr->control.erap_ctl &= ~ERAP_CONTROL_ALLOW_LARGER_RAP ; +if (is_sev_guest (vcpu))sev_vcpu_after_set_cpuid (svm); }
Signature · 1 line
2.55.0.229.g6434b31f56-goog
[email protected]
AI reviewRe: [PATCH 1/3] KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The ERAP_CONTROL_ALLOW_LARGER_RAP bit in the VMCB is permanently lost if the vCPU is reset due to…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The ERAP_CONTROL_ALLOW_LARGER_RAP bit in the VMCB is permanently lost if the vCPU is reset due to a triple-fault (SHUTDOWN intercept). --
--- Patch [1]: [PATCH 1/3] KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid() --- commit 7fe0a6077a4e214f36b5f568e5752ff88dbf1752
Author: Jim Mattson <[email protected]>
KVM: SVM: Configure ALLOW_LARGER_RAP in svm_vcpu_after_set_cpuid()
This commit moves the configuration of the ALLOW_LARGER_RAP bit from init_vmcb() to svm_vcpu_after_set_cpuid() so that it is properly set when userspace configures the guest CPUID. It removes the previous initialization logic from init_vmcb().
arch/x86/kvm/svm/svm.c static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) · line 1248
Quoted 11 lines
arch/x86/kvm/svm/svm.c +0
-3@@ -1243,9 +1243,6 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)svm_clr_intercept (svm,INTERCEPT_PAUSE ); } -if (guest_cpu_cap_has (vcpu,X86_FEATURE_ERAPS )) - svm->vmcb->control.erap_ctl |=ERAP_CONTROL_ALLOW_LARGER_RAP ; -
[Severity: High]
Does removing this initialization cause the ALLOW_LARGER_RAP bit to be permanently lost after a triple-fault?
When a guest triggers a triple fault, KVM intercepts the shutdown and wipes the VMCB:
arch/x86/kvm/svm/svm.c:shutdown_interception() {
...
clear_page(svm->vmcb);
kvm_vcpu_reset(vcpu, true);
...
}kvm_vcpu_reset() then calls init_vmcb() to reinitialize the VMCB. Since the ALLOW_LARGER_RAP setup is removed from init_vmcb(), and userspace typically does not call KVM_SET_CPUID2 during a soft VM reset, will the ERAPS capability remain disabled indefinitely after the reset?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH 2/3 KVM: SVM: Dirty ERAPS register on all ASID TLB flushes
In reply to Jim MattsonPer the AMD APM, ERAPS clears the Return Address Predictor (RAP/RSB) on all implicit TLB invalidations (e.g. modifying certain CR[04] bits).
Move kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS) into svm_flush_tlb_asid() so that any ASID-level TLB flush marks ERAPS dirty. Centralizing the ERAPS dirty call in svm_flush_tlb_asid() ensures that all ASID flushes (flush_tlb_current, flush_tlb_all, flush_tlb_guest) properly set ERAP_CONTROL_CLEAR_RAP on the next VMRUN.
Remove the now redundant svm_flush_tlb_guest() wrapper.
--- arch/x86/kvm/svm/svm.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/svm.c +3 -8
@@ -4168,6 +4168,8 @@ static void svm_flush_tlb_asid(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu); +kvm_register_mark_dirty (vcpu,VCPU_REG_ERAPS ); +/* * Unlike VMX, SVM doesn't provide a way to flush only NPT TLB entries. * A TLB flush for the current ASID flushes both "host" and "guest" TLB
@@ -4226,13 +4228,6 @@ static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva)invlpga (gva, svm->vmcb->control.asid); } -static void svm_flush_tlb_guest (struct kvm_vcpu *vcpu) -{ -kvm_register_mark_dirty (vcpu,VCPU_REG_ERAPS ); - -svm_flush_tlb_asid (vcpu); -} -static inline void sync_cr8_to_lapic (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);
@@ -5380,7 +5375,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .flush_tlb_all = svm_flush_tlb_all, .flush_tlb_current = svm_flush_tlb_current, .flush_tlb_gva = svm_flush_tlb_gva, - .flush_tlb_guest = svm_flush_tlb_guest, + .flush_tlb_guest = svm_flush_tlb_asid, .vcpu_pre_run = svm_vcpu_pre_run, .vcpu_run = svm_vcpu_run,
Signature · 1 line
2.55.0.229.g6434b31f56-goog
PATCH 3/3 KVM: x86: Flush guest TLB on MTRR MSR writes
In reply to Jim MattsonPer both the AMD APM and the Intel SDM, writing to an MTRR with WRMSR is an implicit TLB invalidation that invalidates all TLB entries (including global entries). [The APM uses the word, "update," which could be construed to mean "modify," but it is unclear.]
Previously, kvm_mtrr_set_msr() stored the updated MTRR state without requesting a TLB flush. This broke x86 architectural compliance by leaving existing TLB entries in hardware. Additionally, on AMD CPUs supporting ERAPS, omitting the TLB flush request meant VCPU_REG_ERAPS was never dirtied on MTRR writes, leaving the Return Address Predictor (RAP/RSB) uncleared.
Issue a KVM_REQ_TLB_FLUSH_GUEST request on every MTRR MSR write to match x86 architectural semantics.
--- arch/x86/kvm/mtrr.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/mtrr.c +1 -0
@@ -105,6 +105,7 @@ int kvm_mtrr_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data)return 1 ; *mtrr = data; +kvm_make_request (KVM_REQ_TLB_FLUSH_GUEST , vcpu);return 0 ; }
Signature · 1 line
2.55.0.229.g6434b31f56-goog