KVM: nVMX: Support loading MSRs before nested state
Sean Christopherson
maintainercover KVM: nVMX: Support loading MSRs before nested state
Fix two edge cases in nested VM-Enter where KVM loads garbage into BNDCFGS and/or DEBUGCTL when migrating a vCPU with L2 active. If userspace restores MSRs before nested state, userspace expects KVM to propagate L2's values to vmcs02. KVM already mostly handles this scenario, so even though I personally think it's rather ridiculous, it's easy for KVM to support, and given that our VMM does KVM_SET_MSRS before KVM_SET_NESTED_STATE...
Patches 4 and 5 are minor optimizations to handle BNDCFGS more like DEBUGCTL and/or the PKS MSR.
Sean Christopherson (5):
KVM: nVMX: Snapshot pre-VM-Enter BNDCFGS for !nested_run_pending case
KVM: nVMX: Snapshot pre-VM-Enter DEBUGCTL for !nested_run_pending case
KVM: nVMX: Rename nested.vmcs01_* fields to nested.pre_vmenter_*
KVM: nVMX: Save BNDCFGS to vmcs12 iff relevant controls are exposed to
L1
KVM: nVMX: Update vmcs12 on BNDCFGS write, not at vmcs02=>vmcs12 sync
arch/x86/kvm/vmx/nested.c | 16 ++++++++-------- arch/x86/kvm/vmx/vmx.c | 13 +++++++++++++ arch/x86/kvm/vmx/vmx.h | 15 ++++++++++++--- 3 files changed, 33 insertions(+), 11 deletions(-)
base-commit: 8baacf67c76c560fed954ac972b63e6e59a6fba0
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH 1/5 KVM: nVMX: Snapshot pre-VM-Enter BNDCFGS for !nested_run_pending case
If a nested run isn't pending, snapshot vmcs01.GUEST_BNDCFGS irrespective of whether or not VM_ENTRY_LOAD_BNDCFGS is set in vmcs12. When restoring nested state, e.g. after migration, without a nested run pending, prepare_vmcs02() will propagate nested.vmcs01_guest_bndcfgs to vmcs02, i.e. will load garbage/zeros into vmcs02.GUEST_BNDCFGS.
If userspace restores nested state before MSRs, then loading garbage is a non-issue as loading BNDCFGS will also update vmcs02. But if usersepace restores MSRs first, then KVM is responsible for propagating L2's value, which is actually thrown into vmcs01, into vmcs02.
Restoring L2 MSRs into vmcs01, i.e. loading all MSRs before nested state is all kinds of bizarre and ideally would not be supported. Sadly, some VMMs do exactly that and rely on KVM to make things work.
Note, there's still a lurking SMM bug, as propagating vmcs01.GUEST_BNDFGS to vmcs02 across RSM may corrupt L2's BNDCFGS. But KVM's entire VMX+SMM emulation is flawed as SMI+RSM should not toouch _any_ VMCS when use the "default treatment of SMIs", i.e. when not using an SMI Transfer Monitor.
--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +2 -1
@@ -3382,7 +3382,8 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,if (!(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_DEBUG_CONTROLS )) vmx->nested.vmcs01_debugctl =vmcs_read64 (GUEST_IA32_DEBUGCTL );if (kvm_mpx_supported () && - !(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_BNDCFGS )) + (!vmx->nested.nested_run_pending || + !(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_BNDCFGS ))) vmx->nested.vmcs01_guest_bndcfgs =vmcs_read64 (GUEST_BNDCFGS );/*
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH 2/5 KVM: nVMX: Snapshot pre-VM-Enter DEBUGCTL for !nested_run_pending case
In reply to Sean ChristophersonIf a nested run isn't pending, snapshot vmcs01.GUEST_IA32_DEBUGCTL irrespective of whether or not VM_ENTRY_LOAD_DEBUG_CONTROLS is set in vmcs12. When restoring nested state, e.g. after migration, without a nested run pending, prepare_vmcs02() will propagate nested.vmcs01_debugctl to vmcs02, i.e. will load garbage/zeros into vmcs02.GUEST_IA32_DEBUGCTL.
If userspace restores nested state before MSRs, then loading garbage is a non-issue as loading DEBUGCTL will also update vmcs02. But if usersepace restores MSRs first, then KVM is responsible for propagating L2's value, which is actually thrown into vmcs01, into vmcs02.
Restoring L2 MSRs into vmcs01, i.e. loading all MSRs before nested state is all kinds of bizarre and ideally would not be supported. Sadly, some VMMs do exactly that and rely on KVM to make things work.
Note, there's still a lurking SMM bug, as propagating vmcs01's DEBUGCTL to vmcs02 across RSM may corrupt L2's DEBUGCTL. But KVM's entire VMX+SMM emulation is flawed as SMI+RSM should not toouch _any_ VMCS when use the "default treatment of SMIs", i.e. when not using an SMI Transfer Monitor.
--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +2 -1
@@ -3379,7 +3379,8 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,if (likely (!evaluate_pending_interrupts) &&kvm_vcpu_apicv_active (vcpu)) evaluate_pending_interrupts |=vmx_has_apicv_interrupt (vcpu); -if (!(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_DEBUG_CONTROLS )) +if (!vmx->nested.nested_run_pending || + !(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_DEBUG_CONTROLS )) vmx->nested.vmcs01_debugctl =vmcs_read64 (GUEST_IA32_DEBUGCTL ); if (kvm_mpx_supported() && (!vmx->nested.nested_run_pending ||
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH 3/5 KVM: nVMX: Rename nested.vmcs01_* fields to nested.pre_vmenter_*
In reply to Sean ChristophersonRename the fields in struct nested_vmx used to snapshot pre-VM-Enter values to reflect that they can hold L2's values when restoring nested state, e.g. if userspace restores MSRs before nested state. As crazy as it seems, restoring MSRs before nested state actually works (because KVM goes out if it's way to make it work), even though the initial MSR writes will hit vmcs01 despite holding L2 values.
Add a related comment to vmx_enter_smm() to call out that using the common VM-Exit and VM-Enter helpers to emulate SMI and RSM is wrong and broken. The few MSRs that have snapshots _could_ be fixed by taking a snapshot prior to the forced VM-Exit instead of at forced VM-Enter, but that's just the tip of the iceberg as the rather long list of MSRs that aren't snapshotted (hello, VM-Exit MSR load list) can't be handled this way.
--- arch/x86/kvm/vmx/nested.c | 8 ++++---- arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/vmx/vmx.h | 15 ++++++++++++--- 3 files changed, 23 insertions(+), 7 deletions(-)
arch/x86/kvm/vmx/nested.c +4 -4
@@ -2520,11 +2520,11 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,vmcs_write64 (GUEST_IA32_DEBUGCTL , vmcs12->guest_ia32_debugctl); }else {kvm_set_dr (vcpu,7 , vcpu->arch.dr7); -vmcs_write64 (GUEST_IA32_DEBUGCTL , vmx->nested.vmcs01_debugctl); +vmcs_write64 (GUEST_IA32_DEBUGCTL , vmx->nested.pre_vmenter_debugctl); }if (kvm_mpx_supported () && (!vmx->nested.nested_run_pending || !(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_BNDCFGS ))) -vmcs_write64 (GUEST_BNDCFGS , vmx->nested.vmcs01_guest_bndcfgs); +vmcs_write64 (GUEST_BNDCFGS , vmx->nested.pre_vmenter_bndcfgs);vmx_set_rflags (vcpu, vmcs12->guest_rflags);/* EXCEPTION_BITMAP and CR0_GUEST_HOST_MASK should basically be the
@@ -3381,11 +3381,11 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,if (!vmx->nested.nested_run_pending || !(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_DEBUG_CONTROLS )) - vmx->nested.vmcs01_debugctl =vmcs_read64 (GUEST_IA32_DEBUGCTL ); + vmx->nested.pre_vmenter_debugctl =vmcs_read64 (GUEST_IA32_DEBUGCTL );if (kvm_mpx_supported () && (!vmx->nested.nested_run_pending || !(vmcs12->vm_entry_controls &VM_ENTRY_LOAD_BNDCFGS ))) - vmx->nested.vmcs01_guest_bndcfgs =vmcs_read64 (GUEST_BNDCFGS ); + vmx->nested.pre_vmenter_bndcfgs =vmcs_read64 (GUEST_BNDCFGS );/* * Overwrite vmcs01.GUEST_CR3 with L1's CR3 if EPT is disabled *and*
arch/x86/kvm/vmx/vmx.c +7 -0
@@ -7843,6 +7843,13 @@ static int vmx_enter_smm(struct kvm_vcpu *vcpu, char *smstate) {struct vcpu_vmx *vmx =to_vmx (vcpu); +/* +* TODO: Implement custom flows for forcing the vCPU out/in of L2 on +* SMI and RSM. Using the common VM-Exit + VM-Enter routines is wrong +* SMI and RSM only modify state that is saved and restored via SMRAM. +* E.g. most MSRs are left untouched, but many are modified by VM-Exit +* and VM-Enter, and thus L2's values may be corrupted on SMI+RSM. +*/ vmx->nested.smm.guest_mode =is_guest_mode (vcpu);if (vmx->nested.smm.guest_mode)nested_vmx_vmexit (vcpu,-1 ,0 ,0 );
arch/x86/kvm/vmx/vmx.h +12 -3
@@ -219,9 +219,18 @@ struct nested_vmx {bool has_preemption_timer_deadline;bool preemption_timer_expired; -/* to migrate it to L2 if VM_ENTRY_LOAD_DEBUG_CONTROLS is off */ -u64 vmcs01_debugctl; -u64 vmcs01_guest_bndcfgs; +/* +* Used to snapshot MSRs that are conditionally loaded on VM-Enter in +* order to propagate the guest's pre-VM-Enter value into vmcs02. For +* emulation of VMLAUNCH/VMRESUME, the snapshot will be of L1's value. +* For KVM_SET_NESTED_STATE, the snapshot is of L2's value, _if_ +* userspace restores MSRs before nested state. If userspace restores +* MSRs after nested state, the snapshot holds garbage, but KVM can't +* detect that, and the garbage value in vmcs02 will be overwritten by +* MSR restoration in any case. +*/ +u64 pre_vmenter_debugctl; +u64 pre_vmenter_bndcfgs;/* to migrate it to L1 if L2 writes to L1's CR8 directly */ int l1_tpr_threshold;
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH 4/5 KVM: nVMX: Save BNDCFGS to vmcs12 iff relevant controls are exposed to L1
In reply to Sean ChristophersonSave BNDCFGS to vmcs12 (from vmcs02) if and only if at least of one of the load-on-entry or clear-on-exit fields for BNDCFGS is enumerated as an allowed-1 bit in vmcs12. Skipping the field avoids an unnecessary VMREAD when MPX is supported but not exposed to L1.
Per Intel's SDM:
If the processor supports either the 1-setting of the "load IA32_BNDCFGS" VM-entry control or that of the "clear IA32_BNDCFGS" VM-exit control, the contents of the IA32_BNDCFGS MSR are saved into the corresponding field.
--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +2 -1
@@ -4104,7 +4104,8 @@ static void sync_vmcs02_to_vmcs12_rare(struct kvm_vcpu *vcpu, vmcs12->guest_idtr_base =vmcs_readl (GUEST_IDTR_BASE ); vmcs12->guest_pending_dbg_exceptions =vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ); -if (kvm_mpx_supported ()) +if ((vmx->nested.msrs.entry_ctls_high &VM_ENTRY_LOAD_BNDCFGS ) || + (vmx->nested.msrs.exit_ctls_high &VM_EXIT_CLEAR_BNDCFGS )) vmcs12->guest_bndcfgs =vmcs_read64 (GUEST_BNDCFGS ); vmx->nested.need_sync_vmcs02_to_vmcs12_rare = false;
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH 5/5 KVM: nVMX: Update vmcs12 on BNDCFGS write, not at vmcs02=>vmcs12 sync
In reply to Sean ChristophersonUpdate vmcs12->guest_bndcfgs on intercepted writes to BNDCFGS from L2 instead of waiting until vmcs02 is synchronized to vmcs12. KVM always intercepts BNDCFGS accesses, so the only way the value in vmcs02 can change is via KVM's explicit VMWRITE during emulation.
--- arch/x86/kvm/vmx/nested.c | 3 --- arch/x86/kvm/vmx/vmx.c | 6 ++++++ 2 files changed, 6 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -3
@@ -4104,9 +4104,6 @@ static void sync_vmcs02_to_vmcs12_rare(struct kvm_vcpu *vcpu, vmcs12->guest_idtr_base =vmcs_readl (GUEST_IDTR_BASE ); vmcs12->guest_pending_dbg_exceptions =vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ); -if ((vmx->nested.msrs.entry_ctls_high &VM_ENTRY_LOAD_BNDCFGS ) || - (vmx->nested.msrs.exit_ctls_high &VM_EXIT_CLEAR_BNDCFGS )) - vmcs12->guest_bndcfgs =vmcs_read64 (GUEST_BNDCFGS ); vmx->nested.need_sync_vmcs02_to_vmcs12_rare = false; }
arch/x86/kvm/vmx/vmx.c +6 -0
@@ -2044,6 +2044,12 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)if (is_noncanonical_address (data &PAGE_MASK , vcpu) || (data &MSR_IA32_BNDCFGS_RSVD ))return 1 ; + +if (is_guest_mode (vcpu) && + ((vmx->nested.msrs.entry_ctls_high &VM_ENTRY_LOAD_BNDCFGS ) || + (vmx->nested.msrs.exit_ctls_high &VM_EXIT_CLEAR_BNDCFGS ))) +get_vmcs12 (vcpu)->guest_bndcfgs = data; +vmcs_write64 (GUEST_BNDCFGS , data);break ;case MSR_IA32_UMWAIT_CONTROL :
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Queued, thanks.
Paolo