KVM: nVMX: Fixes for nested state migration when eVMCS is in use
v3 cover KVM: nVMX: Fixes for nested state migration when eVMCS is in use
Changes since v2:
- 'KVM: nVMX: Use '-1' in 'hv_evmcs_vmptr' to indicate that eVMCS is not in use'/ 'KVM: nVMX: Introduce 'EVMPTR_MAP_PENDING' post-migration state' patches instead of 'KVM: nVMX: Introduce nested_evmcs_is_used()' [Paolo]
- 'KVM: nVMX: Don't set 'dirty_vmcs12' flag on enlightened VMPTRLD' patch added [Max]
- 'KVM: nVMX: Release eVMCS when enlightened VMENTRY was disabled' patch added.
- 'KVM: nVMX: Make copy_vmcs12_to_enlightened()/copy_enlightened_to_vmcs12() return 'void'' patch added [Paolo]
- R-b tags added [Max]
Original description:
Commit f5c7e8425f18 ("KVM: nVMX: Always make an attempt to map eVMCS after migration") fixed the most obvious reason why Hyper-V on KVM (e.g. Win10
+ WSL2) was crashing immediately after migration. It was also reported
that we have more issues to fix as, while the failure rate was lowered signifincatly, it was still possible to observe crashes after several dozens of migration. Turns out, the issue arises when we manage to issue KVM_GET_NESTED_STATE right after L2->L2 VMEXIT but before L1 gets a chance to run. This state is tracked with 'need_vmcs12_to_shadow_sync' flag but the flag itself is not part of saved nested state. A few other less significant issues are fixed along the way.
While there's no proof this series fixes all eVMCS related problems, Win10+WSL2 was able to survive 3333 (thanks, Max!) migrations without crashing in testing.
Patches are based on the current kvm/next tree.
Vitaly Kuznetsov (11):
KVM: nVMX: Use '-1' in 'hv_evmcs_vmptr' to indicate that eVMCS is not
in use
KVM: nVMX: Don't set 'dirty_vmcs12' flag on enlightened VMPTRLD
KVM: nVMX: Release eVMCS when enlightened VMENTRY was disabled
KVM: nVMX: Make
copy_vmcs12_to_enlightened()/copy_enlightened_to_vmcs12() return
'void' KVM: nVMX: Introduce 'EVMPTR_MAP_PENDING' post-migration state
KVM: nVMX: Release enlightened VMCS on VMCLEAR
KVM: nVMX: Ignore 'hv_clean_fields' data when eVMCS data is copied in
vmx_get_nested_state()
KVM: nVMX: Force enlightened VMCS sync from nested_vmx_failValid()
KVM: nVMX: Reset eVMCS clean fields data from prepare_vmcs02()
KVM: nVMX: Request to sync eVMCS from VMCS12 after migration
KVM: selftests: evmcs_test: Test that KVM_STATE_NESTED_EVMCS is never
lost
arch/x86/kvm/vmx/evmcs.c | 3 + arch/x86/kvm/vmx/evmcs.h | 8 + arch/x86/kvm/vmx/nested.c | 144 +++++++++++------- arch/x86/kvm/vmx/nested.h | 11 +- arch/x86/kvm/vmx/vmx.c | 1 + .../testing/selftests/kvm/x86_64/evmcs_test.c | 64 ++++---- 6 files changed, 140 insertions(+), 91 deletions(-)
Signature · 1 line
2.31.1
PATCH v3 1/11 KVM: nVMX: Use '-1' in 'hv_evmcs_vmptr' to indicate that eVMCS is not in use
Instead of checking 'vmx->nested.hv_evmcs' use '-1' in 'vmx->nested.hv_evmcs_vmptr' to indicate 'evmcs is not in use' state. This matches how we check 'vmx->nested.current_vmptr'. Introduce EVMPTR_INVALID and evmptr_is_valid() and use it instead of raw '-1' check as a preparation to adding other 'special' values.
No functional change intended.
--- arch/x86/kvm/vmx/evmcs.c | 3 +++ arch/x86/kvm/vmx/evmcs.h | 7 +++++ arch/x86/kvm/vmx/nested.c | 55 ++++++++++++++++++++------------------- arch/x86/kvm/vmx/nested.h | 2 +- arch/x86/kvm/vmx/vmx.c | 1 + 5 files changed, 40 insertions(+), 28 deletions(-)
arch/x86/kvm/vmx/evmcs.c +3 -0
@@ -319,6 +319,9 @@ bool nested_enlightened_vmentry(struct kvm_vcpu *vcpu, u64 *evmcs_gpa)if (unlikely (!assist_page.enlighten_vmentry))return false; +if (unlikely (!evmptr_is_valid (assist_page.current_nested_vmcs))) +return false; + *evmcs_gpa = assist_page.current_nested_vmcs;return true;
arch/x86/kvm/vmx/evmcs.h +7 -0
@@ -197,6 +197,13 @@ static inline void evmcs_load(u64 phys_addr) {}static inline void evmcs_touch_msr_bitmap (void ) {}#endif /* IS_ENABLED(CONFIG_HYPERV) */ +#define EVMPTR_INVALID (-1ULL) + +static inline bool evmptr_is_valid (u64 evmptr) +{ +return evmptr !=EVMPTR_INVALID ; +} +enum nested_evmptrld_status {EVMPTRLD_DISABLED ,EVMPTRLD_SUCCEEDED ,
arch/x86/kvm/vmx/nested.c +28 -27
159 lines not on this page · raw message
arch/x86/kvm/vmx/nested.h +1 -1
@@ -63,7 +63,7 @@ static inline int vmx_has_valid_vmcs12(struct kvm_vcpu *vcpu)* have vmcs12 if it is true. */ return is_guest_mode (vcpu) || vmx->nested.current_vmptr !=-1ull || - vmx->nested.hv_evmcs; +evmptr_is_valid (vmx->nested.hv_evmcs_vmptr); }static inline u16 nested_get_vpid02 (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.c +1 -0
@@ -6941,6 +6941,7 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) vmx->nested.posted_intr_nv =-1 ; vmx->nested.current_vmptr =-1ull ; + vmx->nested.hv_evmcs_vmptr =EVMPTR_INVALID ; vcpu->arch.microcode_version =0x100000000ULL ; vmx->msr_ia32_feature_control_valid_bits =FEAT_CTL_LOCKED ;
Signature · 1 line
2.31.1
PATCH v3 2/11 KVM: nVMX: Don't set 'dirty_vmcs12' flag on enlightened VMPTRLD
In reply to Vitaly Kuznetsov'dirty_vmcs12' is only checked in prepare_vmcs02_early()/prepare_vmcs02() and both checks look like:
'vmx->nested.dirty_vmcs12 || evmptr_is_valid(vmx->nested.hv_evmcs_vmptr)'
so for eVMCS case the flag changes nothing. Drop the assignment to avoid the confusion.
No functional change intended.
--- arch/x86/kvm/vmx/nested.c | 1 - 1 file changed, 1 deletion(-)
arch/x86/kvm/vmx/nested.c +0 -1
@@ -2022,7 +2022,6 @@ static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld(return EVMPTRLD_VMFAIL ; } - vmx->nested.dirty_vmcs12 = true; vmx->nested.hv_evmcs_vmptr = evmcs_gpa; evmcs_gpa_changed = true;
Signature · 1 line
2.31.1
PATCH v3 3/11 KVM: nVMX: Release eVMCS when enlightened VMENTRY was disabled
In reply to Vitaly KuznetsovIn theory, L1 can try to disable enlightened VMENTRY in VP assist page and try to issue VMLAUNCH/VMRESUME. While nested_vmx_handle_enlightened_vmptrld() properly handles this as 'EVMPTRLD_DISABLED', previously mapped eVMCS remains mapped and thus all evmptr_is_valid() checks will still pass and nested_vmx_run() will proceed when it shouldn't.
Release eVMCS immediately when we detect that enlightened vmentry was disabled by L1.
--- arch/x86/kvm/vmx/nested.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +3 -1
@@ -1980,8 +1980,10 @@ static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld(if (likely (!vmx->nested.enlightened_vmcs_enabled))return EVMPTRLD_DISABLED ; -if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa)) +if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa)) { +nested_release_evmcs (vcpu);return EVMPTRLD_DISABLED ; + }if (unlikely (evmcs_gpa != vmx->nested.hv_evmcs_vmptr)) { vmx->nested.current_vmptr =-1ull ;
Signature · 1 line
2.31.1
PATCH v3 4/11 KVM: nVMX: Make copy_vmcs12_to_enlightened()/copy_enlightened_to_vmcs12() return 'void'
In reply to Vitaly Kuznetsovcopy_vmcs12_to_enlightened()/copy_enlightened_to_vmcs12() don't return any result, make them return 'void'.
No functional change intended.
--- arch/x86/kvm/vmx/nested.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/nested.c +4 -4
@@ -1587,7 +1587,7 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)vmcs_load (vmx->loaded_vmcs->vmcs); } -static int copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx) +static void copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx) {struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs;
@@ -1800,10 +1800,10 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx)* vmcs12->exit_io_instruction_eip = evmcs->exit_io_instruction_eip; */ -return 0 ; +return ; } -static int copy_vmcs12_to_enlightened (struct vcpu_vmx *vmx) +static void copy_vmcs12_to_enlightened (struct vcpu_vmx *vmx) {struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs;
@@ -1963,7 +1963,7 @@ static int copy_vmcs12_to_enlightened(struct vcpu_vmx *vmx) evmcs->guest_bndcfgs = vmcs12->guest_bndcfgs; -return 0 ; +return ; }/*
Signature · 1 line
2.31.1
PATCH v3 5/11 KVM: nVMX: Introduce 'EVMPTR_MAP_PENDING' post-migration state
In reply to Vitaly KuznetsovUnlike regular set_current_vmptr(), nested_vmx_handle_enlightened_vmptrld() can not be called directly from vmx_set_nested_state() as KVM may not have all the information yet (e.g. HV_X64_MSR_VP_ASSIST_PAGE MSR may not be restored yet). Enlightened VMCS is mapped later while getting nested state pages. In the meantime, vmx->nested.hv_evmcs_vmptr remains 'EVMPTR_INVALID' and it's indistinguishable from 'evmcs is not in use' case. This leads to certain issues, in particular, if KVM_GET_NESTED_STATE is called right after KVM_SET_NESTED_STATE, KVM_STATE_NESTED_EVMCS flag in the resulting state will be unset (and such state will later fail to load).
Introduce 'EVMPTR_MAP_PENDING' state to detect not-yet-mapped eVMCS after restore. With this, the 'is_guest_mode(vcpu)' hack in vmx_has_valid_vmcs12() is no longer needed.
--- arch/x86/kvm/vmx/evmcs.h | 3 ++- arch/x86/kvm/vmx/nested.c | 6 ++++-- arch/x86/kvm/vmx/nested.h | 11 +++-------- 3 files changed, 9 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx/evmcs.h +2 -1
@@ -198,10 +198,11 @@ static inline void evmcs_touch_msr_bitmap(void) {}#endif /* IS_ENABLED(CONFIG_HYPERV) */ #define EVMPTR_INVALID (-1ULL) +#define EVMPTR_MAP_PENDING (-2ULL)static inline bool evmptr_is_valid (u64 evmptr) { -return evmptr !=EVMPTR_INVALID ; +return evmptr !=EVMPTR_INVALID && evmptr !=EVMPTR_MAP_PENDING ; }enum nested_evmptrld_status {
arch/x86/kvm/vmx/nested.c +4 -2
@@ -3093,7 +3093,7 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)* properly reflected. */ if (vmx->nested.enlightened_vmcs_enabled && - !evmptr_is_valid (vmx->nested.hv_evmcs_vmptr)) { + vmx->nested.hv_evmcs_vmptr ==EVMPTR_MAP_PENDING ) {enum nested_evmptrld_status evmptrld_status =nested_vmx_handle_enlightened_vmptrld (vcpu, false);
@@ -6058,7 +6058,8 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,if (vmx_has_valid_vmcs12 (vcpu)) { kvm_state.size +=sizeof (user_vmx_nested_state->vmcs12); -if (evmptr_is_valid (vmx->nested.hv_evmcs_vmptr)) +/* 'hv_evmcs_vmptr' can also be EVMPTR_MAP_PENDING here */ +if (vmx->nested.hv_evmcs_vmptr !=EVMPTR_INVALID ) kvm_state.flags |=KVM_STATE_NESTED_EVMCS ; if (is_guest_mode(vcpu) &&
@@ -6257,6 +6258,7 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu,* restored yet. EVMCS will be mapped from * nested_get_vmcs12_pages(). */ + vmx->nested.hv_evmcs_vmptr =EVMPTR_MAP_PENDING ;kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); }else {return -EINVAL ;
arch/x86/kvm/vmx/nested.h +3 -8
@@ -56,14 +56,9 @@ static inline int vmx_has_valid_vmcs12(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -/* -* In case we do two consecutive get/set_nested_state()s while L2 was -* running hv_evmcs may end up not being mapped (we map it from -* nested_vmx_run()/vmx_vcpu_run()). Check is_guest_mode() as we always -* have vmcs12 if it is true. -*/ -return is_guest_mode (vcpu) || vmx->nested.current_vmptr !=-1ull || -evmptr_is_valid (vmx->nested.hv_evmcs_vmptr); +/* 'hv_evmcs_vmptr' can also be EVMPTR_MAP_PENDING here */ +return vmx->nested.current_vmptr !=-1ull || + vmx->nested.hv_evmcs_vmptr !=EVMPTR_INVALID ; }static inline u16 nested_get_vpid02 (struct kvm_vcpu *vcpu)
Signature · 1 line
2.31.1
PATCH v3 6/11 KVM: nVMX: Release enlightened VMCS on VMCLEAR
In reply to Vitaly KuznetsovUnlike VMREAD/VMWRITE/VMPTRLD, VMCLEAR is a valid instruction when enlightened VMCS is in use. TLFS has the following brief description:
"The L1 hypervisor can execute a VMCLEAR instruction to transition an enlightened VMCS from the active to the non-active state". Normally, this change can be ignored as unmapping active eVMCS can be postponed until the next VMLAUNCH instruction but in case nested state is migrated with KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE, keeping eVMCS mapped may result in its synchronization with VMCS12 and this is incorrect:
L1 hypervisor is free to reuse inactive eVMCS memory for something else.
Inactive eVMCS after VMCLEAR can just be unmapped.
--- arch/x86/kvm/vmx/nested.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/kvm/vmx/nested.c +2 -0
@@ -4989,6 +4989,8 @@ static int handle_vmclear(struct kvm_vcpu *vcpu) vmptr + offsetof(struct vmcs12 , launch_state), &zero, sizeof(zero)); + }else if (vmx->nested.hv_evmcs && vmptr == vmx->nested.hv_evmcs_vmptr) { +nested_release_evmcs (vcpu); }return nested_vmx_succeed (vcpu);
Signature · 1 line
2.31.1
PATCH v3 7/11 KVM: nVMX: Ignore 'hv_clean_fields' data when eVMCS data is copied in vmx_get_nested_state()
In reply to Vitaly Kuznetsov'Clean fields' data from enlightened VMCS is only valid upon vmentry: L1 hypervisor is not obliged to keep it up-to-date while it is mangling L2's state, KVM_GET_NESTED_STATE request may come at a wrong moment when actual eVMCS changes are unsynchronized with 'hv_clean_fields'. As upon migration VMCS12 is used as a source of ultimate truth, we must make sure we pick all the changes to eVMCS and thus 'clean fields' data must be ignored.
--- arch/x86/kvm/vmx/nested.c | 43 +++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 18 deletions(-)
arch/x86/kvm/vmx/nested.c +25 -18
@@ -1587,7 +1587,7 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)vmcs_load (vmx->loaded_vmcs->vmcs); } -static void copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx) +static void copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx,u32 hv_clean_fields) {struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs;
@@ -1596,7 +1596,7 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->tpr_threshold = evmcs->tpr_threshold; vmcs12->guest_rip = evmcs->guest_rip; -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC ))) { vmcs12->guest_rsp = evmcs->guest_rsp; vmcs12->guest_rflags = evmcs->guest_rflags;
@@ -1604,23 +1604,23 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) evmcs->guest_interruptibility_info; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_PROC ))) { vmcs12->cpu_based_vm_exec_control = evmcs->cpu_based_vm_exec_control; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EXCPN ))) { vmcs12->exception_bitmap = evmcs->exception_bitmap; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_ENTRY ))) { vmcs12->vm_entry_controls = evmcs->vm_entry_controls; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EVENT ))) { vmcs12->vm_entry_intr_info_field = evmcs->vm_entry_intr_info_field;
@@ -1630,7 +1630,7 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) evmcs->vm_entry_instruction_len; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_GRP1 ))) { vmcs12->host_ia32_pat = evmcs->host_ia32_pat; vmcs12->host_ia32_efer = evmcs->host_ia32_efer;
@@ -1650,7 +1650,7 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->host_tr_selector = evmcs->host_tr_selector; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP1 ))) { vmcs12->pin_based_vm_exec_control = evmcs->pin_based_vm_exec_control;
@@ -1659,18 +1659,18 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) evmcs->secondary_vm_exec_control; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP ))) { vmcs12->io_bitmap_a = evmcs->io_bitmap_a; vmcs12->io_bitmap_b = evmcs->io_bitmap_b; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP ))) { vmcs12->msr_bitmap = evmcs->msr_bitmap; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP2 ))) { vmcs12->guest_es_base = evmcs->guest_es_base; vmcs12->guest_cs_base = evmcs->guest_cs_base;
@@ -1710,14 +1710,14 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->guest_tr_selector = evmcs->guest_tr_selector; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP2 ))) { vmcs12->tsc_offset = evmcs->tsc_offset; vmcs12->virtual_apic_page_addr = evmcs->virtual_apic_page_addr; vmcs12->xss_exit_bitmap = evmcs->xss_exit_bitmap; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CRDR ))) { vmcs12->cr0_guest_host_mask = evmcs->cr0_guest_host_mask; vmcs12->cr4_guest_host_mask = evmcs->cr4_guest_host_mask;
@@ -1729,7 +1729,7 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->guest_dr7 = evmcs->guest_dr7; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_POINTER ))) { vmcs12->host_fs_base = evmcs->host_fs_base; vmcs12->host_gs_base = evmcs->host_gs_base;
@@ -1739,13 +1739,13 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->host_rsp = evmcs->host_rsp; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_XLAT ))) { vmcs12->ept_pointer = evmcs->ept_pointer; vmcs12->virtual_processor_id = evmcs->virtual_processor_id; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP1 ))) { vmcs12->vmcs_link_pointer = evmcs->vmcs_link_pointer; vmcs12->guest_ia32_debugctl = evmcs->guest_ia32_debugctl;
@@ -3483,7 +3483,7 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch)return nested_vmx_failInvalid (vcpu);if (evmptr_is_valid (vmx->nested.hv_evmcs_vmptr)) { -copy_enlightened_to_vmcs12 (vmx); +copy_enlightened_to_vmcs12 (vmx, vmx->nested.hv_evmcs->hv_clean_fields);/* Enlightened VMCS doesn't have launch state */ vmcs12->launch_state = !launch; }else if (enable_shadow_vmcs) {
@@ -6118,7 +6118,14 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,copy_vmcs02_to_vmcs12_rare (vcpu,get_vmcs12 (vcpu));if (!vmx->nested.need_vmcs12_to_shadow_sync) {if (evmptr_is_valid (vmx->nested.hv_evmcs_vmptr)) -copy_enlightened_to_vmcs12 (vmx); +/* +* L1 hypervisor is not obliged to keep eVMCS +* clean fields data always up-to-date while +* not in guest mode, 'hv_clean_fields' is only +* supposed to be actual upon vmentry so we need +* to ignore it here and do full copy. +*/ +copy_enlightened_to_vmcs12 (vmx,0 );else if (enable_shadow_vmcs)copy_shadow_to_vmcs12 (vmx); }
Signature · 1 line
2.31.1
PATCH v3 8/11 KVM: nVMX: Force enlightened VMCS sync from nested_vmx_failValid()
In reply to Vitaly Kuznetsov'need_vmcs12_to_shadow_sync' is used for both shadow and enlightened VMCS sync when we exit to L1. The comment in nested_vmx_failValid() validly states why shadow vmcs sync can be omitted but this doesn't apply to enlightened VMCS as it 'shadows' all VMCS12 fields.
--- arch/x86/kvm/vmx/nested.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/nested.c +6 -2
@@ -173,9 +173,13 @@ static int nested_vmx_failValid(struct kvm_vcpu *vcpu, | X86_EFLAGS_ZF);get_vmcs12 (vcpu)->vm_instruction_error = vm_instruction_error;/* -* We don't need to force a shadow sync because -* VM_INSTRUCTION_ERROR is not shadowed +* We don't need to force sync to shadow VMCS because +* VM_INSTRUCTION_ERROR is not shadowed. Enlightened VMCS 'shadows' all +* fields and thus must be synced. */ +if (to_vmx (vcpu)->nested.hv_evmcs_vmptr !=EVMPTR_INVALID ) +to_vmx (vcpu)->nested.need_vmcs12_to_shadow_sync = true; +return kvm_skip_emulated_instruction (vcpu); }
Signature · 1 line
2.31.1
PATCH v3 9/11 KVM: nVMX: Reset eVMCS clean fields data from prepare_vmcs02()
In reply to Vitaly KuznetsovWhen nested state migration happens during L1's execution, it is incorrect to modify eVMCS as it is L1 who 'owns' it at the moment. At least genuine Hyper-V seems to not be very happy when 'clean fields' data changes underneath it.
'Clean fields' data is used in KVM twice: by copy_enlightened_to_vmcs12() and prepare_vmcs02_rare() so we can reset it from prepare_vmcs02() instead.
While at it, update a comment stating why exactly we need to reset 'hv_clean_fields' data from L0.
--- arch/x86/kvm/vmx/nested.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-)
arch/x86/kvm/vmx/nested.c +13 -6
@@ -2060,14 +2060,10 @@ void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -if (evmptr_is_valid (vmx->nested.hv_evmcs_vmptr)) { +if (evmptr_is_valid (vmx->nested.hv_evmcs_vmptr))copy_vmcs12_to_enlightened (vmx); -/* All fields are clean */ - vmx->nested.hv_evmcs->hv_clean_fields |= -HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; - }else { +else copy_vmcs12_to_shadow (vmx); - } vmx->nested.need_vmcs12_to_shadow_sync = false; }
@@ -2607,6 +2603,17 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,kvm_rsp_write (vcpu, vmcs12->guest_rsp);kvm_rip_write (vcpu, vmcs12->guest_rip); + +/* +* It was observed that genuine Hyper-V running in L1 doesn't reset +* 'hv_clean_fields' by itself, it only sets the corresponding dirty +* bits when it changes a field in eVMCS. Mark all fields as clean +* here. +*/ +if (evmptr_is_valid (vmx->nested.hv_evmcs_vmptr)) + vmx->nested.hv_evmcs->hv_clean_fields |= +HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; +return 0 ; }
Signature · 1 line
2.31.1
PATCH v3 10/11 KVM: nVMX: Request to sync eVMCS from VMCS12 after migration
In reply to Vitaly KuznetsovVMCS12 is used to keep the authoritative state during nested state migration. In case 'need_vmcs12_to_shadow_sync' flag is set, we're in between L2->L1 vmexit and L1 guest run when actual sync to enlightened (or shadow) VMCS happens. Nested state, however, has no flag for 'need_vmcs12_to_shadow_sync' so vmx_set_nested_state()-> set_current_vmptr() always sets it. Enlightened vmptrld path, however, doesn't have the quirk so some VMCS12 changes may not get properly reflected to eVMCS and L1 will see an incorrect state.
Note, during L2 execution or when need_vmcs12_to_shadow_sync is not set the change is effectively a nop: in the former case all changes will get reflected during the first L2->L1 vmexit and in the later case VMCS12 and eVMCS are already in sync (thanks to copy_enlightened_to_vmcs12() in vmx_get_nested_state()).
--- arch/x86/kvm/vmx/nested.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/x86/kvm/vmx/nested.c +6 -0
@@ -3111,6 +3111,12 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)if (evmptrld_status ==EVMPTRLD_VMFAIL || evmptrld_status ==EVMPTRLD_ERROR )return false; + +/* +* Post migration VMCS12 always provides the most actual +* information, copy it to eVMCS upon entry. +*/ + vmx->nested.need_vmcs12_to_shadow_sync = true; }return true;
Signature · 1 line
2.31.1
PATCH v3 11/11 KVM: selftests: evmcs_test: Test that KVM_STATE_NESTED_EVMCS is never lost
In reply to Vitaly KuznetsovDo KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE for a freshly restored VM (before the first KVM_RUN) to check that KVM_STATE_NESTED_EVMCS is not lost.
--- .../testing/selftests/kvm/x86_64/evmcs_test.c | 64 +++++++++++-------- 1 file changed, 38 insertions(+), 26 deletions(-)
tools/testing/selftests/kvm/x86_64/evmcs_test.c +38 -26
@@ -121,14 +121,38 @@ void inject_nmi(struct kvm_vm *vm)vcpu_events_set (vm,VCPU_ID , &events); } +static void save_restore_vm (struct kvm_vm *vm) +{ +struct kvm_regs regs1, regs2; +struct kvm_x86_state *state; + + state =vcpu_save_state (vm,VCPU_ID ); +memset (®s1,0 ,sizeof (regs1)); +vcpu_regs_get (vm,VCPU_ID , ®s1); + +kvm_vm_release (vm); + +/* Restore state in a new VM. */ +kvm_vm_restart (vm,O_RDWR ); +vm_vcpu_add (vm,VCPU_ID ); +vcpu_set_hv_cpuid (vm,VCPU_ID ); +vcpu_enable_evmcs (vm,VCPU_ID ); +vcpu_load_state (vm,VCPU_ID , state); +free (state); + +memset (®s2,0 ,sizeof (regs2)); +vcpu_regs_get (vm,VCPU_ID , ®s2); +TEST_ASSERT (!memcmp (®s1, ®s2,sizeof (regs2)), +"Unexpected register values after vcpu_load_state; rdi: %lx rsi: %lx" , + (ulong ) regs2.rdi, (ulong ) regs2.rsi); +} +int main (int argc,char *argv[]) {vm_vaddr_t vmx_pages_gva =0 ; -struct kvm_regs regs1, regs2;struct kvm_vm *vm;struct kvm_run *run; -struct kvm_x86_state *state;struct ucall uc;int stage;
@@ -145,10 +169,6 @@ int main(int argc, char *argv[])vcpu_set_hv_cpuid (vm,VCPU_ID );vcpu_enable_evmcs (vm,VCPU_ID ); - run =vcpu_state (vm,VCPU_ID ); - -vcpu_regs_get (vm,VCPU_ID , ®s1); -vcpu_alloc_vmx (vm, &vmx_pages_gva);vcpu_args_set (vm,VCPU_ID ,1 , vmx_pages_gva);
@@ -160,6 +180,7 @@ int main(int argc, char *argv[])pr_info ("Running L1 which uses EVMCS to run L2\n" );for (stage =1 ;; stage++) { + run =vcpu_state (vm,VCPU_ID );_vcpu_run (vm,VCPU_ID ); TEST_ASSERT(run->exit_reason == KVM_EXIT_IO, "Stage %d: unexpected exit reason: %u (%s),\n",
@@ -184,32 +205,23 @@ int main(int argc, char *argv[]) uc.args[1 ] == stage,"Stage %d: Unexpected register values vmexit, got %lx" , stage, (ulong)uc.args[1]); - state =vcpu_save_state (vm,VCPU_ID ); -memset (®s1,0 ,sizeof (regs1)); -vcpu_regs_get (vm,VCPU_ID , ®s1); - -kvm_vm_release (vm); - -/* Restore state in a new VM. */ -kvm_vm_restart (vm,O_RDWR ); -vm_vcpu_add (vm,VCPU_ID ); -vcpu_set_hv_cpuid (vm,VCPU_ID ); -vcpu_enable_evmcs (vm,VCPU_ID ); -vcpu_load_state (vm,VCPU_ID , state); - run =vcpu_state (vm,VCPU_ID ); -free (state); - -memset (®s2,0 ,sizeof (regs2)); -vcpu_regs_get (vm,VCPU_ID , ®s2); -TEST_ASSERT (!memcmp (®s1, ®s2,sizeof (regs2)), -"Unexpected register values after vcpu_load_state; rdi: %lx rsi: %lx" , - (ulong ) regs2.rdi, (ulong ) regs2.rsi); +save_restore_vm (vm);/* Force immediate L2->L1 exit before resuming */ if (stage ==8 ) {pr_info ("Injecting NMI into L1 before L2 had a chance to run after restore\n" );inject_nmi (vm); } + +/* +* Do KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE for a freshly +* restored VM (before the first KVM_RUN) to check that +* KVM_STATE_NESTED_EVMCS is not lost. +*/ +if (stage ==9 ) { +pr_info ("Trying extra KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE cycle\n" ); +save_restore_vm (vm); + } } done:
Signature · 1 line
2.31.1
Quoted 25 lines · Vitaly Kuznetsov
Changes since v2:
- 'KVM: nVMX: Use '-1' in 'hv_evmcs_vmptr' to indicate that eVMCS is not in use'/ 'KVM: nVMX: Introduce 'EVMPTR_MAP_PENDING' post-migration state' patches instead of 'KVM: nVMX: Introducenested_evmcs_is_used()' [Paolo]
- 'KVM: nVMX: Don't set 'dirty_vmcs12' flag on enlightened VMPTRLD' patch added [Max]
- 'KVM: nVMX: Release eVMCS when enlightened VMENTRY was disabled' patch added.
- 'KVM: nVMX: Make copy_vmcs12_to_enlightened()/copy_enlightened_to_vmcs12() return 'void'' patch added [Paolo]
- R-b tags added [Max]Original description:
Commit
f5c7e8425f18("KVM: nVMX: Always make an attempt to map eVMCS after migration") fixed the most obvious reason why Hyper-V on KVM (e.g. Win10
+ WSL2) was crashing immediately after migration. It was also reported
that we have more issues to fix as, while the failure rate was lowered signifincatly, it was still possible to observe crashes after several dozens of migration. Turns out, the issue arises when we manage to issueKVM_GET_NESTED_STATEright afterL2->L2VMEXIT but before L1 gets a chance to run. This state is tracked with 'need_vmcs12_to_shadow_sync' flag but the flag itself is not part of saved nested state. A few other less significant issues are fixed along the way.
Quoted 5 lines
While there's no proof this series fixes all eVMCS related problems, Win10+WSL2 was able to survive 3333 (thanks, Max!) migrations without crashing in testing.
Patches are based on the current kvm/next tree.
Paolo, Max,
Just to double-check: are we good here? I know there are more improvements/ideas to explore but I'd like to treat this patchset as a set of fixes, it would be unfortunate if we miss 5.14.
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerQuoted 31 lines · Vitaly Kuznetsov
Quoted 29 lines · Vitaly Kuznetsov
Changes since v2:
- 'KVM: nVMX: Use '-1' in 'hv_evmcs_vmptr' to indicate that eVMCS is not in use'/ 'KVM: nVMX: Introduce 'EVMPTR_MAP_PENDING' post-migration state' patches instead of 'KVM: nVMX: Introducenested_evmcs_is_used()' [Paolo]
- 'KVM: nVMX: Don't set 'dirty_vmcs12' flag on enlightened VMPTRLD' patch added [Max]
- 'KVM: nVMX: Release eVMCS when enlightened VMENTRY was disabled' patch added.
- 'KVM: nVMX: Make copy_vmcs12_to_enlightened()/copy_enlightened_to_vmcs12() return 'void'' patch added [Paolo]
- R-b tags added [Max]Original description:
Commit
f5c7e8425f18("KVM: nVMX: Always make an attempt to map eVMCS after migration") fixed the most obvious reason why Hyper-V on KVM (e.g. Win10
+ WSL2) was crashing immediately after migration. It was also reported
that we have more issues to fix as, while the failure rate was lowered signifincatly, it was still possible to observe crashes after several dozens of migration. Turns out, the issue arises when we manage to issueKVM_GET_NESTED_STATEright afterL2->L2VMEXIT but before L1 gets a chance to run. This state is tracked with 'need_vmcs12_to_shadow_sync' flag but the flag itself is not part of saved nested state. A few other less significant issues are fixed along the way.While there's no proof this series fixes all eVMCS related problems, Win10+WSL2 was able to survive 3333 (thanks, Max!) migrations without crashing in testing.
Quoted 8 lines
Quoted 1 line
Patches are based on the current kvm/next tree.
Paolo, Max,
Just to double-check: are we good here? I know there are more improvements/ideas to explore but I'd like to treat this patchset as a set of fixes, it would be unfortunate if we miss 5.14.
Yes, I was busy the last couple of weeks but I am back now.
Paolo