KVM: nVMX: Fixes for nested state migration when eVMCS is in use
cover KVM: nVMX: Fixes for nested state migration when eVMCS is in use
Commit f5c7e8425f18 ("KVM: nVMX: Always make an attempt to map eVMCS after migration") fixed the most obvious reason why Hyper-V on KVM (e.g. Win10
+ WSL2) was crashing immediately after migration. It was also reported
that we have more issues to fix as, while the failure rate was lowered signifincatly, it was still possible to observe crashes after several dozens of migration. Turns out, the issue arises when we manage to issue KVM_GET_NESTED_STATE right after L2->L2 VMEXIT but before L1 gets a chance to run. This state is tracked with 'need_vmcs12_to_shadow_sync' flag but the flag itself is not part of saved nested state. A few other less significant issues are fixed along the way.
While there's no proof this series fixes all eVMCS related problems, Win10+WSL2 was able to survive 500 migrations without crashing in my testing.
Patches are based on the current kvm/next tree.
Vitaly Kuznetsov (7):
KVM: nVMX: Introduce nested_evmcs_is_used()
KVM: nVMX: Release enlightened VMCS on VMCLEAR
KVM: nVMX: Ignore 'hv_clean_fields' data when eVMCS data is copied in
vmx_get_nested_state()
KVM: nVMX: Force enlightened VMCS sync from nested_vmx_failValid()
KVM: nVMX: Reset eVMCS clean fields data from prepare_vmcs02()
KVM: nVMX: Request to sync eVMCS from VMCS12 after migration
KVM: selftests: evmcs_test: Test that KVM_STATE_NESTED_EVMCS is never
lost
arch/x86/kvm/vmx/nested.c | 99 +++++++++++++------ .../testing/selftests/kvm/x86_64/evmcs_test.c | 64 +++++++----- 2 files changed, 109 insertions(+), 54 deletions(-)
Signature · 1 line
2.30.2
PATCH 1/7 KVM: nVMX: Introduce nested_evmcs_is_used()
Unlike regular set_current_vmptr(), nested_vmx_handle_enlightened_vmptrld() can not be called directly from vmx_set_nested_state() as KVM may not have all the information yet (e.g. HV_X64_MSR_VP_ASSIST_PAGE MSR may not be restored yet). Enlightened VMCS is mapped later while getting nested state pages. In the meantime, vmx->nested.hv_evmcs remains NULL and using it for various checks is incorrect. In particular, if KVM_GET_NESTED_STATE is called right after KVM_SET_NESTED_STATE, KVM_STATE_NESTED_EVMCS flag in the resulting state will be unset (and such state will later fail to load).
Introduce nested_evmcs_is_used() and use 'is_guest_mode(vcpu) && vmx->nested.current_vmptr == -1ull' check to detect not-yet-mapped eVMCS after restore.
--- arch/x86/kvm/vmx/nested.c | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-)
arch/x86/kvm/vmx/nested.c +26 -5
@@ -141,6 +141,27 @@ static void init_vmcs_shadow_fields(void) max_shadow_read_write_fields = j; } +static inline bool nested_evmcs_is_used (struct vcpu_vmx *vmx) +{ +struct kvm_vcpu *vcpu = &vmx->vcpu; + +if (vmx->nested.hv_evmcs) +return true; + +/* +* After KVM_SET_NESTED_STATE, enlightened VMCS is mapped during +* KVM_REQ_GET_NESTED_STATE_PAGES handling and until the request is +* processed vmx->nested.hv_evmcs is NULL. It is, however, possible to +* detect such state by checking 'nested.current_vmptr == -1ull' when +* vCPU is in guest mode, it is only possible with eVMCS. +*/ +if (unlikely (vmx->nested.enlightened_vmcs_enabled &&is_guest_mode (vcpu) && + (vmx->nested.current_vmptr ==-1ull ))) +return true; + +return false; +} +/* * The following 3 functions, nested_vmx_succeed()/failValid()/failInvalid(), * set the success or error code of an emulated VMX instruction (as specified
@@ -187,7 +208,7 @@ static int nested_vmx_fail(struct kvm_vcpu *vcpu, u32 vm_instruction_error)* failValid writes the error number to the current VMCS, which * can't be done if there isn't a current VMCS. */ -if (vmx->nested.current_vmptr ==-1ull && !vmx->nested.hv_evmcs) +if (vmx->nested.current_vmptr ==-1ull && !nested_evmcs_is_used (vmx))return nested_vmx_failInvalid (vcpu);return nested_vmx_failValid (vcpu, vm_instruction_error);
@@ -2208,7 +2229,7 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12)u32 exec_control;u64 guest_efer =nested_vmx_calc_efer (vmx, vmcs12); -if (vmx->nested.dirty_vmcs12 || vmx->nested.hv_evmcs) +if (vmx->nested.dirty_vmcs12 ||nested_evmcs_is_used (vmx))prepare_vmcs02_early_rare (vmx, vmcs12);/*
@@ -3437,7 +3458,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,load_vmcs12_host_state (vcpu, vmcs12); vmcs12->vm_exit_reason = exit_reason.full; -if (enable_shadow_vmcs || vmx->nested.hv_evmcs) +if (enable_shadow_vmcs ||nested_evmcs_is_used (vmx)) vmx->nested.need_vmcs12_to_shadow_sync = true;return NVMX_VMENTRY_VMEXIT ; }
@@ -4032,7 +4053,7 @@ static void sync_vmcs02_to_vmcs12(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12) {struct vcpu_vmx *vmx =to_vmx (vcpu); -if (vmx->nested.hv_evmcs) +if (nested_evmcs_is_used (vmx))sync_vmcs02_to_vmcs12_rare (vcpu, vmcs12); vmx->nested.need_sync_vmcs02_to_vmcs12_rare = !vmx->nested.hv_evmcs;
@@ -6056,7 +6077,7 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,if (vmx_has_valid_vmcs12 (vcpu)) { kvm_state.size +=sizeof (user_vmx_nested_state->vmcs12); -if (vmx->nested.hv_evmcs) +if (nested_evmcs_is_used (vmx)) kvm_state.flags |=KVM_STATE_NESTED_EVMCS ; if (is_guest_mode(vcpu) &&
Signature · 1 line
2.30.2
PATCH 2/7 KVM: nVMX: Release enlightened VMCS on VMCLEAR
In reply to Vitaly KuznetsovUnlike VMREAD/VMWRITE/VMPTRLD, VMCLEAR is a valid instruction when enlightened VMCS is in use. TLFS has the following brief description:
"The L1 hypervisor can execute a VMCLEAR instruction to transition an enlightened VMCS from the active to the non-active state". Normally, this change can be ignored as unmapping active eVMCS can be postponed until the next VMLAUNCH instruction but in case nested state is migrated with KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE, keeping eVMCS mapped may result in its synchronization with VMCS12 and this is incorrect:
L1 hypervisor is free to reuse inactive eVMCS memory for something else.
Inactive eVMCS after VMCLEAR can just be unmapped.
--- arch/x86/kvm/vmx/nested.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/kvm/vmx/nested.c +2 -0
@@ -5008,6 +5008,8 @@ static int handle_vmclear(struct kvm_vcpu *vcpu) vmptr + offsetof(struct vmcs12 , launch_state), &zero, sizeof(zero)); + }else if (vmx->nested.hv_evmcs && vmptr == vmx->nested.hv_evmcs_vmptr) { +nested_release_evmcs (vcpu); }return nested_vmx_succeed (vcpu);
Signature · 1 line
2.30.2
PATCH 3/7 KVM: nVMX: Ignore 'hv_clean_fields' data when eVMCS data is copied in vmx_get_nested_state()
In reply to Vitaly Kuznetsov'Clean fields' data from enlightened VMCS is only valid upon vmentry: L1 hypervisor is not obliged to keep it up-to-date while it is mangling L2's state, KVM_GET_NESTED_STATE request may come at a wrong moment when actual eVMCS changes are unsynchronized with 'hv_clean_fields'. As upon migration VMCS12 is used as a source of ultimate truce, we must make sure we pick all the changes to eVMCS and thus 'clean fields' data must be ignored.
--- arch/x86/kvm/vmx/nested.c | 43 +++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 18 deletions(-)
arch/x86/kvm/vmx/nested.c +25 -18
@@ -1607,16 +1607,23 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)vmcs_load (vmx->loaded_vmcs->vmcs); } -static int copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx) +static int copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx,bool from_vmentry) {struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs; +u32 hv_clean_fields;/* HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE */ vmcs12->tpr_threshold = evmcs->tpr_threshold; vmcs12->guest_rip = evmcs->guest_rip; -if (unlikely (!(evmcs->hv_clean_fields & +/* Clean fields data can only be trusted upon vmentry */ +if (likely (from_vmentry)) + hv_clean_fields = evmcs->hv_clean_fields; +else + hv_clean_fields =0 ; + +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC ))) { vmcs12->guest_rsp = evmcs->guest_rsp; vmcs12->guest_rflags = evmcs->guest_rflags;
@@ -1624,23 +1631,23 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) evmcs->guest_interruptibility_info; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_PROC ))) { vmcs12->cpu_based_vm_exec_control = evmcs->cpu_based_vm_exec_control; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EXCPN ))) { vmcs12->exception_bitmap = evmcs->exception_bitmap; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_ENTRY ))) { vmcs12->vm_entry_controls = evmcs->vm_entry_controls; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EVENT ))) { vmcs12->vm_entry_intr_info_field = evmcs->vm_entry_intr_info_field;
@@ -1650,7 +1657,7 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) evmcs->vm_entry_instruction_len; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_GRP1 ))) { vmcs12->host_ia32_pat = evmcs->host_ia32_pat; vmcs12->host_ia32_efer = evmcs->host_ia32_efer;
@@ -1670,7 +1677,7 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->host_tr_selector = evmcs->host_tr_selector; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP1 ))) { vmcs12->pin_based_vm_exec_control = evmcs->pin_based_vm_exec_control;
@@ -1679,18 +1686,18 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) evmcs->secondary_vm_exec_control; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP ))) { vmcs12->io_bitmap_a = evmcs->io_bitmap_a; vmcs12->io_bitmap_b = evmcs->io_bitmap_b; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP ))) { vmcs12->msr_bitmap = evmcs->msr_bitmap; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP2 ))) { vmcs12->guest_es_base = evmcs->guest_es_base; vmcs12->guest_cs_base = evmcs->guest_cs_base;
@@ -1730,14 +1737,14 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->guest_tr_selector = evmcs->guest_tr_selector; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP2 ))) { vmcs12->tsc_offset = evmcs->tsc_offset; vmcs12->virtual_apic_page_addr = evmcs->virtual_apic_page_addr; vmcs12->xss_exit_bitmap = evmcs->xss_exit_bitmap; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CRDR ))) { vmcs12->cr0_guest_host_mask = evmcs->cr0_guest_host_mask; vmcs12->cr4_guest_host_mask = evmcs->cr4_guest_host_mask;
@@ -1749,7 +1756,7 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->guest_dr7 = evmcs->guest_dr7; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_POINTER ))) { vmcs12->host_fs_base = evmcs->host_fs_base; vmcs12->host_gs_base = evmcs->host_gs_base;
@@ -1759,13 +1766,13 @@ static int copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx) vmcs12->host_rsp = evmcs->host_rsp; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_XLAT ))) { vmcs12->ept_pointer = evmcs->ept_pointer; vmcs12->virtual_processor_id = evmcs->virtual_processor_id; } -if (unlikely (!(evmcs->hv_clean_fields & +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP1 ))) { vmcs12->vmcs_link_pointer = evmcs->vmcs_link_pointer; vmcs12->guest_ia32_debugctl = evmcs->guest_ia32_debugctl;
@@ -3503,7 +3510,7 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch)return nested_vmx_failInvalid (vcpu);if (vmx->nested.hv_evmcs) { -copy_enlightened_to_vmcs12 (vmx); +copy_enlightened_to_vmcs12 (vmx, true);/* Enlightened VMCS doesn't have launch state */ vmcs12->launch_state = !launch; }else if (enable_shadow_vmcs) {
@@ -6136,7 +6143,7 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,copy_vmcs02_to_vmcs12_rare (vcpu,get_vmcs12 (vcpu));if (!vmx->nested.need_vmcs12_to_shadow_sync) {if (vmx->nested.hv_evmcs) -copy_enlightened_to_vmcs12 (vmx); +copy_enlightened_to_vmcs12 (vmx, false);else if (enable_shadow_vmcs)copy_shadow_to_vmcs12 (vmx); }
Signature · 1 line
2.30.2
Sean Christopherson
maintainerRe: [PATCH 3/7] KVM: nVMX: Ignore 'hv_clean_fields' data when eVMCS data is copied in vmx_get_nested_state()
Quoted 37 lines · Vitaly Kuznetsov
'Clean fields' data from enlightened VMCS is only valid upon vmentry: L1 hypervisor is not obliged to keep it up-to-date while it is mangling L2's state,
KVM_GET_NESTED_STATErequest may come at a wrong moment when actual eVMCS changes are unsynchronized with 'hv_clean_fields'. As upon migration VMCS12 is used as a source of ultimate truce, we must make sure we pick all the changes to eVMCS and thus 'clean fields' data must be ignored.Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 43 +++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 18 deletions(-)arch/x86/kvm/vmx/nested.c +7
-2@@ -1607,16 +1607,23 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)vmcs_load (vmx->loaded_vmcs->vmcs); } -static int copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx) +static int copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx,bool from_vmentry) {struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs; +u32 hv_clean_fields;/* HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE */ vmcs12->tpr_threshold = evmcs->tpr_threshold; vmcs12->guest_rip = evmcs->guest_rip; - if (unlikely(!(evmcs->hv_clean_fields & +/* Clean fields data can only be trusted upon vmentry */ +if (likely (from_vmentry)) + hv_clean_fields = evmcs->hv_clean_fields; +else + hv_clean_fields =0 ;
...
arch/x86/kvm/vmx/nested.c static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch) · line 3513
Quoted 6 lines
arch/x86/kvm/vmx/nested.c +1
-1@@ -3503,7 +3510,7 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch)return nested_vmx_failInvalid (vcpu);if (vmx->nested.hv_evmcs) { -copy_enlightened_to_vmcs12 (vmx); +copy_enlightened_to_vmcs12 (vmx, true);
Rather than pass a bool, what about having the caller explicitly specify the clean fields? Then the migration path can have a comment about needing to assume all fields are dirty, and the normal path would be self-documenting. E.g. with evmcs captured in a local var:
if (evmcs) {
copy_enlightened_to_vmcs12(vmx, evmcs->hv_clean_fields);
} else if (...) {
}Quoted 15 lines
arch/x86/kvm/vmx/nested.c +1
-1/* Enlightened VMCS doesn't have launch state */ vmcs12->launch_state = !launch; }else if (enable_shadow_vmcs) {@@ -6136,7 +6143,7 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,copy_vmcs02_to_vmcs12_rare (vcpu,get_vmcs12 (vcpu));if (!vmx->nested.need_vmcs12_to_shadow_sync) {if (vmx->nested.hv_evmcs) -copy_enlightened_to_vmcs12 (vmx); +copy_enlightened_to_vmcs12 (vmx, false);else if (enable_shadow_vmcs)copy_shadow_to_vmcs12 (vmx); }Signature · 1 line
2.30.2
Re: [PATCH 3/7] KVM: nVMX: Ignore 'hv_clean_fields' data when eVMCS data is copied in vmx_get_nested_state()
Quoted 53 lines · Sean Christopherson
Quoted 37 lines · Vitaly Kuznetsov
'Clean fields' data from enlightened VMCS is only valid upon vmentry: L1 hypervisor is not obliged to keep it up-to-date while it is mangling L2's state,
KVM_GET_NESTED_STATErequest may come at a wrong moment when actual eVMCS changes are unsynchronized with 'hv_clean_fields'. As upon migration VMCS12 is used as a source of ultimate truce, we must make sure we pick all the changes to eVMCS and thus 'clean fields' data must be ignored.Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 43 +++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 18 deletions(-)arch/x86/kvm/vmx/nested.c +7
-2@@ -1607,16 +1607,23 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)vmcs_load (vmx->loaded_vmcs->vmcs); } -static int copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx) +static int copy_enlightened_to_vmcs12 (struct vcpu_vmx *vmx,bool from_vmentry) {struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs; +u32 hv_clean_fields;/* HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE */ vmcs12->tpr_threshold = evmcs->tpr_threshold; vmcs12->guest_rip = evmcs->guest_rip; - if (unlikely(!(evmcs->hv_clean_fields & +/* Clean fields data can only be trusted upon vmentry */ +if (likely (from_vmentry)) + hv_clean_fields = evmcs->hv_clean_fields; +else + hv_clean_fields =0 ;...
arch/x86/kvm/vmx/nested.c static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch) · line 3513
Quoted 6 lines
arch/x86/kvm/vmx/nested.c +1
-1@@ -3503,7 +3510,7 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch)return nested_vmx_failInvalid (vcpu);if (vmx->nested.hv_evmcs) { -copy_enlightened_to_vmcs12 (vmx); +copy_enlightened_to_vmcs12 (vmx, true);Rather than pass a bool, what about having the caller explicitly specify the clean fields? Then the migration path can have a comment about needing to assume all fields are dirty, and the normal path would be self-documenting. E.g. with evmcs captured in a local var:
Quoted 5 lines
if (evmcs) {
copy_enlightened_to_vmcs12(vmx, evmcs->hv_clean_fields);
} else if (...) {
}I like the idea, thanks! Will incorporate into v2.
Quoted 16 lines
Quoted 15 lines
arch/x86/kvm/vmx/nested.c +1
-1/* Enlightened VMCS doesn't have launch state */ vmcs12->launch_state = !launch; }else if (enable_shadow_vmcs) {@@ -6136,7 +6143,7 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,copy_vmcs02_to_vmcs12_rare (vcpu,get_vmcs12 (vcpu));if (!vmx->nested.need_vmcs12_to_shadow_sync) {if (vmx->nested.hv_evmcs) -copy_enlightened_to_vmcs12 (vmx); +copy_enlightened_to_vmcs12 (vmx, false);else if (enable_shadow_vmcs)copy_shadow_to_vmcs12 (vmx); }Signature · 1 line
2.30.2
Signature · 1 line
Vitaly
PATCH 4/7 KVM: nVMX: Force enlightened VMCS sync from nested_vmx_failValid()
In reply to Vitaly Kuznetsov'need_vmcs12_to_shadow_sync' is used for both shadow and enlightened VMCS sync when we exit to L1. The comment in nested_vmx_failValid() validly states why shadow vmcs sync can be omitted but this doesn't apply to enlightened VMCS as it 'shadows' all VMCS12 fields.
--- arch/x86/kvm/vmx/nested.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/nested.c +6 -2
@@ -194,9 +194,13 @@ static int nested_vmx_failValid(struct kvm_vcpu *vcpu, | X86_EFLAGS_ZF);get_vmcs12 (vcpu)->vm_instruction_error = vm_instruction_error;/* -* We don't need to force a shadow sync because -* VM_INSTRUCTION_ERROR is not shadowed +* We don't need to force sync to shadow VMCS because +* VM_INSTRUCTION_ERROR is not shadowed. Enlightened VMCS 'shadows' all +* fields and thus must be synced. */ +if (nested_evmcs_is_used (to_vmx (vcpu))) +to_vmx (vcpu)->nested.need_vmcs12_to_shadow_sync = true; +return kvm_skip_emulated_instruction (vcpu); }
Signature · 1 line
2.30.2
PATCH 5/7 KVM: nVMX: Reset eVMCS clean fields data from prepare_vmcs02()
In reply to Vitaly KuznetsovWhen nested state migration happens during L1's execution, it is incorrect to modify eVMCS as it is L1 who 'owns' it at the moment. At lease genuine Hyper-v seems to not be very happy when 'clean fields' data changes underneath it.
'Clean fields' data is used in KVM twice: by copy_enlightened_to_vmcs12() and prepare_vmcs02_rare() so we can reset it from prepare_vmcs02() instead.
--- arch/x86/kvm/vmx/nested.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/nested.c +6 -3
@@ -2090,9 +2090,6 @@ void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu)if (vmx->nested.hv_evmcs) {copy_vmcs12_to_enlightened (vmx); -/* All fields are clean */ - vmx->nested.hv_evmcs->hv_clean_fields |= -HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; }else {copy_vmcs12_to_shadow (vmx); }
@@ -2636,6 +2633,12 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,kvm_rsp_write (vcpu, vmcs12->guest_rsp);kvm_rip_write (vcpu, vmcs12->guest_rip); + +/* Mark all fields as clean so L1 hypervisor can set what's dirty */ +if (hv_evmcs) + vmx->nested.hv_evmcs->hv_clean_fields |= +HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; +return 0 ; }
Signature · 1 line
2.30.2
Sean Christopherson
maintainerRe: [PATCH 5/7] KVM: nVMX: Reset eVMCS clean fields data from prepare_vmcs02()
Quoted 18 lines · Vitaly Kuznetsov
When nested state migration happens during L1's execution, it is incorrect to modify eVMCS as it is L1 who 'owns' it at the moment. At lease genuine Hyper-v seems to not be very happy when 'clean fields' data changes underneath it.
'Clean fields' data is used in KVM twice: by
copy_enlightened_to_vmcs12()andprepare_vmcs02_rare()so we can reset it fromprepare_vmcs02()instead.Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)arch/x86/kvm/vmx/nested.c +0
-0
arch/x86/kvm/vmx/nested.c void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu) · line 2095
Quoted 9 lines
arch/x86/kvm/vmx/nested.c +0
-3@@ -2090,9 +2090,6 @@ void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu)if (vmx->nested.hv_evmcs) {copy_vmcs12_to_enlightened (vmx); -/* All fields are clean */ - vmx->nested.hv_evmcs->hv_clean_fields |= -HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; }else {copy_vmcs12_to_shadow (vmx); }
Looks like this makes both paths single lines, i.e. the curly braces can be dropped.
Quoted 16 lines
arch/x86/kvm/vmx/nested.c +6
-0@@ -2636,6 +2633,12 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,kvm_rsp_write (vcpu, vmcs12->guest_rsp);kvm_rip_write (vcpu, vmcs12->guest_rip); + +/* Mark all fields as clean so L1 hypervisor can set what's dirty */ +if (hv_evmcs) + vmx->nested.hv_evmcs->hv_clean_fields |= +HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; +return 0 ; }Signature · 1 line
2.30.2
Re: [PATCH 5/7] KVM: nVMX: Reset eVMCS clean fields data from prepare_vmcs02()
Quoted 20 lines · Sean Christopherson
Quoted 19 lines · Vitaly Kuznetsov
When nested state migration happens during L1's execution, it is incorrect to modify eVMCS as it is L1 who 'owns' it at the moment. At lease genuine Hyper-v seems to not be very happy when 'clean fields' data changes underneath it.
'Clean fields' data is used in KVM twice: by
copy_enlightened_to_vmcs12()andprepare_vmcs02_rare()so we can reset it fromprepare_vmcs02()instead.Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)arch/x86/kvm/vmx/nested.c +0
-0@@ -2090,9 +2090,6 @@ void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu)
Quoted 12 lines
arch/x86/kvm/vmx/nested.c void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu) · line 2095
Quoted 8 lines
arch/x86/kvm/vmx/nested.c +0
-3if (vmx->nested.hv_evmcs) {copy_vmcs12_to_enlightened (vmx); -/* All fields are clean */ - vmx->nested.hv_evmcs->hv_clean_fields |= -HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; }else {copy_vmcs12_to_shadow (vmx); }Looks like this makes both paths single lines, i.e. the curly braces can be dropped.
Yea, I've noticed that but I wasn't exactly sure the benefit of removing one line here overwhelms the pain of needing an extra step while doing code archeology with 'git blame' :-) Will drop on v2.
Quoted 17 lines
Quoted 16 lines
arch/x86/kvm/vmx/nested.c +6
-0@@ -2636,6 +2633,12 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,kvm_rsp_write (vcpu, vmcs12->guest_rsp);kvm_rip_write (vcpu, vmcs12->guest_rip); + +/* Mark all fields as clean so L1 hypervisor can set what's dirty */ +if (hv_evmcs) + vmx->nested.hv_evmcs->hv_clean_fields |= +HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; +return 0 ; }Signature · 1 line
2.30.2
Signature · 1 line
Vitaly
PATCH 6/7 KVM: nVMX: Request to sync eVMCS from VMCS12 after migration
In reply to Vitaly KuznetsovVMCS12 is used to keep the authoritative state during nested state migration. In case 'need_vmcs12_to_shadow_sync' flag is set, we're in between L2->L1 vmexit and L1 guest run when actual sync to enlightened (or shadow) VMCS happens. Nested state, however, has no flag for 'need_vmcs12_to_shadow_sync' so vmx_set_nested_state()-> set_current_vmptr() always sets it. Enlightened vmptrld path, however, doesn't have the quirk so some VMCS12 changes may not get properly reflected to eVMCS and L1 will see an incorrect state.
Note, during L2 execution or when need_vmcs12_to_shadow_sync is not set the change is effectively a nop: in the former case all changes will get reflected during the first L2->L1 vmexit and in the later case VMCS12 and eVMCS are already in sync (thanks to copy_enlightened_to_vmcs12() in vmx_get_nested_state()).
--- arch/x86/kvm/vmx/nested.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/x86/kvm/vmx/nested.c +6 -0
@@ -3135,6 +3135,12 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)if (evmptrld_status ==EVMPTRLD_VMFAIL || evmptrld_status ==EVMPTRLD_ERROR )return false; + +/* +* Post migration VMCS12 always provides the most actual +* information, copy it to eVMCS upon entry. +*/ + vmx->nested.need_vmcs12_to_shadow_sync = true; }return true;
Signature · 1 line
2.30.2
PATCH 7/7 KVM: selftests: evmcs_test: Test that KVM_STATE_NESTED_EVMCS is never lost
In reply to Vitaly KuznetsovDo KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE for a freshly restored VM (before the first KVM_RUN) to check that KVM_STATE_NESTED_EVMCS is not lost.
--- .../testing/selftests/kvm/x86_64/evmcs_test.c | 64 +++++++++++-------- 1 file changed, 38 insertions(+), 26 deletions(-)
tools/testing/selftests/kvm/x86_64/evmcs_test.c +38 -26
@@ -121,14 +121,38 @@ void inject_nmi(struct kvm_vm *vm)vcpu_events_set (vm,VCPU_ID , &events); } +static void save_restore_vm (struct kvm_vm *vm) +{ +struct kvm_regs regs1, regs2; +struct kvm_x86_state *state; + + state =vcpu_save_state (vm,VCPU_ID ); +memset (®s1,0 ,sizeof (regs1)); +vcpu_regs_get (vm,VCPU_ID , ®s1); + +kvm_vm_release (vm); + +/* Restore state in a new VM. */ +kvm_vm_restart (vm,O_RDWR ); +vm_vcpu_add (vm,VCPU_ID ); +vcpu_set_hv_cpuid (vm,VCPU_ID ); +vcpu_enable_evmcs (vm,VCPU_ID ); +vcpu_load_state (vm,VCPU_ID , state); +free (state); + +memset (®s2,0 ,sizeof (regs2)); +vcpu_regs_get (vm,VCPU_ID , ®s2); +TEST_ASSERT (!memcmp (®s1, ®s2,sizeof (regs2)), +"Unexpected register values after vcpu_load_state; rdi: %lx rsi: %lx" , + (ulong ) regs2.rdi, (ulong ) regs2.rsi); +} +int main (int argc,char *argv[]) {vm_vaddr_t vmx_pages_gva =0 ; -struct kvm_regs regs1, regs2;struct kvm_vm *vm;struct kvm_run *run; -struct kvm_x86_state *state;struct ucall uc;int stage;
@@ -145,10 +169,6 @@ int main(int argc, char *argv[])vcpu_set_hv_cpuid (vm,VCPU_ID );vcpu_enable_evmcs (vm,VCPU_ID ); - run =vcpu_state (vm,VCPU_ID ); - -vcpu_regs_get (vm,VCPU_ID , ®s1); -vcpu_alloc_vmx (vm, &vmx_pages_gva);vcpu_args_set (vm,VCPU_ID ,1 , vmx_pages_gva);
@@ -160,6 +180,7 @@ int main(int argc, char *argv[])pr_info ("Running L1 which uses EVMCS to run L2\n" );for (stage =1 ;; stage++) { + run =vcpu_state (vm,VCPU_ID );_vcpu_run (vm,VCPU_ID ); TEST_ASSERT(run->exit_reason == KVM_EXIT_IO, "Stage %d: unexpected exit reason: %u (%s),\n",
@@ -184,32 +205,23 @@ int main(int argc, char *argv[]) uc.args[1 ] == stage,"Stage %d: Unexpected register values vmexit, got %lx" , stage, (ulong)uc.args[1]); - state =vcpu_save_state (vm,VCPU_ID ); -memset (®s1,0 ,sizeof (regs1)); -vcpu_regs_get (vm,VCPU_ID , ®s1); - -kvm_vm_release (vm); - -/* Restore state in a new VM. */ -kvm_vm_restart (vm,O_RDWR ); -vm_vcpu_add (vm,VCPU_ID ); -vcpu_set_hv_cpuid (vm,VCPU_ID ); -vcpu_enable_evmcs (vm,VCPU_ID ); -vcpu_load_state (vm,VCPU_ID , state); - run =vcpu_state (vm,VCPU_ID ); -free (state); - -memset (®s2,0 ,sizeof (regs2)); -vcpu_regs_get (vm,VCPU_ID , ®s2); -TEST_ASSERT (!memcmp (®s1, ®s2,sizeof (regs2)), -"Unexpected register values after vcpu_load_state; rdi: %lx rsi: %lx" , - (ulong ) regs2.rdi, (ulong ) regs2.rsi); +save_restore_vm (vm);/* Force immediate L2->L1 exit before resuming */ if (stage ==8 ) {pr_info ("Injecting NMI into L1 before L2 had a chance to run after restore\n" );inject_nmi (vm); } + +/* +* Do KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE for a freshly +* restored VM (before the first KVM_RUN) to check that +* KVM_STATE_NESTED_EVMCS is not lost. +*/ +if (stage ==9 ) { +pr_info ("Trying extra KVM_GET_NESTED_STATE/KVM_SET_NESTED_STATE cycle\n" ); +save_restore_vm (vm); + } } done:
Signature · 1 line
2.30.2