x86/kvm/nVMX: fix Enlightened VMCLEAR
v2 cover x86/kvm/nVMX: fix Enlightened VMCLEAR
VMCLEAR implementation for Enlightened VMCS is not entirely correct when something else than the currently active eVMCS on the calling vCPU is targeted. In case there's no currently active eVMCS on the calling vCPU we are corrupting the targeted area by writing to the non-existent launch_state field.
Fix the logic by always treating the targeted area as 'enlightened' in case Enlightened VMEntry is enabled on the calling vCPU.
Changes since v1:
- 'evmcs_vmptr' -> 'evmcs_gpa' [Paolo Bonzini]
- avoid nested_release_evmcs() in handle_vmclear even for the currently active eVMCS on the calling vCPU [Liran Alon], PATCH1 added to support the change.
Vitaly Kuznetsov (2):
x86/KVM/nVMX: don't use clean fields data on enlightened VMLAUNCH x86/kvm/nVMX: fix VMCLEAR when Enlightened VMCS is in use
arch/x86/kvm/vmx/evmcs.c | 18 ++++++++++++++ arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/nested.c | 52 ++++++++++++++++++++++----------------- 3 files changed, 49 insertions(+), 22 deletions(-)
Signature · 1 line
2.20.1
PATCH v2 1/2 x86/KVM/nVMX: don't use clean fields data on enlightened VMLAUNCH
Apparently, Windows doesn't maintain clean fields data after it does VMCLEAR for an enlightened VMCS so we can only use it on VMRESUME. The issue went unnoticed because currently we do nested_release_evmcs() in handle_vmclear() and the consecutive enlightened VMPTRLD invalidates clean fields when a new eVMCS is mapped but we're going to change the logic.
--- arch/x86/kvm/vmx/nested.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/nested.c +12 -8
@@ -1766,6 +1766,7 @@ static int nested_vmx_handle_enlightened_vmptrld(struct kvm_vcpu *vcpu, {struct vcpu_vmx *vmx =to_vmx (vcpu);struct hv_vp_assist_page assist_page; +bool evmcs_gpa_changed = false;if (likely (!vmx->nested.enlightened_vmcs_enabled))return 1 ;
@@ -1819,15 +1820,9 @@ static int nested_vmx_handle_enlightened_vmptrld(struct kvm_vcpu *vcpu, } vmx->nested.dirty_vmcs12 = true; -/* -* As we keep L2 state for one guest only 'hv_clean_fields' mask -* can't be used when we switch between them. Reset it here for -* simplicity. -*/ - vmx->nested.hv_evmcs->hv_clean_fields &= - ~HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; vmx->nested.hv_evmcs_vmptr = assist_page.current_nested_vmcs; + evmcs_gpa_changed = true;/* * Unlike normal vmcs12, enlightened vmcs12 is not fully * reloaded from guest's memory (read only fields, fields not
@@ -1841,6 +1836,15 @@ static int nested_vmx_handle_enlightened_vmptrld(struct kvm_vcpu *vcpu, } } + +/* +* Clean fields data can't de used on VMLAUNCH and when we switch +* between different L2 guests as KVM keeps a single VMCS12 per L1. +*/ +if (from_launch || evmcs_gpa_changed) + vmx->nested.hv_evmcs->hv_clean_fields &= + ~HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; +return 1 ; }
@@ -3074,7 +3078,7 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch)if (!nested_vmx_check_permission (vcpu))return 1 ; -if (!nested_vmx_handle_enlightened_vmptrld (vcpu, true)) +if (!nested_vmx_handle_enlightened_vmptrld (vcpu, launch))return 1 ;if (!vmx->nested.hv_evmcs && vmx->nested.current_vmptr ==-1ull )
Signature · 1 line
2.20.1
PATCH v2 2/2 x86/kvm/nVMX: fix VMCLEAR when Enlightened VMCS is in use
In reply to Vitaly KuznetsovWhen Enlightened VMCS is in use, it is valid to do VMCLEAR and, according to TLFS, this should "transition an enlightened VMCS from the active to the non-active state". It is, however, wrong to assume that it is only valid to do VMCLEAR for the eVMCS which is currently active on the vCPU performing VMCLEAR.
Currently, the logic in handle_vmclear() is broken: in case, there is no active eVMCS on the vCPU doing VMCLEAR we treat the argument as a 'normal' VMCS and kvm_vcpu_write_guest() to the 'launch_state' field irreversibly corrupts the memory area.
So, in case the VMCLEAR argument is not the current active eVMCS on the vCPU, how can we know if the area it is pointing to is a normal or an enlightened VMCS?
Thanks to the bug in Hyper-V (see commit 72aeb60c52bf7 ("KVM: nVMX: Verify eVMCS revision id match supported eVMCS version on eVMCS VMPTRLD")) we can not, the revision can't be used to distinguish between them. So let's assume it is always enlightened in case enlightened vmentry is enabled in the assist page. Also, check if vmx->nested.enlightened_vmcs_enabled to minimize the impact for 'unenlightened' workloads.
--- arch/x86/kvm/vmx/evmcs.c | 18 ++++++++++++++++++ arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/nested.c | 32 ++++++++++++++++++-------------- 3 files changed, 37 insertions(+), 14 deletions(-)
arch/x86/kvm/vmx/evmcs.c +18 -0
@@ -3,6 +3,7 @@#include <linux/errno.h> #include <linux/smp.h> +#include "../hyperv.h" #include "evmcs.h" #include "vmcs.h" #include "vmx.h"
@@ -309,6 +310,23 @@ void evmcs_sanitize_exec_ctrls(struct vmcs_config *vmcs_conf) }#endif +bool nested_enlightened_vmentry (struct kvm_vcpu *vcpu,u64 *evmcs_gpa) +{ +struct hv_vp_assist_page assist_page; + + *evmcs_gpa =-1ull ; + +if (unlikely (!kvm_hv_get_assist_page (vcpu, &assist_page))) +return false; + +if (unlikely (!assist_page.enlighten_vmentry)) +return false; + + *evmcs_gpa = assist_page.current_nested_vmcs; + +return true; +} +uint16_t nested_get_evmcs_version (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
arch/x86/kvm/vmx/evmcs.h +1 -0
@@ -195,6 +195,7 @@ static inline void evmcs_sanitize_exec_ctrls(struct vmcs_config *vmcs_conf) {}static inline void evmcs_touch_msr_bitmap (void ) {}#endif /* IS_ENABLED(CONFIG_HYPERV) */ +bool nested_enlightened_vmentry (struct kvm_vcpu *vcpu,u64 *evmcs_gpa);uint16_t nested_get_evmcs_version (struct kvm_vcpu *vcpu);int nested_enable_evmcs (struct kvm_vcpu *vcpu,uint16_t *vmcs_version);
arch/x86/kvm/vmx/nested.c +18 -14
@@ -1765,27 +1765,22 @@ static int nested_vmx_handle_enlightened_vmptrld(struct kvm_vcpu *vcpu, bool from_launch) {struct vcpu_vmx *vmx =to_vmx (vcpu); -struct hv_vp_assist_page assist_page;bool evmcs_gpa_changed = false; +u64 evmcs_gpa;if (likely (!vmx->nested.enlightened_vmcs_enabled))return 1 ; -if (unlikely (!kvm_hv_get_assist_page (vcpu, &assist_page))) +if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa))return 1 ; -if (unlikely (!assist_page.enlighten_vmentry)) -return 1 ; - -if (unlikely (assist_page.current_nested_vmcs != - vmx->nested.hv_evmcs_vmptr)) { - +if (unlikely (evmcs_gpa != vmx->nested.hv_evmcs_vmptr)) {if (!vmx->nested.hv_evmcs) vmx->nested.current_vmptr =-1ull ;nested_release_evmcs (vcpu); -if (kvm_vcpu_map (vcpu,gpa_to_gfn (assist_page.current_nested_vmcs), +if (kvm_vcpu_map (vcpu,gpa_to_gfn (evmcs_gpa), &vmx->nested.hv_evmcs_map))return 0 ;
@@ -1820,7 +1815,7 @@ static int nested_vmx_handle_enlightened_vmptrld(struct kvm_vcpu *vcpu, } vmx->nested.dirty_vmcs12 = true; - vmx->nested.hv_evmcs_vmptr = assist_page.current_nested_vmcs; + vmx->nested.hv_evmcs_vmptr = evmcs_gpa; evmcs_gpa_changed = true;/*
@@ -4335,6 +4330,7 @@ static int handle_vmclear(struct kvm_vcpu *vcpu)struct vcpu_vmx *vmx =to_vmx (vcpu);u32 zero =0 ;gpa_t vmptr; +u64 evmcs_gpa;if (!nested_vmx_check_permission (vcpu))return 1 ;
@@ -4350,10 +4346,18 @@ static int handle_vmclear(struct kvm_vcpu *vcpu)return nested_vmx_failValid (vcpu,VMXERR_VMCLEAR_VMXON_POINTER ); -if (vmx->nested.hv_evmcs_map.hva) { -if (vmptr == vmx->nested.hv_evmcs_vmptr) -nested_release_evmcs (vcpu); - }else { +/* +* When Enlightened VMEntry is enabled on the calling CPU we treat +* memory area pointer by vmptr as Enlightened VMCS (as there's no good +* way to distinguish it from VMCS12) and we must not corrupt it by +* writing to the non-existent 'launch_state' field. The area doesn't +* have to be the currently active EVMCS on the calling CPU and there's +* nothing KVM has to do to transition it from 'active' to 'non-active' +* state. It is possible that the area will stay mapped as +* vmx->nested.hv_evmcs but this shouldn't be a problem. +*/ +if (likely (!vmx->nested.enlightened_vmcs_enabled || + !nested_enlightened_vmentry (vcpu, &evmcs_gpa))) {if (vmptr == vmx->nested.current_vmptr)nested_release_vmcs12 (vcpu);
Signature · 1 line
2.20.1
Quoted 15 lines · Vitaly Kuznetsov
VMCLEAR implementation for Enlightened VMCS is not entirely correct when something else than the currently active eVMCS on the calling vCPU is targeted. In case there's no currently active eVMCS on the calling vCPU we are corrupting the targeted area by writing to the non-existent
launch_statefield.Fix the logic by always treating the targeted area as 'enlightened' in case Enlightened VMEntry is enabled on the calling vCPU.
Changes since v1:
- 'evmcs_vmptr' -> 'evmcs_gpa' [Paolo Bonzini]
- avoidnested_release_evmcs()inhandle_vmcleareven for the currently active eVMCS on the calling vCPU [Liran Alon], PATCH1 added to support the change.
Quoted 9 lines
Vitaly Kuznetsov (2):
x86/KVM/nVMX: don't use clean fields data on enlightened VMLAUNCH x86/kvm/nVMX: fix VMCLEAR when Enlightened VMCS is in usearch/x86/kvm/vmx/evmcs.c | 18 ++++++++++++++ arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/nested.c | 52 ++++++++++++++++++++++----------------- 3 files changed, 49 insertions(+), 22 deletions(-)
Queued, thanks.
Paolo