KVM: nVMX: Filter out all unsupported controls when eVMCS was activated
PATCH 1/1 KVM: nVMX: Filter out all unsupported controls when eVMCS was activated
Windows Server 2022 with Hyper-V role enabled failed to boot on KVM when enlightened VMCS is advertised. Debugging revealed there are two exposed secondary controls it is not happy with: SECONDARY_EXEC_ENABLE_VMFUNC and SECONDARY_EXEC_SHADOW_VMCS. These controls are known to be unsupported, as there are no corresponding fields in eVMCSv1 (see the comment above EVMCS1_UNSUPPORTED_2NDEXEC definition).
Previously, commit 31de3d2500e4 ("x86/kvm/hyper-v: move VMX controls sanitization out of nested_enable_evmcs()") introduced the required filtering mechanism for VMX MSRs but for some reason put only known to be problematic (and not full EVMCS1_UNSUPPORTED_* lists) controls there.
Note, Windows Server 2022 seems to have gained some sanity check for VMX
MSRs: it doesn't even try to launch a guest when there's something it doesn't like, nested_evmcs_check_controls() mechanism can't catch the problem.
Let's be bold this time and instead of playing whack-a-mole just filter out all unsupported controls from VMX MSRs.
--- arch/x86/kvm/vmx/evmcs.c | 12 +++++++++--- arch/x86/kvm/vmx/vmx.c | 9 +++++---- 2 files changed, 14 insertions(+), 7 deletions(-)
arch/x86/kvm/vmx/evmcs.c +9 -3
@@ -353,14 +353,20 @@ void nested_evmcs_filter_control_msr(u32 msr_index, u64 *pdata)switch (msr_index) {case MSR_IA32_VMX_EXIT_CTLS :case MSR_IA32_VMX_TRUE_EXIT_CTLS : - ctl_high &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ; + ctl_high &= ~EVMCS1_UNSUPPORTED_VMEXIT_CTRL ;break ;case MSR_IA32_VMX_ENTRY_CTLS :case MSR_IA32_VMX_TRUE_ENTRY_CTLS : - ctl_high &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ; + ctl_high &= ~EVMCS1_UNSUPPORTED_VMENTRY_CTRL ;break ;case MSR_IA32_VMX_PROCBASED_CTLS2 : - ctl_high &= ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ; + ctl_high &= ~EVMCS1_UNSUPPORTED_2NDEXEC ; +break ; +case MSR_IA32_VMX_PINBASED_CTLS : + ctl_high &= ~EVMCS1_UNSUPPORTED_PINCTRL ; +break ; +case MSR_IA32_VMX_VMFUNC : + ctl_low &= ~EVMCS1_UNSUPPORTED_VMFUNC ;break ; }
arch/x86/kvm/vmx/vmx.c +5 -4
@@ -1837,10 +1837,11 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) &msr_info->data))return 1 ;/* -* Enlightened VMCS v1 doesn't have certain fields, but buggy -* Hyper-V versions are still trying to use corresponding -* features when they are exposed. Filter out the essential -* minimum. +* Enlightened VMCS v1 doesn't have certain VMCS fields but +* instead of just ignoring the features, different Hyper-V +* versions are either trying to use them and fail or do some +* sanity checking and refuse to boot. Filter all unsupported +* features out. */ if (!msr_info->host_initiated && vmx->nested.enlightened_vmcs_enabled)
Signature · 1 line
2.31.1
Sean Christopherson
maintainerQuoted 2 lines · Vitaly Kuznetsov
Let's be bold this time and instead of playing whack-a-mole just filter out all unsupported controls from VMX MSRs.
Out of curiosity, why didn't we do this from the get-go?
Quoted 5 lines · Sean Christopherson
Quoted 2 lines · Vitaly Kuznetsov
Let's be bold this time and instead of playing whack-a-mole just filter out all unsupported controls from VMX MSRs.
Out of curiosity, why didn't we do this from the get-go?
We actually did, the initial implementation (57b119da3594f) was filtering out everything but then things changed in "only clear controls which are known to cause issues" (31de3d2500e4). I forgot everything already but was able to google this suggestion from Paolo:
https://www.lkml.org/lkml/2020/1/22/1108
so finally we've settled on a shortened list. Now as new Windows version is out, we have new problems to solve)
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerQuoted 6 lines · Vitaly Kuznetsov
Quoted 4 lines · Sean Christopherson
Quoted 2 lines · Vitaly Kuznetsov
Let's be bold this time and instead of playing whack-a-mole just filter out all unsupported controls from VMX MSRs.
Quoted 8 lines
Quoted 1 line
Out of curiosity, why didn't we do this from the get-go?
We actually did, the initial implementation (
57b119da3594f) was filtering out everything but then things changed in "only clear controls which are known to cause issues" (31de3d2500e4). I forgot everything already but was able to google this suggestion from Paolo:
The doubt was whether userspaces could be enabling eVMCS blindly, and thus would lose features for Linux guests.
Paolo
Quoted 67 lines · Vitaly Kuznetsov
Windows Server 2022 with Hyper-V role enabled failed to boot on KVM when enlightened VMCS is advertised. Debugging revealed there are two exposed secondary controls it is not happy with:
SECONDARY_EXEC_ENABLE_VMFUNCandSECONDARY_EXEC_SHADOW_VMCS. These controls are known to be unsupported, as there are no corresponding fields in eVMCSv1 (see the comment aboveEVMCS1_UNSUPPORTED_2NDEXECdefinition).Previously, commit
31de3d2500e4("x86/kvm/hyper-v: move VMX controls sanitization out ofnested_enable_evmcs()") introduced the required filtering mechanism for VMX MSRs but for some reason put only known to be problematic (and not full EVMCS1_UNSUPPORTED_* lists) controls there.Note, Windows Server 2022 seems to have gained some sanity check for VMX
MSRs: it doesn't even try to launch a guest when there's something it doesn't like,nested_evmcs_check_controls()mechanism can't catch the problem.Let's be bold this time and instead of playing whack-a-mole just filter out all unsupported controls from VMX MSRs.
Fixes 31de3d2500e4 ("x86/kvm/hyper-v: move VMX controls sanitization out of nested_enable_evmcs()")Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/evmcs.c | 12 +++++++++--- arch/x86/kvm/vmx/vmx.c | 9 +++++---- 2 files changed, 14 insertions(+), 7 deletions(-)arch/x86/kvm/vmx/evmcs.c +9
-3@@ -353,14 +353,20 @@ void nested_evmcs_filter_control_msr(u32 msr_index, u64 *pdata)switch (msr_index) {case MSR_IA32_VMX_EXIT_CTLS :case MSR_IA32_VMX_TRUE_EXIT_CTLS : - ctl_high &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL ; + ctl_high &= ~EVMCS1_UNSUPPORTED_VMEXIT_CTRL ;break ;case MSR_IA32_VMX_ENTRY_CTLS :case MSR_IA32_VMX_TRUE_ENTRY_CTLS : - ctl_high &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL ; + ctl_high &= ~EVMCS1_UNSUPPORTED_VMENTRY_CTRL ;break ;case MSR_IA32_VMX_PROCBASED_CTLS2 : - ctl_high &= ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ; + ctl_high &= ~EVMCS1_UNSUPPORTED_2NDEXEC ; +break ; +case MSR_IA32_VMX_PINBASED_CTLS : + ctl_high &= ~EVMCS1_UNSUPPORTED_PINCTRL ; +break ; +case MSR_IA32_VMX_VMFUNC : + ctl_low &= ~EVMCS1_UNSUPPORTED_VMFUNC ;break ; }arch/x86/kvm/vmx/vmx.c +0
-2@@ -1837,10 +1837,11 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) &msr_info->data))return 1 ;/* -* Enlightened VMCS v1 doesn't have certain fields, but buggy -* Hyper-V versions are still trying to use corresponding
arch/x86/kvm/vmx/vmx.c static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 1847
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +5
-2-* features when they are exposed. Filter out the essential -* minimum. +* Enlightened VMCS v1 doesn't have certain VMCS fields but +* instead of just ignoring the features, different Hyper-V +* versions are either trying to use them and fail or do some +* sanity checking and refuse to boot. Filter all unsupported +* features out. */ if (!msr_info->host_initiated && vmx->nested.enlightened_vmcs_enabled)
Queued, thanks.
Paolo