KVM: nVMX: Enlightened MSR Bitmap feature for Hyper-V on KVM
Subsystems (4)
Vitaly Kuznetsov
maintainerv2 cover KVM: nVMX: Enlightened MSR Bitmap feature for Hyper-V on KVM
Changes since v1:
- Added Maxim's R-b tags to patches 1,2
- Moved nested.msr_bitmap_changed clearing from nested_get_vmcs12_pages() to nested_vmx_prepare_msr_bitmap [Maxim Levitsky]
- Rebased to current kvm/queue.
Updating MSR bitmap for L2 is not cheap and rearly needed. TLFS for Hyper-V offers 'Enlightened MSR Bitmap' feature which allows L1 hypervisor to inform L0 when it changes MSR bitmap, this eliminates the need to examine L1's MSR bitmap for L2 every time when 'real' MSR bitmap for L2 gets constructed.
When the feature is enabled for Win10+WSL2, it shaves off around 700 CPU cycles from a nested vmexit cost (tight cpuid loop test).
First patch of the series is unrelated to the newly implemented feature, it fixes a bug in Enlightened MSR Bitmap usage when KVM runs as a nested hypervisor on top of Hyper-V.
Vitaly Kuznetsov (4):
KVM: nVMX: Don't use Enlightened MSR Bitmap for L3
KVM: VMX: Introduce vmx_msr_bitmap_l01_changed() helper
KVM: nVMX: Track whether changes in L0 require MSR bitmap for L2 to be
rebuilt
KVM: nVMX: Implement Enlightened MSR Bitmap feature
arch/x86/kvm/hyperv.c | 2 ++ arch/x86/kvm/vmx/nested.c | 29 +++++++++++++++++++++++++--- arch/x86/kvm/vmx/vmx.c | 40 ++++++++++++++++++++++++++------------- arch/x86/kvm/vmx/vmx.h | 6 ++++++ 4 files changed, 61 insertions(+), 16 deletions(-)
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v2 1/4 KVM: nVMX: Don't use Enlightened MSR Bitmap for L3
When KVM runs as a nested hypervisor on top of Hyper-V it uses Enlightened VMCS and enables Enlightened MSR Bitmap feature for its L1s and L2s (which are actually L2s and L3s from Hyper-V's perspective). When MSR bitmap is updated, KVM has to reset HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP from clean fields to make Hyper-V aware of the change. For KVM's L1s, this is done in vmx_disable_intercept_for_msr()/vmx_enable_intercept_for_msr(). MSR bitmap for L2 is build in nested_vmx_prepare_msr_bitmap() by blending MSR bitmap for L1 and L1's idea of MSR bitmap for L2. KVM, however, doesn't check if the resulting bitmap is different and never cleans HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP in eVMCS02. This is incorrect and may result in Hyper-V missing the update.
The issue could've been solved by calling evmcs_touch_msr_bitmap() for eVMCS02 from nested_vmx_prepare_msr_bitmap() unconditionally but doing so would not give any performance benefits (compared to not using Enlightened MSR Bitmap at all). 3-level nesting is also not a very common setup nowadays.
Don't enable 'Enlightened MSR Bitmap' feature for KVM's L2s (real L3s) for now.
--- arch/x86/kvm/vmx/vmx.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/vmx.c +12 -9
@@ -2655,15 +2655,6 @@ int alloc_loaded_vmcs(struct loaded_vmcs *loaded_vmcs)if (!loaded_vmcs->msr_bitmap) goto out_vmcs;memset (loaded_vmcs->msr_bitmap,0xff ,PAGE_SIZE ); - -if (IS_ENABLED (CONFIG_HYPERV ) && -static_branch_unlikely (&enable_evmcs) && - (ms_hyperv.nested_features &HV_X64_NESTED_MSR_BITMAP )) { -struct hv_enlightened_vmcs *evmcs = - (struct hv_enlightened_vmcs *)loaded_vmcs->vmcs; - - evmcs->hv_enlightenments_control.msr_bitmap =1 ; - } }memset (&loaded_vmcs->host_state,0 ,sizeof (struct vmcs_host_state ));
@@ -6903,6 +6894,18 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu) vmx->loaded_vmcs = &vmx->vmcs01; +/* +* Use Hyper-V 'Enlightened MSR Bitmap' feature when KVM runs as a +* nested (L1) hypervisor and Hyper-V in L0 supports it. +*/ +if (IS_ENABLED (CONFIG_HYPERV ) &&static_branch_unlikely (&enable_evmcs) + && (ms_hyperv.nested_features &HV_X64_NESTED_MSR_BITMAP )) { +struct hv_enlightened_vmcs *evmcs = + (struct hv_enlightened_vmcs *)vmx->loaded_vmcs->vmcs; + + evmcs->hv_enlightenments_control.msr_bitmap =1 ; + } +if (cpu_need_virtualize_apic_accesses (vcpu)) { err =alloc_apic_access_page (vcpu->kvm);if (err)
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v2 2/4 KVM: VMX: Introduce vmx_msr_bitmap_l01_changed() helper
In reply to Vitaly KuznetsovIn preparation to enabling 'Enlightened MSR Bitmap' feature for Hyper-V guests move MSR bitmap update tracking to a dedicated helper.
Note: vmx_msr_bitmap_l01_changed() is called when MSR bitmap might be updated. KVM doesn't check if the bit we're trying to set is already set (or the bit it's trying to clear is already cleared). Such situations should not be common and a few false positives should not be a problem.
No functional change intended.
--- arch/x86/kvm/vmx/vmx.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/vmx.c +13 -4
@@ -3726,6 +3726,17 @@ static void vmx_set_msr_bitmap_write(ulong *msr_bitmap, u32 msr)__set_bit (msr &0x1fff , msr_bitmap +0xc00 / f); } +static void vmx_msr_bitmap_l01_changed (struct vcpu_vmx *vmx) +{ +/* +* When KVM is a nested hypervisor on top of Hyper-V and uses +* 'Enlightened MSR Bitmap' feature L0 needs to know that MSR +* bitmap has changed. +*/ +if (static_branch_unlikely (&enable_evmcs)) +evmcs_touch_msr_bitmap (); +} +void vmx_disable_intercept_for_msr (struct kvm_vcpu *vcpu,u32 msr,int type) {struct vcpu_vmx *vmx =to_vmx (vcpu);
@@ -3734,8 +3745,7 @@ void vmx_disable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type)if (!cpu_has_vmx_msr_bitmap ())return ; -if (static_branch_unlikely (&enable_evmcs)) -evmcs_touch_msr_bitmap (); +vmx_msr_bitmap_l01_changed (vmx);/* * Mark the desired intercept state in shadow bitmap, this is needed
@@ -3779,8 +3789,7 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type)if (!cpu_has_vmx_msr_bitmap ())return ; -if (static_branch_unlikely (&enable_evmcs)) -evmcs_touch_msr_bitmap (); +vmx_msr_bitmap_l01_changed (vmx);/* * Mark the desired intercept state in shadow bitmap, this is needed
Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerPATCH v2 3/4 KVM: nVMX: Track whether changes in L0 require MSR bitmap for L2 to be rebuilt
In reply to Vitaly KuznetsovIntroduce a flag to keep track of whether MSR bitmap for L2 needs to be rebuilt due to changes in MSR bitmap for L1 or switching to a different L2. This information will be used for Enlightened MSR Bitmap feature for Hyper-V guests.
Note, setting msr_bitmap_changed to 'true' from set_current_vmptr() is not really needed for Enlightened MSR Bitmap as the feature can only be used in conjunction with Enlightened VMCS but let's keep tracking information complete, it's cheap and in the future similar PV feature can easily be implemented for KVM on KVM too.
No functional change intended.
--- arch/x86/kvm/vmx/nested.c | 9 ++++++++- arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/vmx/vmx.h | 6 ++++++ 3 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +8 -1
@@ -700,6 +700,8 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,kvm_vcpu_unmap (vcpu, &to_vmx (vcpu)->nested.msr_bitmap_map, false); + vmx->nested.msr_bitmap_changed = false; +return true; }
@@ -2054,10 +2056,13 @@ static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld(* Clean fields data can't be used on VMLAUNCH and when we switch * between different L2 guests as KVM keeps a single VMCS12 per L1. */ -if (from_launch || evmcs_gpa_changed) +if (from_launch || evmcs_gpa_changed) { vmx->nested.hv_evmcs->hv_clean_fields &= ~HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL ; + vmx->nested.msr_bitmap_changed = true; + } +return EVMPTRLD_SUCCEEDED ; }
@@ -5274,6 +5279,7 @@ static void set_current_vmptr(struct vcpu_vmx *vmx, gpa_t vmptr) vmx->nested.need_vmcs12_to_shadow_sync = true; } vmx->nested.dirty_vmcs12 = true; + vmx->nested.msr_bitmap_changed = true; }/* Emulate the VMPTRLD instruction */
@@ -6400,6 +6406,7 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu, goto error_guest_mode; vmx->nested.dirty_vmcs12 = true; + vmx->nested.msr_bitmap_changed = true; ret =nested_vmx_enter_non_root_mode (vcpu, false);if (ret) goto error_guest_mode;
arch/x86/kvm/vmx/vmx.c +2 -0
@@ -3735,6 +3735,8 @@ static void vmx_msr_bitmap_l01_changed(struct vcpu_vmx *vmx)*/ if (static_branch_unlikely (&enable_evmcs))evmcs_touch_msr_bitmap (); + + vmx->nested.msr_bitmap_changed = true; }void vmx_disable_intercept_for_msr (struct kvm_vcpu *vcpu,u32 msr,int type)
arch/x86/kvm/vmx/vmx.h +6 -0
@@ -148,6 +148,12 @@ struct nested_vmx {bool need_vmcs12_to_shadow_sync;bool dirty_vmcs12; +/* +* Indicates whether MSR bitmap for L2 needs to be rebuilt due to +* changes in MSR bitmap for L1 or switching to a different L2. +*/ +bool msr_bitmap_changed; +/* * Indicates lazily loaded guest state has not yet been decached from * vmcs02.
Signature · 1 line
2.31.1
Sean Christopherson
maintainerRe: [PATCH v2 3/4] KVM: nVMX: Track whether changes in L0 require MSR bitmap for L2 to be rebuilt
Quoted 15 lines · Vitaly Kuznetsov
Introduce a flag to keep track of whether MSR bitmap for L2 needs to be rebuilt due to changes in MSR bitmap for L1 or switching to a different L2. This information will be used for Enlightened MSR Bitmap feature for Hyper-V guests.
Note, setting
msr_bitmap_changedto 'true' fromset_current_vmptr()is not really needed for Enlightened MSR Bitmap as the feature can only be used in conjunction with Enlightened VMCS but let's keep tracking information complete, it's cheap and in the future similar PV feature can easily be implemented for KVM on KVM too.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>---
...
Quoted 6 lines
void
vmx_disable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type)arch/x86/kvm/vmx/vmx.h +0
-0
arch/x86/kvm/vmx/vmx.h struct nested_vmx { · line 155
Quoted 8 lines
arch/x86/kvm/vmx/vmx.h +5
-0@@ -148,6 +148,12 @@ struct nested_vmx {bool need_vmcs12_to_shadow_sync;bool dirty_vmcs12; +/* +* Indicates whether MSR bitmap for L2 needs to be rebuilt due to +* changes in MSR bitmap for L1 or switching to a different L2. +*/ +bool msr_bitmap_changed;
This is misleading, and arguably wrong. It's only accurate when used in conjuction with a paravirt L1 that states if a VMCS has a dirty MSR bitmap. E.g. this flag will be wrong if L1 changes the address of the bitmap in the VMCS, and it's obviously wrong if L1 changes the MSR bitmap itself.
The changelog kind of covers that, but those details will be completely lost to readers of the code.
Would it be illegal from KVM to simply clear the CLEAN bit in the eVMCS at the appropriate points?
Quoted 7 lines
arch/x86/kvm/vmx/vmx.h +1
-0+/* * Indicates lazily loaded guest state has not yet been decached from * vmcs02. Signature · 1 line
2.31.1
Vitaly Kuznetsov
maintainerRe: [PATCH v2 3/4] KVM: nVMX: Track whether changes in L0 require MSR bitmap for L2 to be rebuilt
Quoted 39 lines · Sean Christopherson
Quoted 15 lines · Vitaly Kuznetsov
Introduce a flag to keep track of whether MSR bitmap for L2 needs to be rebuilt due to changes in MSR bitmap for L1 or switching to a different L2. This information will be used for Enlightened MSR Bitmap feature for Hyper-V guests.
Note, setting
msr_bitmap_changedto 'true' fromset_current_vmptr()is not really needed for Enlightened MSR Bitmap as the feature can only be used in conjunction with Enlightened VMCS but let's keep tracking information complete, it's cheap and in the future similar PV feature can easily be implemented for KVM on KVM too.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>---
...
Quoted 6 lines
void
vmx_disable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type)arch/x86/kvm/vmx/vmx.h +0
-0arch/x86/kvm/vmx/vmx.h struct nested_vmx { · line 155
Quoted 8 lines
arch/x86/kvm/vmx/vmx.h +5
-0@@ -148,6 +148,12 @@ struct nested_vmx {bool need_vmcs12_to_shadow_sync;bool dirty_vmcs12; +/* +* Indicates whether MSR bitmap for L2 needs to be rebuilt due to +* changes in MSR bitmap for L1 or switching to a different L2. +*/ +bool msr_bitmap_changed;This is misleading, and arguably wrong. It's only accurate when used in conjuction with a paravirt L1 that states if a VMCS has a dirty MSR bitmap. E.g. this flag will be wrong if L1 changes the address of the bitmap in the VMCS, and it's obviously wrong if L1 changes the MSR bitmap itself.
Quoted 2 lines
The changelog kind of covers that, but those details will be completely lost to readers of the code.
Would it help if we rename 'msr_bitmap_changed' to something?
Quoted 3 lines
Would it be illegal from KVM to simply clear the CLEAN bit in the eVMCS at the appropriate points?
It would probably be OK to do that while we're in L2, however, in case we're running L1 things can get messy. E.g. MSR-bitmap for L1 is changed and we clear the clean bit in the currently mapped eVMCS for L2. Later, before L1 runs L2, it sets the bit back again indicating 'no changes in MSR-bitmap-12' and we (erroneously) skip updating MSR-Bitmap-02.
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerRe: [PATCH v2 3/4] KVM: nVMX: Track whether changes in L0 require MSR bitmap for L2 to be rebuilt
Quoted 4 lines · Vitaly Kuznetsov
Quoted 3 lines
The changelog kind of covers that, but those details will be completely lost to readers of the code.
Would it help if we rename '
msr_bitmap_changed' to something?
Yeah, what about 'msr_bitmap_force_recalc'?
Paolo
Vitaly Kuznetsov
maintainerPATCH v2 4/4 KVM: nVMX: Implement Enlightened MSR Bitmap feature
In reply to Vitaly KuznetsovUpdating MSR bitmap for L2 is not cheap and rearly needed. TLFS for Hyper-V offers 'Enlightened MSR Bitmap' feature which allows L1 hypervisor to inform L0 when it changes MSR bitmap, this eliminates the need to examine L1's MSR bitmap for L2 every time when 'real' MSR bitmap for L2 gets constructed.
Use 'vmx->nested.msr_bitmap_changed' flag to implement the feature.
Note, KVM already uses 'Enlightened MSR bitmap' feature when it runs as a nested hypervisor on top of Hyper-V. The newly introduced feature is going to be used by Hyper-V guests on KVM.
When the feature is enabled for Win10+WSL2, it shaves off around 700 CPU cycles from a nested vmexit cost (tight cpuid loop test).
--- arch/x86/kvm/hyperv.c | 2 ++ arch/x86/kvm/vmx/nested.c | 20 ++++++++++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-)
arch/x86/kvm/hyperv.c +2 -0
@@ -2516,6 +2516,8 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,case HYPERV_CPUID_NESTED_FEATURES : ent->eax = evmcs_ver; +if (evmcs_ver) + ent->eax |=HV_X64_NESTED_MSR_BITMAP ;break ;
arch/x86/kvm/vmx/nested.c +18 -2
@@ -608,15 +608,30 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12) {int msr; +struct vcpu_vmx *vmx =to_vmx (vcpu);unsigned long *msr_bitmap_l1; -unsigned long *msr_bitmap_l0 =to_vmx (vcpu)->nested.vmcs02.msr_bitmap; -struct kvm_host_map *map = &to_vmx (vcpu)->nested.msr_bitmap_map; +unsigned long *msr_bitmap_l0 = vmx->nested.vmcs02.msr_bitmap; +struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs; +struct kvm_host_map *map = &vmx->nested.msr_bitmap_map;/* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap () || !nested_cpu_has (vmcs12,CPU_BASED_USE_MSR_BITMAPS ))return false; +/* +* MSR bitmap update can be skipped when: +* - MSR bitmap for L1 hasn't changed. +* - Nested hypervisor (L1) is attempting to launch the same L2 as +* before. +* - Nested hypervisor (L1) has enabled 'Enlightened MSR Bitmap' feature +* and tells KVM (L0) there were no changes in MSR bitmap for L2. +*/ +if (!vmx->nested.msr_bitmap_changed && evmcs && + evmcs->hv_enlightenments_control.msr_bitmap && + evmcs->hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP ) + goto out_clear_msr_bitmap_changed; +if (kvm_vcpu_map (vcpu,gpa_to_gfn (vmcs12->msr_bitmap), map))return false;
@@ -700,6 +715,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,kvm_vcpu_unmap (vcpu, &to_vmx (vcpu)->nested.msr_bitmap_map, false); +out_clear_msr_bitmap_changed: vmx->nested.msr_bitmap_changed = false;return true;
Signature · 1 line
2.31.1