Skip to contentSkip to conversation

KVM: SVM: nSVM: Implement Enlightened MSR-Bitmap for Hyper-V-on-KVM and fix it for KVM-on-Hyper-V

PATCH Applied Vitaly Kuznetsov 2 participants 14 messages Dec 20, 2021 – Feb 2, 2022

Subsystems (4)

v1 of 2 Applied Latest is v2 →

lore
Share & tools
lore

Vitaly Kuznetsov

author maintainer
#0 lore raw

cover KVM: SVM: nSVM: Implement Enlightened MSR-Bitmap for Hyper-V-on-KVM and fix it for KVM-on-Hyper-V

Enlightened MSR-Bitmap feature implements a PV protocol for L0 and L1 hypervisors to collaborate and skip unneeded updates to MSR-Bitmap. KVM implements the feature for KVM-on-Hyper-V but it seems there was a flaw in the implementation and the feature may not be fully functional. PATCHes 1-2 fix the problem. The rest of the series implements the same feature for Hyper-V-on-KVM.

Vitaly Kuznetsov (5):
KVM: SVM: Drop stale comment from
svm_hv_vmcb_dirty_nested_enlightenments()
KVM: SVM: hyper-v: Enable Enlightened MSR-Bitmap support for real
KVM: nSVM: Track whether changes in L0 require MSR bitmap for L2 to be
rebuilt
KVM: x86: Make kvm_hv_hypercall_enabled() static inline
KVM: nSVM: Implement Enlightened MSR-Bitmap feature

 arch/x86/kvm/hyperv.c           | 12 +--------
 arch/x86/kvm/hyperv.h           |  6 ++++-
 arch/x86/kvm/svm/nested.c       | 47 ++++++++++++++++++++++++++++-----
 arch/x86/kvm/svm/svm.c          |  3 ++-
 arch/x86/kvm/svm/svm.h          | 16 +++++++----
 arch/x86/kvm/svm/svm_onhyperv.h | 12 +++------
 6 files changed, 63 insertions(+), 33 deletions(-)
Signature · 1 line
2.33.1
author maintainer
#1 lore raw

PATCH 1/5 KVM: SVM: Drop stale comment from svm_hv_vmcb_dirty_nested_enlightenments()

Commit 3fa5e8fd0a0e4 ("KVM: SVM: delay svm_vcpu_init_msrpm after svm->vmcb is initialized") re-arranged svm_vcpu_init_msrpm() call in svm_create_vcpu() making the comment about vmcb being NULL obsolete. Drop it.

While on it, drop superfluous vmcb_is_clean() check: vmcb_mark_dirty() is a bit flip, an extra check is unlikely to bring any performance gain. Drop now-unneeded vmcb_is_clean() helper as well.

Fixes 3fa5e8fd0a0e4 ("KVM: SVM: delay svm_vcpu_init_msrpm after svm->vmcb is initialized")
---
 arch/x86/kvm/svm/svm.h          | 5 -----
 arch/x86/kvm/svm/svm_onhyperv.h | 9 +--------
 2 files changed, 1 insertion(+), 13 deletions(-)
arch/x86/kvm/svm/svm.h +0 -5
index daa8ca84afcc..5d197aae3a19 100644
@@ -305,11 +305,6 @@ static inline void vmcb_mark_all_clean(struct vmcb *vmcb)
 			       & ~VMCB_ALWAYS_DIRTY_MASK;
 }
 
-static inline bool vmcb_is_clean(struct vmcb *vmcb, int bit)
-{
-	return (vmcb->control.clean & (1 << bit));
-}
-
 static inline void vmcb_mark_dirty(struct vmcb *vmcb, int bit)
 {
 	vmcb->control.clean &= ~(1 << bit);
arch/x86/kvm/svm/svm_onhyperv.h +1 -8
index c53b8bf8d013..cdbcfc63d171 100644
@@ -83,14 +83,7 @@ static inline void svm_hv_vmcb_dirty_nested_enlightenments(
 	struct hv_enlightenments *hve =
 		(struct hv_enlightenments *)vmcb->control.reserved_sw;
 
-	/*
-	 * vmcb can be NULL if called during early vcpu init.
-	 * And its okay not to mark vmcb dirty during vcpu init
-	 * as we mark it dirty unconditionally towards end of vcpu
-	 * init phase.
-	 */
-	if (vmcb_is_clean(vmcb, VMCB_HV_NESTED_ENLIGHTENMENTS) &&
-	    hve->hv_enlightenments_control.msr_bitmap)
+	if (hve->hv_enlightenments_control.msr_bitmap)
 		vmcb_mark_dirty(vmcb, VMCB_HV_NESTED_ENLIGHTENMENTS);
 }
 
Signature · 1 line
2.33.1
maintainer Applied
#2 lore raw

Re: [PATCH 1/5] KVM: SVM: Drop stale comment from svm_hv_vmcb_dirty_nested_enlightenments()

Quoted 11 lines · Vitaly Kuznetsov

Commit 3fa5e8fd0a0e4 ("KVM: SVM: delay svm_vcpu_init_msrpm after svm->vmcb is initialized") re-arranged svm_vcpu_init_msrpm() call in svm_create_vcpu() making the comment about vmcb being NULL obsolete. Drop it.

While on it, drop superfluous vmcb_is_clean() check: vmcb_mark_dirty() is a bit flip, an extra check is unlikely to bring any performance gain. Drop now-unneeded vmcb_is_clean() helper as well.

Fixes 3fa5e8fd0a0e4 ("KVM: SVM: delay svm_vcpu_init_msrpm after svm->vmcb is initialized")

Queued, but with subject changed to "KVM: SVM: clean up beginning of svm_hv_vmcb_dirty_nested_enlightenments()".

Paolo

Quoted 40 lines
  arch/x86/kvm/svm/svm.h          | 5 -----
  arch/x86/kvm/svm/svm_onhyperv.h | 9 +--------
  2 files changed, 1 insertion(+), 13 deletions(-)
arch/x86/kvm/svm/svm.h +0 -5
index daa8ca84afcc..5d197aae3a19 100644
@@ -305,11 +305,6 @@ static inline void vmcb_mark_all_clean(struct vmcb *vmcb)
  			       & ~VMCB_ALWAYS_DIRTY_MASK;
  }
 
-static inline bool vmcb_is_clean(struct vmcb *vmcb, int bit)
-{
-	return (vmcb->control.clean & (1 << bit));
-}
-
  static inline void vmcb_mark_dirty(struct vmcb *vmcb, int bit)
  {
  	vmcb->control.clean &= ~(1 << bit);
arch/x86/kvm/svm/svm_onhyperv.h +1 -8
index c53b8bf8d013..cdbcfc63d171 100644
@@ -83,14 +83,7 @@ static inline void svm_hv_vmcb_dirty_nested_enlightenments(
  	struct hv_enlightenments *hve =
  		(struct hv_enlightenments *)vmcb->control.reserved_sw;
 
-	/*
-	 * vmcb can be NULL if called during early vcpu init.
-	 * And its okay not to mark vmcb dirty during vcpu init
-	 * as we mark it dirty unconditionally towards end of vcpu
-	 * init phase.
-	 */
-	if (vmcb_is_clean(vmcb, VMCB_HV_NESTED_ENLIGHTENMENTS) &&
-	    hve->hv_enlightenments_control.msr_bitmap)
+	if (hve->hv_enlightenments_control.msr_bitmap)
  		vmcb_mark_dirty(vmcb, VMCB_HV_NESTED_ENLIGHTENMENTS);
  }
 
author maintainer
#3 lore raw

PATCH 2/5 KVM: SVM: hyper-v: Enable Enlightened MSR-Bitmap support for real

In reply to Vitaly Kuznetsov

Commit c4327f15dfc7 ("KVM: SVM: hyper-v: Enlightened MSR-Bitmap support") introduced enlightened MSR-Bitmap support for KVM-on-Hyper-V but it didn't actually enable the support. Similar to enlightened NPT TLB flush and direct TLB flush features, the guest (KVM) has to tell L0 (Hyper-V) that it's using the feature by setting the appropriate feature fit in VMCB control area (sw reserved fields).

Fixes c4327f15dfc7 ("KVM: SVM: hyper-v: Enlightened MSR-Bitmap support")
---
 arch/x86/kvm/svm/svm_onhyperv.h | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/svm_onhyperv.h +3 -0
index cdbcfc63d171..489ca56212c6 100644
@@ -46,6 +46,9 @@ static inline void svm_hv_init_vmcb(struct vmcb *vmcb)
 	if (npt_enabled &&
 	    ms_hyperv.nested_features & HV_X64_NESTED_ENLIGHTENED_TLB)
 		hve->hv_enlightenments_control.enlightened_npt_tlb = 1;
+
+	if (ms_hyperv.nested_features & HV_X64_NESTED_MSR_BITMAP)
+		hve->hv_enlightenments_control.msr_bitmap = 1;
 }
 
 static inline void svm_hv_hardware_setup(void)
Signature · 1 line
2.33.1
maintainer
#4 lore raw

Re: [PATCH 2/5] KVM: SVM: hyper-v: Enable Enlightened MSR-Bitmap support for real

Quoted 18 lines · Vitaly Kuznetsov

Commit c4327f15dfc7 ("KVM: SVM: hyper-v: Enlightened MSR-Bitmap support") introduced enlightened MSR-Bitmap support for KVM-on-Hyper-V but it didn't actually enable the support. Similar to enlightened NPT TLB flush and direct TLB flush features, the guest (KVM) has to tell L0 (Hyper-V) that it's using the feature by setting the appropriate feature fit in VMCB control area (sw reserved fields).

Fixes c4327f15dfc7 ("KVM: SVM: hyper-v: Enlightened MSR-Bitmap support")
---
  arch/x86/kvm/svm/svm_onhyperv.h | 3 +++
  1 file changed, 3 insertions(+)
arch/x86/kvm/svm/svm_onhyperv.h +0 -0
index cdbcfc63d171..489ca56212c6 100644

arch/x86/kvm/svm/svm_onhyperv.h static inline void svm_hv_init_vmcb(struct vmcb *vmcb) · line 54

Quoted 9 lines
arch/x86/kvm/svm/svm_onhyperv.h +3 -0
@@ -46,6 +46,9 @@ static inline void svm_hv_init_vmcb(struct vmcb *vmcb)
  	if (npt_enabled &&
  	    ms_hyperv.nested_features & HV_X64_NESTED_ENLIGHTENED_TLB)
  		hve->hv_enlightenments_control.enlightened_npt_tlb = 1;
+
+	if (ms_hyperv.nested_features & HV_X64_NESTED_MSR_BITMAP)
+		hve->hv_enlightenments_control.msr_bitmap = 1;
  }
 
  static inline void svm_hv_hardware_setup(void)

Queued this one for now.

Paolo

author maintainer
#5 lore raw

PATCH 3/5 KVM: nSVM: Track whether changes in L0 require MSR bitmap for L2 to be rebuilt

In reply to Vitaly Kuznetsov

Similar to nVMX commit ed2a4800ae9d ("KVM: nVMX: Track whether changes in L0 require MSR bitmap for L2 to be rebuilt"), introduce a flag to keep track of whether MSR bitmap for L2 needs to be rebuilt due to changes in MSR bitmap for L1 or switching to a different L2.

---
 arch/x86/kvm/svm/nested.c | 6 ++++++
 arch/x86/kvm/svm/svm.c    | 3 ++-
 arch/x86/kvm/svm/svm.h    | 9 +++++++++
 3 files changed, 17 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +6 -0
index cf206855ebf0..f27323728be2 100644
@@ -193,6 +193,8 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 		svm->nested.msrpm[p] = svm->msrpm[p] | value;
 	}
 
+	svm->nested.force_msr_bitmap_recalc = false;
+
 	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
 
 	return true;
@@ -494,6 +496,7 @@ static void nested_vmcb02_prepare_save(struct vcpu_svm *svm, struct vmcb *vmcb12
 	if (svm->nested.vmcb12_gpa != svm->nested.last_vmcb12_gpa) {
 		new_vmcb12 = true;
 		svm->nested.last_vmcb12_gpa = svm->nested.vmcb12_gpa;
+		svm->nested.force_msr_bitmap_recalc = true;
 	}
 
 	if (unlikely(new_vmcb12 || vmcb_is_dirty(vmcb12, VMCB_SEG))) {
@@ -1494,6 +1497,9 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
 	svm_switch_vmcb(svm, &svm->nested.vmcb02);
 	nested_vmcb02_prepare_control(svm);
+
+	svm->nested.force_msr_bitmap_recalc = true;
+
 	kvm_make_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu);
 	ret = 0;
 out_free:
arch/x86/kvm/svm/svm.c +2 -1
index e57e6857e063..24cae7e43316 100644
@@ -668,6 +668,7 @@ static bool msr_write_intercepted(struct kvm_vcpu *vcpu, u32 msr)
 static void set_msr_interception_bitmap(struct kvm_vcpu *vcpu, u32 *msrpm,
 					u32 msr, int read, int write)
 {
+	struct vcpu_svm *svm = to_svm(vcpu);
 	u8 bit_read, bit_write;
 	unsigned long tmp;
 	u32 offset;
@@ -698,7 +699,7 @@ static void set_msr_interception_bitmap(struct kvm_vcpu *vcpu, u32 *msrpm,
 	msrpm[offset] = tmp;
 
 	svm_hv_vmcb_dirty_nested_enlightenments(vcpu);
-
+	svm->nested.force_msr_bitmap_recalc = true;
 }
 
 void set_msr_interception(struct kvm_vcpu *vcpu, u32 *msrpm, u32 msr,
arch/x86/kvm/svm/svm.h +9 -0
index 5d197aae3a19..e5040daaa968 100644
@@ -164,6 +164,15 @@ struct svm_nested_state {
 	struct vmcb_save_area_cached save;
 
 	bool initialized;
+
+	/*
+	 * Indicates whether MSR bitmap for L2 needs to be rebuilt due to
+	 * changes in MSR bitmap for L1 or switching to a different L2. Note,
+	 * this flag can only be used reliably in conjunction with a paravirt L1
+	 * which informs L0 whether any changes to MSR bitmap for L2 were done
+	 * on its side.
+	 */
+	bool force_msr_bitmap_recalc;
 };
 
 struct vcpu_sev_es_state {
Signature · 1 line
2.33.1
author maintainer
#6 lore raw

PATCH 4/5 KVM: x86: Make kvm_hv_hypercall_enabled() static inline

In reply to Vitaly Kuznetsov

In preparation for using kvm_hv_hypercall_enabled() from SVM code, make it static inline to avoid the need to export it. The function is a simple check with only two call sites currently.

---
 arch/x86/kvm/hyperv.c | 5 -----
 arch/x86/kvm/hyperv.h | 6 +++++-
 2 files changed, 5 insertions(+), 6 deletions(-)
arch/x86/kvm/hyperv.c +0 -5
index a91424ed436d..c008522112f6 100644
@@ -2014,11 +2014,6 @@ int kvm_hv_set_enforce_cpuid(struct kvm_vcpu *vcpu, bool enforce)
 	return ret;
 }
 
-bool kvm_hv_hypercall_enabled(struct kvm_vcpu *vcpu)
-{
-	return vcpu->arch.hyperv_enabled && to_kvm_hv(vcpu->kvm)->hv_guest_os_id;
-}
-
 static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result)
 {
 	bool longmode;
arch/x86/kvm/hyperv.h +5 -1
index ed1c4e546d04..e19c00ee9ab3 100644
@@ -89,7 +89,11 @@ static inline u32 kvm_hv_get_vpindex(struct kvm_vcpu *vcpu)
 int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host);
 int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, bool host);
 
-bool kvm_hv_hypercall_enabled(struct kvm_vcpu *vcpu);
+static inline bool kvm_hv_hypercall_enabled(struct kvm_vcpu *vcpu)
+{
+	return vcpu->arch.hyperv_enabled && to_kvm_hv(vcpu->kvm)->hv_guest_os_id;
+}
+
 int kvm_hv_hypercall(struct kvm_vcpu *vcpu);
 
 void kvm_hv_irq_routing_update(struct kvm *kvm);
Signature · 1 line
2.33.1
author maintainer
#7 lore raw

PATCH 5/5 KVM: nSVM: Implement Enlightened MSR-Bitmap feature

In reply to Vitaly Kuznetsov

Similar to nVMX commit 502d2bf5f2fd ("KVM: nVMX: Implement Enlightened MSR Bitmap feature"), add support for the feature for nSVM (Hyper-V on KVM).

Notable differences from nVMX implementation:
- As the feature uses SW reserved fields in VMCB control, KVM needs to make sure it's dealing with a Hyper-V guest (kvm_hv_hypercall_enabled()).

- 'msrpm_base_pa' needs to be always be overwritten in nested_svm_vmrun_msrpm(), even when the update is skipped. As an optimization, nested_vmcb02_prepare_control() copies it from VMCB01 so when MSR-Bitmap feature for L2 is disabled nothing needs to be done.

- 'struct vmcb_ctrl_area_cached' needs to be extended with clean fields/sw reserved data and __nested_copy_vmcb_control_to_cache() needs to copy it so nested_svm_vmrun_msrpm() can use it later.

---
 arch/x86/kvm/hyperv.c     |  7 +------
 arch/x86/kvm/svm/nested.c | 41 ++++++++++++++++++++++++++++++++-------
 arch/x86/kvm/svm/svm.h    |  2 ++
 3 files changed, 37 insertions(+), 13 deletions(-)
arch/x86/kvm/hyperv.c +1 -6
index c008522112f6..18af9e67be40 100644
@@ -2409,10 +2409,6 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
 	if (kvm_x86_ops.nested_ops->get_evmcs_version)
 		evmcs_ver = kvm_x86_ops.nested_ops->get_evmcs_version(vcpu);
 
-	/* Skip NESTED_FEATURES if eVMCS is not supported */
-	if (!evmcs_ver)
-		--nent;
-
 	if (cpuid->nent < nent)
 		return -E2BIG;
 
@@ -2512,8 +2508,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
 
 		case HYPERV_CPUID_NESTED_FEATURES:
 			ent->eax = evmcs_ver;
-			if (evmcs_ver)
-				ent->eax |= HV_X64_NESTED_MSR_BITMAP;
+			ent->eax |= HV_X64_NESTED_MSR_BITMAP;
 
 			break;
 
arch/x86/kvm/svm/nested.c +34 -7

92 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +2 -0
index e5040daaa968..40b0b92d4e3e 100644
@@ -138,6 +138,8 @@ struct vmcb_ctrl_area_cached {
 	u32 event_inj_err;
 	u64 nested_cr3;
 	u64 virt_ext;
+	u32 clean;
+	u8 reserved_sw[32];
 };
 
 struct svm_nested_state {
Signature · 1 line
2.33.1
maintainer
#8 lore raw
In reply to Vitaly Kuznetsov
Quoted 16 lines · Vitaly Kuznetsov

Enlightened MSR-Bitmap feature implements a PV protocol for L0 and L1 hypervisors to collaborate and skip unneeded updates to MSR-Bitmap. KVM implements the feature for KVM-on-Hyper-V but it seems there was a flaw in the implementation and the feature may not be fully functional. PATCHes 1-2 fix the problem. The rest of the series implements the same feature for Hyper-V-on-KVM.

Vitaly Kuznetsov (5):
KVM: SVM: Drop stale comment from
svm_hv_vmcb_dirty_nested_enlightenments()
KVM: SVM: hyper-v: Enable Enlightened MSR-Bitmap support for real
KVM: nSVM: Track whether changes in L0 require MSR bitmap for L2 to be
rebuilt
KVM: x86: Make kvm_hv_hypercall_enabled() static inline
KVM: nSVM: Implement Enlightened MSR-Bitmap feature

Quoted 8 lines
  arch/x86/kvm/hyperv.c           | 12 +--------
  arch/x86/kvm/hyperv.h           |  6 ++++-
  arch/x86/kvm/svm/nested.c       | 47 ++++++++++++++++++++++++++++-----
  arch/x86/kvm/svm/svm.c          |  3 ++-
  arch/x86/kvm/svm/svm.h          | 16 +++++++----
  arch/x86/kvm/svm/svm_onhyperv.h | 12 +++------
  6 files changed, 63 insertions(+), 33 deletions(-)

Queued 3-5 now, but it would be nice to have some testcases.

Paolo

author maintainer
#9 lore raw
Quoted 26 lines · Paolo Bonzini
Quoted 16 lines · Vitaly Kuznetsov

Enlightened MSR-Bitmap feature implements a PV protocol for L0 and L1 hypervisors to collaborate and skip unneeded updates to MSR-Bitmap. KVM implements the feature for KVM-on-Hyper-V but it seems there was a flaw in the implementation and the feature may not be fully functional. PATCHes 1-2 fix the problem. The rest of the series implements the same feature for Hyper-V-on-KVM.

Vitaly Kuznetsov (5):
KVM: SVM: Drop stale comment from
svm_hv_vmcb_dirty_nested_enlightenments()
KVM: SVM: hyper-v: Enable Enlightened MSR-Bitmap support for real
KVM: nSVM: Track whether changes in L0 require MSR bitmap for L2 to be
rebuilt
KVM: x86: Make kvm_hv_hypercall_enabled() static inline
KVM: nSVM: Implement Enlightened MSR-Bitmap feature

Quoted 8 lines
  arch/x86/kvm/hyperv.c           | 12 +--------
  arch/x86/kvm/hyperv.h           |  6 ++++-
  arch/x86/kvm/svm/nested.c       | 47 ++++++++++++++++++++++++++++-----
  arch/x86/kvm/svm/svm.c          |  3 ++-
  arch/x86/kvm/svm/svm.h          | 16 +++++++----
  arch/x86/kvm/svm/svm_onhyperv.h | 12 +++------
  6 files changed, 63 insertions(+), 33 deletions(-)
Quoted 2 lines

Queued 3-5 now, but it would be nice to have some testcases.

Thanks, indeed, I'll try to draft something up, both for nVMX and nSVM.

Signature · 1 line
Vitaly
maintainer
#10 lore raw
Quoted 28 lines · Vitaly Kuznetsov
Quoted 26 lines · Paolo Bonzini
Quoted 24 lines · Vitaly Kuznetsov

Enlightened MSR-Bitmap feature implements a PV protocol for L0 and L1 hypervisors to collaborate and skip unneeded updates to MSR-Bitmap. KVM implements the feature for KVM-on-Hyper-V but it seems there was a flaw in the implementation and the feature may not be fully functional. PATCHes 1-2 fix the problem. The rest of the series implements the same feature for Hyper-V-on-KVM.

Vitaly Kuznetsov (5):
KVM: SVM: Drop stale comment from
svm_hv_vmcb_dirty_nested_enlightenments()
KVM: SVM: hyper-v: Enable Enlightened MSR-Bitmap support for real
KVM: nSVM: Track whether changes in L0 require MSR bitmap for L2 to be
rebuilt
KVM: x86: Make kvm_hv_hypercall_enabled() static inline
KVM: nSVM: Implement Enlightened MSR-Bitmap feature

   arch/x86/kvm/hyperv.c           | 12 +--------
   arch/x86/kvm/hyperv.h           |  6 ++++-
   arch/x86/kvm/svm/nested.c       | 47 ++++++++++++++++++++++++++++-----
   arch/x86/kvm/svm/svm.c          |  3 ++-
   arch/x86/kvm/svm/svm.h          | 16 +++++++----
   arch/x86/kvm/svm/svm_onhyperv.h | 12 +++------
   6 files changed, 63 insertions(+), 33 deletions(-)
Quoted 1 line
Quoted 1 line

Queued 3-5 now, but it would be nice to have some testcases.

Hmm, it fails to compile with CONFIG_HYPERV disabled, and a trivial #if also fails due to an unused goto label. Does this look good to you?

arch/x86/kvm/svm/nested.c +9 -8
index e3759a79d39a..a2b5267b3e73 100644
@@ -173,9 +173,16 @@ void recalc_intercepts(struct vcpu_svm *svm)
  */
 static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 {
+	int i;
+
+	if (!(vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT)))
+		return true;
+
+	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
+
+#if IS_ENABLED(CONFIG_HYPERV)
 	struct hv_enlightenments *hve =
 		(struct hv_enlightenments *)svm->nested.ctl.reserved_sw;
-	int i;
 
 	/*
 	 * MSR bitmap update can be skipped when:
@@ -185,10 +192,8 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 	    kvm_hv_hypercall_enabled(&svm->vcpu) &&
 	    hve->hv_enlightenments_control.msr_bitmap &&
 	    (svm->nested.ctl.clean & VMCB_HV_NESTED_ENLIGHTENMENTS))
-		goto set_msrpm_base_pa;
-
-	if (!(vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT)))
 		return true;
+#endif
 
 	for (i = 0; i < MSRPM_OFFSETS; i++) {
 		u32 value, p;
@@ -213,10 +216,6 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
 	}
 
 	svm->nested.force_msr_bitmap_recalc = false;
-
-set_msrpm_base_pa:
-	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
-
 	return true;
 }
 

Thanks,

Paolo

author maintainer
#11 lore raw
Quoted 29 lines · Paolo Bonzini
Quoted 28 lines · Vitaly Kuznetsov
Quoted 26 lines · Paolo Bonzini
Quoted 24 lines · Vitaly Kuznetsov

Enlightened MSR-Bitmap feature implements a PV protocol for L0 and L1 hypervisors to collaborate and skip unneeded updates to MSR-Bitmap. KVM implements the feature for KVM-on-Hyper-V but it seems there was a flaw in the implementation and the feature may not be fully functional. PATCHes 1-2 fix the problem. The rest of the series implements the same feature for Hyper-V-on-KVM.

Vitaly Kuznetsov (5):
KVM: SVM: Drop stale comment from
svm_hv_vmcb_dirty_nested_enlightenments()
KVM: SVM: hyper-v: Enable Enlightened MSR-Bitmap support for real
KVM: nSVM: Track whether changes in L0 require MSR bitmap for L2 to be
rebuilt
KVM: x86: Make kvm_hv_hypercall_enabled() static inline
KVM: nSVM: Implement Enlightened MSR-Bitmap feature

   arch/x86/kvm/hyperv.c           | 12 +--------
   arch/x86/kvm/hyperv.h           |  6 ++++-
   arch/x86/kvm/svm/nested.c       | 47 ++++++++++++++++++++++++++++-----
   arch/x86/kvm/svm/svm.c          |  3 ++-
   arch/x86/kvm/svm/svm.h          | 16 +++++++----
   arch/x86/kvm/svm/svm_onhyperv.h | 12 +++------
   6 files changed, 63 insertions(+), 33 deletions(-)
Quoted 5 lines
Quoted 1 line
Quoted 1 line

Queued 3-5 now, but it would be nice to have some testcases.

Hmm, it fails to compile with CONFIG_HYPERV disabled, and a trivial #if also fails due to an unused goto label. Does this look good to you?

Hm, it does but honestly I did not anticipate this dependency -- CONFIG_HYPERV is needed for KVM-on-Hyper-V but this feature is for Hyper-V-on-KVM. Let me take a look tomorrow.

Quoted 51 lines
arch/x86/kvm/svm/nested.c +9 -8
index e3759a79d39a..a2b5267b3e73 100644
@@ -173,9 +173,16 @@ void recalc_intercepts(struct vcpu_svm *svm)
   */
  static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
  {
+	int i;
+
+	if (!(vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT)))
+		return true;
+
+	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
+
+#if IS_ENABLED(CONFIG_HYPERV)
  	struct hv_enlightenments *hve =
  		(struct hv_enlightenments *)svm->nested.ctl.reserved_sw;
-	int i;
 
  	/*
  	 * MSR bitmap update can be skipped when:
@@ -185,10 +192,8 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
  	    kvm_hv_hypercall_enabled(&svm->vcpu) &&
  	    hve->hv_enlightenments_control.msr_bitmap &&
  	    (svm->nested.ctl.clean & VMCB_HV_NESTED_ENLIGHTENMENTS))
-		goto set_msrpm_base_pa;
-
-	if (!(vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT)))
  		return true;
+#endif
 
  	for (i = 0; i < MSRPM_OFFSETS; i++) {
  		u32 value, p;
@@ -213,10 +216,6 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)
  	}
 
  	svm->nested.force_msr_bitmap_recalc = false;
-
-set_msrpm_base_pa:
-	svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm));
-
  	return true;
  }
 

Thanks,

Paolo

Signature · 1 line
Vitaly
maintainer
#12 lore raw
Quoted 7 lines · Vitaly Kuznetsov
Quoted 3 lines

Hmm, it fails to compile with CONFIG_HYPERV disabled, and a trivial #if also fails due to an unused goto label. Does this look good to you?

Hm, it does but honestly I did not anticipate this dependency -- CONFIG_HYPERV is needed for KVM-on-Hyper-V but this feature is for Hyper-V-on-KVM. Let me take a look tomorrow.

It's because, without it, the relevant structs are not defined by svm_onhyperv.h. Go ahead and send a new version if you prefer, I can unqueue it (really, just not push to kvm/queue).

Paolo

author maintainer
#13 lore raw
Quoted 11 lines · Paolo Bonzini
Quoted 7 lines · Vitaly Kuznetsov
Quoted 3 lines

Hmm, it fails to compile with CONFIG_HYPERV disabled, and a trivial #if also fails due to an unused goto label. Does this look good to you?

Hm, it does but honestly I did not anticipate this dependency -- CONFIG_HYPERV is needed for KVM-on-Hyper-V but this feature is for Hyper-V-on-KVM. Let me take a look tomorrow.

It's because, without it, the relevant structs are not defined by svm_onhyperv.h. Go ahead and send a new version if you prefer

Should be fixed in v2. I still think it makes sense to keep this KVM-on-Hyper-V and Hyper-V-on-KVM separation as it's really confusing to an unprepared reader. There's still room for improvement in nVMX I believe but it's orthogonal to this nSVM feature.

Signature · 1 line
Vitaly