Skip to contentSkip to conversation

KVM: SVM: A fix and cleanups for VMCB intercepts

PATCH Applied Sean Christopherson 2 participants 15 messages Feb 18 – Mar 5

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v2 cover KVM: SVM: A fix and cleanups for VMCB intercepts

Fix a likely-benign bug where KVM fails to mark vmcb01 intercepts as dirty after recalculating intercepts while L2 is active, then do a bunch of related cleanup, e.g. to split recalc_intercepts() into nested vs. non-nested functionality.

v2:
- Fix the aforementioned bug.
- Split recalc_intercepts() instead of simply renaming it.
- Move the new WARN in nested_vmcb02_recalc_intercepts() to its own patch.
- Use less weird local variables even if they aren't consistent with the existing code...
- ... and then change some names in the existing code to provide consistency.

v1: https://lkml.kernel.org/r/20260112182022.771276-1-yosry.ahmed%40linux.dev

Sean Christopherson (6):
KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts
KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested
parts
KVM: nSVM: Directly (re)calc vmcb02 intercepts from
nested_vmcb02_prepare_control()
KVM: nSVM: Use intuitive local variables in
nested_vmcb02_recalc_intercepts()
KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state
KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing

    vmcb02

Yosry Ahmed (2):
KVM: nSVM: WARN and abort vmcb02 intercepts recalc if vmcb02 isn't
active
KVM: nSVM: Use vmcb12_is_intercept() in
nested_sync_control_from_vmcb02()

 arch/x86/kvm/svm/nested.c | 88 +++++++++++++++++++--------------------
 arch/x86/kvm/svm/sev.c    |  2 +-
 arch/x86/kvm/svm/svm.c    |  6 +--
 arch/x86/kvm/svm/svm.h    | 28 +++++++++----
 4 files changed, 67 insertions(+), 57 deletions(-)

base-commit: 183bb0ce8c77b0fd1fb25874112bc8751a461e49

Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
author maintainer
#1 lore raw

PATCH v2 1/8 KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts

When reacting to an intercept update, explicitly mark vmcb01's intercepts dirty, as KVM always initially operates on vmcb01, and nested_svm_vmexit() isn't guaranteed to mark VMCB_INTERCEPTS as dirty. I.e. if L2 is active, KVM will modify the intercepts for L1, but might not mark them as dirty before the next VMRUN of L1.

Fixes 116a0a23676e ("KVM: SVM: Add clean-bit for intercetps, tsc-offset and pause filter count")
---
 arch/x86/kvm/svm/nested.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +3 -1
index de90b104a0dd..66701106a51b 100644
@@ -128,11 +128,13 @@ void recalc_intercepts(struct vcpu_svm *svm)
 	struct vmcb_ctrl_area_cached *g;
 	unsigned int i;
 
-	vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
+	vmcb_mark_dirty(svm->vmcb01.ptr, VMCB_INTERCEPTS);
 
 	if (!is_guest_mode(&svm->vcpu))
 		return;
 
+	vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
+
 	c = &svm->vmcb->control;
 	h = &svm->vmcb01.ptr->control;
 	g = &svm->nested.ctl;
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Reviewed-by on v2 1/8
#2 lore raw

Re: [PATCH v2 1/8] KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts

Quoted 9 lines · Sean Christopherson

When reacting to an intercept update, explicitly mark vmcb01's intercepts dirty, as KVM always initially operates on vmcb01, and nested_svm_vmexit() isn't guaranteed to mark VMCB_INTERCEPTS as dirty. I.e. if L2 is active, KVM will modify the intercepts for L1, but might not mark them as dirty before the next VMRUN of L1.

Fixes 116a0a23676e ("KVM: SVM: Add clean-bit for intercetps, tsc-offset and pause filter count")
Quoted 26 lines
---
 arch/x86/kvm/svm/nested.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +3 -1
index de90b104a0dd..66701106a51b 100644
@@ -128,11 +128,13 @@ void recalc_intercepts(struct vcpu_svm *svm)
 	struct vmcb_ctrl_area_cached *g;
 	unsigned int i;
 
-	vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
+	vmcb_mark_dirty(svm->vmcb01.ptr, VMCB_INTERCEPTS);
 
 	if (!is_guest_mode(&svm->vcpu))
 		return;
 
+	vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
+
 	c = &svm->vmcb->control;
 	h = &svm->vmcb01.ptr->control;
 	g = &svm->nested.ctl;
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
author maintainer
#3 lore raw

PATCH v2 2/8 KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested parts

In reply to Sean Christopherson

Extract the non-nested aspects of recalc_intercepts() into a separate helper, svm_mark_intercepts_dirty(), to make it clear that the call isn't *just* recalculating (vmcb02's) intercepts, and to not bury non-nested code in nested.c.

As suggested by Yosry, opportunistically prepend "nested_vmbc02_" to recalc_intercepts() so that it's obvious the function specifically deals with recomputing intercepts for L2.

No functional change intended.

---
 arch/x86/kvm/svm/nested.c |  9 ++-------
 arch/x86/kvm/svm/sev.c    |  2 +-
 arch/x86/kvm/svm/svm.c    |  4 ++--
 arch/x86/kvm/svm/svm.h    | 26 ++++++++++++++++++++------
 4 files changed, 25 insertions(+), 16 deletions(-)
arch/x86/kvm/svm/nested.c +2 -7
index 66701106a51b..48b60dd6e7a3 100644
@@ -122,17 +122,12 @@ static bool nested_vmcb_needs_vls_intercept(struct vcpu_svm *svm)
 	return false;
 }
 
-void recalc_intercepts(struct vcpu_svm *svm)
+void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
 {
 	struct vmcb_control_area *c, *h;
 	struct vmcb_ctrl_area_cached *g;
 	unsigned int i;
 
-	vmcb_mark_dirty(svm->vmcb01.ptr, VMCB_INTERCEPTS);
-
-	if (!is_guest_mode(&svm->vcpu))
-		return;
-
 	vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
 
 	c = &svm->vmcb->control;
@@ -918,7 +913,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	 * Merge guest and host intercepts - must be called with vcpu in
 	 * guest-mode to take effect.
 	 */
-	recalc_intercepts(svm);
+	svm_mark_intercepts_dirty(svm);
 }
 
 static void nested_svm_copy_common_state(struct vmcb *from_vmcb, struct vmcb *to_vmcb)
arch/x86/kvm/svm/sev.c +1 -1
index ea515cf41168..03b6dc75a6e8 100644
@@ -4639,7 +4639,7 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm, bool init_event)
 	if (!sev_vcpu_has_debug_swap(svm)) {
 		vmcb_set_intercept(&vmcb->control, INTERCEPT_DR7_READ);
 		vmcb_set_intercept(&vmcb->control, INTERCEPT_DR7_WRITE);
-		recalc_intercepts(svm);
+		svm_mark_intercepts_dirty(svm);
 	} else {
 		/*
 		 * Disable #DB intercept iff DebugSwap is enabled.  KVM doesn't
arch/x86/kvm/svm/svm.c +2 -2
index 8f8bc863e214..9e76bf1671da 100644
@@ -635,7 +635,7 @@ static void set_dr_intercepts(struct vcpu_svm *svm)
 	vmcb_set_intercept(&vmcb->control, INTERCEPT_DR7_READ);
 	vmcb_set_intercept(&vmcb->control, INTERCEPT_DR7_WRITE);
 
-	recalc_intercepts(svm);
+	svm_mark_intercepts_dirty(svm);
 }
 
 static void clr_dr_intercepts(struct vcpu_svm *svm)
@@ -644,7 +644,7 @@ static void clr_dr_intercepts(struct vcpu_svm *svm)
 
 	vmcb->control.intercepts[INTERCEPT_DR] = 0;
 
-	recalc_intercepts(svm);
+	svm_mark_intercepts_dirty(svm);
 }
 
 static bool msr_write_intercepted(struct kvm_vcpu *vcpu, u32 msr)
arch/x86/kvm/svm/svm.h +20 -6
index ebd7b36b1ceb..92a1691dc7be 100644
@@ -357,8 +357,6 @@ struct svm_cpu_data {
 
 DECLARE_PER_CPU(struct svm_cpu_data, svm_data);
 
-void recalc_intercepts(struct vcpu_svm *svm);
-
 static __always_inline struct kvm_svm *to_kvm_svm(struct kvm *kvm)
 {
 	return container_of(kvm, struct kvm_svm, kvm);
@@ -486,6 +484,22 @@ static inline bool vmcb12_is_intercept(struct vmcb_ctrl_area_cached *control, u3
 	return __vmcb_is_intercept((unsigned long *)&control->intercepts, bit);
 }
 
+void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm);
+
+static inline void svm_mark_intercepts_dirty(struct vcpu_svm *svm)
+{
+	vmcb_mark_dirty(svm->vmcb01.ptr, VMCB_INTERCEPTS);
+
+	/*
+	 * If L2 is active, recalculate the intercepts for vmcb02 to account
+	 * for the changes made to vmcb01.  All intercept configuration is done
+	 * for vmcb01 and then propagated to vmcb02 to combine KVM's intercepts
+	 * with L1's intercepts (from the vmcb12 snapshot).
+	 */
+	if (is_guest_mode(&svm->vcpu))
+		nested_vmcb02_recalc_intercepts(svm);
+}
+
 static inline void set_exception_intercept(struct vcpu_svm *svm, u32 bit)
 {
 	struct vmcb *vmcb = svm->vmcb01.ptr;
@@ -493,7 +507,7 @@ static inline void set_exception_intercept(struct vcpu_svm *svm, u32 bit)
 	WARN_ON_ONCE(bit >= 32);
 	vmcb_set_intercept(&vmcb->control, INTERCEPT_EXCEPTION_OFFSET + bit);
 
-	recalc_intercepts(svm);
+	svm_mark_intercepts_dirty(svm);
 }
 
 static inline void clr_exception_intercept(struct vcpu_svm *svm, u32 bit)
@@ -503,7 +517,7 @@ static inline void clr_exception_intercept(struct vcpu_svm *svm, u32 bit)
 	WARN_ON_ONCE(bit >= 32);
 	vmcb_clr_intercept(&vmcb->control, INTERCEPT_EXCEPTION_OFFSET + bit);
 
-	recalc_intercepts(svm);
+	svm_mark_intercepts_dirty(svm);
 }
 
 static inline void svm_set_intercept(struct vcpu_svm *svm, int bit)
@@ -512,7 +526,7 @@ static inline void svm_set_intercept(struct vcpu_svm *svm, int bit)
 
 	vmcb_set_intercept(&vmcb->control, bit);
 
-	recalc_intercepts(svm);
+	svm_mark_intercepts_dirty(svm);
 }
 
 static inline void svm_clr_intercept(struct vcpu_svm *svm, int bit)
@@ -521,7 +535,7 @@ static inline void svm_clr_intercept(struct vcpu_svm *svm, int bit)
 
 	vmcb_clr_intercept(&vmcb->control, bit);
 
-	recalc_intercepts(svm);
+	svm_mark_intercepts_dirty(svm);
 }
 
 static inline bool svm_is_intercept(struct vcpu_svm *svm, int bit)
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Reviewed-by on v2 2/8
#4 lore raw

Re: [PATCH v2 2/8] KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested parts

Quoted 5 lines · Sean Christopherson

Extract the non-nested aspects of recalc_intercepts() into a separate helper, svm_mark_intercepts_dirty(), to make it clear that the call isn't *just* recalculating (vmcb02's) intercepts, and to not bury non-nested code in nested.c.

Quoted 8 lines

As suggested by Yosry, opportunistically prepend "nested_vmbc02_" to recalc_intercepts() so that it's obvious the function specifically deals with recomputing intercepts for L2.

No functional change intended.

author maintainer
#5 lore raw

PATCH v2 3/8 KVM: nSVM: WARN and abort vmcb02 intercepts recalc if vmcb02 isn't active

In reply to Sean Christopherson

From: Yosry Ahmed <[email protected]>

WARN and bail early from nested_vmcb02_recalc_intercepts() if vmcb02 isn't the active/current VMCB, as recalculating intercepts for vmcb01 using logic intended for merging vmcb12 and vmcb01 intercepts can yield unexpected and unwanted results.

In addition to hardening against general bugs, this will provide additional safeguards "if" nested_vmcb02_recalc_intercepts() is invoked directly from nested_vmcb02_prepare_control().

[sean: split to separate patch, bail early on "failure"]
---
 arch/x86/kvm/svm/nested.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/nested.c +3 -0
index 48b60dd6e7a3..793f5d2eed3a 100644
@@ -128,6 +128,9 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
 	struct vmcb_ctrl_area_cached *g;
 	unsigned int i;
 
+	if (WARN_ON_ONCE(svm->vmcb != svm->nested.vmcb02.ptr))
+		return;
+
 	vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
 
 	c = &svm->vmcb->control;
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
author maintainer
#6 lore raw

PATCH v2 4/8 KVM: nSVM: Directly (re)calc vmcb02 intercepts from nested_vmcb02_prepare_control()

In reply to Sean Christopherson

Now that nested_vmcb02_recalc_intercepts() provides guardrails against it being incorrectly called without vmcb02 active, invoke it directly from nested_vmcb02_recalc_intercepts() instead of bouncing through svm_mark_intercepts_dirty(), which unnecessarily marks vmcb01 as dirty.

---
 arch/x86/kvm/svm/nested.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +1 -1
index 793f5d2eed3a..e8512de5aef7 100644
@@ -916,7 +916,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	 * Merge guest and host intercepts - must be called with vcpu in
 	 * guest-mode to take effect.
 	 */
-	svm_mark_intercepts_dirty(svm);
+	nested_vmcb02_recalc_intercepts(svm);
 }
 
 static void nested_svm_copy_common_state(struct vmcb *from_vmcb, struct vmcb *to_vmcb)
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Reviewed-by on v2 4/8
#7 lore raw

Re: [PATCH v2 4/8] KVM: nSVM: Directly (re)calc vmcb02 intercepts from nested_vmcb02_prepare_control()

Quoted 6 lines · Sean Christopherson

Now that nested_vmcb02_recalc_intercepts() provides guardrails against it being incorrectly called without vmcb02 active, invoke it directly from nested_vmcb02_recalc_intercepts() instead of bouncing through svm_mark_intercepts_dirty(), which unnecessarily marks vmcb01 as dirty.

Quoted 20 lines
---
 arch/x86/kvm/svm/nested.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +1 -1
index 793f5d2eed3a..e8512de5aef7 100644
@@ -916,7 +916,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	 * Merge guest and host intercepts - must be called with vcpu in
 	 * guest-mode to take effect.
 	 */
-	svm_mark_intercepts_dirty(svm);
+	nested_vmcb02_recalc_intercepts(svm);
 }
 
 static void nested_svm_copy_common_state(struct vmcb *from_vmcb, struct vmcb *to_vmcb)
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
author maintainer
#8 lore raw

PATCH v2 5/8 KVM: nSVM: Use intuitive local variables in nested_vmcb02_recalc_intercepts()

In reply to Sean Christopherson

Now that nested_vmcb02_recalc_intercepts() is explicitly scoped to deal with *only* recalculating vmcb02 intercepts, rename its local variables to use more intuivite names. The current "c", "h", and "g" local variables, for the current VMCB, vmcb01, and (cached) vmcb12 respectively, are short and sweet, but don't do much to help unfamiliar readers understand what the code is doing.

Use vmcb12_ctrl/vmcb01/vmcb02/vmcb12_ctrl in lieu of c/h/g to make it clear the function is updating intercepts in vmcb02 based on the intercepts in vmcb01 and (cached) vmcb12.

Opportunistically change the existing WARN_ON to a WARN_ON_ONCE so that a KVM bug doesn't unintentionally DoS the host.

No functional change intended.

[sean: use WARN_ON_ONCE, keep local vmcb12 cache as vmcb12_ctrl]
---
 arch/x86/kvm/svm/nested.c | 35 ++++++++++++++++-------------------
 1 file changed, 16 insertions(+), 19 deletions(-)
arch/x86/kvm/svm/nested.c +16 -19
index e8512de5aef7..bda2d6d613c9 100644
@@ -124,23 +124,20 @@ static bool nested_vmcb_needs_vls_intercept(struct vcpu_svm *svm)
 
 void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
 {
-	struct vmcb_control_area *c, *h;
-	struct vmcb_ctrl_area_cached *g;
+	struct vmcb_ctrl_area_cached *vmcb12_ctrl = &svm->nested.ctl;
+	struct vmcb *vmcb02 = svm->nested.vmcb02.ptr;
+	struct vmcb *vmcb01 = svm->vmcb01.ptr;
 	unsigned int i;
 
-	if (WARN_ON_ONCE(svm->vmcb != svm->nested.vmcb02.ptr))
+	if (WARN_ON_ONCE(svm->vmcb != vmcb02))
 		return;
 
-	vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
-
-	c = &svm->vmcb->control;
-	h = &svm->vmcb01.ptr->control;
-	g = &svm->nested.ctl;
+	vmcb_mark_dirty(vmcb02, VMCB_INTERCEPTS);
 
 	for (i = 0; i < MAX_INTERCEPT; i++)
-		c->intercepts[i] = h->intercepts[i];
+		vmcb02->control.intercepts[i] = vmcb01->control.intercepts[i];
 
-	if (g->int_ctl & V_INTR_MASKING_MASK) {
+	if (vmcb12_ctrl->int_ctl & V_INTR_MASKING_MASK) {
 		/*
 		 * If L2 is active and V_INTR_MASKING is enabled in vmcb12,
 		 * disable intercept of CR8 writes as L2's CR8 does not affect
@@ -151,9 +148,9 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
 		 * the effective RFLAGS.IF for L1 interrupts will never be set
 		 * while L2 is running (L2's RFLAGS.IF doesn't affect L1 IRQs).
 		 */
-		vmcb_clr_intercept(c, INTERCEPT_CR8_WRITE);
-		if (!(svm->vmcb01.ptr->save.rflags & X86_EFLAGS_IF))
-			vmcb_clr_intercept(c, INTERCEPT_VINTR);
+		vmcb_clr_intercept(&vmcb02->control, INTERCEPT_CR8_WRITE);
+		if (!(vmcb01->save.rflags & X86_EFLAGS_IF))
+			vmcb_clr_intercept(&vmcb02->control, INTERCEPT_VINTR);
 	}
 
 	/*
@@ -161,14 +158,14 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
 	 * flush feature is enabled.
 	 */
 	if (!nested_svm_l2_tlb_flush_enabled(&svm->vcpu))
-		vmcb_clr_intercept(c, INTERCEPT_VMMCALL);
+		vmcb_clr_intercept(&vmcb02->control, INTERCEPT_VMMCALL);
 
 	for (i = 0; i < MAX_INTERCEPT; i++)
-		c->intercepts[i] |= g->intercepts[i];
+		vmcb02->control.intercepts[i] |= vmcb12_ctrl->intercepts[i];
 
 	/* If SMI is not intercepted, ignore guest SMI intercept as well  */
 	if (!intercept_smi)
-		vmcb_clr_intercept(c, INTERCEPT_SMI);
+		vmcb_clr_intercept(&vmcb02->control, INTERCEPT_SMI);
 
 	if (nested_vmcb_needs_vls_intercept(svm)) {
 		/*
@@ -176,10 +173,10 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
 		 * we must intercept these instructions to correctly
 		 * emulate them in case L1 doesn't intercept them.
 		 */
-		vmcb_set_intercept(c, INTERCEPT_VMLOAD);
-		vmcb_set_intercept(c, INTERCEPT_VMSAVE);
+		vmcb_set_intercept(&vmcb02->control, INTERCEPT_VMLOAD);
+		vmcb_set_intercept(&vmcb02->control, INTERCEPT_VMSAVE);
 	} else {
-		WARN_ON(!(c->virt_ext & VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK));
+		WARN_ON_ONCE(!(vmcb02->control.virt_ext & VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK));
 	}
 }
 
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
author maintainer
#9 lore raw

PATCH v2 6/8 KVM: nSVM: Use vmcb12_is_intercept() in nested_sync_control_from_vmcb02()

In reply to Sean Christopherson

From: Yosry Ahmed <[email protected]>

Use vmcb12_is_intercept() instead of open-coding the intercept check.

No functional change intended.

---
 arch/x86/kvm/svm/nested.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +1 -1
index bda2d6d613c9..bbb8dfc9979b 100644
@@ -530,7 +530,7 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm)
 	 * int_ctl (because it was never recognized while L2 was running).
 	 */
 	if (svm_is_intercept(svm, INTERCEPT_VINTR) &&
-	    !test_bit(INTERCEPT_VINTR, (unsigned long *)svm->nested.ctl.intercepts))
+	    !vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_VINTR))
 		mask &= ~V_IRQ_MASK;
 
 	if (nested_vgif_enabled(svm))
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
author maintainer
#10 lore raw

PATCH v2 7/8 KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state

In reply to Sean Christopherson

Move "bus_lock_rip" from "vmcb_ctrl_area_cached" to "svm_nested_state" as "last_bus_lock_rip" to more accurately reflect what it tracks, and because it is NOT a cached vmcb12 control field. The misplaced field isn't all that apparent in the current code base, as KVM uses "svm->nested.ctl" broadly, but the bad placement becomes glaringly obvious if "svm->nested.ctl" is captured as a local "vmcb12_ctrl" variable.

No functional change intended.

---
 arch/x86/kvm/svm/nested.c | 8 ++++----
 arch/x86/kvm/svm/svm.c    | 2 +-
 arch/x86/kvm/svm/svm.h    | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/nested.c +4 -4
index bbb8dfc9979b..bcd6304f3c0c 100644
@@ -806,7 +806,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	 * L1 re-enters L2, the same instruction will trigger a VM-Exit and the
 	 * entire cycle start over.
 	 */
-	if (vmcb02->save.rip && (svm->nested.ctl.bus_lock_rip == vmcb02->save.rip))
+	if (vmcb02->save.rip && (svm->nested.last_bus_lock_rip == vmcb02->save.rip))
 		vmcb02->control.bus_lock_counter = 1;
 	else
 		vmcb02->control.bus_lock_counter = 0;
@@ -1191,11 +1191,11 @@ int nested_svm_vmexit(struct vcpu_svm *svm)
 	}
 
 	/*
-	 * Invalidate bus_lock_rip unless KVM is still waiting for the guest
-	 * to make forward progress before re-enabling bus lock detection.
+	 * Invalidate last_bus_lock_rip unless KVM is still waiting for the
+	 * guest to make forward progress before re-enabling bus lock detection.
 	 */
 	if (!vmcb02->control.bus_lock_counter)
-		svm->nested.ctl.bus_lock_rip = INVALID_GPA;
+		svm->nested.last_bus_lock_rip = INVALID_GPA;
 
 	nested_svm_copy_common_state(svm->nested.vmcb02.ptr, svm->vmcb01.ptr);
 
arch/x86/kvm/svm/svm.c +1 -1
index 9e76bf1671da..7c832a0decc2 100644
@@ -3223,7 +3223,7 @@ static int bus_lock_exit(struct kvm_vcpu *vcpu)
 	vcpu->arch.complete_userspace_io = complete_userspace_buslock;
 
 	if (is_guest_mode(vcpu))
-		svm->nested.ctl.bus_lock_rip = vcpu->arch.cui_linear_rip;
+		svm->nested.last_bus_lock_rip = vcpu->arch.cui_linear_rip;
 
 	return 0;
 }
arch/x86/kvm/svm/svm.h +1 -1
index 92a1691dc7be..c4ed1be38ceb 100644
@@ -173,7 +173,6 @@ struct vmcb_ctrl_area_cached {
 	u64 nested_cr3;
 	u64 virt_ext;
 	u32 clean;
-	u64 bus_lock_rip;
 	union {
 #if IS_ENABLED(CONFIG_HYPERV) || IS_ENABLED(CONFIG_KVM_HYPERV)
 		struct hv_vmcb_enlightenments hv_enlightenments;
@@ -188,6 +187,7 @@ struct svm_nested_state {
 	u64 vm_cr_msr;
 	u64 vmcb12_gpa;
 	u64 last_vmcb12_gpa;
+	u64 last_bus_lock_rip;
 
 	/*
 	 * The MSR permissions map used for vmcb02, which is the merge result
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Reviewed-by on v2 7/8
#11 lore raw

Re: [PATCH v2 7/8] KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state

Quoted 10 lines · Sean Christopherson

Move "bus_lock_rip" from "vmcb_ctrl_area_cached" to "svm_nested_state" as "last_bus_lock_rip" to more accurately reflect what it tracks, and because it is NOT a cached vmcb12 control field. The misplaced field isn't all that apparent in the current code base, as KVM uses "svm->nested.ctl" broadly, but the bad placement becomes glaringly obvious if "svm->nested.ctl" is captured as a local "vmcb12_ctrl" variable.

No functional change intended.

author maintainer
#12 lore raw

PATCH v2 8/8 KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing vmcb02

In reply to Sean Christopherson

Grab svm->nested.ctl as vmcb12_ctrl when preparing the vmcb02 controls to make it more obvious that much of the data is coming from vmcb12 (or rather, a snapshot of vmcb12 at the time of L1's VMRUN).

Opportunistically reorder the variable definitions to create a pretty reverse fir tree.

No functional change intended.

---
 arch/x86/kvm/svm/nested.c | 39 +++++++++++++++++++--------------------
 1 file changed, 19 insertions(+), 20 deletions(-)
arch/x86/kvm/svm/nested.c +19 -20
index bcd6304f3c0c..1814522db6b4 100644
@@ -745,11 +745,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	u32 int_ctl_vmcb01_bits = V_INTR_MASKING_MASK;
 	u32 int_ctl_vmcb12_bits = V_TPR_MASK | V_IRQ_INJECTION_BITS_MASK;
 
-	struct kvm_vcpu *vcpu = &svm->vcpu;
-	struct vmcb *vmcb01 = svm->vmcb01.ptr;
+	struct vmcb_ctrl_area_cached *vmcb12_ctrl = &svm->nested.ctl;
 	struct vmcb *vmcb02 = svm->nested.vmcb02.ptr;
-	u32 pause_count12;
-	u32 pause_thresh12;
+	struct vmcb *vmcb01 = svm->vmcb01.ptr;
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	u32 pause_count12, pause_thresh12;
 
 	nested_svm_transition_tlb_flush(vcpu);
 
@@ -762,7 +762,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	 */
 
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_VGIF) &&
-	    (svm->nested.ctl.int_ctl & V_GIF_ENABLE_MASK))
+	    (vmcb12_ctrl->int_ctl & V_GIF_ENABLE_MASK))
 		int_ctl_vmcb12_bits |= (V_GIF_MASK | V_GIF_ENABLE_MASK);
 	else
 		int_ctl_vmcb01_bits |= (V_GIF_MASK | V_GIF_ENABLE_MASK);
@@ -820,10 +820,9 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	if (nested_npt_enabled(svm))
 		nested_svm_init_mmu_context(vcpu);
 
-	vcpu->arch.tsc_offset = kvm_calc_nested_tsc_offset(
-			vcpu->arch.l1_tsc_offset,
-			svm->nested.ctl.tsc_offset,
-			svm->tsc_ratio_msr);
+	vcpu->arch.tsc_offset = kvm_calc_nested_tsc_offset(vcpu->arch.l1_tsc_offset,
+							   vmcb12_ctrl->tsc_offset,
+							   svm->tsc_ratio_msr);
 
 	vmcb02->control.tsc_offset = vcpu->arch.tsc_offset;
 
@@ -832,13 +831,13 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 		nested_svm_update_tsc_ratio_msr(vcpu);
 
 	vmcb02->control.int_ctl             =
-		(svm->nested.ctl.int_ctl & int_ctl_vmcb12_bits) |
+		(vmcb12_ctrl->int_ctl & int_ctl_vmcb12_bits) |
 		(vmcb01->control.int_ctl & int_ctl_vmcb01_bits);
 
-	vmcb02->control.int_vector          = svm->nested.ctl.int_vector;
-	vmcb02->control.int_state           = svm->nested.ctl.int_state;
-	vmcb02->control.event_inj           = svm->nested.ctl.event_inj;
-	vmcb02->control.event_inj_err       = svm->nested.ctl.event_inj_err;
+	vmcb02->control.int_vector          = vmcb12_ctrl->int_vector;
+	vmcb02->control.int_state           = vmcb12_ctrl->int_state;
+	vmcb02->control.event_inj           = vmcb12_ctrl->event_inj;
+	vmcb02->control.event_inj_err       = vmcb12_ctrl->event_inj_err;
 
 	/*
 	 * next_rip is consumed on VMRUN as the return address pushed on the
@@ -849,7 +848,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	 * prior to injecting the event).
 	 */
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_NRIPS))
-		vmcb02->control.next_rip    = svm->nested.ctl.next_rip;
+		vmcb02->control.next_rip    = vmcb12_ctrl->next_rip;
 	else if (boot_cpu_has(X86_FEATURE_NRIPS))
 		vmcb02->control.next_rip    = vmcb12_rip;
 
@@ -859,7 +858,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 		svm->soft_int_csbase = vmcb12_csbase;
 		svm->soft_int_old_rip = vmcb12_rip;
 		if (guest_cpu_cap_has(vcpu, X86_FEATURE_NRIPS))
-			svm->soft_int_next_rip = svm->nested.ctl.next_rip;
+			svm->soft_int_next_rip = vmcb12_ctrl->next_rip;
 		else
 			svm->soft_int_next_rip = vmcb12_rip;
 	}
@@ -870,11 +869,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 		vmcb02->control.virt_ext |= VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK;
 
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_PAUSEFILTER))
-		pause_count12 = svm->nested.ctl.pause_filter_count;
+		pause_count12 = vmcb12_ctrl->pause_filter_count;
 	else
 		pause_count12 = 0;
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_PFTHRESHOLD))
-		pause_thresh12 = svm->nested.ctl.pause_filter_thresh;
+		pause_thresh12 = vmcb12_ctrl->pause_filter_thresh;
 	else
 		pause_thresh12 = 0;
 	if (kvm_pause_in_guest(svm->vcpu.kvm)) {
@@ -888,7 +887,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 		vmcb02->control.pause_filter_thresh = vmcb01->control.pause_filter_thresh;
 
 		/* ... but ensure filtering is disabled if so requested.  */
-		if (vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_PAUSE)) {
+		if (vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_PAUSE)) {
 			if (!pause_count12)
 				vmcb02->control.pause_filter_count = 0;
 			if (!pause_thresh12)
@@ -905,7 +904,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,
 	 * L2 is the "guest").
 	 */
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))
-		vmcb02->control.erap_ctl = (svm->nested.ctl.erap_ctl &
+		vmcb02->control.erap_ctl = (vmcb12_ctrl->erap_ctl &
 					    ERAP_CONTROL_ALLOW_LARGER_RAP) |
 					   ERAP_CONTROL_CLEAR_RAP;
 
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Reviewed-by on v2 8/8
#13 lore raw

Re: [PATCH v2 8/8] KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing vmcb02

Quoted 11 lines · Sean Christopherson

Grab svm->nested.ctl as vmcb12_ctrl when preparing the vmcb02 controls to make it more obvious that much of the data is coming from vmcb12 (or rather, a snapshot of vmcb12 at the time of L1's VMRUN).

Opportunistically reorder the variable definitions to create a pretty reverse fir tree.

No functional change intended.

author maintainer Applied
#14 lore raw
In reply to Sean Christopherson
Quoted 13 lines · Sean Christopherson

Fix a likely-benign bug where KVM fails to mark vmcb01 intercepts as dirty after recalculating intercepts while L2 is active, then do a bunch of related cleanup, e.g. to split recalc_intercepts() into nested vs. non-nested functionality.

v2:
- Fix the aforementioned bug.
- Split recalc_intercepts() instead of simply renaming it.
- Move the new WARN in nested_vmcb02_recalc_intercepts() to its own patch.
- Use less weird local variables even if they aren't consistent with the existing code...
- ... and then change some names in the existing code to provide consistency.

Quoted 1 line

[...]

Applied to kvm-x86 nested, thanks!

[1/8] KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts

      https://github.com/kvm-x86/linux/commit/d5bde6113aed
[2/8] KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested parts
      https://github.com/kvm-x86/linux/commit/0b97f929831a
[3/8] KVM: nSVM: WARN and abort vmcb02 intercepts recalc if vmcb02 isn't active
      https://github.com/kvm-x86/linux/commit/a367b6e10372
[4/8] KVM: nSVM: Directly (re)calc vmcb02 intercepts from nested_vmcb02_prepare_control()
      https://github.com/kvm-x86/linux/commit/4a80c4bc1f10
[5/8] KVM: nSVM: Use intuitive local variables in nested_vmcb02_recalc_intercepts()
      https://github.com/kvm-x86/linux/commit/586160b75091
[6/8] KVM: nSVM: Use vmcb12_is_intercept() in nested_sync_control_from_vmcb02()
      https://github.com/kvm-x86/linux/commit/ef09eebc5736
[7/8] KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state
      https://github.com/kvm-x86/linux/commit/af75470944f4
[8/8] KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing vmcb02
      https://github.com/kvm-x86/linux/commit/56bfbe68f78e

--
https://github.com/kvm-x86/linux/tree/next