Skip to contentSkip to conversation

KVM: nSVM: Intercept STGI/CLGI as needed to inject #UD

PATCH Applied Sean Christopherson 2 participants 6 messages Mar 4 – Mar 5

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v5 cover KVM: nSVM: Intercept STGI/CLGI as needed to inject #UD

The STGI/CLGI fixes from Kevin's broader "Align SVM with APM defined behaviors" series.

v5:
- Separate the STGI/CLGI fixes from everything else.
- Apply the Intel compatibiliy weirdness only to VMLOAD/VMSAVE (to fixup SYSENTER MSRs).
- Re-remove STGI/CLGI intercept clearing in init_vmcb().

v4:
- https://lore.kernel.org/all/[email protected]
- Dropped "KVM: SVM: Inject #UD for STGI if EFER.SVME=0 and SVM Lock and DEV are not available" as per Sean
- Added back STGI and CLGI intercept clearing in init_vmcb to maintain previous behavior on intel guests. Previously intel guests always had STGI and CLGI intercepts cleared if vgif was enabled. In V3, because the clearing of the intercepts was moved from init_vmcb() to the !guest_cpuid_is_intel_compatible() case in svm_recalc_instruction_intercepts(), the CLGI intercept would be indefinitely set on intel guests. I added back the clearing to init_vmcb() to retain intel guest behavior before this patch.
- In "Raise #UD if VMMCALL instruction is not intercepted" patch:
- Exempt Hyper-V L2 TLB flush hypercalls from the #UD injection, as L0 intentionally intercepts these VMMCALLs on behalf of L1 via the direct hypercall enlightenment.
- Added nested_svm_is_l2_tlb_flush_hcall() which just returns true if the hypercall was a Hyper-V L2 TLB flush hypercall.

v3:
- https://lore.kernel.org/kvm/[email protected]/
- Elaborated on 'Move STGI and CLGI intercept handling' commit message as per Sean
- Fixed bug due to interaction with svm_enable_nmi_window() and 'Move STGI and CLGI intercept handling' as pointed out by Yosry. Code changes suggested by Sean/Yosry.
- Removed open-coded nested_svm_check_permissions() in STGI interception function as per Yosry

v2:
- https://lore.kernel.org/all/[email protected]
- Split up the series into smaller more logical changes as suggested by Sean
- Added patch for injecting #UD for STGI under APM defined conditions as suggested by Sean
- Combined EFER.SVME=0 conditional with intel CPU logic in svm_recalc_instruction_intercepts

Kevin Cheng (2):
KVM: SVM: Move STGI and CLGI intercept handling
KVM: SVM: Recalc instructions intercepts when EFER.SVME is toggled

 arch/x86/kvm/svm/svm.c | 53 +++++++++++++++++++++++++++++++-----------
 1 file changed, 40 insertions(+), 13 deletions(-)

base-commit: 11439c4635edd669ae435eec308f4ab8a0804808

Signature · 1 line
2.53.0.473.g4a7958ca14-goog
author maintainer
#1 lore raw

PATCH v5 1/2 KVM: SVM: Move STGI and CLGI intercept handling

From: Kevin Cheng <[email protected]>

Move STGI/CLGI intercept handling to svm_recalc_instruction_intercepts() in preparation for making the function EFER.SVME-aware. This will allow configuring STGI/CLGI intercepts along with other intercepts for other SVM instructions when EFER.SVME is toggled (KVM needs to intercept SVM instructions when EFER.SVME=0 to inject #UD).

When clearing the STGI intercept in particular, request KVM_REQ_EVENT if there is at least one a pending GIF-controlled event. This avoids breaking NMI/SMI window tracking, as enable_{nmi,smi}_window() sets INTERCEPT_STGI to detect when NMIs become unblocked. KVM_REQ_EVENT forces kvm_check_and_inject_events() to re-evaluate pending events and re-enable the intercept if needed.

Extract the pending GIF event check into a helper function svm_has_pending_gif_event() to deduplicate the logic between svm_recalc_instruction_intercepts() and svm_set_gif().

[sean: keep vgif handling out of the "Intel CPU model" path]
---
 arch/x86/kvm/svm/svm.c | 32 ++++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/svm.c +24 -8
index 8f8bc863e214..5975a1e14ac9 100644
@@ -1009,6 +1009,14 @@ void svm_write_tsc_multiplier(struct kvm_vcpu *vcpu)
 	preempt_enable();
 }
 
+static bool svm_has_pending_gif_event(struct vcpu_svm *svm)
+{
+	return svm->vcpu.arch.smi_pending ||
+	       svm->vcpu.arch.nmi_pending ||
+	       kvm_cpu_has_injectable_intr(&svm->vcpu) ||
+	       kvm_apic_has_pending_init_or_sipi(&svm->vcpu);
+}
+
 /* Evaluate instruction intercepts that depend on guest CPUID features. */
 static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)
 {
@@ -1052,6 +1060,20 @@ static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)
 		}
 	}
 
+	if (vgif) {
+		svm_clr_intercept(svm, INTERCEPT_STGI);
+		svm_clr_intercept(svm, INTERCEPT_CLGI);
+
+		/*
+		 * Process pending events when clearing STGI/CLGI intercepts if
+		 * there's at least one pending event that is masked by GIF, so
+		 * that KVM re-evaluates if the intercept needs to be set again
+		 * to track when GIF is re-enabled (e.g. for NMI injection).
+		 */
+		if (svm_has_pending_gif_event(svm))
+			kvm_make_request(KVM_REQ_EVENT, &svm->vcpu);
+	}
+
 	if (kvm_need_rdpmc_intercept(vcpu))
 		svm_set_intercept(svm, INTERCEPT_RDPMC);
 	else
@@ -1195,11 +1217,8 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
 	if (vnmi)
 		svm->vmcb->control.int_ctl |= V_NMI_ENABLE_MASK;
 
-	if (vgif) {
-		svm_clr_intercept(svm, INTERCEPT_STGI);
-		svm_clr_intercept(svm, INTERCEPT_CLGI);
+	if (vgif)
 		svm->vmcb->control.int_ctl |= V_GIF_ENABLE_MASK;
-	}
 
 	if (vls)
 		svm->vmcb->control.virt_ext |= VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK;
@@ -2320,10 +2339,7 @@ void svm_set_gif(struct vcpu_svm *svm, bool value)
 			svm_clear_vintr(svm);
 
 		enable_gif(svm);
-		if (svm->vcpu.arch.smi_pending ||
-		    svm->vcpu.arch.nmi_pending ||
-		    kvm_cpu_has_injectable_intr(&svm->vcpu) ||
-		    kvm_apic_has_pending_init_or_sipi(&svm->vcpu))
+		if (svm_has_pending_gif_event(svm))
 			kvm_make_request(KVM_REQ_EVENT, &svm->vcpu);
 	} else {
 		disable_gif(svm);
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Reviewed-by on v5 1/2
#2 lore raw

Re: [PATCH v5 1/2] KVM: SVM: Move STGI and CLGI intercept handling

Quoted 15 lines · Sean Christopherson

From: Kevin Cheng <[email protected]>

Move STGI/CLGI intercept handling to svm_recalc_instruction_intercepts() in preparation for making the function EFER.SVME-aware. This will allow configuring STGI/CLGI intercepts along with other intercepts for other SVM instructions when EFER.SVME is toggled (KVM needs to intercept SVM instructions when EFER.SVME=0 to inject #UD).

When clearing the STGI intercept in particular, request KVM_REQ_EVENT if there is at least one a pending GIF-controlled event. This avoids breaking NMI/SMI window tracking, as enable_{nmi,smi}_window() sets INTERCEPT_STGI to detect when NMIs become unblocked. KVM_REQ_EVENT forces kvm_check_and_inject_events() to re-evaluate pending events and re-enable the intercept if needed.

Quoted 7 lines

Extract the pending GIF event check into a helper function svm_has_pending_gif_event() to deduplicate the logic between svm_recalc_instruction_intercepts() and svm_set_gif().

[sean: keep vgif handling out of the "Intel CPU model" path]
author maintainer
#3 lore raw

PATCH v5 2/2 KVM: SVM: Recalc instructions intercepts when EFER.SVME is toggled

In reply to Sean Christopherson

From: Kevin Cheng <[email protected]>

The AMD APM states that VMRUN, VMLOAD, VMSAVE, CLGI, VMMCALL, and INVLPGA instructions should generate a #UD when EFER.SVME is cleared. Currently, when VMLOAD, VMSAVE, or CLGI are executed in L1 with EFER.SVME cleared, no #UD is generated in certain cases. This is because the intercepts for these instructions are cleared based on whether or not vls or vgif is enabled. The #UD fails to be generated when the intercepts are absent.

Fix the missing #UD generation by ensuring that all relevant instructions have intercepts set when SVME.EFER is disabled.

VMMCALL is special because KVM's ABI is that VMCALL/VMMCALL are always supported for L1 and never fault.

[sean: isolate Intel CPU "compatibility" in EFER.SVME=1 path]
---
 arch/x86/kvm/svm/svm.c | 35 +++++++++++++++++++++++------------
 1 file changed, 23 insertions(+), 12 deletions(-)
arch/x86/kvm/svm/svm.c +23 -12
index 5975a1e14ac9..07b595487caf 100644
@@ -244,6 +244,8 @@ int svm_set_efer(struct kvm_vcpu *vcpu, u64 efer)
 			if (svm_gp_erratum_intercept && !sev_guest(vcpu->kvm))
 				set_exception_intercept(svm, GP_VECTOR);
 		}
+
+		kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu);
 	}
 
 	svm->vmcb->save.efer = efer | EFER_SVME;
@@ -1042,27 +1044,31 @@ static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)
 	}
 
 	/*
-	 * No need to toggle VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK here, it is
-	 * always set if vls is enabled. If the intercepts are set, the bit is
-	 * meaningless anyway.
+	 * Intercept instructions that #UD if EFER.SVME=0, as SVME must be set
+	 * even when running the guest, i.e. hardware will only ever see
+	 * EFER.SVME=1.
+	 *
+	 * No need to toggle any of the vgif/vls/etc. enable bits here, as they
+	 * are set when the VMCB is initialized and never cleared (if the
+	 * relevant intercepts are set, the enablements are meaningless anyway).
 	 */
-	if (guest_cpuid_is_intel_compatible(vcpu)) {
+	if (!(vcpu->arch.efer & EFER_SVME)) {
 		svm_set_intercept(svm, INTERCEPT_VMLOAD);
 		svm_set_intercept(svm, INTERCEPT_VMSAVE);
+		svm_set_intercept(svm, INTERCEPT_CLGI);
+		svm_set_intercept(svm, INTERCEPT_STGI);
 	} else {
 		/*
 		 * If hardware supports Virtual VMLOAD VMSAVE then enable it
 		 * in VMCB and clear intercepts to avoid #VMEXIT.
 		 */
-		if (vls) {
+		if (guest_cpuid_is_intel_compatible(vcpu)) {
+			svm_set_intercept(svm, INTERCEPT_VMLOAD);
+			svm_set_intercept(svm, INTERCEPT_VMSAVE);
+		} else if (vls) {
 			svm_clr_intercept(svm, INTERCEPT_VMLOAD);
 			svm_clr_intercept(svm, INTERCEPT_VMSAVE);
 		}
-	}
-
-	if (vgif) {
-		svm_clr_intercept(svm, INTERCEPT_STGI);
-		svm_clr_intercept(svm, INTERCEPT_CLGI);
 
 		/*
 		 * Process pending events when clearing STGI/CLGI intercepts if
@@ -1070,8 +1076,13 @@ static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)
 		 * that KVM re-evaluates if the intercept needs to be set again
 		 * to track when GIF is re-enabled (e.g. for NMI injection).
 		 */
-		if (svm_has_pending_gif_event(svm))
-			kvm_make_request(KVM_REQ_EVENT, &svm->vcpu);
+		if (vgif) {
+			svm_clr_intercept(svm, INTERCEPT_CLGI);
+			svm_clr_intercept(svm, INTERCEPT_STGI);
+
+			if (svm_has_pending_gif_event(svm))
+				kvm_make_request(KVM_REQ_EVENT, &svm->vcpu);
+		}
 	}
 
 	if (kvm_need_rdpmc_intercept(vcpu))
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Reviewed-by on v5 2/2
#4 lore raw

Re: [PATCH v5 2/2] KVM: SVM: Recalc instructions intercepts when EFER.SVME is toggled

Quoted 13 lines · Sean Christopherson

From: Kevin Cheng <[email protected]>

The AMD APM states that VMRUN, VMLOAD, VMSAVE, CLGI, VMMCALL, and INVLPGA instructions should generate a #UD when EFER.SVME is cleared. Currently, when VMLOAD, VMSAVE, or CLGI are executed in L1 with EFER.SVME cleared, no #UD is generated in certain cases. This is because the intercepts for these instructions are cleared based on whether or not vls or vgif is enabled. The #UD fails to be generated when the intercepts are absent.

Fix the missing #UD generation by ensuring that all relevant instructions have intercepts set when SVME.EFER is disabled.

Quoted 6 lines

VMMCALL is special because KVM's ABI is that VMCALL/VMMCALL are always supported for L1 and never fault.

[sean: isolate Intel CPU "compatibility" in EFER.SVME=1 path]
author maintainer Applied
#5 lore raw
In reply to Sean Christopherson
Quoted 10 lines · Sean Christopherson

The STGI/CLGI fixes from Kevin's broader "Align SVM with APM defined behaviors" series.

v5:
- Separate the STGI/CLGI fixes from everything else.
- Apply the Intel compatibiliy weirdness only to VMLOAD/VMSAVE (to fixup SYSENTER MSRs).
- Re-remove STGI/CLGI intercept clearing in init_vmcb().

[...]

Applied to kvm-x86 nested, thanks!

[1/2] KVM: SVM: Move STGI and CLGI intercept handling

      https://github.com/kvm-x86/linux/commit/69f779f79e0d
[2/2] KVM: SVM: Recalc instructions intercepts when EFER.SVME is toggled
      https://github.com/kvm-x86/linux/commit/460c7eb2e759

--
https://github.com/kvm-x86/linux/tree/next