Skip to contentSkip to conversation

KVM: x86: Emulate L2 triple fault without killing L1

PATCH Applied Sean Christopherson 2 participants 6 messages Mar 2, 2021

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

cover KVM: x86: Emulate L2 triple fault without killing L1

If KVM (L0) intercepts #GP, but L1 does not, then L2 can kill L1 by triggering triple fault. On both VMX and SVM, if the CPU hits a fault while vectoring an injected #DF (or I supposed any #DF), any intercept from the hypervisor takes priority over triple fault. #PF is unlikely to be intercepted by L0 but not L1. The bigger problem is #GP, which is intercepted on both VMX and SVM if enable_vmware_backdoor=1, and is also now intercepted for the lovely VMRUN/VMLOAD/VMSAVE errata.

Based on kvm/queue, commit fe5f0041c026 ("KVM/SVM: Move vmenter.S exception fixups out of line"). x86.c and svm/nested.c conflict with kvm/master. They are minor and straighforward, but let me know if you want me to post a version based on kvm/master for easier inclusion into 5.12.

Sean Christopherson (2):
KVM: x86: Handle triple fault in L2 without killing L1
KVM: nSVM: Add helper to synthesize nested VM-Exit without collateral

 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/lapic.c            |  2 +-
 arch/x86/kvm/svm/nested.c       | 57 ++++++++-------------------------
 arch/x86/kvm/svm/svm.c          |  6 +---
 arch/x86/kvm/svm/svm.h          |  9 ++++++
 arch/x86/kvm/vmx/nested.c       |  9 ++++++
 arch/x86/kvm/x86.c              | 29 +++++++++++++----
 arch/x86/kvm/x86.h              |  2 ++
 8 files changed, 60 insertions(+), 55 deletions(-)
Signature · 1 line
2.30.1.766.gb4fecdf3b7-goog
maintainer
#1 lore raw
Quoted 8 lines · Sean Christopherson

If KVM (L0) intercepts #GP, but L1 does not, then L2 can kill L1 by triggering triple fault. On both VMX and SVM, if the CPU hits a fault while vectoring an injected #DF (or I supposed any #DF), any intercept from the hypervisor takes priority over triple fault. #PF is unlikely to be intercepted by L0 but not L1. The bigger problem is #GP, which is intercepted on both VMX and SVM if enable_vmware_backdoor=1, and is also now intercepted for the lovely VMRUN/VMLOAD/VMSAVE errata.

Quoted 4 lines

Based on kvm/queue, commit fe5f0041c026 ("KVM/SVM: Move vmenter.S exception fixups out of line"). x86.c and svm/nested.c conflict with kvm/master. They are minor and straighforward, but let me know if you want me to post a version based on kvm/master for easier inclusion into 5.12.

I think it would be too intrusive. Let's stick this in 5.13 only.

Paolo

Quoted 14 lines

Sean Christopherson (2):
KVM: x86: Handle triple fault in L2 without killing L1
KVM: nSVM: Add helper to synthesize nested VM-Exit without collateral

  arch/x86/include/asm/kvm_host.h |  1 +
  arch/x86/kvm/lapic.c            |  2 +-
  arch/x86/kvm/svm/nested.c       | 57 ++++++++-------------------------
  arch/x86/kvm/svm/svm.c          |  6 +---
  arch/x86/kvm/svm/svm.h          |  9 ++++++
  arch/x86/kvm/vmx/nested.c       |  9 ++++++
  arch/x86/kvm/x86.c              | 29 +++++++++++++----
  arch/x86/kvm/x86.h              |  2 ++
  8 files changed, 60 insertions(+), 55 deletions(-)
author maintainer
#2 lore raw
Quoted 9 lines · Paolo Bonzini
Quoted 8 lines · Sean Christopherson

If KVM (L0) intercepts #GP, but L1 does not, then L2 can kill L1 by triggering triple fault. On both VMX and SVM, if the CPU hits a fault while vectoring an injected #DF (or I supposed any #DF), any intercept from the hypervisor takes priority over triple fault. #PF is unlikely to be intercepted by L0 but not L1. The bigger problem is #GP, which is intercepted on both VMX and SVM if enable_vmware_backdoor=1, and is also now intercepted for the lovely VMRUN/VMLOAD/VMSAVE errata.

Quoted 6 lines
Quoted 4 lines

Based on kvm/queue, commit fe5f0041c026 ("KVM/SVM: Move vmenter.S exception fixups out of line"). x86.c and svm/nested.c conflict with kvm/master. They are minor and straighforward, but let me know if you want me to post a version based on kvm/master for easier inclusion into 5.12.

I think it would be too intrusive. Let's stick this in 5.13 only.

Hmm, agreed, especially since most of the paths are not properly tested. In that case, probably best to also drop [email protected]?

author maintainer
#3 lore raw

PATCH 1/2 KVM: x86: Handle triple fault in L2 without killing L1

In reply to Sean Christopherson

Synthesize a nested VM-Exit if L2 triggers an emulated triple fault instead of exiting to userspace, which likely will kill L1. Any flow that does KVM_REQ_TRIPLE_FAULT is suspect, but the most common scenario for L2 killing L1 is if L0 (KVM) intercepts a contributory exception that is _not_intercepted by L1. E.g. if KVM is intercepting #GPs for the VMware backdoor, a #GP that occurs in L2 while vectoring an injected #DF will cause KVM to emulate triple fault.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/lapic.c            |  2 +-
 arch/x86/kvm/svm/nested.c       | 14 ++++++++++++++
 arch/x86/kvm/vmx/nested.c       |  9 +++++++++
 arch/x86/kvm/x86.c              | 29 +++++++++++++++++++++++------
 arch/x86/kvm/x86.h              |  2 ++
 6 files changed, 50 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 6db60ea8ee5b..3668f38bad60 100644
@@ -1333,6 +1333,7 @@ struct kvm_x86_ops {
 struct kvm_x86_nested_ops {
 	int (*check_events)(struct kvm_vcpu *vcpu);
 	bool (*hv_timer_pending)(struct kvm_vcpu *vcpu);
+	void (*triple_fault)(struct kvm_vcpu *vcpu);
 	int (*get_state)(struct kvm_vcpu *vcpu,
 			 struct kvm_nested_state __user *user_kvm_nested_state,
 			 unsigned user_data_size);
arch/x86/kvm/lapic.c +1 -1
index 45d40bfacb7c..9faed6e6960a 100644
@@ -2859,7 +2859,7 @@ void kvm_apic_accept_events(struct kvm_vcpu *vcpu)
 		return;
 
 	if (is_guest_mode(vcpu)) {
-		r = kvm_x86_ops.nested_ops->check_events(vcpu);
+		r = kvm_check_nested_events(vcpu);
 		if (r < 0)
 			return;
 		/*
arch/x86/kvm/svm/nested.c +14 -0
index 90a1704b5752..93a61ed76e5b 100644
@@ -671,6 +671,9 @@ int nested_svm_vmexit(struct vcpu_svm *svm)
 	struct kvm_host_map map;
 	int rc;
 
+	/* Triple faults in L2 should never escape. */
+	WARN_ON_ONCE(kvm_check_request(KVM_REQ_TRIPLE_FAULT, vcpu));
+
 	rc = kvm_vcpu_map(vcpu, gpa_to_gfn(svm->nested.vmcb12_gpa), &map);
 	if (rc) {
 		if (rc == -EINVAL)
@@ -792,6 +795,16 @@ int nested_svm_vmexit(struct vcpu_svm *svm)
 	return 0;
 }
 
+static void nested_svm_triple_fault(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	svm->vmcb->control.exit_code   = SVM_EXIT_SHUTDOWN;
+	svm->vmcb->control.exit_info_1 = 0;
+	svm->vmcb->control.exit_info_2 = 0;
+	nested_svm_vmexit(svm);
+}
+
 int svm_allocate_nested(struct vcpu_svm *svm)
 {
 	struct page *vmcb02_page;
@@ -1327,6 +1340,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
 struct kvm_x86_nested_ops svm_nested_ops = {
 	.check_events = svm_check_nested_events,
+	.triple_fault = nested_svm_triple_fault,
 	.get_nested_state_pages = svm_get_nested_state_pages,
 	.get_state = svm_get_nested_state,
 	.set_state = svm_set_nested_state,
arch/x86/kvm/vmx/nested.c +9 -0
index fdd80dd8e781..3493a48411a8 100644
@@ -4416,6 +4416,9 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 	/* trying to cancel vmlaunch/vmresume is a bug */
 	WARN_ON_ONCE(vmx->nested.nested_run_pending);
 
+	/* Similarly, triple faults in L2 should never escape. */
+	WARN_ON_ONCE(kvm_check_request(KVM_REQ_TRIPLE_FAULT, vcpu));
+
 	kvm_clear_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu);
 
 	/* Service the TLB flush request for L2 before switching to L1. */
@@ -4552,6 +4555,11 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 	vmx->fail = 0;
 }
 
+static void nested_vmx_triple_fault(struct kvm_vcpu *vcpu)
+{
+	nested_vmx_vmexit(vcpu, EXIT_REASON_TRIPLE_FAULT, 0, 0);
+}
+
 /*
  * Decode the memory-address operand of a vmx instruction, as recorded on an
  * exit caused by such an instruction (run by a guest hypervisor).
@@ -6593,6 +6601,7 @@ __init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *))
 struct kvm_x86_nested_ops vmx_nested_ops = {
 	.check_events = vmx_check_nested_events,
 	.hv_timer_pending = nested_vmx_preemption_timer_pending,
+	.triple_fault = nested_vmx_triple_fault,
 	.get_state = vmx_get_nested_state,
 	.set_state = vmx_set_nested_state,
 	.get_nested_state_pages = vmx_get_nested_state_pages,
arch/x86/kvm/x86.c +23 -6
index 828de7d65074..11746f488f04 100644
@@ -8349,6 +8349,19 @@ static void update_cr8_intercept(struct kvm_vcpu *vcpu)
 	static_call(kvm_x86_update_cr8_intercept)(vcpu, tpr, max_irr);
 }
 
+int kvm_check_nested_events(struct kvm_vcpu *vcpu)
+{
+	if (WARN_ON_ONCE(!is_guest_mode(vcpu)))
+		return -EIO;
+
+	if (kvm_check_request(KVM_REQ_TRIPLE_FAULT, vcpu)) {
+		kvm_x86_ops.nested_ops->triple_fault(vcpu);
+		return 1;
+	}
+
+	return kvm_x86_ops.nested_ops->check_events(vcpu);
+}
+
 static void kvm_inject_exception(struct kvm_vcpu *vcpu)
 {
 	if (vcpu->arch.exception.error_code && !is_protmode(vcpu))
@@ -8401,7 +8414,7 @@ static void inject_pending_event(struct kvm_vcpu *vcpu, bool *req_immediate_exit
 	 * from L2 to L1.
 	 */
 	if (is_guest_mode(vcpu)) {
-		r = kvm_x86_ops.nested_ops->check_events(vcpu);
+		r = kvm_check_nested_events(vcpu);
 		if (r < 0)
 			goto busy;
 	}
@@ -8964,10 +8977,14 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 			goto out;
 		}
 		if (kvm_check_request(KVM_REQ_TRIPLE_FAULT, vcpu)) {
-			vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
-			vcpu->mmio_needed = 0;
-			r = 0;
-			goto out;
+			if (is_guest_mode(vcpu)) {
+				kvm_x86_ops.nested_ops->triple_fault(vcpu);
+			} else {
+				vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN;
+				vcpu->mmio_needed = 0;
+				r = 0;
+				goto out;
+			}
 		}
 		if (kvm_check_request(KVM_REQ_APF_HALT, vcpu)) {
 			/* Page is swapped out. Do synthetic halt */
@@ -9265,7 +9282,7 @@ static inline int vcpu_block(struct kvm *kvm, struct kvm_vcpu *vcpu)
 static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu)
 {
 	if (is_guest_mode(vcpu))
-		kvm_x86_ops.nested_ops->check_events(vcpu);
+		kvm_check_nested_events(vcpu);
 
 	return (vcpu->arch.mp_state == KVM_MP_STATE_RUNNABLE &&
 		!vcpu->arch.apf.halted);
arch/x86/kvm/x86.h +2 -0
index ee6e01067884..daccf20fbcd5 100644
@@ -56,6 +56,8 @@ static inline unsigned int __shrink_ple_window(unsigned int val,
 
 #define MSR_IA32_CR_PAT_DEFAULT  0x0007040600070406ULL
 
+int kvm_check_nested_events(struct kvm_vcpu *vcpu);
+
 static inline void kvm_clear_exception_queue(struct kvm_vcpu *vcpu)
 {
 	vcpu->arch.exception.pending = false;
Signature · 1 line
2.30.1.766.gb4fecdf3b7-goog
author maintainer
#4 lore raw

PATCH 2/2 KVM: nSVM: Add helper to synthesize nested VM-Exit without collateral

In reply to Sean Christopherson

Add a helper to consolidate boilerplate for nested VM-Exits that don't provide any data in exit_info_*.

No functional change intended.

---
 arch/x86/kvm/svm/nested.c | 55 +++++----------------------------------
 arch/x86/kvm/svm/svm.c    |  6 +----
 arch/x86/kvm/svm/svm.h    |  9 +++++++
 3 files changed, 16 insertions(+), 54 deletions(-)
arch/x86/kvm/svm/nested.c +6 -49
index 93a61ed76e5b..307c11125391 100644
@@ -797,12 +797,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm)
 
 static void nested_svm_triple_fault(struct kvm_vcpu *vcpu)
 {
-	struct vcpu_svm *svm = to_svm(vcpu);
-
-	svm->vmcb->control.exit_code   = SVM_EXIT_SHUTDOWN;
-	svm->vmcb->control.exit_info_1 = 0;
-	svm->vmcb->control.exit_info_2 = 0;
-	nested_svm_vmexit(svm);
+	nested_svm_simple_vmexit(to_svm(vcpu), SVM_EXIT_SHUTDOWN);
 }
 
 int svm_allocate_nested(struct vcpu_svm *svm)
@@ -1027,50 +1022,11 @@ static void nested_svm_inject_exception_vmexit(struct vcpu_svm *svm)
 	nested_svm_vmexit(svm);
 }
 
-static void nested_svm_smi(struct vcpu_svm *svm)
-{
-	svm->vmcb->control.exit_code = SVM_EXIT_SMI;
-	svm->vmcb->control.exit_info_1 = 0;
-	svm->vmcb->control.exit_info_2 = 0;
-
-	nested_svm_vmexit(svm);
-}
-
-static void nested_svm_nmi(struct vcpu_svm *svm)
-{
-	svm->vmcb->control.exit_code = SVM_EXIT_NMI;
-	svm->vmcb->control.exit_info_1 = 0;
-	svm->vmcb->control.exit_info_2 = 0;
-
-	nested_svm_vmexit(svm);
-}
-
-static void nested_svm_intr(struct vcpu_svm *svm)
-{
-	trace_kvm_nested_intr_vmexit(svm->vmcb->save.rip);
-
-	svm->vmcb->control.exit_code   = SVM_EXIT_INTR;
-	svm->vmcb->control.exit_info_1 = 0;
-	svm->vmcb->control.exit_info_2 = 0;
-
-	nested_svm_vmexit(svm);
-}
-
 static inline bool nested_exit_on_init(struct vcpu_svm *svm)
 {
 	return vmcb_is_intercept(&svm->nested.ctl, INTERCEPT_INIT);
 }
 
-static void nested_svm_init(struct vcpu_svm *svm)
-{
-	svm->vmcb->control.exit_code   = SVM_EXIT_INIT;
-	svm->vmcb->control.exit_info_1 = 0;
-	svm->vmcb->control.exit_info_2 = 0;
-
-	nested_svm_vmexit(svm);
-}
-
-
 static int svm_check_nested_events(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
@@ -1084,7 +1040,7 @@ static int svm_check_nested_events(struct kvm_vcpu *vcpu)
 			return -EBUSY;
 		if (!nested_exit_on_init(svm))
 			return 0;
-		nested_svm_init(svm);
+		nested_svm_simple_vmexit(svm, SVM_EXIT_INIT);
 		return 0;
 	}
 
@@ -1102,7 +1058,7 @@ static int svm_check_nested_events(struct kvm_vcpu *vcpu)
 			return -EBUSY;
 		if (!nested_exit_on_smi(svm))
 			return 0;
-		nested_svm_smi(svm);
+		nested_svm_simple_vmexit(svm, SVM_EXIT_SMI);
 		return 0;
 	}
 
@@ -1111,7 +1067,7 @@ static int svm_check_nested_events(struct kvm_vcpu *vcpu)
 			return -EBUSY;
 		if (!nested_exit_on_nmi(svm))
 			return 0;
-		nested_svm_nmi(svm);
+		nested_svm_simple_vmexit(svm, SVM_EXIT_NMI);
 		return 0;
 	}
 
@@ -1120,7 +1076,8 @@ static int svm_check_nested_events(struct kvm_vcpu *vcpu)
 			return -EBUSY;
 		if (!nested_exit_on_intr(svm))
 			return 0;
-		nested_svm_intr(svm);
+		trace_kvm_nested_intr_vmexit(svm->vmcb->save.rip);
+		nested_svm_simple_vmexit(svm, SVM_EXIT_INTR);
 		return 0;
 	}
 
arch/x86/kvm/svm/svm.c +1 -5
index 54610270f66a..9fe9076d4b8b 100644
@@ -2203,12 +2203,8 @@ static int emulate_svm_instr(struct kvm_vcpu *vcpu, int opcode)
 	int ret;
 
 	if (is_guest_mode(vcpu)) {
-		svm->vmcb->control.exit_code = guest_mode_exit_codes[opcode];
-		svm->vmcb->control.exit_info_1 = 0;
-		svm->vmcb->control.exit_info_2 = 0;
-
 		/* Returns '1' or -errno on failure, '0' on success. */
-		ret = nested_svm_vmexit(svm);
+		ret = nested_svm_simple_vmexit(svm, guest_mode_exit_codes[opcode]);
 		if (ret)
 			return ret;
 		return 1;
arch/x86/kvm/svm/svm.h +9 -0
index fbbb26dd0f73..c4a433c66a33 100644
@@ -444,6 +444,15 @@ int svm_allocate_nested(struct vcpu_svm *svm);
 int nested_svm_vmrun(struct kvm_vcpu *vcpu);
 void nested_svm_vmloadsave(struct vmcb *from_vmcb, struct vmcb *to_vmcb);
 int nested_svm_vmexit(struct vcpu_svm *svm);
+
+static inline int nested_svm_simple_vmexit(struct vcpu_svm *svm, u32 exit_code)
+{
+	svm->vmcb->control.exit_code   = exit_code;
+	svm->vmcb->control.exit_info_1 = 0;
+	svm->vmcb->control.exit_info_2 = 0;
+	return nested_svm_vmexit(svm);
+}
+
 int nested_svm_exit_handled(struct vcpu_svm *svm);
 int nested_svm_check_permissions(struct kvm_vcpu *vcpu);
 int nested_svm_check_exception(struct vcpu_svm *svm, unsigned nr,
Signature · 1 line
2.30.1.766.gb4fecdf3b7-goog
maintainer Applied
#5 lore raw
In reply to Sean Christopherson
Quoted 17 lines · Sean Christopherson

If KVM (L0) intercepts #GP, but L1 does not, then L2 can kill L1 by triggering triple fault. On both VMX and SVM, if the CPU hits a fault while vectoring an injected #DF (or I supposed any #DF), any intercept from the hypervisor takes priority over triple fault. #PF is unlikely to be intercepted by L0 but not L1. The bigger problem is #GP, which is intercepted on both VMX and SVM if enable_vmware_backdoor=1, and is also now intercepted for the lovely VMRUN/VMLOAD/VMSAVE errata.

Based on kvm/queue, commit fe5f0041c026 ("KVM/SVM: Move vmenter.S exception fixups out of line"). x86.c and svm/nested.c conflict with kvm/master. They are minor and straighforward, but let me know if you want me to post a version based on kvm/master for easier inclusion into 5.12.

Sean Christopherson (2):
KVM: x86: Handle triple fault in L2 without killing L1
KVM: nSVM: Add helper to synthesize nested VM-Exit without collateral

Quoted 10 lines
  arch/x86/include/asm/kvm_host.h |  1 +
  arch/x86/kvm/lapic.c            |  2 +-
  arch/x86/kvm/svm/nested.c       | 57 ++++++++-------------------------
  arch/x86/kvm/svm/svm.c          |  6 +---
  arch/x86/kvm/svm/svm.h          |  9 ++++++
  arch/x86/kvm/vmx/nested.c       |  9 ++++++
  arch/x86/kvm/x86.c              | 29 +++++++++++++----
  arch/x86/kvm/x86.h              |  2 ++
  8 files changed, 60 insertions(+), 55 deletions(-)

Queued, thanks.

Paolo