Skip to contentSkip to conversation

KVM: x86: Fix emulation of (some) L2 instructions

PATCH Applied Sean Christopherson 1 participant 13 messages Feb 1, 2025 – Feb 28, 2025

Subsystems (4)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v2 cover KVM: x86: Fix emulation of (some) L2 instructions

Fix a variety of bugs related to emulating instructions on behalf of L2, and (finally) add support for synthesizing nested VM-Exit to L1 when L1 wants to intercept an instruction (KVM currently injects a #UD into L2).

There's no real motivation behind this series. I spotted the PAUSE_EXITING vs. BUS_LOCK_DETECTION goof when sorting out a report/question about HLT emulation in L2 doing weird things, and then stupidly thought "how hard can it be to generate a VM-Exit?". Turns out, not that hard, but definitely a bit harder than I was anticipating due to the annoying RIP vs. next RIP flaw.

Given that VMX has literally never done the right thing, and SVM was quite broken since the beginning, I doubt anyone cares about this, but we have the code, so why not...

Sean Christopherson (11):
KVM: nVMX: Check PAUSE_EXITING, not BUS_LOCK_DETECTION, on PAUSE
emulation
KVM: nSVM: Pass next RIP, not current RIP, for nested VM-Exit on
emulation
KVM: nVMX: Allow emulating RDPID on behalf of L2
KVM: nVMX: Emulate HLT in L2 if it's not intercepted
KVM: nVMX: Consolidate missing X86EMUL_INTERCEPTED logic in L2
emulation
KVM: x86: Plumb the src/dst operand types through to
.check_intercept()
KVM: x86: Plumb the emulator's starting RIP into nested intercept
checks
KVM: x86: Add a #define for the architectural max instruction length
KVM: nVMX: Allow the caller to provide instruction length on nested
VM-Exit
KVM: nVMX: Synthesize nested VM-Exit for supported emulation
intercepts
KVM: selftests: Add a nested (forced) emulation intercept test for x86

 arch/x86/kvm/emulate.c                        |   5 +-
 arch/x86/kvm/kvm_emulate.h                    |   7 +-
 arch/x86/kvm/trace.h                          |  14 +-
 arch/x86/kvm/vmx/nested.c                     |  14 +-
 arch/x86/kvm/vmx/nested.h                     |  22 ++-
 arch/x86/kvm/vmx/vmx.c                        | 102 ++++++++----
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/nested_emulation_test.c | 146 ++++++++++++++++++
 8 files changed, 265 insertions(+), 46 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/nested_emulation_test.c

base-commit: eb723766b1030a23c38adf2348b7c3d1409d11f0

Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#1 lore raw

PATCH v2 1/11 KVM: nVMX: Check PAUSE_EXITING, not BUS_LOCK_DETECTION, on PAUSE emulation

When emulating PAUSE on behalf of L2, check for interception in vmcs12 by looking at primary execution controls, not secondary execution controls. Checking for PAUSE_EXITING in secondary execution controls effectively results in KVM looking for BUS_LOCK_DETECTION, which KVM doesn't expose to L1, i.e. is always off in vmcs12, and ultimately results in KVM failing to "intercept" PAUSE.

Because KVM doesn't handle interception during emulation correctly on VMX, i.e. the "fixed" code is still quite broken, and not intercepting PAUSE is relatively benign, for all intents and purposes the bug means that L2 gets to live when it would otherwise get an unexpected #UD.

Fixes 4984563823f0 ("KVM: nVMX: Emulate NOPs in L2, and PAUSE if it's not intercepted")
---
 arch/x86/kvm/vmx/vmx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
index f72835e85b6d..3654c08cfa31 100644
@@ -8092,7 +8092,7 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 		 * the PAUSE.
 		 */
 		if ((info->rep_prefix != REPE_PREFIX) ||
-		    !nested_cpu_has2(vmcs12, CPU_BASED_PAUSE_EXITING))
+		    !nested_cpu_has(vmcs12, CPU_BASED_PAUSE_EXITING))
 			return X86EMUL_CONTINUE;
 
 		break;
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#2 lore raw

PATCH v2 2/11 KVM: nSVM: Pass next RIP, not current RIP, for nested VM-Exit on emulation

In reply to Sean Christopherson

Set "next_rip" in the emulation interception info passed to vendor code using the emulator context's "_eip", not "eip". "eip" holds RIP from the start of emulation, i.e. the RIP of the instruction that's being emulated, whereas _eip tracks the context's current position in decoding the code stream, which at the time of the intercept checks is effectively the RIP of the next instruction.

Passing the current RIP as next_rip causes SVM to stuff the wrong value value into vmcb12->control.next_rip if a nested VM-Exit is generated, i.e. if L1 wants to intercept the instruction, and could result in L1 putting L2 into an infinite loop due to restarting L2 with the same RIP over and over.

Fixes 8a76d7f25f8f ("KVM: x86: Add x86 callback for intercept check")
---
 arch/x86/kvm/emulate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/emulate.c +1 -1
index 60986f67c35a..0915b5e8aa71 100644
@@ -478,7 +478,7 @@ static int emulator_check_intercept(struct x86_emulate_ctxt *ctxt,
 		.src_bytes  = ctxt->src.bytes,
 		.dst_bytes  = ctxt->dst.bytes,
 		.ad_bytes   = ctxt->ad_bytes,
-		.next_rip   = ctxt->eip,
+		.next_rip   = ctxt->_eip,
 	};
 
 	return ctxt->ops->intercept(ctxt, &info, stage);
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#3 lore raw

PATCH v2 3/11 KVM: nVMX: Allow emulating RDPID on behalf of L2

In reply to Sean Christopherson

Return X86EMUL_CONTINUE instead X86EMUL_UNHANDLEABLE when emulating RDPID on behalf of L2 and L1 _does_ expose RDPID/RDTSCP to L2. When RDPID emulation was added by commit fb6d4d340e05 ("KVM: x86: emulate RDPID"), KVM incorrectly allowed emulation by default. Commit 07721feee46b ("KVM:
nVMX: Don't emulate instructions in guest mode") fixed that flaw, but missed that RDPID emulation was relying on the common return path to allow emulation on behalf of L2.

Fixes 07721feee46b ("KVM: nVMX: Don't emulate instructions in guest mode")
---
 arch/x86/kvm/vmx/vmx.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
arch/x86/kvm/vmx/vmx.c +7 -6
index 3654c08cfa31..9773287acade 100644
@@ -8049,18 +8049,19 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
 
 	switch (info->intercept) {
-	/*
-	 * RDPID causes #UD if disabled through secondary execution controls.
-	 * Because it is marked as EmulateOnUD, we need to intercept it here.
-	 * Note, RDPID is hidden behind ENABLE_RDTSCP.
-	 */
 	case x86_intercept_rdpid:
+		/*
+		 * RDPID causes #UD if not enabled through secondary execution
+		 * controls (ENABLE_RDTSCP).  Note, the implicit MSR access to
+		 * TSC_AUX is NOT subject to interception, i.e. checking only
+		 * the dedicated execution control is architecturally correct.
+		 */
 		if (!nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENABLE_RDTSCP)) {
 			exception->vector = UD_VECTOR;
 			exception->error_code_valid = false;
 			return X86EMUL_PROPAGATE_FAULT;
 		}
-		break;
+		return X86EMUL_CONTINUE;
 
 	case x86_intercept_in:
 	case x86_intercept_ins:
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#4 lore raw

PATCH v2 4/11 KVM: nVMX: Emulate HLT in L2 if it's not intercepted

In reply to Sean Christopherson

Extend VMX's nested intercept logic for emulated instructions to handle HLT interception, primarily for testing purposes. Failure to allow emulation of HLT isn't all that interesting, as emulating HLT while L2 is active either requires forced emulation (and no #UD intercept in L1), TLB games in the guest to coerce KVM into emulating the wrong instruction, or a bug elsewhere in KVM.

E.g. without commit 47ef3ef843c0 ("KVM: VMX: Handle event vectoring error in check_emulate_instruction()"), KVM can end up trying to emulate HLT if RIP happens to point at a HLT when a vectored event arrives with L2's IDT pointing at emulated MMIO.

Note, vmx_check_intercept() is still broken when L1 wants to intercept an instruction, as KVM injects a #UD instead of synthesizing a nested VM-Exit. That issue extends far beyond HLT, punt on it for now.

---
 arch/x86/kvm/vmx/vmx.c | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/kvm/vmx/vmx.c +5 -0
index 9773287acade..fb4e9290e6c4 100644
@@ -8083,6 +8083,11 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 		/* FIXME: produce nested vmexit and return X86EMUL_INTERCEPTED.  */
 		break;
 
+	case x86_intercept_hlt:
+		if (!nested_cpu_has(vmcs12, CPU_BASED_HLT_EXITING))
+			return X86EMUL_CONTINUE;
+		break;
+
 	case x86_intercept_pause:
 		/*
 		 * PAUSE is a single-byte NOP with a REPE prefix, i.e. collides
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#5 lore raw

PATCH v2 5/11 KVM: nVMX: Consolidate missing X86EMUL_INTERCEPTED logic in L2 emulation

In reply to Sean Christopherson

Refactor the handling of port I/O interception checks when emulating on behalf of L2 in anticipation of synthesizing a nested VM-Exit to L1 instead of injecting a #UD into L2.

No functional change intended.

---
 arch/x86/kvm/vmx/vmx.c | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx/vmx.c +7 -11
index fb4e9290e6c4..dba22536eea3 100644
@@ -8007,12 +8007,11 @@ static __init void vmx_set_cpu_caps(void)
 		kvm_cpu_cap_check_and_set(X86_FEATURE_WAITPKG);
 }
 
-static int vmx_check_intercept_io(struct kvm_vcpu *vcpu,
+static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
 				  struct x86_instruction_info *info)
 {
 	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
 	unsigned short port;
-	bool intercept;
 	int size;
 
 	if (info->intercept == x86_intercept_in ||
@@ -8032,13 +8031,9 @@ static int vmx_check_intercept_io(struct kvm_vcpu *vcpu,
 	 * Otherwise, IO instruction VM-exits are controlled by the IO bitmaps.
 	 */
 	if (!nested_cpu_has(vmcs12, CPU_BASED_USE_IO_BITMAPS))
-		intercept = nested_cpu_has(vmcs12,
-					   CPU_BASED_UNCOND_IO_EXITING);
-	else
-		intercept = nested_vmx_check_io_bitmaps(vcpu, port, size);
+		return nested_cpu_has(vmcs12, CPU_BASED_UNCOND_IO_EXITING);
 
-	/* FIXME: produce nested vmexit and return X86EMUL_INTERCEPTED.  */
-	return intercept ? X86EMUL_UNHANDLEABLE : X86EMUL_CONTINUE;
+	return nested_vmx_check_io_bitmaps(vcpu, port, size);
 }
 
 int vmx_check_intercept(struct kvm_vcpu *vcpu,
@@ -8067,7 +8062,9 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 	case x86_intercept_ins:
 	case x86_intercept_out:
 	case x86_intercept_outs:
-		return vmx_check_intercept_io(vcpu, info);
+		if (!vmx_is_io_intercepted(vcpu, info))
+			return X86EMUL_CONTINUE;
+		break;
 
 	case x86_intercept_lgdt:
 	case x86_intercept_lidt:
@@ -8079,8 +8076,6 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 	case x86_intercept_str:
 		if (!nested_cpu_has2(vmcs12, SECONDARY_EXEC_DESC))
 			return X86EMUL_CONTINUE;
-
-		/* FIXME: produce nested vmexit and return X86EMUL_INTERCEPTED.  */
 		break;
 
 	case x86_intercept_hlt:
@@ -8108,6 +8103,7 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 		break;
 	}
 
+	/* FIXME: produce nested vmexit and return X86EMUL_INTERCEPTED.  */
 	return X86EMUL_UNHANDLEABLE;
 }
 
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#6 lore raw

PATCH v2 6/11 KVM: x86: Plumb the src/dst operand types through to .check_intercept()

In reply to Sean Christopherson

When checking for intercept when emulating an instruction on behalf of L2, forward the source and destination operand types to vendor code so that VMX can synthesize the correct EXIT_QUALIFICATION for port I/O VM-Exits.

---
 arch/x86/kvm/emulate.c     | 2 ++
 arch/x86/kvm/kvm_emulate.h | 2 ++
 2 files changed, 4 insertions(+)
arch/x86/kvm/emulate.c +2 -0
index 0915b5e8aa71..ca613796b5af 100644
@@ -477,6 +477,8 @@ static int emulator_check_intercept(struct x86_emulate_ctxt *ctxt,
 		.dst_val    = ctxt->dst.val64,
 		.src_bytes  = ctxt->src.bytes,
 		.dst_bytes  = ctxt->dst.bytes,
+		.src_type   = ctxt->src.type,
+		.dst_type   = ctxt->dst.type,
 		.ad_bytes   = ctxt->ad_bytes,
 		.next_rip   = ctxt->_eip,
 	};
arch/x86/kvm/kvm_emulate.h +2 -0
index 73072585e164..49ab8b060137 100644
@@ -44,6 +44,8 @@ struct x86_instruction_info {
 	u64 dst_val;            /* value of destination operand         */
 	u8  src_bytes;          /* size of source operand               */
 	u8  dst_bytes;          /* size of destination operand          */
+	u8  src_type;		/* type of source operand		*/
+	u8  dst_type;		/* type of destination operand		*/
 	u8  ad_bytes;           /* size of src/dst address              */
 	u64 next_rip;           /* rip following the instruction        */
 };
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#7 lore raw

PATCH v2 7/11 KVM: x86: Plumb the emulator's starting RIP into nested intercept checks

In reply to Sean Christopherson

When checking for intercept when emulating an instruction on behalf of L2, pass the emulator's view of the RIP of the instruction being emulated to vendor code. Unlike SVM, which communicates the next RIP on VM-Exit, VMX communicates the length of the instruction that generated the VM-Exit, i.e. requires the current and next RIPs.

Note, unless userspace modifies RIP during a userspace exit that requires completion, kvm_rip_read() will contain the same information. Pass the emulator's view largely out of a paranoia, and because there is no meaningful cost in doing so.

---
 arch/x86/kvm/emulate.c     | 1 +
 arch/x86/kvm/kvm_emulate.h | 1 +
 2 files changed, 2 insertions(+)
arch/x86/kvm/emulate.c +1 -0
index ca613796b5af..1349e278cd2a 100644
@@ -480,6 +480,7 @@ static int emulator_check_intercept(struct x86_emulate_ctxt *ctxt,
 		.src_type   = ctxt->src.type,
 		.dst_type   = ctxt->dst.type,
 		.ad_bytes   = ctxt->ad_bytes,
+		.rip	    = ctxt->eip,
 		.next_rip   = ctxt->_eip,
 	};
 
arch/x86/kvm/kvm_emulate.h +1 -0
index 49ab8b060137..35029b12667f 100644
@@ -47,6 +47,7 @@ struct x86_instruction_info {
 	u8  src_type;		/* type of source operand		*/
 	u8  dst_type;		/* type of destination operand		*/
 	u8  ad_bytes;           /* size of src/dst address              */
+	u64 rip;		/* rip of the instruction		*/
 	u64 next_rip;           /* rip following the instruction        */
 };
 
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#8 lore raw

PATCH v2 8/11 KVM: x86: Add a #define for the architectural max instruction length

In reply to Sean Christopherson

Add a #define to capture x86's architecturally defined max instruction length instead of open coding the literal in a variety of places.

No functional change intended.

---
 arch/x86/kvm/kvm_emulate.h |  4 +++-
 arch/x86/kvm/trace.h       | 14 +++++++-------
 arch/x86/kvm/vmx/nested.c  |  2 +-
 3 files changed, 11 insertions(+), 9 deletions(-)
arch/x86/kvm/kvm_emulate.h +3 -1
index 35029b12667f..c1df5acfacaf 100644
@@ -275,8 +275,10 @@ struct operand {
 	};
 };
 
+#define X86_MAX_INSTRUCTION_LENGTH	15
+
 struct fetch_cache {
-	u8 data[15];
+	u8 data[X86_MAX_INSTRUCTION_LENGTH];
 	u8 *ptr;
 	u8 *end;
 };
arch/x86/kvm/trace.h +7 -7
index 0b844cb97978..ccda95e53f62 100644
@@ -830,12 +830,12 @@ TRACE_EVENT(kvm_emulate_insn,
 	TP_ARGS(vcpu, failed),
 
 	TP_STRUCT__entry(
-		__field(    __u64, rip                       )
-		__field(    __u32, csbase                    )
-		__field(    __u8,  len                       )
-		__array(    __u8,  insn,    15	             )
-		__field(    __u8,  flags       	   	     )
-		__field(    __u8,  failed                    )
+		__field(    __u64, rip                              )
+		__field(    __u32, csbase                           )
+		__field(    __u8,  len                              )
+		__array(    __u8,  insn, X86_MAX_INSTRUCTION_LENGTH )
+		__field(    __u8,  flags       	   	            )
+		__field(    __u8,  failed                           )
 		),
 
 	TP_fast_assign(
@@ -846,7 +846,7 @@ TRACE_EVENT(kvm_emulate_insn,
 		__entry->rip = vcpu->arch.emulate_ctxt->_eip - __entry->len;
 		memcpy(__entry->insn,
 		       vcpu->arch.emulate_ctxt->fetch.data,
-		       15);
+		       X86_MAX_INSTRUCTION_LENGTH);
 		__entry->flags = kei_decode_mode(vcpu->arch.emulate_ctxt->mode);
 		__entry->failed = failed;
 		),
arch/x86/kvm/vmx/nested.c +1 -1
index 8a7af02d466e..fb4fd96ce0f8 100644
@@ -2970,7 +2970,7 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 		case INTR_TYPE_SOFT_EXCEPTION:
 		case INTR_TYPE_SOFT_INTR:
 		case INTR_TYPE_PRIV_SW_EXCEPTION:
-			if (CC(vmcs12->vm_entry_instruction_len > 15) ||
+			if (CC(vmcs12->vm_entry_instruction_len > X86_MAX_INSTRUCTION_LENGTH) ||
 			    CC(vmcs12->vm_entry_instruction_len == 0 &&
 			    CC(!nested_cpu_has_zero_length_injection(vcpu))))
 				return -EINVAL;
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#9 lore raw

PATCH v2 9/11 KVM: nVMX: Allow the caller to provide instruction length on nested VM-Exit

In reply to Sean Christopherson

Rework the nested VM-Exit helper to take the instruction length as a parameter, and convert nested_vmx_vmexit() into a "default" wrapper that grabs the length from vmcs02 as appropriate. This will allow KVM to set the correct instruction length when synthesizing a nested VM-Exit when emulating an instruction that L1 wants to intercept.

No functional change intended, as the path to prepare_vmcs12()'s reading of vmcs02.VM_EXIT_INSTRUCTION_LEN is gated on the same set of conditions as the VMREAD in the new nested_vmx_vmexit().

---
 arch/x86/kvm/vmx/nested.c | 12 +++++++-----
 arch/x86/kvm/vmx/nested.h | 22 ++++++++++++++++++++--
 2 files changed, 27 insertions(+), 7 deletions(-)
arch/x86/kvm/vmx/nested.c +7 -5
index fb4fd96ce0f8..791e00d467df 100644
@@ -4618,7 +4618,7 @@ static void sync_vmcs02_to_vmcs12(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12)
  */
 static void prepare_vmcs12(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
 			   u32 vm_exit_reason, u32 exit_intr_info,
-			   unsigned long exit_qualification)
+			   unsigned long exit_qualification, u32 exit_insn_len)
 {
 	/* update exit information fields: */
 	vmcs12->vm_exit_reason = vm_exit_reason;
@@ -4646,7 +4646,7 @@ static void prepare_vmcs12(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
 					  vm_exit_reason, exit_intr_info);
 
 		vmcs12->vm_exit_intr_info = exit_intr_info;
-		vmcs12->vm_exit_instruction_len = vmcs_read32(VM_EXIT_INSTRUCTION_LEN);
+		vmcs12->vm_exit_instruction_len = exit_insn_len;
 		vmcs12->vmx_instruction_info = vmcs_read32(VMX_INSTRUCTION_INFO);
 
 		/*
@@ -4930,8 +4930,9 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)
  * and modify vmcs12 to make it see what it would expect to see there if
  * L2 was its real guest. Must only be called when in L2 (is_guest_mode())
  */
-void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
-		       u32 exit_intr_info, unsigned long exit_qualification)
+void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
+			 u32 exit_intr_info, unsigned long exit_qualification,
+			 u32 exit_insn_len)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
@@ -4981,7 +4982,8 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
 
 		if (vm_exit_reason != -1)
 			prepare_vmcs12(vcpu, vmcs12, vm_exit_reason,
-				       exit_intr_info, exit_qualification);
+				       exit_intr_info, exit_qualification,
+				       exit_insn_len);
 
 		/*
 		 * Must happen outside of sync_vmcs02_to_vmcs12() as it will
arch/x86/kvm/vmx/nested.h +20 -2
index 2c296b6abb8c..6eedcfc91070 100644
@@ -26,8 +26,26 @@ void nested_vmx_free_vcpu(struct kvm_vcpu *vcpu);
 enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,
 						     bool from_vmentry);
 bool nested_vmx_reflect_vmexit(struct kvm_vcpu *vcpu);
-void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
-		       u32 exit_intr_info, unsigned long exit_qualification);
+void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
+			 u32 exit_intr_info, unsigned long exit_qualification,
+			 u32 exit_insn_len);
+
+static inline void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
+				     u32 exit_intr_info,
+				     unsigned long exit_qualification)
+{
+	u32 exit_insn_len;
+
+	if (to_vmx(vcpu)->fail || vm_exit_reason == -1 ||
+	    (vm_exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY))
+		exit_insn_len = 0;
+	else
+		exit_insn_len = vmcs_read32(VM_EXIT_INSTRUCTION_LEN);
+
+	__nested_vmx_vmexit(vcpu, vm_exit_reason, exit_intr_info,
+			    exit_qualification, exit_insn_len);
+}
+
 void nested_sync_vmcs12_to_shadow(struct kvm_vcpu *vcpu);
 int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data);
 int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata);
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#10 lore raw

PATCH v2 10/11 KVM: nVMX: Synthesize nested VM-Exit for supported emulation intercepts

In reply to Sean Christopherson

When emulating an instruction on behalf of L2 that L1 wants to intercept, generate a nested VM-Exit instead of injecting a #UD into L2. Now that (most of) the necessary information is available, synthesizing a VM-Exit isn't terribly difficult.

Punt on decoding the ModR/M for descriptor table exits for now. There is no evidence that any hypervisor intercepts descriptor table accesses *and* uses the EXIT_QUALIFICATION to expedite emulation, i.e. it's not worth delaying basic support for.

To avoid doing more harm than good, e.g. by putting L2 into an infinite or effectively corrupting its code stream, inject #UD if the instruction length is nonsensical.

---
 arch/x86/kvm/vmx/vmx.c | 70 +++++++++++++++++++++++++++++++++---------
 1 file changed, 56 insertions(+), 14 deletions(-)
arch/x86/kvm/vmx/vmx.c +56 -14
index dba22536eea3..7b2a6921f156 100644
@@ -8008,20 +8008,13 @@ static __init void vmx_set_cpu_caps(void)
 }
 
 static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
-				  struct x86_instruction_info *info)
+				  struct x86_instruction_info *info,
+				  unsigned long *exit_qualification)
 {
 	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
 	unsigned short port;
 	int size;
-
-	if (info->intercept == x86_intercept_in ||
-	    info->intercept == x86_intercept_ins) {
-		port = info->src_val;
-		size = info->dst_bytes;
-	} else {
-		port = info->dst_val;
-		size = info->src_bytes;
-	}
+	bool imm;
 
 	/*
 	 * If the 'use IO bitmaps' VM-execution control is 0, IO instruction
@@ -8033,6 +8026,30 @@ static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
 	if (!nested_cpu_has(vmcs12, CPU_BASED_USE_IO_BITMAPS))
 		return nested_cpu_has(vmcs12, CPU_BASED_UNCOND_IO_EXITING);
 
+	if (info->intercept == x86_intercept_in ||
+	    info->intercept == x86_intercept_ins) {
+		port = info->src_val;
+		size = info->dst_bytes;
+		imm  = info->src_type == OP_IMM;
+	} else {
+		port = info->dst_val;
+		size = info->src_bytes;
+		imm  = info->dst_type == OP_IMM;
+	}
+
+
+	*exit_qualification = ((unsigned long)port << 16) | (size - 1);
+
+	if (info->intercept == x86_intercept_ins ||
+	    info->intercept == x86_intercept_outs)
+		*exit_qualification |= BIT(4);
+
+	if (info->rep_prefix)
+		*exit_qualification |= BIT(5);
+
+	if (imm)
+		*exit_qualification |= BIT(6);
+
 	return nested_vmx_check_io_bitmaps(vcpu, port, size);
 }
 
@@ -8042,6 +8059,9 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 			struct x86_exception *exception)
 {
 	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
+	unsigned long exit_qualification = 0;
+	u32 vm_exit_reason;
+	u64 exit_insn_len;
 
 	switch (info->intercept) {
 	case x86_intercept_rdpid:
@@ -8062,8 +8082,10 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 	case x86_intercept_ins:
 	case x86_intercept_out:
 	case x86_intercept_outs:
-		if (!vmx_is_io_intercepted(vcpu, info))
+		if (!vmx_is_io_intercepted(vcpu, info, &exit_qualification))
 			return X86EMUL_CONTINUE;
+
+		vm_exit_reason = EXIT_REASON_IO_INSTRUCTION;
 		break;
 
 	case x86_intercept_lgdt:
@@ -8076,11 +8098,25 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 	case x86_intercept_str:
 		if (!nested_cpu_has2(vmcs12, SECONDARY_EXEC_DESC))
 			return X86EMUL_CONTINUE;
+
+		if (info->intercept == x86_intercept_lldt ||
+		    info->intercept == x86_intercept_ltr ||
+		    info->intercept == x86_intercept_sldt ||
+		    info->intercept == x86_intercept_str)
+			vm_exit_reason = EXIT_REASON_LDTR_TR;
+		else
+			vm_exit_reason = EXIT_REASON_GDTR_IDTR;
+		/*
+		 * FIXME: Decode the ModR/M to generate the correct exit
+		 *        qualification for memory operands.
+		 */
 		break;
 
 	case x86_intercept_hlt:
 		if (!nested_cpu_has(vmcs12, CPU_BASED_HLT_EXITING))
 			return X86EMUL_CONTINUE;
+
+		vm_exit_reason = EXIT_REASON_HLT;
 		break;
 
 	case x86_intercept_pause:
@@ -8096,15 +8132,21 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu,
 		    !nested_cpu_has(vmcs12, CPU_BASED_PAUSE_EXITING))
 			return X86EMUL_CONTINUE;
 
+		vm_exit_reason = EXIT_REASON_PAUSE_INSTRUCTION;
 		break;
 
 	/* TODO: check more intercepts... */
 	default:
-		break;
+		return X86EMUL_UNHANDLEABLE;
 	}
 
-	/* FIXME: produce nested vmexit and return X86EMUL_INTERCEPTED.  */
-	return X86EMUL_UNHANDLEABLE;
+	exit_insn_len = abs_diff((s64)info->next_rip, (s64)info->rip);
+	if (!exit_insn_len || exit_insn_len > X86_MAX_INSTRUCTION_LENGTH)
+		return X86EMUL_UNHANDLEABLE;
+
+	__nested_vmx_vmexit(vcpu, vm_exit_reason, 0, exit_qualification,
+			    exit_insn_len);
+	return X86EMUL_INTERCEPTED;
 }
 
 #ifdef CONFIG_X86_64
Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer
#11 lore raw

PATCH v2 11/11 KVM: selftests: Add a nested (forced) emulation intercept test for x86

In reply to Sean Christopherson

Add a rudimentary test for validating KVM's handling of L1 hypervisor intercepts during instruction emulation on behalf of L2. To minimize complexity and avoid overlap with other tests, only validate KVM's handling of instructions that L1 wants to intercept, i.e. that generate a nested VM-Exit. Full testing of emulation on behalf of L2 is better achieved by running existing (forced) emulation tests in a VM, (although on VMX, getting L0 to emulate on #UD requires modifying either L1 KVM to not intercept #UD, or modifying L0 KVM to prioritize L0's exception intercepts over L1's intercepts, as is done by KVM for SVM).

Since emulation should never be successful, i.e. L2 always exits to L1, dynamically generate the L2 code stream instead of adding a helper for each instruction. Doing so requires hand coding instruction opcodes, but makes it significantly easier for the test to compute the expected "next RIP" and instruction length.

---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/nested_emulation_test.c | 146 ++++++++++++++++++
 2 files changed, 147 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/nested_emulation_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 4277b983cace..f773f8f99249 100644
@@ -69,6 +69,7 @@ TEST_GEN_PROGS_x86 += x86/hyperv_tlb_flush
 TEST_GEN_PROGS_x86 += x86/kvm_clock_test
 TEST_GEN_PROGS_x86 += x86/kvm_pv_test
 TEST_GEN_PROGS_x86 += x86/monitor_mwait_test
+TEST_GEN_PROGS_x86 += x86/nested_emulation_test
 TEST_GEN_PROGS_x86 += x86/nested_exceptions_test
 TEST_GEN_PROGS_x86 += x86/platform_info_test
 TEST_GEN_PROGS_x86 += x86/pmu_counters_test
tools/testing/selftests/kvm/x86/nested_emulation_test.c new file +146 -0

146 lines not on this page · raw message

Signature · 1 line
2.48.1.362.g079036d154-goog
author maintainer Applied
#12 lore raw
In reply to Sean Christopherson
Quoted 4 lines · Sean Christopherson

Fix a variety of bugs related to emulating instructions on behalf of L2, and (finally) add support for synthesizing nested VM-Exit to L1 when L1 wants to intercept an instruction (KVM currently injects a #UD into L2).

Quoted 8 lines

There's no real motivation behind this series. I spotted the PAUSE_EXITING vs. BUS_LOCK_DETECTION goof when sorting out a report/question about HLT emulation in L2 doing weird things, and then stupidly thought "how hard can it be to generate a VM-Exit?". Turns out, not that hard, but definitely a bit harder than I was anticipating due to the annoying RIP vs. next RIP flaw.

[...]

Applied to kvm-x86 misc, thanks!

[01/11] KVM: nVMX: Check PAUSE_EXITING, not BUS_LOCK_DETECTION, on PAUSE emulation

        https://github.com/kvm-x86/linux/commit/f002a97ec8c9
[02/11] KVM: nSVM: Pass next RIP, not current RIP, for nested VM-Exit on emulation
        https://github.com/kvm-x86/linux/commit/c8e612bfedff
[03/11] KVM: nVMX: Allow emulating RDPID on behalf of L2
        https://github.com/kvm-x86/linux/commit/3244616aac8d
[04/11] KVM: nVMX: Emulate HLT in L2 if it's not intercepted
        https://github.com/kvm-x86/linux/commit/f43f7a215af0
[05/11] KVM: nVMX: Consolidate missing X86EMUL_INTERCEPTED logic in L2 emulation
        https://github.com/kvm-x86/linux/commit/08e3d89eb330
[06/11] KVM: x86: Plumb the src/dst operand types through to .check_intercept()
        https://github.com/kvm-x86/linux/commit/407d03fe924c
[07/11] KVM: x86: Plumb the emulator's starting RIP into nested intercept checks
        https://github.com/kvm-x86/linux/commit/9aeb9d8a6738
[08/11] KVM: x86: Add a #define for the architectural max instruction length
        https://github.com/kvm-x86/linux/commit/d4aea23fd0ff
[09/11] KVM: nVMX: Allow the caller to provide instruction length on nested VM-Exit
        https://github.com/kvm-x86/linux/commit/fbd1e0f19546
[10/11] KVM: nVMX: Synthesize nested VM-Exit for supported emulation intercepts
        https://github.com/kvm-x86/linux/commit/79a14afc6090
[11/11] KVM: selftests: Add a nested (forced) emulation intercept test for x86
        https://github.com/kvm-x86/linux/commit/2428865bf0af

--
https://github.com/kvm-x86/linux/tree/next