Skip to contentSkip to conversation

KVM: nVMX: Leave VMX mode on apparent CPU reset

RFC PATCH In review Jan Kiszka 3 participants 7 messages Dec 16, 2013 – Dec 30, 2013

Subsystems (3)
lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

RFC PATCH 1/1 KVM: nVMX: Leave VMX mode on apparent CPU reset

As long as we do not expose all the VMX related states to user space, there is no way to properly reset a VCPU when VMX is enabled. Emulate this for now by catching host-side clearings of the feature control MSR. This allows to reboot a VM while it is running some hypervisor code.

---

Better ideas? Or continue to leave it as it is?

 arch/x86/kvm/vmx.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
arch/x86/kvm/vmx.c +35 -0
index f90320b..da04247 100644
@@ -2455,6 +2455,8 @@ static int vmx_get_vmx_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 *pdata)
 	return 1;
 }
 
+static void vmx_reset_nested(struct kvm_vcpu *vcpu);
+
 static int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	u32 msr_index = msr_info->index;
@@ -2470,6 +2472,12 @@ static int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 				& FEATURE_CONTROL_LOCKED)
 			return 0;
 		to_vmx(vcpu)->nested.msr_ia32_feature_control = data;
+		/*
+		 * Detect reset and allow to leave VMX mode this way until we
+		 * expose all related states to user space.
+		 */
+		if (host_initialized && data == 0)
+			vmx_reset_nested(vcpu);
 		return 1;
 	}
 
@@ -8487,6 +8495,33 @@ static void nested_vmx_vmexit(struct kvm_vcpu *vcpu)
 		vmx->nested.sync_shadow_vmcs = true;
 }
 
+static void vmx_reset_nested(struct kvm_vcpu *vcpu)
+{
+	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+
+	if (!vmx->nested.vmxon)
+		return;
+
+	if (is_guest_mode(vcpu)) {
+		vmcs12->host_cr0 = X86_CR0_NW | X86_CR0_CD | X86_CR0_ET;
+		vmcs12->host_cr3 = 0;
+		vmcs12->host_cr4 = 0;
+		vmcs12->host_rsp = 0;
+		vmcs12->vm_exit_controls = 0;
+		nested_vmx_vmexit(vcpu);
+	}
+
+	free_nested(vmx);
+
+	/*
+	 * If we were in guest mode, the reset state user space wrote so far
+	 * is now inconsistent. If we were in host mode, some state update may
+	 * have been rejected. So simply repeat the reset her.
+	 */
+	vmx_vcpu_reset(vcpu);
+}
+
 /*
  * L1's failure to enter L2 is a subset of a normal exit, as explained in
  * 23.7 "VM-entry failures during or after loading guest state" (this also
Signature · 1 line
1.8.1.1.298.ge7eed54
maintainer
#1 lore raw

Il 16/12/2013 10:32, Jan Kiszka ha scritto:

Quoted 9 lines

As long as we do not expose all the VMX related states to user space, there is no way to properly reset a VCPU when VMX is enabled. Emulate this for now by catching host-side clearings of the feature control MSR. This allows to reboot a VM while it is running some hypervisor code.

---

Better ideas? Or continue to leave it as it is?

The final vmx_vcpu_reset is the only really ugly part, but it is _really_ ugly... Can you modify QEMU to restore MSRs first, and reduce vmx_reset_nested to just

	if (is_guest_mode(vcpu))
		nested_vmx_vmexit(vcpu);

	free_nested(vmx);

?

Paolo

author
#2 lore raw
Quoted 15 lines · Paolo Bonzini

Il 16/12/2013 10:32, Jan Kiszka ha scritto:

Quoted 9 lines

As long as we do not expose all the VMX related states to user space, there is no way to properly reset a VCPU when VMX is enabled. Emulate this for now by catching host-side clearings of the feature control MSR. This allows to reboot a VM while it is running some hypervisor code.

---

Better ideas? Or continue to leave it as it is?

The final vmx_vcpu_reset is the only really ugly part, but it is _really_ ugly... Can you modify QEMU to restore MSRs first, and reduce vmx_reset_nested to just

Quoted 6 lines
	if (is_guest_mode(vcpu))
		nested_vmx_vmexit(vcpu);

	free_nested(vmx);

?

Well, I could make setting of MSR_IA32_FEATURE_CONTROL to 0 an official "clear VMX" interface. Then QEMU would have to issue this MSR set request before doing any other CPU state manipulation. Is that what you have in mind?

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
maintainer
#3 lore raw

Il 17/12/2013 15:40, Jan Kiszka ha scritto:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

The final vmx_vcpu_reset is the only really ugly part, but it is _really_ ugly... Can you modify QEMU to restore MSRs first, and reduce vmx_reset_nested to just

	if (is_guest_mode(vcpu))
		nested_vmx_vmexit(vcpu);
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
	free_nested(vmx);

?

Well, I could make setting of MSR_IA32_FEATURE_CONTROL to 0 an official "clear VMX" interface. Then QEMU would have to issue this MSR set request before doing any other CPU state manipulation. Is that what you have in mind?

Yes, that was the idea.

Paolo

#4 lore raw
In reply to Jan Kiszka
Quoted 9 lines · Jan Kiszka

As long as we do not expose all the VMX related states to user space, there is no way to properly reset a VCPU when VMX is enabled. Emulate this for now by catching host-side clearings of the feature control MSR. This allows to reboot a VM while it is running some hypervisor code.

---

Better ideas? Or continue to leave it as it is?

The hidden backport might be problematic... How about an explicit notification to leave guest mode?
Such as a new flag to KVM_SET_REGS (or even a separate ioctl?).

#5 lore raw
Quoted 6 lines · Marcelo Tosatti
Quoted 5 lines · Jan Kiszka

As long as we do not expose all the VMX related states to user space, there is no way to properly reset a VCPU when VMX is enabled. Emulate this for now by catching host-side clearings of the feature control MSR. This allows to reboot a VM while it is running some hypervisor code.

Quoted 9 lines
Quoted 4 lines

---

Better ideas? Or continue to leave it as it is?

The hidden backport might be problematic... How about an explicit notification to leave guest mode?
Such as a new flag to KVM_SET_REGS (or even a separate ioctl?).

Nevermind.

#6 lore raw
In reply to Jan Kiszka
Quoted 9 lines · Jan Kiszka

As long as we do not expose all the VMX related states to user space, there is no way to properly reset a VCPU when VMX is enabled. Emulate this for now by catching host-side clearings of the feature control MSR. This allows to reboot a VM while it is running some hypervisor code.

---

Better ideas? Or continue to leave it as it is?

So basically reset is broken, with nested virt?