Skip to contentSkip to conversation

kvm: nvmx: limit atomic switch MSRs

PATCH Superseded Marc Orr 2 participants 3 messages Sep 12, 2019 – Sep 13, 2019

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Marc Orr

author
#0 lore raw

PATCH 1/1 kvm: nvmx: limit atomic switch MSRs

Allowing an unlimited number of MSRs to be specified via the VMX load/store MSR lists (e.g., vm-entry MSR load list) is bad for two reasons. First, a guest can specify an unreasonable number of MSRs, forcing KVM to process all of them in software. Second, the SDM bounds the number of MSRs allowed to be packed into the atomic switch MSR lists. Quoting the appendix chapter, titled "MISCELLANEOUS DATA":

"Bits 27:25 is used to compute the recommended maximum number of MSRs that should appear in the VM-exit MSR-store list, the VM-exit MSR-load list, or the VM-entry MSR-load list. Specifically, if the value bits 27:25 of IA32_VMX_MISC is N, then 512 * (N + 1) is the recommended maximum number of MSRs to be included in each list. If the limit is exceeded, undefined processor behavior may result (including a machine check during the VMX transition)."

Thus, force a VM-entry to fail due to MSR loading when the MSR load list is too large. Similarly, trigger an abort during a VM exit that encounters an MSR load list or MSR store list that is too large.

Test these new checks with the kvm-unit-test "x86: nvmx: test max atomic switch MSRs".

---
 arch/x86/include/asm/vmx.h |  1 +
 arch/x86/kvm/vmx/nested.c  | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
arch/x86/include/asm/vmx.h +1 -0
index a39136b0d509..21c2a1d982e8 100644
@@ -110,6 +110,7 @@
 #define VMX_MISC_SAVE_EFER_LMA			0x00000020
 #define VMX_MISC_ACTIVITY_HLT			0x00000040
 #define VMX_MISC_ZERO_LEN_INS			0x40000000
+#define VMX_MISC_MSR_LIST_INCREMENT             512
 
 /* VMFUNC functions */
 #define VMX_VMFUNC_EPTP_SWITCHING               0x00000001
arch/x86/kvm/vmx/nested.c +19 -0
index ced9fba32598..69c6fc5557d8 100644
@@ -856,6 +856,17 @@ static int nested_vmx_store_msr_check(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static u64 vmx_control_msr(u32 low, u32 high);
+
+static u32 nested_vmx_max_atomic_switch_msrs(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	u64 vmx_misc = vmx_control_msr(vmx->nested.msrs.misc_low,
+				       vmx->nested.msrs.misc_high);
+
+	return (vmx_misc_max_msr(vmx_misc) + 1) * VMX_MISC_MSR_LIST_INCREMENT;
+}
+
 /*
  * Load guest's/host's msr at nested entry/exit.
  * return 0 for success, entry index for failure.
@@ -865,9 +876,13 @@ static u32 nested_vmx_load_msr(struct kvm_vcpu *vcpu, u64 gpa, u32 count)
 	u32 i;
 	struct vmx_msr_entry e;
 	struct msr_data msr;
+	u32 max_msr_list_size = nested_vmx_max_atomic_switch_msrs(vcpu);
 
 	msr.host_initiated = false;
 	for (i = 0; i < count; i++) {
+		if (unlikely(i >= max_msr_list_size))
+			goto fail;
+
 		if (kvm_vcpu_read_guest(vcpu, gpa + i * sizeof(e),
 					&e, sizeof(e))) {
 			pr_debug_ratelimited(
@@ -899,6 +914,10 @@ static int nested_vmx_store_msr(struct kvm_vcpu *vcpu, u64 gpa, u32 count)
 {
 	u32 i;
 	struct vmx_msr_entry e;
+	u32 max_msr_list_size = nested_vmx_max_atomic_switch_msrs(vcpu);
+
+	if (unlikely(count > max_msr_list_size))
+		return -EINVAL;
 
 	for (i = 0; i < count; i++) {
 		struct msr_data msr_info;
Signature · 1 line
2.23.0.237.gc6a4ce50a0-goog
#1 lore raw
Quoted 6 lines · Marc Orr

Allowing an unlimited number of MSRs to be specified via the VMX load/store MSR lists (e.g., vm-entry MSR load list) is bad for two reasons. First, a guest can specify an unreasonable number of MSRs, forcing KVM to process all of them in software. Second, the SDM bounds the number of MSRs allowed to be packed into the atomic switch MSR lists. Quoting the appendix chapter, titled "MISCELLANEOUS DATA":

Super Nit: There are multiple appendices in the SDM, maybe this?

Quoting the "Miscellaneous Data" section in the "VMX Capability Reporting Facility" appendix:

Quoted 29 lines

"Bits 27:25 is used to compute the recommended maximum number of MSRs that should appear in the VM-exit MSR-store list, the VM-exit MSR-load list, or the VM-entry MSR-load list. Specifically, if the value bits 27:25 of IA32_VMX_MISC is N, then 512 * (N + 1) is the recommended maximum number of MSRs to be included in each list. If the limit is exceeded, undefined processor behavior may result (including a machine check during the VMX transition)."

Thus, force a VM-entry to fail due to MSR loading when the MSR load list is too large. Similarly, trigger an abort during a VM exit that encounters an MSR load list or MSR store list that is too large.

Test these new checks with the kvm-unit-test "x86: nvmx: test max atomic switch MSRs".

---
 arch/x86/include/asm/vmx.h |  1 +
 arch/x86/kvm/vmx/nested.c  | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
arch/x86/include/asm/vmx.h +0 -0
index a39136b0d509..21c2a1d982e8 100644

arch/x86/include/asm/vmx.h line 113

Quoted 4 lines
arch/x86/include/asm/vmx.h +1 -0
@@ -110,6 +110,7 @@
 #define VMX_MISC_SAVE_EFER_LMA			0x00000020
 #define VMX_MISC_ACTIVITY_HLT			0x00000040
 #define VMX_MISC_ZERO_LEN_INS			0x40000000
+#define VMX_MISC_MSR_LIST_INCREMENT             512

VMX_MISC_MSR_LIST_MULTIPLIER seems more appropriate. INCREMENT makes it sound like X number of entries get tacked on to the end.

arch/x86/kvm/vmx/nested.c static int nested_vmx_store_msr_check(struct kvm_vcpu *vcpu, · line 859

Quoted 12 lines
arch/x86/include/asm/vmx.h +0 -0
 /* VMFUNC functions */
 #define VMX_VMFUNC_EPTP_SWITCHING               0x00000001
arch/x86/kvm/vmx/nested.c +1 -0
index ced9fba32598..69c6fc5557d8 100644
@@ -856,6 +856,17 @@ static int nested_vmx_store_msr_check(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static u64 vmx_control_msr(u32 low, u32 high);

vmx_control_msr() is a 'static inline', just hoist it above this function. It probably makes sense to move vmx_control_verify() too, maybe to just below nested_vmx_abort()?

Quoted 13 lines
arch/x86/kvm/vmx/nested.c +9 -0
+static u32 nested_vmx_max_atomic_switch_msrs(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	u64 vmx_misc = vmx_control_msr(vmx->nested.msrs.misc_low,
+				       vmx->nested.msrs.misc_high);
+
+	return (vmx_misc_max_msr(vmx_misc) + 1) * VMX_MISC_MSR_LIST_INCREMENT;
+}
+
 /*
  * Load guest's/host's msr at nested entry/exit.
  * return 0 for success, entry index for failure.

arch/x86/kvm/vmx/nested.c static u32 nested_vmx_load_msr(struct kvm_vcpu *vcpu, u64 gpa, u32 count) · line 883

Quoted 8 lines
arch/x86/kvm/vmx/nested.c +2 -0
@@ -865,9 +876,13 @@ static u32 nested_vmx_load_msr(struct kvm_vcpu *vcpu, u64 gpa, u32 count)
 	u32 i;
 	struct vmx_msr_entry e;
 	struct msr_data msr;
+	u32 max_msr_list_size = nested_vmx_max_atomic_switch_msrs(vcpu);
 
 	msr.host_initiated = false;
 	for (i = 0; i < count; i++) {
+		if (unlikely(i >= max_msr_list_size))

Although the SDM gives us leeway to do whatever we please since it states this is undefined behavior, KVM should at least be consistent between nested_vmx_load_msr() and nested_vmx_load_msr(). Here it fails only after processing the first N MSRs, while in the store case it fails immediately.

I'm guessing you opted for the divergent behavior because nested_vmx_load_msr() returns the failing index for VM-Enter, but nested_vmx_store_msr() has visible side effects too. E.g. storing the MSRs modifies memory, which could theoretically be read by other CPUs since VMX abort only brings down the current logical CPU.

Quoted 19 lines
arch/x86/kvm/vmx/nested.c +6 -0
+			goto fail;
+
 		if (kvm_vcpu_read_guest(vcpu, gpa + i * sizeof(e),
 					&e, sizeof(e))) {
 			pr_debug_ratelimited(
@@ -899,6 +914,10 @@ static int nested_vmx_store_msr(struct kvm_vcpu *vcpu, u64 gpa, u32 count)
 {
 	u32 i;
 	struct vmx_msr_entry e;
+	u32 max_msr_list_size = nested_vmx_max_atomic_switch_msrs(vcpu);
+
+	if (unlikely(count > max_msr_list_size))
+		return -EINVAL;
 
 	for (i = 0; i < count; i++) {
 		struct msr_data msr_info;
Signature · 1 line
2.23.0.237.gc6a4ce50a0-goog
author
#2 lore raw

All of this feedback sounds straightforward. I'll send a v2 out later today. Thanks!

Quoted 8 lines · Marc Orr
Quoted 6 lines

Allowing an unlimited number of MSRs to be specified via the VMX load/store MSR lists (e.g., vm-entry MSR load list) is bad for two reasons. First, a guest can specify an unreasonable number of MSRs, forcing KVM to process all of them in software. Second, the SDM bounds the number of MSRs allowed to be packed into the atomic switch MSR lists. Quoting the appendix chapter, titled "MISCELLANEOUS DATA":

Quoted 4 lines

Super Nit: There are multiple appendices in the SDM, maybe this?

Quoting the "Miscellaneous Data" section in the "VMX Capability Reporting Facility" appendix:

Will do in v2.

Quoted 4 lines
Quoted 1 line

+#define VMX_MISC_MSR_LIST_INCREMENT 512

VMX_MISC_MSR_LIST_MULTIPLIER seems more appropriate. INCREMENT makes it sound like X number of entries get tacked on to the end.

Will do in v2.

Quoted 5 lines
Quoted 1 line

+static u64 vmx_control_msr(u32 low, u32 high);

vmx_control_msr() is a 'static inline', just hoist it above this function. It probably makes sense to move vmx_control_verify() too, maybe to just below nested_vmx_abort()?

Will do in v2.

Quoted 9 lines
Quoted 3 lines
      msr.host_initiated = false;
      for (i = 0; i < count; i++) {

+ if (unlikely(i >= max_msr_list_size))

Although the SDM gives us leeway to do whatever we please since it states this is undefined behavior, KVM should at least be consistent between nested_vmx_load_msr() and nested_vmx_load_msr(). Here it fails only after processing the first N MSRs, while in the store case it fails immediately.

Quoted 5 lines

I'm guessing you opted for the divergent behavior because nested_vmx_load_msr() returns the failing index for VM-Enter, but nested_vmx_store_msr() has visible side effects too. E.g. storing the MSRs modifies memory, which could theoretically be read by other CPUs since VMX abort only brings down the current logical CPU.

Ok. I'll update nested_vmx_store_msr() to iterate over as many MSRs as it can in v2.