x86/kvm: Enable MCE injection in the guest

PATCH Applied Borislav Petkov 3 participants 6 messages Nov 14, 2018 – Jan 13, 2019

Subsystems (3)

x86 KVM KVM x86

lore

Share & tools

lore

Borislav Petkov

author maintainer

2018-11-14 18:15 UTC #0 lore raw

v3 cover x86/kvm: Enable MCE injection in the guest

From: Borislav Petkov <[email protected]>

Hi all,

here's a rediff ontop of -rc2. No changes, only added Yazen's Tested-by.

Please queue,
thx.

Changelog:
==========

v2:

here's v2, dropping patch 3 and incorporating hopefully all of Radim's feedback.

v1:

there's this mce-inject.ko module in the kernel which allows for injecting real MCEs and thus test the MCE handling code.

It is doubly useful to be able to inject same MCEs in a guest so that testing of the MCE handling code can happen even easier/faster. In order to be able to do that on an AMD guest, we need to emulate some bits and pieces like the HWCR[McStatusWrEn] bit which allows writes to the MCi_STATUS registers without a #GP.

The below does that and with it I'm able to properly inject MCEs in said guest.

Borislav Petkov (2):
kvm/x86: Move MSR_K7_HWCR to svm.c
x86/kvm: Implement MSR_HWCR support

 arch/x86/kvm/svm.c | 20 ++++++++++++++++++++
 arch/x86/kvm/x86.c | 46 +++++++++++++++++++++++++++++++---------------
 2 files changed, 51 insertions(+), 15 deletions(-)

Signature · 1 line

2.19.1

Borislav Petkov

author maintainer

2018-11-14 18:15 UTC #1 lore raw

PATCH v3 1/2 kvm/x86: Move MSR_K7_HWCR to svm.c

From: Borislav Petkov <[email protected]>

This is an AMD-specific MSR. Put it where it belongs.

Signed-off-by Borislav Petkov <[email protected]>

Tested-by Yazen Ghannam <[email protected]>

---
 arch/x86/kvm/svm.c | 14 ++++++++++++++
 arch/x86/kvm/x86.c | 12 ------------
 2 files changed, 14 insertions(+), 12 deletions(-)

arch/x86/kvm/svm.c +14 -0

index 0e21ccc46792..5c9c26dc7d84 100644
@@ -4154,6 +4154,9 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_F10H_DECFG:
 		msr_info->data = svm->msr_decfg;
 		break;
+	case MSR_K7_HWCR:
+		msr_info->data = 0;
+		break;
 	default:
 		return kvm_get_msr_common(vcpu, msr_info);
 	}

@@ -4358,6 +4361,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 		svm->msr_decfg = data;
 		break;
 	}
+	case MSR_K7_HWCR:
+		data &= ~(u64)0x40;	/* ignore flush filter disable */
+		data &= ~(u64)0x100;	/* ignore ignne emulation enable */
+		data &= ~(u64)0x8;	/* ignore TLB cache disable */
+		data &= ~(u64)0x40000;  /* ignore Mc status write enable */
+		if (data != 0) {
+			vcpu_unimpl(vcpu, "unimplemented HWCR wrmsr: 0x%llx\n",
+				    data);
+			return 1;
+		}
+		break;
 	case MSR_IA32_APICBASE:
 		if (kvm_vcpu_apicv_active(vcpu))
 			avic_update_vapic_bar(to_svm(vcpu), data);

arch/x86/kvm/x86.c +0 ~~-12~~

index 5cd5647120f2..d9abbe24deb5 100644
@@ -2434,17 +2434,6 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		break;
 	case MSR_EFER:
 		return set_efer(vcpu, data);
-	case MSR_K7_HWCR:
-		data &= ~(u64)0x40;	/* ignore flush filter disable */
-		data &= ~(u64)0x100;	/* ignore ignne emulation enable */
-		data &= ~(u64)0x8;	/* ignore TLB cache disable */
-		data &= ~(u64)0x40000;  /* ignore Mc status write enable */
-		if (data != 0) {
-			vcpu_unimpl(vcpu, "unimplemented HWCR wrmsr: 0x%llx\n",
-				    data);
-			return 1;
-		}
-		break;
 	case MSR_FAM10H_MMIO_CONF_BASE:
 		if (data != 0) {
 			vcpu_unimpl(vcpu, "unimplemented MMIO_CONF_BASE wrmsr: "

@@ -2713,7 +2702,6 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_K8_SYSCFG:
 	case MSR_K8_TSEG_ADDR:
 	case MSR_K8_TSEG_MASK:
-	case MSR_K7_HWCR:
 	case MSR_VM_HSAVE_PA:
 	case MSR_K8_INT_PENDING_MSG:
 	case MSR_AMD64_NB_CFG:

Signature · 1 line

2.19.1

David Hildenbrand

Reviewed-by on v3 1/2

2018-11-26 15:33 UTC #2 lore raw

Re: [PATCH v3 1/2] kvm/x86: Move MSR_K7_HWCR to svm.c

Quoted 26 lines · Borislav Petkov

From: Borislav Petkov <[email protected]>

This is an AMD-specific MSR. Put it where it belongs.

Signed-off-by Borislav Petkov <[email protected]>

Tested-by Yazen Ghannam <[email protected]>

---
 arch/x86/kvm/svm.c | 14 ++++++++++++++
 arch/x86/kvm/x86.c | 12 ------------
 2 files changed, 14 insertions(+), 12 deletions(-)

arch/x86/kvm/svm.c +3 -0

index 0e21ccc46792..5c9c26dc7d84 100644
@@ -4154,6 +4154,9 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_F10H_DECFG:
 		msr_info->data = svm->msr_decfg;
 		break;
+	case MSR_K7_HWCR:
+		msr_info->data = 0;
+		break;
 	default:
 		return kvm_get_msr_common(vcpu, msr_info);
 	}

arch/x86/kvm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · line 4369

Quoted 9 lines

arch/x86/kvm/svm.c +6 -0

@@ -4358,6 +4361,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 		svm->msr_decfg = data;
 		break;
 	}
+	case MSR_K7_HWCR:
+		data &= ~(u64)0x40;	/* ignore flush filter disable */
+		data &= ~(u64)0x100;	/* ignore ignne emulation enable */
+		data &= ~(u64)0x8;	/* ignore TLB cache disable */
+		data &= ~(u64)0x40000;  /* ignore Mc status write enable */
+		if (data != 0) {

While touching this, I would turn this into "if (data)". Up to you.

Reviewed-by David Hildenbrand <[email protected]>

Quoted 39 lines

arch/x86/kvm/svm.c +5 -0

+			vcpu_unimpl(vcpu, "unimplemented HWCR wrmsr: 0x%llx\n",
+				    data);
+			return 1;
+		}
+		break;
 	case MSR_IA32_APICBASE:
 		if (kvm_vcpu_apicv_active(vcpu))
 			avic_update_vapic_bar(to_svm(vcpu), data);

arch/x86/kvm/x86.c +0 ~~-12~~

index 5cd5647120f2..d9abbe24deb5 100644
@@ -2434,17 +2434,6 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		break;
 	case MSR_EFER:
 		return set_efer(vcpu, data);
-	case MSR_K7_HWCR:
-		data &= ~(u64)0x40;	/* ignore flush filter disable */
-		data &= ~(u64)0x100;	/* ignore ignne emulation enable */
-		data &= ~(u64)0x8;	/* ignore TLB cache disable */
-		data &= ~(u64)0x40000;  /* ignore Mc status write enable */
-		if (data != 0) {
-			vcpu_unimpl(vcpu, "unimplemented HWCR wrmsr: 0x%llx\n",
-				    data);
-			return 1;
-		}
-		break;
 	case MSR_FAM10H_MMIO_CONF_BASE:
 		if (data != 0) {
 			vcpu_unimpl(vcpu, "unimplemented MMIO_CONF_BASE wrmsr: "

@@ -2713,7 +2702,6 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_K8_SYSCFG:
 	case MSR_K8_TSEG_ADDR:
 	case MSR_K8_TSEG_MASK:
-	case MSR_K7_HWCR:
 	case MSR_VM_HSAVE_PA:
 	case MSR_K8_INT_PENDING_MSG:
 	case MSR_AMD64_NB_CFG:

Signature · 3 lines

Thanks,

David / dhildenb

Borislav Petkov

author maintainer

2018-11-14 18:15 UTC #3 lore raw

PATCH v3 2/2 x86/kvm: Implement MSR_HWCR support

In reply to Borislav Petkov

From: Borislav Petkov <[email protected]>

The hardware configuration register has some useful bits which can be used by guests. Implement McStatusWrEn which can be used by guests when injecting MCEs with the in-kernel mce-inject module.

For that, we need to set bit 18 - McStatusWrEn - first, before writing the MCi_STATUS registers (otherwise we #GP).

Add the required machinery to do so.

Signed-off-by Borislav Petkov <[email protected]>

Tested-by Yazen Ghannam <[email protected]>

---
 arch/x86/kvm/svm.c | 12 +++++++++---
 arch/x86/kvm/x86.c | 34 +++++++++++++++++++++++++++++++---
 2 files changed, 40 insertions(+), 6 deletions(-)

arch/x86/kvm/svm.c +9 -3

index 5c9c26dc7d84..db8e943f7576 100644
@@ -251,6 +251,9 @@ struct vcpu_svm {
 
 	/* which host CPU was used for running this vcpu */
 	unsigned int last_cpu;
+
+	/* MSRC001_0015 Hardware Configuration */
+	u64 msr_hwcr;
 };
 
 /*

@@ -4155,7 +4158,7 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		msr_info->data = svm->msr_decfg;
 		break;
 	case MSR_K7_HWCR:
-		msr_info->data = 0;
+		msr_info->data = svm->msr_hwcr;
 		break;
 	default:
 		return kvm_get_msr_common(vcpu, msr_info);

@@ -4365,8 +4368,11 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
 		data &= ~(u64)0x40;	/* ignore flush filter disable */
 		data &= ~(u64)0x100;	/* ignore ignne emulation enable */
 		data &= ~(u64)0x8;	/* ignore TLB cache disable */
-		data &= ~(u64)0x40000;  /* ignore Mc status write enable */
-		if (data != 0) {
+
+		/* Handle McStatusWrEn */
+		if (data == BIT_ULL(18)) {
+			svm->msr_hwcr = data;
+		} else if (data != 0) {
 			vcpu_unimpl(vcpu, "unimplemented HWCR wrmsr: 0x%llx\n",
 				    data);
 			return 1;

arch/x86/kvm/x86.c +31 -3

index d9abbe24deb5..607fe883113c 100644
@@ -2262,6 +2262,30 @@ static void kvmclock_sync_fn(struct work_struct *work)
 					KVMCLOCK_SYNC_PERIOD);
 }
 
+/*
+ * On AMD, HWCR[McStatusWrEn] controls whether setting MCi_STATUS results in #GP.
+ */
+static bool __set_mci_status(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
+{
+	if (guest_cpuid_is_amd(vcpu)) {
+		struct msr_data tmp;
+
+		tmp.index = MSR_K7_HWCR;
+
+		if (kvm_x86_ops->get_msr(vcpu, &tmp))
+			return false;
+
+		/* McStatusWrEn enabled? */
+		if (tmp.data & BIT_ULL(18))
+			return true;
+	}
+
+	if (msr_info->data != 0)
+		return false;
+
+	return true;
+}
+
 static int set_msr_mce(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	u64 mcg_cap = vcpu->arch.mcg_cap;

@@ -2293,9 +2317,13 @@ static int set_msr_mce(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			if ((offset & 0x3) == 0 &&
 			    data != 0 && (data | (1 << 10)) != ~(u64)0)
 				return -1;
-			if (!msr_info->host_initiated &&
-				(offset & 0x3) == 1 && data != 0)
-				return -1;
+
+			/* MCi_STATUS */
+			if ((offset & 0x3) == 1 && !msr_info->host_initiated) {
+				if (!__set_mci_status(vcpu, msr_info))
+						return -1;
+			}
+
 			vcpu->arch.mce_banks[offset] = data;
 			break;
 		}

Signature · 1 line

2.19.1

Paolo Bonzini

maintainer Applied

2018-11-26 16:41 UTC #4 lore raw

In reply to Borislav Petkov

Quoted 33 lines · Borislav Petkov

From: Borislav Petkov <[email protected]>
Hi all,
here's a rediff ontop of -rc2. No changes, only added Yazen's Tested-by.
Please queue,
thx.
Changelog:
==========
v2:
here's v2, dropping patch 3 and incorporating hopefully all of Radim's feedback.
v1:
there's this mce-inject.ko module in the kernel which allows for injecting real MCEs and thus test the MCE handling code.
It is doubly useful to be able to inject same MCEs in a guest so that testing of the MCE handling code can happen even easier/faster. In order to be able to do that on an AMD guest, we need to emulate some bits and pieces like the HWCR[McStatusWrEn] bit which allows writes to the MCi_STATUS registers without a #GP.
The below does that and with it I'm able to properly inject MCEs in said guest.
--

Quoted 8 lines

Borislav Petkov (2):
kvm/x86: Move MSR_K7_HWCR to svm.c
x86/kvm: Implement MSR_HWCR support
 arch/x86/kvm/svm.c | 20 ++++++++++++++++++++
 arch/x86/kvm/x86.c | 46 +++++++++++++++++++++++++++++++---------------
 2 files changed, 51 insertions(+), 15 deletions(-)

Queued, thanks.

Paolo

Borislav Petkov

author maintainer

2019-01-13 14:37 UTC #5 lore raw

Quoted 38 lines · Paolo Bonzini

Quoted 37 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Hi all,
here's a rediff ontop of -rc2. No changes, only added Yazen's Tested-by.
Please queue,
thx.
Changelog:
==========
v2:
here's v2, dropping patch 3 and incorporating hopefully all of Radim's feedback.
v1:
there's this mce-inject.ko module in the kernel which allows for injecting real MCEs and thus test the MCE handling code.
It is doubly useful to be able to inject same MCEs in a guest so that testing of the MCE handling code can happen even easier/faster. In order to be able to do that on an AMD guest, we need to emulate some bits and pieces like the HWCR[McStatusWrEn] bit which allows writes to the MCi_STATUS registers without a #GP.
The below does that and with it I'm able to properly inject MCEs in said guest.
--
Borislav Petkov (2):
kvm/x86: Move MSR_K7_HWCR to svm.c
x86/kvm: Implement MSR_HWCR support

Quoted 6 lines

Quoted 4 lines

 arch/x86/kvm/svm.c | 20 ++++++++++++++++++++
 arch/x86/kvm/x86.c | 46 +++++++++++++++++++++++++++++++---------------
 2 files changed, 51 insertions(+), 15 deletions(-)

Queued, thanks.

Where?

I don't see them in rc1...

Signature · 4 lines

Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.