Skip to contentSkip to conversation

KVM: x86: hyper-v: XMM fast hypercalls fixes

PATCH In review Vitaly Kuznetsov 4 participants 20 messages Feb 22, 2022 – Feb 28, 2022

Subsystems (4)
lore
Share & tools
lore

Vitaly Kuznetsov

author maintainer
#0 lore raw

cover KVM: x86: hyper-v: XMM fast hypercalls fixes

While working on some Hyper-V TLB flush improvements and Direct TLB flush feature for Hyper-V on KVM I experienced Windows Server 2019 crashes on boot when XMM fast hypercall input feature is advertised. Turns out, HVCALL_SEND_IPI_EX is also an XMM fast hypercall and returning an error kills the guest. This is fixed in PATCH4. PATCH3 fixes erroneous capping of sparse CPU banks for XMM fast TLB flush hypercalls. The problem should be reproducible with >360 vCPUs.

Vitaly Kuznetsov (4):
KVM: x86: hyper-v: Drop redundant 'ex' parameter from

    kvm_hv_send_ipi()
  KVM: x86: hyper-v: Drop redundant 'ex' parameter from
    kvm_hv_flush_tlb()

KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast
TLB flush hypercalls
KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall

 arch/x86/kvm/hyperv.c | 84 +++++++++++++++++++++++--------------------
 1 file changed, 45 insertions(+), 39 deletions(-)
Signature · 1 line
2.35.1
author maintainer
#1 lore raw

PATCH 1/4 KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()

'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.

No functional change intended.

---
 arch/x86/kvm/hyperv.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/kvm/hyperv.c +4 -4
index 6e38a7d22e97..15b6a7bd2346 100644
@@ -1875,7 +1875,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector,
 	}
 }
 
-static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 {
 	struct kvm *kvm = vcpu->kvm;
 	struct hv_send_ipi_ex send_ipi_ex;
@@ -1889,7 +1889,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 	u32 vector;
 	bool all_cpus;
 
-	if (!ex) {
+	if (hc->code == HVCALL_SEND_IPI) {
 		if (!hc->fast) {
 			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi,
 						    sizeof(send_ipi))))
@@ -2279,14 +2279,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, false);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
 		break;
 	case HVCALL_SEND_IPI_EX:
 		if (unlikely(hc.fast || hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, true);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
 		break;
 	case HVCALL_POST_DEBUG_DATA:
 	case HVCALL_RETRIEVE_DEBUG_DATA:
Signature · 1 line
2.35.1
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()

Quoted 24 lines · Vitaly Kuznetsov

'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.

No functional change intended.

---
 arch/x86/kvm/hyperv.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/kvm/hyperv.c +1 -1
index 6e38a7d22e97..15b6a7bd2346 100644
@@ -1875,7 +1875,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector,
 	}
 }
 
-static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 {
 	struct kvm *kvm = vcpu->kvm;
 	struct hv_send_ipi_ex send_ipi_ex;

arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1892

Quoted 5 lines
arch/x86/kvm/hyperv.c +1 -1
@@ -1889,7 +1889,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 	u32 vector;
 	bool all_cpus;
 
-	if (!ex) {
+	if (hc->code == HVCALL_SEND_IPI) {

I am thinking, if we already touch this code,
why not to use switch here instead on the hc->code,
so that we can catch this function being called with something else than HVCALL_SEND_IPI_EX

Quoted 10 lines
arch/x86/kvm/hyperv.c +1 -1
 		if (!hc->fast) {
 			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi,
 						    sizeof(send_ipi))))
@@ -2279,14 +2279,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, false);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
 		break;

arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2292

Quoted 10 lines
arch/x86/kvm/hyperv.c +1 -1
 	case HVCALL_SEND_IPI_EX:
 		if (unlikely(hc.fast || hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, true);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
 		break;
 	case HVCALL_POST_DEBUG_DATA:
 	case HVCALL_RETRIEVE_DEBUG_DATA:

Other than this minor nitpick:

Best regards,
Maxim Levitsky

author maintainer
#3 lore raw

Re: [PATCH 1/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()

Quoted 28 lines · Maxim Levitsky
Quoted 27 lines · Vitaly Kuznetsov

'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.

No functional change intended.

---
 arch/x86/kvm/hyperv.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/kvm/hyperv.c +1 -1
index 6e38a7d22e97..15b6a7bd2346 100644
@@ -1875,7 +1875,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector,
 	}
 }
 
-static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 {
 	struct kvm *kvm = vcpu->kvm;
 	struct hv_send_ipi_ex send_ipi_ex;
@@ -1889,7 +1889,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 	u32 vector;
 	bool all_cpus;
 
Quoted 7 lines

arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1892

Quoted 2 lines
arch/x86/kvm/hyperv.c +1 -1
-	if (!ex) {
+	if (hc->code == HVCALL_SEND_IPI) {

I am thinking, if we already touch this code,
why not to use switch here instead on the hc->code,
so that we can catch this function being called with something else than HVCALL_SEND_IPI_EX

I'm not against this second line of defense but kvm_hv_send_ipi() is only called explicitly from kvm_hv_hypercall()'s switch so something is really screwed up if we end up seeing something different from HVCALL_SEND_IPI_EX/HVCALL_SEND_IPI here.

I'm now working on a bigger series for TLB flush improvements, will use your suggestion there, thanks!

Quoted 32 lines
Quoted 11 lines
arch/x86/kvm/hyperv.c +1 -1
 		if (!hc->fast) {
 			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi,
 						    sizeof(send_ipi))))
@@ -2279,14 +2279,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, false);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
 		break;

arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2292

Quoted 10 lines
arch/x86/kvm/hyperv.c +1 -1
 	case HVCALL_SEND_IPI_EX:
 		if (unlikely(hc.fast || hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, true);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
 		break;
 	case HVCALL_POST_DEBUG_DATA:
 	case HVCALL_RETRIEVE_DEBUG_DATA:

Other than this minor nitpick:

Best regards,
Maxim Levitsky

Signature · 1 line
Vitaly
Reviewed-by on v1 1/4
#4 lore raw

Re: [PATCH 1/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_send_ipi()

In reply to Vitaly Kuznetsov
Quoted 10 lines · Vitaly Kuznetsov

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.

No functional change intended.

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

author maintainer
#5 lore raw

PATCH 2/4 KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()

In reply to Vitaly Kuznetsov

'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.

No functional change intended.

---
 arch/x86/kvm/hyperv.c | 23 ++++++-----------------
 1 file changed, 6 insertions(+), 17 deletions(-)
arch/x86/kvm/hyperv.c +6 -17
index 15b6a7bd2346..714af3b94f31 100644
@@ -1750,7 +1750,7 @@ struct kvm_hv_hcall {
 	sse128_t xmm[HV_HYPERCALL_MAX_XMM_REGISTERS];
 };
 
-static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 {
 	int i;
 	gpa_t gpa;
@@ -1765,7 +1765,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 	int sparse_banks_len;
 	bool all_cpus;
 
-	if (!ex) {
+	if (hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST ||
+	    hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE) {
 		if (hc->fast) {
 			flush.address_space = hc->ingpa;
 			flush.flags = hc->outgpa;
@@ -2247,32 +2248,20 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 				kvm_hv_hypercall_complete_userspace;
 		return 0;
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST:
-		if (unlikely(!hc.rep_cnt || hc.rep_idx)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
-	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
-		if (unlikely(hc.rep)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
 		if (unlikely(!hc.rep_cnt || hc.rep_idx)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
+	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
 		if (unlikely(hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
 	case HVCALL_SEND_IPI:
 		if (unlikely(hc.rep)) {
Signature · 1 line
2.35.1
Reviewed-by on v1 2/4
#6 lore raw

Re: [PATCH 2/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()

Quoted 59 lines · Vitaly Kuznetsov

'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.

No functional change intended.

---
 arch/x86/kvm/hyperv.c | 23 ++++++-----------------
 1 file changed, 6 insertions(+), 17 deletions(-)
arch/x86/kvm/hyperv.c +5 -16
index 15b6a7bd2346..714af3b94f31 100644
@@ -1750,7 +1750,7 @@ struct kvm_hv_hcall {
 	sse128_t xmm[HV_HYPERCALL_MAX_XMM_REGISTERS];
 };
 
-static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 {
 	int i;
 	gpa_t gpa;
@@ -1765,7 +1765,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 	int sparse_banks_len;
 	bool all_cpus;
 
-	if (!ex) {
+	if (hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST ||
+	    hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE) {
 		if (hc->fast) {
 			flush.address_space = hc->ingpa;
 			flush.flags = hc->outgpa;
@@ -2247,32 +2248,20 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 				kvm_hv_hypercall_complete_userspace;
 		return 0;
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST:
-		if (unlikely(!hc.rep_cnt || hc.rep_idx)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
-	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
-		if (unlikely(hc.rep)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
 		if (unlikely(!hc.rep_cnt || hc.rep_idx)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
+	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:

arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2267

Quoted 10 lines
arch/x86/kvm/hyperv.c +1 -1
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
 		if (unlikely(hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
 	case HVCALL_SEND_IPI:
 		if (unlikely(hc.rep)) {

Best regards,
Maxim Levitsky

Reviewed-by on v1 2/4
#7 lore raw

Re: [PATCH 2/4] KVM: x86: hyper-v: Drop redundant 'ex' parameter from kvm_hv_flush_tlb()

In reply to Vitaly Kuznetsov
Quoted 10 lines · Vitaly Kuznetsov

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

'struct kvm_hv_hcall' has all the required information already, there's no need to pass 'ex' additionally.

No functional change intended.

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

author maintainer
#8 lore raw

PATCH 3/4 KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast TLB flush hypercalls

In reply to Vitaly Kuznetsov

When TLB flush hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}_EX are issued in 'XMM fast' mode, the maximum number of allowed sparse_banks is not 'HV_HYPERCALL_MAX_XMM_REGISTERS - 1' (5) but twice as many (10) as each XMM register is 128 bit long and can hold two 64 bit long banks.

Fixes 5974565bc26d ("KVM: x86: kvm_hv_flush_tlb use inputs from XMM registers")
---
 arch/x86/kvm/hyperv.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/hyperv.c +2 -1
index 714af3b94f31..6dda93bf98ae 100644
@@ -1820,7 +1820,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 
 		if (!all_cpus) {
 			if (hc->fast) {
-				if (sparse_banks_len > HV_HYPERCALL_MAX_XMM_REGISTERS - 1)
+				/* XMM0 is already consumed, each XMM holds two sparse banks. */
+				if (sparse_banks_len > 2 * (HV_HYPERCALL_MAX_XMM_REGISTERS - 1))
 					return HV_STATUS_INVALID_HYPERCALL_INPUT;
 				for (i = 0; i < sparse_banks_len; i += 2) {
 					sparse_banks[i] = sse128_lo(hc->xmm[i / 2 + 1]);
Signature · 1 line
2.35.1
Reviewed-by on v1 3/4
#9 lore raw

Re: [PATCH 3/4] KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast TLB flush hypercalls

Quoted 11 lines · Vitaly Kuznetsov

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

When TLB flush hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}_EX are issued in 'XMM fast' mode, the maximum number of allowed sparse_banks is not 'HV_HYPERCALL_MAX_XMM_REGISTERS - 1' (5) but twice as many (10) as each XMM register is 128 bit long and can hold two 64 bit long banks.

Fixes 5974565bc26d ("KVM: x86: kvm_hv_flush_tlb use inputs from XMM registers")

Indeed :)

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

author maintainer
#10 lore raw

PATCH 4/4 KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall

In reply to Vitaly Kuznetsov

It has been proven on practice that at least Windows Server 2019 tries using HVCALL_SEND_IPI_EX in 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.

Note, HVCALL_SEND_IPI is a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.

Fixes d8f5537a8816 ("KVM: hyper-v: Advertise support for fast XMM hypercalls")
---
 arch/x86/kvm/hyperv.c | 52 ++++++++++++++++++++++++++++---------------
 1 file changed, 34 insertions(+), 18 deletions(-)
arch/x86/kvm/hyperv.c +34 -18
index 6dda93bf98ae..3060057bdfd4 100644
@@ -1890,6 +1890,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 	int sparse_banks_len;
 	u32 vector;
 	bool all_cpus;
+	int i;
 
 	if (hc->code == HVCALL_SEND_IPI) {
 		if (!hc->fast) {
@@ -1910,9 +1911,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 
 		trace_kvm_hv_send_ipi(vector, sparse_banks[0]);
 	} else {
-		if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
-					    sizeof(send_ipi_ex))))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
+						    sizeof(send_ipi_ex))))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			send_ipi_ex.vector = (u32)hc->ingpa;
+			send_ipi_ex.vp_set.format = hc->outgpa;
+			send_ipi_ex.vp_set.valid_bank_mask = sse128_lo(hc->xmm[0]);
+		}
 
 		trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector,
 					 send_ipi_ex.vp_set.format,
@@ -1920,8 +1927,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 
 		vector = send_ipi_ex.vector;
 		valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask;
-		sparse_banks_len = bitmap_weight(&valid_bank_mask, 64) *
-			sizeof(sparse_banks[0]);
+		sparse_banks_len = bitmap_weight(&valid_bank_mask, 64);
 
 		all_cpus = send_ipi_ex.vp_set.format == HV_GENERIC_SET_ALL;
 
@@ -1931,12 +1937,27 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 		if (!sparse_banks_len)
 			goto ret_success;
 
-		if (kvm_read_guest(kvm,
-				   hc->ingpa + offsetof(struct hv_send_ipi_ex,
-							vp_set.bank_contents),
-				   sparse_banks,
-				   sparse_banks_len))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (kvm_read_guest(kvm,
+					   hc->ingpa + offsetof(struct hv_send_ipi_ex,
+								vp_set.bank_contents),
+					   sparse_banks,
+					   sparse_banks_len * sizeof(sparse_banks[0])))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			/*
+			 * The lower half of XMM0 is already consumed, each XMM holds
+			 * two sparse banks.
+			 */
+			if (sparse_banks_len > (2 * HV_HYPERCALL_MAX_XMM_REGISTERS - 1))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+			for (i = 0; i < sparse_banks_len; i++) {
+				if (i % 2)
+					sparse_banks[i] = sse128_lo(hc->xmm[(i + 1) / 2]);
+				else
+					sparse_banks[i] = sse128_hi(hc->xmm[i / 2]);
+			}
+		}
 	}
 
 check_and_send_ipi:
@@ -2098,6 +2119,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
+	case HVCALL_SEND_IPI_EX:
 		return true;
 	}
 
@@ -2265,14 +2287,8 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
 	case HVCALL_SEND_IPI:
-		if (unlikely(hc.rep)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_send_ipi(vcpu, &hc);
-		break;
 	case HVCALL_SEND_IPI_EX:
-		if (unlikely(hc.fast || hc.rep)) {
+		if (unlikely(hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
Signature · 1 line
2.35.1
#11 lore raw

Re: [PATCH 4/4] KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall

Quoted 48 lines · Vitaly Kuznetsov

It has been proven on practice that at least Windows Server 2019 tries using HVCALL_SEND_IPI_EX in 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.

Note, HVCALL_SEND_IPI is a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.

Fixes d8f5537a8816 ("KVM: hyper-v: Advertise support for fast XMM hypercalls")
---
 arch/x86/kvm/hyperv.c | 52 ++++++++++++++++++++++++++++---------------
 1 file changed, 34 insertions(+), 18 deletions(-)
arch/x86/kvm/hyperv.c +10 -3
index 6dda93bf98ae..3060057bdfd4 100644
@@ -1890,6 +1890,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 	int sparse_banks_len;
 	u32 vector;
 	bool all_cpus;
+	int i;
 
 	if (hc->code == HVCALL_SEND_IPI) {
 		if (!hc->fast) {
@@ -1910,9 +1911,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 
 		trace_kvm_hv_send_ipi(vector, sparse_banks[0]);
 	} else {
-		if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
-					    sizeof(send_ipi_ex))))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
+						    sizeof(send_ipi_ex))))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			send_ipi_ex.vector = (u32)hc->ingpa;
+			send_ipi_ex.vp_set.format = hc->outgpa;
+			send_ipi_ex.vp_set.valid_bank_mask = sse128_lo(hc->xmm[0]);
+		}
 
 		trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector,
 					 send_ipi_ex.vp_set.format,

arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1930

Quoted 6 lines
arch/x86/kvm/hyperv.c +1 -2
@@ -1920,8 +1927,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 
 		vector = send_ipi_ex.vector;
 		valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask;
-		sparse_banks_len = bitmap_weight(&valid_bank_mask, 64) *
-			sizeof(sparse_banks[0]);
+		sparse_banks_len = bitmap_weight(&valid_bank_mask, 64);

Is this change intentional?

I haven't fully reviewed this, because kvm/queue seem to have a bit different version of this, and I didn't fully follow on all of this.

Best regards,
Maxim Levitsky

Quoted 61 lines
arch/x86/kvm/hyperv.c +23 -13
 		all_cpus = send_ipi_ex.vp_set.format == HV_GENERIC_SET_ALL;
 
@@ -1931,12 +1937,27 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 		if (!sparse_banks_len)
 			goto ret_success;
 
-		if (kvm_read_guest(kvm,
-				   hc->ingpa + offsetof(struct hv_send_ipi_ex,
-							vp_set.bank_contents),
-				   sparse_banks,
-				   sparse_banks_len))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (kvm_read_guest(kvm,
+					   hc->ingpa + offsetof(struct hv_send_ipi_ex,
+								vp_set.bank_contents),
+					   sparse_banks,
+					   sparse_banks_len * sizeof(sparse_banks[0])))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			/*
+			 * The lower half of XMM0 is already consumed, each XMM holds
+			 * two sparse banks.
+			 */
+			if (sparse_banks_len > (2 * HV_HYPERCALL_MAX_XMM_REGISTERS - 1))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+			for (i = 0; i < sparse_banks_len; i++) {
+				if (i % 2)
+					sparse_banks[i] = sse128_lo(hc->xmm[(i + 1) / 2]);
+				else
+					sparse_banks[i] = sse128_hi(hc->xmm[i / 2]);
+			}
+		}
 	}
 
 check_and_send_ipi:
@@ -2098,6 +2119,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
+	case HVCALL_SEND_IPI_EX:
 		return true;
 	}
 
@@ -2265,14 +2287,8 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
 	case HVCALL_SEND_IPI:
-		if (unlikely(hc.rep)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_send_ipi(vcpu, &hc);
-		break;
 	case HVCALL_SEND_IPI_EX:
-		if (unlikely(hc.fast || hc.rep)) {
+		if (unlikely(hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
author maintainer
#12 lore raw

Re: [PATCH 4/4] KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall

Quoted 50 lines · Maxim Levitsky
Quoted 49 lines · Vitaly Kuznetsov

It has been proven on practice that at least Windows Server 2019 tries using HVCALL_SEND_IPI_EX in 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.

Note, HVCALL_SEND_IPI is a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.

Fixes d8f5537a8816 ("KVM: hyper-v: Advertise support for fast XMM hypercalls")
---
 arch/x86/kvm/hyperv.c | 52 ++++++++++++++++++++++++++++---------------
 1 file changed, 34 insertions(+), 18 deletions(-)
arch/x86/kvm/hyperv.c +10 -3
index 6dda93bf98ae..3060057bdfd4 100644
@@ -1890,6 +1890,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 	int sparse_banks_len;
 	u32 vector;
 	bool all_cpus;
+	int i;
 
 	if (hc->code == HVCALL_SEND_IPI) {
 		if (!hc->fast) {
@@ -1910,9 +1911,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 
 		trace_kvm_hv_send_ipi(vector, sparse_banks[0]);
 	} else {
-		if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
-					    sizeof(send_ipi_ex))))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
+						    sizeof(send_ipi_ex))))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			send_ipi_ex.vector = (u32)hc->ingpa;
+			send_ipi_ex.vp_set.format = hc->outgpa;
+			send_ipi_ex.vp_set.valid_bank_mask = sse128_lo(hc->xmm[0]);
+		}
 
 		trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector,
 					 send_ipi_ex.vp_set.format,
@@ -1920,8 +1927,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 
Quoted 7 lines

arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1930

Quoted 5 lines
arch/x86/kvm/hyperv.c +1 -2
 		vector = send_ipi_ex.vector;
 		valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask;
-		sparse_banks_len = bitmap_weight(&valid_bank_mask, 64) *
-			sizeof(sparse_banks[0]);
+		sparse_banks_len = bitmap_weight(&valid_bank_mask, 64);

Is this change intentional?

Yes it is. Previously, 'sparse_banks_len' was the number of bytes to read, now it's in u64-s.

(see below)

Quoted 10 lines

I haven't fully reviewed this, because kvm/queue seem to have a bit different version of this, and I didn't fully follow on all of this.

Quoted 7 lines
arch/x86/kvm/hyperv.c +0 -0
 		all_cpus = send_ipi_ex.vp_set.format == HV_GENERIC_SET_ALL;
 
@@ -1931,12 +1937,27 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 		if (!sparse_banks_len)
 			goto ret_success;
 
Quoted 12 lines

arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1945

Quoted 12 lines
arch/x86/kvm/hyperv.c +6 -6
-		if (kvm_read_guest(kvm,
-				   hc->ingpa + offsetof(struct hv_send_ipi_ex,
-							vp_set.bank_contents),
-				   sparse_banks,
-				   sparse_banks_len))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (kvm_read_guest(kvm,
+					   hc->ingpa + offsetof(struct hv_send_ipi_ex,
+								vp_set.bank_contents),
+					   sparse_banks,
+					   sparse_banks_len * sizeof(sparse_banks[0])))

^^^ here ^^^

Quoted 8 lines

arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 1953

Quoted 8 lines
arch/x86/kvm/hyperv.c +8 -0
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			/*
+			 * The lower half of XMM0 is already consumed, each XMM holds
+			 * two sparse banks.
+			 */
+			if (sparse_banks_len > (2 * HV_HYPERCALL_MAX_XMM_REGISTERS - 1))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;

And here. This is the reason for change: it's more convenient to count it 'xmm halves' than in bytes.

Quoted 36 lines
Quoted 34 lines
arch/x86/kvm/hyperv.c +9 -7
+			for (i = 0; i < sparse_banks_len; i++) {
+				if (i % 2)
+					sparse_banks[i] = sse128_lo(hc->xmm[(i + 1) / 2]);
+				else
+					sparse_banks[i] = sse128_hi(hc->xmm[i / 2]);
+			}
+		}
 	}
 
 check_and_send_ipi:
@@ -2098,6 +2119,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
+	case HVCALL_SEND_IPI_EX:
 		return true;
 	}
 
@@ -2265,14 +2287,8 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
 	case HVCALL_SEND_IPI:
-		if (unlikely(hc.rep)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_send_ipi(vcpu, &hc);
-		break;
 	case HVCALL_SEND_IPI_EX:
-		if (unlikely(hc.fast || hc.rep)) {
+		if (unlikely(hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
Signature · 1 line
Vitaly
Reviewed-by on v1 4/4
#13 lore raw

Re: [PATCH 4/4] KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall

In reply to Vitaly Kuznetsov
Quoted 11 lines · Vitaly Kuznetsov

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

It has been proven on practice that at least Windows Server 2019 tries using HVCALL_SEND_IPI_EX in 'XMM fast' mode when it has more than 64 vCPUs and it needs to send an IPI to a vCPU > 63. Similarly to other XMM Fast hypercalls (HVCALL_FLUSH_VIRTUAL_ADDRESS_{LIST,SPACE}{,_EX}), this information is missing in TLFS as of 6.0b. Currently, KVM returns an error (HV_STATUS_INVALID_HYPERCALL_INPUT) and Windows crashes.

Quoted 5 lines

Note, HVCALL_SEND_IPI is a 'standard' fast hypercall (not 'XMM fast') as all its parameters fit into RDX:R8 and this is handled by KVM correctly.

Fixes d8f5537a8816 ("KVM: hyper-v: Advertise support for fast XMM hypercalls")

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

maintainer
#14 lore raw
In reply to Vitaly Kuznetsov
Quoted 13 lines · Vitaly Kuznetsov

While working on some Hyper-V TLB flush improvements and Direct TLB flush feature for Hyper-V on KVM I experienced Windows Server 2019 crashes on boot when XMM fast hypercall input feature is advertised. Turns out, HVCALL_SEND_IPI_EX is also an XMM fast hypercall and returning an error kills the guest. This is fixed in PATCH4. PATCH3 fixes erroneous capping of sparse CPU banks for XMM fast TLB flush hypercalls. The problem should be reproducible with >360 vCPUs.

Vitaly Kuznetsov (4):
KVM: x86: hyper-v: Drop redundant 'ex' parameter from

     kvm_hv_send_ipi()
   KVM: x86: hyper-v: Drop redundant 'ex' parameter from
     kvm_hv_flush_tlb()
Quoted 7 lines

KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast
TLB flush hypercalls
KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall

  arch/x86/kvm/hyperv.c | 84 +++++++++++++++++++++++--------------------
  1 file changed, 45 insertions(+), 39 deletions(-)

Merging this in 5.18 is a bit messy. Please check that the below patch against kvm/next makes sense:

arch/x86/kvm/hyperv.c +50 -37
index 653e08c993c4..98fb998c31ce 100644
@@ -1770,9 +1770,11 @@ struct kvm_hv_hcall {
 };
 
 static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc,
+				 int consumed_xmm_halves,
 				 u64 *sparse_banks, gpa_t offset)
 {
 	u16 var_cnt;
+	int i;
 
 	if (hc->var_cnt > 64)
 		return -EINVAL;
@@ -1780,13 +1782,29 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc,
 	/* Ignore banks that cannot possibly contain a legal VP index. */
 	var_cnt = min_t(u16, hc->var_cnt, KVM_HV_MAX_SPARSE_VCPU_SET_BITS);
 
+	if (hc->fast) {
+		/*
+		 * Each XMM holds two sparse banks, but do not count halves that
+		 * have already been consumed for hypercall parameters.
+		 */
+		if (hc->var_cnt > 2 * HV_HYPERCALL_MAX_XMM_REGISTERS - consumed_xmm_halves)
+			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		for (i = 0; i < var_cnt; i++) {
+			int j = i + consumed_xmm_halves;
+			if (j % 2)
+				sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);
+			else
+				sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);
+		}
+		return 0;
+	}
+
 	return kvm_read_guest(kvm, hc->ingpa + offset, sparse_banks,
 			      var_cnt * sizeof(*sparse_banks));
 }
 
-static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 {
-	int i;
 	struct kvm *kvm = vcpu->kvm;
 	struct hv_tlb_flush_ex flush_ex;
 	struct hv_tlb_flush flush;
@@ -1803,7 +1821,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 	 */
 	BUILD_BUG_ON(KVM_HV_MAX_SPARSE_VCPU_SET_BITS > 64);
 
-	if (!ex) {
+	if (hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST ||
+	    hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE) {
 		if (hc->fast) {
 			flush.address_space = hc->ingpa;
 			flush.flags = hc->outgpa;
@@ -1859,17 +1878,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 		if (!hc->var_cnt)
 			goto ret_success;
 
-		if (hc->fast) {
-			if (hc->var_cnt > HV_HYPERCALL_MAX_XMM_REGISTERS - 1)
-				return HV_STATUS_INVALID_HYPERCALL_INPUT;
-			for (i = 0; i < hc->var_cnt; i += 2) {
-				sparse_banks[i] = sse128_lo(hc->xmm[i / 2 + 1]);
-				sparse_banks[i + 1] = sse128_hi(hc->xmm[i / 2 + 1]);
-			}
-			goto do_flush;
-		}
-
-		if (kvm_get_sparse_vp_set(kvm, hc, sparse_banks,
+		if (kvm_get_sparse_vp_set(kvm, hc, 2, sparse_banks,
 					  offsetof(struct hv_tlb_flush_ex,
 						   hv_vp_set.bank_contents)))
 			return HV_STATUS_INVALID_HYPERCALL_INPUT;
@@ -1913,7 +1922,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector,
 	}
 }
 
-static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
 {
 	struct kvm *kvm = vcpu->kvm;
 	struct hv_send_ipi_ex send_ipi_ex;
@@ -1924,7 +1933,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 	u32 vector;
 	bool all_cpus;
 
-	if (!ex) {
+	if (hc->code == HVCALL_SEND_IPI) {
 		if (!hc->fast) {
 			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi,
 						    sizeof(send_ipi))))
@@ -1943,9 +1952,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 
 		trace_kvm_hv_send_ipi(vector, sparse_banks[0]);
 	} else {
-		if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
-					    sizeof(send_ipi_ex))))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
+						    sizeof(send_ipi_ex))))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			send_ipi_ex.vector = (u32)hc->ingpa;
+			send_ipi_ex.vp_set.format = hc->outgpa;
+			send_ipi_ex.vp_set.valid_bank_mask = sse128_lo(hc->xmm[0]);
+		}
 
 		trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector,
 					 send_ipi_ex.vp_set.format,
@@ -1964,7 +1979,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 		if (!hc->var_cnt)
 			goto ret_success;
 
-		if (kvm_get_sparse_vp_set(kvm, hc, sparse_banks,
+		if (kvm_get_sparse_vp_set(kvm, hc, 1, sparse_banks,
 					  offsetof(struct hv_send_ipi_ex,
 						   vp_set.bank_contents)))
 			return HV_STATUS_INVALID_HYPERCALL_INPUT;
@@ -2126,6 +2141,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
+	case HVCALL_SEND_IPI_EX:
 		return true;
 	}
 
@@ -2283,46 +2299,43 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
 				kvm_hv_hypercall_complete_userspace;
 		return 0;
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST:
-		if (unlikely(!hc.rep_cnt || hc.rep_idx || hc.var_cnt)) {
+		if (unlikely(hc.var_cnt)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
-	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
-		if (unlikely(hc.rep || hc.var_cnt)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
+		fallthrough;
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
 		if (unlikely(!hc.rep_cnt || hc.rep_idx)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
+	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
+		if (unlikely(hc.var_cnt)) {
+			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
+			break;
+		}
+		fallthrough;
 	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
 		if (unlikely(hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
 		break;
 	case HVCALL_SEND_IPI:
-		if (unlikely(hc.rep || hc.var_cnt)) {
+		if (unlikely(hc.var_cnt)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, false);
-		break;
+		fallthrough;
 	case HVCALL_SEND_IPI_EX:
-		if (unlikely(hc.fast || hc.rep)) {
+		if (unlikely(hc.rep)) {
 			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
 			break;
 		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, true);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
 		break;
 	case HVCALL_POST_DEBUG_DATA:
 	case HVCALL_RETRIEVE_DEBUG_DATA:

The resulting merge commit is already in kvm/queue shortly (which should become the next kvm/next as soon as tests complete).

Paolo

author maintainer
#15 lore raw
Quoted 18 lines · Paolo Bonzini
Quoted 17 lines · Vitaly Kuznetsov

While working on some Hyper-V TLB flush improvements and Direct TLB flush feature for Hyper-V on KVM I experienced Windows Server 2019 crashes on boot when XMM fast hypercall input feature is advertised. Turns out, HVCALL_SEND_IPI_EX is also an XMM fast hypercall and returning an error kills the guest. This is fixed in PATCH4. PATCH3 fixes erroneous capping of sparse CPU banks for XMM fast TLB flush hypercalls. The problem should be reproducible with >360 vCPUs.

Vitaly Kuznetsov (4):
KVM: x86: hyper-v: Drop redundant 'ex' parameter from

     kvm_hv_send_ipi()
   KVM: x86: hyper-v: Drop redundant 'ex' parameter from
     kvm_hv_flush_tlb()

KVM: x86: hyper-v: Fix the maximum number of sparse banks for XMM fast
TLB flush hypercalls
KVM: x86: hyper-v: HVCALL_SEND_IPI_EX is an XMM fast hypercall

Quoted 6 lines
Quoted 3 lines
  arch/x86/kvm/hyperv.c | 84 +++++++++++++++++++++++--------------------
  1 file changed, 45 insertions(+), 39 deletions(-)

Merging this in 5.18 is a bit messy. Please check that the below patch against kvm/next makes sense:

Something is wrong with the diff as it doesn't apply :-(

Quoted 24 lines
arch/x86/kvm/hyperv.c +5 -0
index 653e08c993c4..98fb998c31ce 100644
@@ -1770,9 +1770,11 @@ struct kvm_hv_hcall {
  };
 
  static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc,
+				 int consumed_xmm_halves,
  				 u64 *sparse_banks, gpa_t offset)
  {
  	u16 var_cnt;
+	int i;
 
  	if (hc->var_cnt > 64)
  		return -EINVAL;
@@ -1780,13 +1782,29 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc,
  	/* Ignore banks that cannot possibly contain a legal VP index. */
  	var_cnt = min_t(u16, hc->var_cnt, KVM_HV_MAX_SPARSE_VCPU_SET_BITS);
 
+	if (hc->fast) {
+		/*
+		 * Each XMM holds two sparse banks, but do not count halves that

arch/x86/kvm/hyperv.c static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc, · line 1797

Quoted 10 lines
arch/x86/kvm/hyperv.c +10 -0
+		 * have already been consumed for hypercall parameters.
+		 */
+		if (hc->var_cnt > 2 * HV_HYPERCALL_MAX_XMM_REGISTERS - consumed_xmm_halves)
+			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		for (i = 0; i < var_cnt; i++) {
+			int j = i + consumed_xmm_halves;
+			if (j % 2)
+				sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);
+			else
+				sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);

Let's say we have 1 half of XMM0 consumed. Now:

i = 0;
j = 1;
if (1)

     sparse_banks[0] = sse128_lo(hc->xmm[0]); 

This doesn't look right as we need to get the upper half of XMM0.

I guess it should be reversed,

     if (j % 2)
         sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);
     else
         sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);
Quoted 16 lines
arch/x86/kvm/hyperv.c +5 -2
+		}
+		return 0;
+	}
+
  	return kvm_read_guest(kvm, hc->ingpa + offset, sparse_banks,
  			      var_cnt * sizeof(*sparse_banks));
  }
 
-static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
  {
-	int i;
  	struct kvm *kvm = vcpu->kvm;
  	struct hv_tlb_flush_ex flush_ex;
  	struct hv_tlb_flush flush;

arch/x86/kvm/hyperv.c static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1825

Quoted 6 lines
arch/x86/kvm/hyperv.c +2 -1
@@ -1803,7 +1821,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
  	 */
  	BUILD_BUG_ON(KVM_HV_MAX_SPARSE_VCPU_SET_BITS > 64);
 
-	if (!ex) {
+	if (hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST ||
+	    hc->code == HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE) {

In case you're trying to come up with a smaller patch for 5.18, we can certainly drop these 'ex'/'non-ex' changes as these are merely cosmetic.

Quoted 11 lines
arch/x86/kvm/hyperv.c +0 -4
  		if (hc->fast) {
  			flush.address_space = hc->ingpa;
  			flush.flags = hc->outgpa;
@@ -1859,17 +1878,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
  		if (!hc->var_cnt)
  			goto ret_success;
 
-		if (hc->fast) {
-			if (hc->var_cnt > HV_HYPERCALL_MAX_XMM_REGISTERS - 1)
-				return HV_STATUS_INVALID_HYPERCALL_INPUT;
-			for (i = 0; i < hc->var_cnt; i += 2) {

arch/x86/kvm/hyperv.c static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool · line 1883

Quoted 10 lines
arch/x86/kvm/hyperv.c +1 -7
-				sparse_banks[i] = sse128_lo(hc->xmm[i / 2 + 1]);
-				sparse_banks[i + 1] = sse128_hi(hc->xmm[i / 2 + 1]);
-			}
-			goto do_flush;
-		}
-
-		if (kvm_get_sparse_vp_set(kvm, hc, sparse_banks,
+		if (kvm_get_sparse_vp_set(kvm, hc, 2, sparse_banks,
  					  offsetof(struct hv_tlb_flush_ex,
  						   hv_vp_set.bank_contents)))

I like your idea to put 'consumed_xmm_halves' into kvm_get_sparse_vp_set() as kvm_hv_flush_tlb is getting too big.

Quoted 106 lines
arch/x86/kvm/hyperv.c +25 -21
  			return HV_STATUS_INVALID_HYPERCALL_INPUT;
@@ -1913,7 +1922,7 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector,
  	}
  }
 
-static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool ex)
+static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)
  {
  	struct kvm *kvm = vcpu->kvm;
  	struct hv_send_ipi_ex send_ipi_ex;
@@ -1924,7 +1933,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
  	u32 vector;
  	bool all_cpus;
 
-	if (!ex) {
+	if (hc->code == HVCALL_SEND_IPI) {
  		if (!hc->fast) {
  			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi,
  						    sizeof(send_ipi))))
@@ -1943,9 +1952,15 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
 
  		trace_kvm_hv_send_ipi(vector, sparse_banks[0]);
  	} else {
-		if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
-					    sizeof(send_ipi_ex))))
-			return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		if (!hc->fast) {
+			if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex,
+						    sizeof(send_ipi_ex))))
+				return HV_STATUS_INVALID_HYPERCALL_INPUT;
+		} else {
+			send_ipi_ex.vector = (u32)hc->ingpa;
+			send_ipi_ex.vp_set.format = hc->outgpa;
+			send_ipi_ex.vp_set.valid_bank_mask = sse128_lo(hc->xmm[0]);
+		}
 
  		trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector,
  					 send_ipi_ex.vp_set.format,
@@ -1964,7 +1979,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc, bool
  		if (!hc->var_cnt)
  			goto ret_success;
 
-		if (kvm_get_sparse_vp_set(kvm, hc, sparse_banks,
+		if (kvm_get_sparse_vp_set(kvm, hc, 1, sparse_banks,
  					  offsetof(struct hv_send_ipi_ex,
  						   vp_set.bank_contents)))
  			return HV_STATUS_INVALID_HYPERCALL_INPUT;
@@ -2126,6 +2141,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc)
  	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
  	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
  	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
+	case HVCALL_SEND_IPI_EX:
  		return true;
  	}
 
@@ -2283,46 +2299,43 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
  				kvm_hv_hypercall_complete_userspace;
  		return 0;
  	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST:
-		if (unlikely(!hc.rep_cnt || hc.rep_idx || hc.var_cnt)) {
+		if (unlikely(hc.var_cnt)) {
  			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
  			break;
  		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
-	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
-		if (unlikely(hc.rep || hc.var_cnt)) {
-			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
-			break;
-		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, false);
-		break;
+		fallthrough;
  	case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX:
  		if (unlikely(!hc.rep_cnt || hc.rep_idx)) {
  			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
  			break;
  		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
  		break;
+	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE:
+		if (unlikely(hc.var_cnt)) {
+			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
+			break;
+		}
+		fallthrough;
  	case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX:
  		if (unlikely(hc.rep)) {
  			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
  			break;
  		}
-		ret = kvm_hv_flush_tlb(vcpu, &hc, true);
+		ret = kvm_hv_flush_tlb(vcpu, &hc);
  		break;
  	case HVCALL_SEND_IPI:
-		if (unlikely(hc.rep || hc.var_cnt)) {
+		if (unlikely(hc.var_cnt)) {
  			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
  			break;
  		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, false);
-		break;
+		fallthrough;
  	case HVCALL_SEND_IPI_EX:

arch/x86/kvm/hyperv.c int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · line 2341

Quoted 10 lines
arch/x86/kvm/hyperv.c +2 -2
-		if (unlikely(hc.fast || hc.rep)) {
+		if (unlikely(hc.rep)) {
  			ret = HV_STATUS_INVALID_HYPERCALL_INPUT;
  			break;
  		}
-		ret = kvm_hv_send_ipi(vcpu, &hc, true);
+		ret = kvm_hv_send_ipi(vcpu, &hc);
  		break;
  	case HVCALL_POST_DEBUG_DATA:
  	case HVCALL_RETRIEVE_DEBUG_DATA:

I've smoke tested this (with the change I've mentioned above) and WS2019 booted with 65 vCPUs. This is a good sign)

Quoted 5 lines

The resulting merge commit is already in kvm/queue shortly (which should become the next kvm/next as soon as tests complete).

I see, please swap sse128_lo()/sse128_hi() there too :-)

Signature · 1 line
Vitaly
maintainer
#16 lore raw
Quoted 7 lines · Vitaly Kuznetsov

Let's say we have 1 half of XMM0 consumed. Now:

  i = 0;
  j = 1;
  if (1)
      sparse_banks[0] = sse128_lo(hc->xmm[0]);
Quoted 8 lines

This doesn't look right as we need to get the upper half of XMM0.

I guess it should be reversed,

      if (j % 2)
          sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);
      else
          sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);

Yeah, of course.

Thanks!

Paolo

#17 lore raw
Quoted 8 lines · Paolo Bonzini
Quoted 7 lines · Vitaly Kuznetsov

Let's say we have 1 half of XMM0 consumed. Now:

  i = 0;
  j = 1;
  if (1)
      sparse_banks[0] = sse128_lo(hc->xmm[0]);
Quoted 8 lines
Quoted 8 lines

This doesn't look right as we need to get the upper half of XMM0.

I guess it should be reversed,

      if (j % 2)
          sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);
      else
          sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);

Maybe I am missing parts of this series.. I dont see this change in any of the 4 patches Vitaly sent. Yes, they look swapped to me too.

~ Sid.

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

author maintainer
#18 lore raw
Quoted 13 lines · Siddharth Chandrasekaran
Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · Vitaly Kuznetsov

Let's say we have 1 half of XMM0 consumed. Now:

  i = 0;
  j = 1;
  if (1)
      sparse_banks[0] = sse128_lo(hc->xmm[0]);

This doesn't look right as we need to get the upper half of XMM0.

I guess it should be reversed,

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
      if (j % 2)
          sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);
      else
          sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);

Maybe I am missing parts of this series.. I dont see this change in any of the 4 patches Vitaly sent. Yes, they look swapped to me too.

There was a conflict with a patch series from Sean:
https://lore.kernel.org/kvm/[email protected]/

and this is a part of the resolution:

commit c0f1eaeb9e628bf86bf50f11cb4a2b671528391e
Merge: 4dfc4ec2b7f5 47d3e5cdfe60
Author: Paolo Bonzini <[email protected]>
Date: Fri Feb 25 06:28:10 2022 -0500

Merge branch 'kvm-hv-xmm-hypercall-fixes' into HEAD

Signature · 1 line
Vitaly
#19 lore raw
Quoted 26 lines · Vitaly Kuznetsov
Quoted 13 lines · Siddharth Chandrasekaran
Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · Vitaly Kuznetsov

Let's say we have 1 half of XMM0 consumed. Now:

  i = 0;
  j = 1;
  if (1)
      sparse_banks[0] = sse128_lo(hc->xmm[0]);

This doesn't look right as we need to get the upper half of XMM0.

I guess it should be reversed,

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
      if (j % 2)
          sparse_banks[i] = sse128_hi(hc->xmm[j / 2]);
      else
          sparse_banks[i] = sse128_lo(hc->xmm[j / 2]);

Maybe I am missing parts of this series.. I dont see this change in any of the 4 patches Vitaly sent. Yes, they look swapped to me too.

There was a conflict with a patch series from Sean:
https://lore.kernel.org/kvm/[email protected]/

Quoted 8 lines

and this is a part of the resolution:

commit c0f1eaeb9e628bf86bf50f11cb4a2b671528391e
Merge: 4dfc4ec2b7f5 47d3e5cdfe60
Author: Paolo Bonzini <[email protected]>
Date: Fri Feb 25 06:28:10 2022 -0500

Merge branch 'kvm-hv-xmm-hypercall-fixes' into HEAD

Got it, thank you!

~ Sid.

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879